I did try exactly that, but the GC overhead was still prohibitively high for my use case, made because big arrays in practice are composed from shorter non-contiguous arrays to make garbage collection even remotely possible.

Hmmm, if the typed array only contains primitives, the GC isn't looking inside the array. If the thing pointed to, can never contain a pointer, it won't need to be scanned.