That's exactly what ARM did with SVE, so it's easy to discuss the pros and cons.
The biggest problem, to my understanding, is that SIMD extensions are mostly targeted with optimized hand-written assembly code. It's fine to say that the SIMD is variable-width, but you still have to run this on a processor where the hardware is not variable-width. Not just the SIMD unit, but also the feedpaths/etc. And some things a "generic" framework might let you do, might have catastrophic performance consequences.
In practice a lot of this will be avoided by just knowing the hardware you're going to target and not doing the things that hurt performance, but that also completely misses the point of variable-width, you might as well just target the hardware directly at that point.
And actually your data may not be "variable-width" either.
It seems to me it's extremely common to work with vectors that are much larger than the host CPU's SIMD width. In that case you could get much better performance letting the CPU schedule a large batch of work rather than trying to micro-optimize for multiple possible targets.
Software is typically compiled for x86-64, but not necessarily for each specific x86-64 processor available. You often just get a generic precompiled binary. That's not really optimized for the machine you're running the software on.
Maybe you want to have some restrictions on the size of the vectors you're working with to make it more convenient for the hardware, like "should be a multiple of 4x64 bits", but I still think there could be big gains with variable width in most cases.
I thought the same but that’s not how that works. It’s variable width at the assembly level. You give it entire buffers with the total length and it internally uses the appropriate width (or tells you the width it used during the loop? I forget that detail). Anyway, it’s not a SW framework but a core part of the CPU implemented in Hw.
That's not how SVE works. Any given SVE CPU still has a fixed vector width at runtime, but the CPU exposes instructions which tell you the vector width and other convenience instructions for performing loop counter math and handling the "final" iteration which may not use a full vector.
Then you need to write your loops in such a way that they work with a variable width: i.e., doing more iterations if the vector width is smaller.
The biggest problem, to my understanding, is that SIMD extensions are mostly targeted with optimized hand-written assembly code. It's fine to say that the SIMD is variable-width, but you still have to run this on a processor where the hardware is not variable-width. Not just the SIMD unit, but also the feedpaths/etc. And some things a "generic" framework might let you do, might have catastrophic performance consequences.
In practice a lot of this will be avoided by just knowing the hardware you're going to target and not doing the things that hurt performance, but that also completely misses the point of variable-width, you might as well just target the hardware directly at that point.
And actually your data may not be "variable-width" either.