We would probably get a good amount of better performance out of it, but our current Heroku bill is low enough that even cutting it in half would be a small thing compared to changing the ruby version every developer uses.

While technically interesting, it just doesn't make any sense for us from a business perspective to put time into it.

I bet it would be a lot faster though. At least it is according to all the performance measurements I've seen in the past.