It would be rather pointless to ship Perl4, as Perl5 is almost completely backwards compatible with Perl4. Most of the parts where backwards compatibility was removed weren't actually used to a large extent. Also Perl5 didn't add that much syntax, and a big reason it was ugly is that Perl4 didn't leave a lot of space for the extra features. Which is also why Perl6 breaks backwards compatibility.
There are also some features that were eventually removed from Perl5, but not really. For example setting $[ was removed (it was a pretty stupid idea), but if you try to use it Perl loads the module arybase which re-implements that feature. (removing it simplified the internals)
There are also some features that were eventually removed from Perl5, but not really. For example setting $[ was removed (it was a pretty stupid idea), but if you try to use it Perl loads the module arybase which re-implements that feature. (removing it simplified the internals)