you need a cross-platform USB stick program, with all your secrets on it encrypted properly, for that kind of thing. (The simplest hack I can think of would probably be python binaries, with a local-webserver based interface.)
I don't know of such an application, or whether the approach is rigorous, I'm afraid. But I think that's the shape of the solution.
First, I'm not remotely interested in running my own mail infrastructur anymore. Been there, done that. Today it's much too hard to get mails accepted by others.
But more important: iPads don't have an USB connector, my mobile phone doesn't have one. Friends have Macs, in other places there might be other crippled devices.
The web is a universal building block. USB sticks are not.
Sorry, I wasn't clear. I use last pass to transfer my private key to the PGP app on my iPad (and elsewhere) (as opposed going through Dropbox or whatever).
Relies upon trusting last pass and trusting the iPad of course, both of which are questionable.
Why rely on the (possibly compromised) OS of the host computer's hard drive when you can boot your own OS straight from the USB itself? What you are looking for is tails (https://tails.boum.org/) with a luks encrypted persistent partition.
I don't know of such an application, or whether the approach is rigorous, I'm afraid. But I think that's the shape of the solution.