It looks like I need to regenerate the auth key every 90 days, which kind of kills this for me. I definitely don't want to have to update all my docker stuff every 90 days, and it's almost assuredly going to go offline right when I can't deal with it.
The trick is to persist the tailscale var volume. The auth key is only used when setting up a particular client the first time, once it's connected to your network the auth key is irrelevant.
If you're doing this with ephemeral containers then yes you'll need a way to roll auth keys. OAuth credentials don't expire and Tailscale has a command line single purpose tool to get an auth key given OAuth credentials, so that can be a viable alternative.