I dare you to tell my why lxc-docker silently fails to start at boot in jessie. Without a single shred of clue in any log, no matter what logging options are turned up - at least with a shell script I can add some echo/exit/touch etc. statements to convince myself it's at least being noticed at startup: https://groups.google.com/forum/#!topic/docker-user/bg5-hlmR...
> I dare you to tell my why lxc-docker silently fails to start at boot in jessie. Without a single shred of clue in any log, no matter what logging options are turned up - at least with a shell script I can add some echo/exit/touch etc. statements to convince myself it's at least being noticed at startup: https://groups.google.com/forum/#!topic/docker-user/bg5-hlmR....
Have you tried adding "touch" commands in ExecStartPre/ExecStartPost? Systemd also gives you a hint of why something didn't start via "systemctl status $servicename".
No offence, but systemctl status isn't exactly new to me. The issue is that network.target apparently never fires any service which requires it (in Jessie at least). For what it's worth systemctl status docker shows it's just inactive (dead) - journalctl similarly shows no clues at all. I have a Debian pressed.cfg & virt-install script to reproduce if you're interested, https://gist.github.com/csirac2/f6e4ac6e7c715243d4b8 and https://gist.github.com/csirac2/57a1ddf7931fa3e9d17a although these aren't up to date with what I've been preparing for a bug report
The next step in troubleshooting that one is to run strace and find out what's happening. I agree it's frustrating but if no one debugs these things, problems won't get solved. (and yes, scorching the earth and installing a different distro is a valid solution to the problem at hand)
I'm not sure how to make strace pid 1 :-) And I'm not quite ready to abandon Debian yet. I've created a virt-install script which reproduces the problem reliably (honestly all it does is install jessie, then lxc-docker). Hopefully my workload eases and I can put this trivially reproducible thing into a bug report somewhere (debian? systemd? docker) for somebody to see what's going on.
FWIW systemctl start docker works fine after the system has booted. It's configured to require network.target, and even though I can see network target scroll by during boot, it's still not being noticed.
The work-around is to make multiuser want docker, then I've got docker starting at bootup.
Sounds like lxc-docker.service simply wasn't enabled. "enabling" is basically creating a symlink from multi-user.target.wants to lxc-docker.service. This is done by 'systemctl enable lxc-docker.service' once that service declares WantedBy=multi-user.target in its [Install] section.
Other symptoms that would indicate this was the problem would have been that 'systemctl status lxc-docker.serivce' would have said "Loaded: loaded (...; disabled)" and "Active: inactive (dead)". i.e., systemd never started the service because nothing ever wanted it.
strace -p1 should do the trick, you can attach after start, but you will have to get it started early, probably as a systemd service. Or if its in a container, strace form outside the container.
Without doing any further investigation: does the type of service you're setting in your user file match the way you're starting up the daemon? For a "normal" service, you need to start the service as "normal" and have it run without forking / exiting (as systemd handles daemonization).
If you start a "normal" service using a command that double-forks and immediately exits (like a `--daemon` flag), systemd will see that the process has exited to mean that the service has finished, and will terminate any of its child processes (as you want runaway dependencies culled when a service crashes). This sounds like the behavior you describe.
