> Resident memory usage dropped from 87MB down to a mere 3MB, a 29x reduction!
This isn't so much Java vs. Go as it is JIT/interpreted vs. AOT-compiled. The numbers are entirely typical across a wide range of such comparisons.
With an interpreter or JIT, you need to load all your code at startup and process it. Generally, _all_ of your dependencies need to be loaded and parsed upfront, either converted to some internal in-memory representation or JIT'd directly to machine code. This will allocate a bunch of heap structures during the processing, and the end result is a bunch of data that needs to stay resident and can't easily be shared with other processes.
With AOT, you mmap() in a file. The OS only pages in the code you execute, and can page it back out as needed. Pages are shared between all processes running the same executable.
At sandstorm.io our rule of thumb is that an app written in Node, Ruby, Python, PHP, etc. will take 100MB of RAM while an app written in C++, Rust, or Go will take 2MB. Since Sandstorm runs per-user (and even per-document) app instances, this is a pretty big deal.
The good news is that https://github.com/google/snappy-startshould fix this problem: by checkpointing the process after it finishes its parsing/JITing but before it starts handling requests, we can get an mmap-able starting state that is very much like an AOT-compiled binary. At least, in theory -- there's still a bunch of work to do for this to actually work in practice.
> The resulting docker image shrunk from 668MB to 4.3MB
While I would expect the Go image to be smaller (since Go builds static binaries, so literally all you need in the image is the binary), I suspect that the 668MB Java image was at least 90% unnecessary garbage that was not actually needed at runtime. Unfortunately the package managers we all use are not optimized for containers; instead they evolved targeting systems with dedicated disks that can easily absorb gigabytes of bloat.
In Debian, for example, every package implicitly depends on coreutils. That's perfectly reasonable when installing an OS on a machine: you almost certainly need a working shell to boot and administer your machine. But a container can get by just fine without coreutils, and a typical web server probably (hopefully) doesn't need to call out to a shell. Even if a shell is needed, busybox/toybox is probably sufficient and will take a lot less space.
Packages also often contain things like documentation, unit tests, etc. which obviously aren't needed in a container.
For Sandstorm.io we deal with this problem by running the app in a mode where we trace all the files it actually uses, and then we build a package containing only those. It mostly works and manages to keep packages reasonably-sized, but it does lead to bugs of the form: "I forgot to test this feature while tracing, so the assets it requires didn't make it into the package." We're looking for better options.
> This isn't so much Java vs. Go as it is JIT/interpreted vs. AOT-compiled. The numbers are entirely typical across a wide range of such comparisons.
> [...] while an app written in C++, Rust, or Go will take 2MB.
Agreed. As mentioned in the blog post, I considered Rust but decided against it because of I found it much less mature than Go. I did not consider C++ because, as mentioned in the blog post, part of the point was to experiment with new language/tools and even though I consider myself proficient with it, I learned the hard way that the lack of memory safety is rarely worth it.
> I suspect that the 668MB Java image was at least 90% unnecessary garbage that was not actually needed at runtime. Unfortunately the package managers we all use are not optimized for containers
Exactly. That was part of the point of this blog post, which I may not have been successful at getting across. Switching to go was, if not the path of least resistance to solve this issue, at least one of a few relatively easy routes. It also happened to be a great deal of fun.
I assume you didn't use the normal Dockerfile-based build system to build your super-small Docker images for the Go-based services. So how did you do it?
We're not doing anything too fancy. Basically, we spawn a container to build a statically linked binary and do a regular Dockerfile-based build inside that container. The result is an image which contains only a single binary (any maybe some static assets like config files or images).
We're planning to open source our build script shortly.
We need to run the app up until the point when it diverges -- i.e. when it first observes input that will be different across different runs of the app. For that, we need to be watching the syscalls and evaluating each one for potential divergence. As long as we are doing that, we might as well at the same record a log of those syscalls which we can replay later. Then once a divergent syscall happens, we dump the state of memory. Later, we can restore the memory and replay the syscalls to reproduce an identical starting process.
CRIU has no concept of divergence. CRIU takes an already-running process with arbitrary state and snapshots it whole.
CRIU's problem is actually orders of magnitude more complicated than snappy-start's: it needs to understand every possible file descriptor type that the process could have open, every aspect of process state, etc. snappy-start only needs to understand the specific syscalls that we care to implement; it can simply consider any call it doesn't recognize as divergent, and stop there. Adding support for more syscalls is then merely an optimization.
CRIU also requires special kernel features to support, which means more attack surface. Sandstorm wants to block everything except the most common kernel APIs for security reasons. snappy-start requires no new kernel features; it uses the well-understood APIs debuggers use, and we know we can still prohibit apps themselves from using those APIs.
Meanwhile, CRIU is much harder to customize. How would we decide when to do the snapshot? We'd have to re-implement much of snappy-start just for that purpose. And how do we teach CRIU about the specific assumptions that are safe and useful to make given our particular environment?
None of this is to say that CRIU is bad -- it's actually pretty amazing. But it's not the best fit for this specific problem.
This isn't so much Java vs. Go as it is JIT/interpreted vs. AOT-compiled. The numbers are entirely typical across a wide range of such comparisons.
With an interpreter or JIT, you need to load all your code at startup and process it. Generally, _all_ of your dependencies need to be loaded and parsed upfront, either converted to some internal in-memory representation or JIT'd directly to machine code. This will allocate a bunch of heap structures during the processing, and the end result is a bunch of data that needs to stay resident and can't easily be shared with other processes.
With AOT, you mmap() in a file. The OS only pages in the code you execute, and can page it back out as needed. Pages are shared between all processes running the same executable.
At sandstorm.io our rule of thumb is that an app written in Node, Ruby, Python, PHP, etc. will take 100MB of RAM while an app written in C++, Rust, or Go will take 2MB. Since Sandstorm runs per-user (and even per-document) app instances, this is a pretty big deal.
The good news is that https://github.com/google/snappy-start should fix this problem: by checkpointing the process after it finishes its parsing/JITing but before it starts handling requests, we can get an mmap-able starting state that is very much like an AOT-compiled binary. At least, in theory -- there's still a bunch of work to do for this to actually work in practice.
> The resulting docker image shrunk from 668MB to 4.3MB
While I would expect the Go image to be smaller (since Go builds static binaries, so literally all you need in the image is the binary), I suspect that the 668MB Java image was at least 90% unnecessary garbage that was not actually needed at runtime. Unfortunately the package managers we all use are not optimized for containers; instead they evolved targeting systems with dedicated disks that can easily absorb gigabytes of bloat.
In Debian, for example, every package implicitly depends on coreutils. That's perfectly reasonable when installing an OS on a machine: you almost certainly need a working shell to boot and administer your machine. But a container can get by just fine without coreutils, and a typical web server probably (hopefully) doesn't need to call out to a shell. Even if a shell is needed, busybox/toybox is probably sufficient and will take a lot less space.
Packages also often contain things like documentation, unit tests, etc. which obviously aren't needed in a container.
For Sandstorm.io we deal with this problem by running the app in a mode where we trace all the files it actually uses, and then we build a package containing only those. It mostly works and manages to keep packages reasonably-sized, but it does lead to bugs of the form: "I forgot to test this feature while tracing, so the assets it requires didn't make it into the package." We're looking for better options.