LLVM is a good deal lower-level than the JVM. It lets you build representations of a program in an internal representation that's higher-level than assembly but doesn't limit you as much as the JVM; for example, it doesn't come with a built-in garbage collector and allows tail-call optimization. LLVM makes a great compiler back-end, does a bunch of optimizations, and can be used at runtime to do on-the-fly compilation.

Once VMKit is more stable, it'll be able to run Java code and .NET code on the same LLVM-based VM, which is nice.

LLVM supports a bunch of different processors and platforms as backends. If you're writing a compiler, this is a godsend.