You may find this page at OSdev.org enlightening on the subject. Basically, the main reason Java in its usual form is problematic for systems programming is that it compiles to a bytecode rather than to native code. While this by itself is not a showstopper - it is possible to compile Java natively, or convert the bytecode to native code (this is what the JIT compiler does, in fact) - it makes it less desireable as a systems language.
There are other reasons, though. It is a garbage collected language, which means that the memory management has to be be in place and running before any Java programs can be run. It does not have programmer accessible pointers, which makes accessing memory-mapped hardware problematic at best. It has in-language threading, which means that the process management and scheduling needs to be in place ahead of time as well. Overall, it is a language with a high overhead.
Mind you, even C++ isn't particularly good for systems programming; there are reasons beyond just Linus' dislike of C++ that C remains the predominant systems language. C++ exception handling and other features are overhead that require a lot of systme support as well.
All in all, either Java or C++ can be used for systems programming, but it would require a lot of prep work to get the system ready for either one, work that would have to be done ain assembly or a true systems programming language like C.