It does, to some extend. There aren't JVMs available for all hardware architectures and operating systems, and there are some differences in the scope of the available JVMs that do exist.

So how can I find this out while choosing one?

Check the specs for the machines you're interested in. I don't have pointers ready.
Did some checking and found the situation confusing to say the least, we ended up creating a web application instead with HTML specifically tailored for the less capable browsers and smaller screens of PDAs.
That way we could use the full power of J2EE while still targeting low spec hardware.

Right now I'm not sure what I'm going to get. I don't really now anything about them, so I'll have to do some research on what to get. I'll probably get a descent one, but get it off of ebay so I don't have to pay so much.