What you want is something like Janino. We've used it for years. You give it (near standard) code and it gives you the classes so you can use them. It actually has quite a few different modes and supports the 1.5 syntactic sugar and auto-boxing and such.
If you call javac, not only will you have to be ready for anything it does, you'll then have to handle putting the class in the right place or making an additional classloader.
Janino is very easy. It should be exactly what you are looking for.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…