How about switching the focus from your JButton to JFrame which will do the trick

Just add this code before setting your JFrame visibility
setFocusable(true);
before your
setVisible(true);...