> Does this mean that i need to do this for every servlet? Which is not efficient.
Yes, it's not efficient. The best approach when dealing with database resources would be using the connection pooling capabilities offered by your container. In case of application servers like Glassfish, Jboss etc. look into the application server documentation for creating a connection pool and exposing it over JNDI using the admin console of that app server.
In case you are using a simple container like Tomcat, Jetty etc. look into the documentation provided to get a list of configuration files which need to be changed/created to expose a connection pool over JNDI.
In case you are required to programatically manage your connection pools, look into the discussion present in this thread and post if you still have issues.