You should always close resources after you're through with them, and scarce resources like database connections should be treated more carefully than most.
While an application should release them when closing, it's always good practice to do so deliberately, and make certain you don't keep them open longer than needed.
Case in point from the real world:
A major customer (this was a multinational) had a weird problem. One of their application servers (and the database server it was hooked up with) would crash every morning at around 9AM.
It would require a hard reboot of the database to get back online.
When investigating we found that the application was opening hundreds of database connections, and kept trying to open more and more until the database license ran out, after which it would keep polling and queueing requests for more connections until the database server crashed because of the overload.
All clients would then of course hang or crash as well.
The application was opening connections for every request to the database, and never closing them again.
Adding a few lines of code to handle connections properly cured the problem (which had been costing them tens of thousands of Euros per day for weeks) in minutes.
A more complete solution was complete and tested a few weeks later using connection pooling for better performance.