There are books written on this subject alone, it is very difficult to give an answer in single post in a volunteer forum. There are people @ £750 per day to do this work, so you won't get it free here.

Also without intimate knowledge of your application and database it is impossible to tell you where the bottlenecks are.

But typically you are looking to improve in several areas, I see already you try bigger hardware.

You also need to consider indexing, this involves analysing every single query for table joins and criteria to ensure you have indexes on the right columns so the database can find records quicker.

How does the application retreive data ? is it using disconnected data ? or forward only cursors that hold a connection open for longer locking the db for longer.

Also what kind of database do you have is it OLTP (ie a database receiving many transactions from many users) but they mostly only access the latest data, in which case you want to move some of your 17gb of older data into another database optimized for reporting. A data warehouse as it's often refered to.

Your other option to consider is building a tier between you application and the database (SOA) Service Oriented Architecture (or something like that) This can cache frequent data requests and relieve the database from some it's load.