Nope... as long as the domain name is the same it dont matter who hosts the site. The only thing you might notice is that the site may be down for a couple days due to DNS updating.

If you upload your site to the new server, change your DNS, and leave the old site up for a week you won't have any downtime.

You shouldn't experience a penalty or drop in ranking simply because you're moving hosts so long as you don't leave the domain 'sitting' for a period of time.

Define "sitting"

I've dealt with quite a few clients that refer to it as sitting, others refer to it with other terms. More so when the domain is left unattended or the DNS is pointed to a canceled/dead account without content.

You'd be surprised how many people I've seen cancel a hosting account before switching the DNS only to find that their domain no longer points to the correct location. They either forgot to update the DNS, were not informed they needed to by the host or if the domain is hosted with their current host, the host simply neglected to change the information.

generally there arent any ranking problems when moving hosts:
you may also want to read matt cutts' blog which provides a nice step-by-step guide in this regards
http://www.mattcutts.com/blog/moving-to-a-new-web-host/