What’s the difference between a Bastion Host and a Proxy Server, as part of a firewall strategy?

A bastion host is one where 3rd party clients are allowed an SSH session. From this 'bastion' host, the client could then open a new session to internal hosts. Using a Bastion is a way to control all external traffic entering your site. Using this along with a VPN is a nice control mechanism.