is there a performance penalty when running a 32 bit application on a 64 bit machine becasue in Windows, it uses wow64 to switch modes?

For the most part no, there is no penalty. The x64 instruction set is an extension to the x86 (32bit). This means that all the x86 instructions are still supported and executed natively by the CPU, i.e., there is no "virtual machine" or other emulation layer. There is no reason for the 32bit instructions to be executed slower than the 64bit ones. There might be some negligible overhead, but nothing noticeable.

The WoW64 thing mostly pertains the OS environment outside the application (virtual addressing, using drivers, kernel calls, etc.), but it doesn't get in-between the application and the CPU. In other words, it makes the application think that it is running in a classic Win32 environment, and since all those OS-related calls are always very slow, the overhead created by wow64 is negligible (just executes a few arithmetic conversions, which is nothing in comparison to making a kernel call).