If you are using it for educational purposes then there are two options mentioned above by Mr. iamthwee.
And if using for some real application and u think that doing the swap without temporary variable would reduce the space required or the time then do think again. Its not that way. Current compilers are so optimised that using a temp varible doesnt even a fractiion affect the program.
But still if u are so peculiar about performance then usign Assembly Language is the way to go.
So u have two options:
1) use normal temp variable
2) use assembly language procedures
since using XOR and += have their own shortcomings.
For Each ctrl As Control In Me.Controls("pnlMainPanel").Controls
If ctrl.GetType Is GetType(System.Windows.Forms.Panel) Then
For Each subCtrl As Control In ctrl.Controls
If subCtrl.GetType Is GetType(System.Windows.Forms.TextBox) Then
If subCtrl.GetType Is ...