Without using a library, you're simply going to have to do it yourself.
Frankly, you need to start by thinking rather than coding. You can read the xml file easily. Open it, read in data. But that just gets you a lot of strings. How do you intend to organise what you read?
The thing about xml is that each tag doesn't have to be on a separate line - an entire file can be one line. You have to parse it directly. You can either do this yourself, or use a library such as xerxes. I have done it both ways. Generally, for really high speed or high volume needs, I roll my own parser and generic structures to model the xml found.
For Each ctrl As Control In Me.Controls("pnlMainPanel").Controls
If ctrl.GetType Is GetType(System.Windows.Forms.Panel) Then
For Each subCtrl As Control In ctrl.Controls
If subCtrl.GetType Is GetType(System.Windows.Forms.TextBox) Then
If subCtrl.GetType Is ...