C# test app - does it need user agent header

[frank754]

I've written a C# app in Visual Studio 2008 C# Windows forms which fetches the text html from a webpage and it works fine in most cases. I needed the multipart/form data, as the main interest is sending text queries back and forth from a test server in Visual Studio.
For most other websites it works fine, I get the full output of the index.html page or whatever page I want. (I know C# has a GUI graphical web browser like the picture box, but I'm more interested in my own coding).
Works fine with my Apache server, and my website on Bravenet.
Does not work with my IIS server nor with www.google.com
Both give me a 405 "Method not allowed"
Edit: It does work though with
"http://google.com/gmail" so maybe it depends on the exact page.

I have been thinking that I need to add a header with user-agent or some sort of browser identification. In my searches, some say this is not enabled in this format.
I've been trying all afternoon to no avail but if any of you have an easy solution,
please let me know. As I said, the app works fine, but it's the pickiness of the servers that make it hit and miss.
Thanks in advance.

var request = WebRequest.Create(url);   //url = the text, i.e. http://etc....
                request.ContentType = "multipart/form-data";
                request.Method = "POST";
                string postData = requesttext;
                byte[] bytes = Encoding.ASCII.GetBytes(postData);
                request.ContentLength = bytes.Length;
                var requestStream = request.GetRequestStream();
                requestStream.Write(bytes, 0, bytes.Length);
                requestStream.Close();
                var response = request.GetResponse();
                if (response == null) return;
                var reader = new StreamReader(response.GetResponseStream());
                outputtext += reader.ReadToEnd().Trim();

[gashtio]

The User-agent field is mainly used for formatting the response accordingly (e.g. for mobile phones) and shouldn't result in an error code(4xx). The 405 you're getting means that the method used in the request (POST) is not allowed. If you examine the headers of the response from google.com you'll see "Allow: GET, HEAD" so you'll have to change the request method to either of those.

On a side note, if you're only going to be sending text and not binary data, you don't need the "multipart/form-data" content type.