http headers - Faking browser request in ASP.net C#

Question

I'm using the code below to pull one of our 3rd party developed pages in so I can parse it as XML for my random bits of work.

Irritatingly we stil have a browser detection level set on the server that only allows certain browsers on to the site; so the question is how would I fake it so that the server thinks its a browser request?

   static string GetHtmlPage(string strURL)
    {

        String strResult;
        System.Net.WebResponse objResponse;

        System.Net.WebRequest objRequest = System.Net.HttpWebRequest.Create(strURL);

        objResponse = objRequest.GetResponse();
        using (System.IO.StreamReader sr = new System.IO.StreamReader(objResponse.GetResponseStream()))
        {
            strResult = sr.ReadToEnd();
            sr.Close();
        }
        return strResult;
    }

Answer 1

Browser detection is done based on a header in the request to the server. All you need to do is set that header. However, with HttpWebRequest you don't set that through the headers collection but rather with the .UserAgent property.

...
System.Net.WebRequest objRequest = 
   System.Net.HttpWebRequest.Create(strURL);

//Pretend to be IE7
((System.Net.HttpWebRequest)objRequest).UserAgent = 
   "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)";

objResponse = objRequest.GetResponse();
...