Retrieving HTTP content in .NET

HTTP Cookies

HTTP Cookies are a state management implementation of the HTTP protocol and many Web pages require them. If you're using remote HTTP functionality to drive a Web site (following URLs and the like) you will in many case have to be able to support cookies.

Cookies work by storing tokens on the client side, so the client side is really responsible for managing any cookie created. Normally a browser manages all of this for you, but here there's no browser to help out in an application front end and we're responsible for tracking this state ourselves. This means when the server assigns a cookie for one request, the client must hang on to it and send it back to the server on the next request where it applies (based on the Web site and virtual directory). HttpWebRequest and HttpWebResponse provide the container to hold cookies both for the sending and receiving ends but it doesn't automatically persist them so that becomes your responsibility.

Because the Cookie collections are nicely abstracted in these objects it's fairly easy to save and restore them. The key to make this work is to have a persistent object reference to the cookie collection and then reuse the same cookie store each time.

To do this let's assume you are running the request on a form (or some other class – this in the example below). You'd create a property called Cookies:

CookieCollection Cookies;

On the Request end of the connection before the request is sent to the server you can then check whether there's a previously saved set of cookies and if so use them:

Request.CookieContainer = new CookieContainer();
if (this.Cookies != null &&
  this.Cookies.Count > 0)
   Request.CookieContainer.Add(this.Cookies);

So, if you previously had retrieved cookies, they were stored in the Cookies property and then added back into the Request's CookieContainer property. CookieContainer is a collection of cookie collections – it's meant to be able to store cookies for multiple sites. Here I only deal with tracking a single set of cookies for a single set of requests.

On the receiving end once the request headers have been retrieved after the call to GetWebResponse(), you then use code like the following:

// *** Save the cookies on the persistent object
if (Response.Cookies.Count > 0)
  this.Cookies = Response.Cookies;

This saves the cookies collection until the next request when it is then reassigned to the Request which sends it to the server. Note, that this is a very simplistic cookie management approach that will work only if a single or a single set of cookies is set on a given Web site. If multiple cookies are set in multiple different places of the site you will actually have to retrieve the individual cookies and individually store them into the Cookie collection. Here's some code that demonstrates:

if (loWebResponse.Cookies.Count > 0)
  if (this.Cookies == null)
  {
   this.Cookies = loWebResponse.Cookies;
  }
  else
  {
   // If we already have cookies update list
   foreach (Cookie oRespCookie in
        loWebResponse.Cookies)
   {
     bool bMatch = false;
     foreach(Cookie oReqCookie in
         this.oCookies) {
      if (oReqCookie.Name ==
        oRespCookie.Name)  {
        oReqCookie.Value =
               oRespCookie.Name;
        bMatch = true;
        break;
      }
     }
     if (!bMatch)
      this.Cookies.Add(oRespCookie);
   }
  }
}

This should give you a good starting point. This code still doesn't deal with things like domains and virtual paths and also doesn't deal with saved cookies, but for most applications the above should be sufficient.

You might also like...

Comments

Contribute

Why not write for us? Or you could submit an event or a user group in your area. Alternatively just tell us what you think!

Our tools

We've got automatic conversion tools to convert C# to VB.NET, VB.NET to C#. Also you can compress javascript and compress css and generate sql connection strings.

“Debugging is anticipated with distaste, performed with reluctance, and bragged about forever.” - Dan Kaminsky