Retrieving HTTP content in .NET

Wrapping it up

By now you're probably getting an idea of the power that is provided by the HttpWebRequest object. While using these objects is a straight forward process it does require a fair amount of code and , using these objects required a fair amount of code and knowledge of a number of classes.

Since I use HTTP access in just about every application I create, I decided to create a wrapper class called wwHttp that simplifies the whole process quite a bit (the class is included in the code download for this article). Rather than creating two separate Request and Response objects the single class handles in a single object with simple string properties. The class handles setting up POST variables for you, creating any authentication and proxy settings from strings rather than objects, manages cookie and provides an optional simplified error handler that sets properties instead of throwing exceptions. It does this while allowing access to the base objects – you can pass in a WebRequest object and retrieve a reference to both the Request and Response objects, so you can get the best of both worlds with simplicity without giving up any of the features of the framework classes. The class also provides several overloaded methods for returning strings, streams and running output to file. The class can be set up to fire events at buffer retrieval points when data is downloaded to provide feed back in a GUI application.

Start by adding the namespace:

using Westwind.Tools.Http;

Using the class you can retrieve content from a site as simply as this:

wwHttp loHttp = new wwHttp();

loHttp.Username = "ricks";
loHttp.Password = "password";

loHttp.ProxyAddress = "";

loHttp.AddPostKey("Name","Rick Strahl");
loHttp.AddPostKey("Company","West Wind Technologies");
loHttp.HandleCookies = true; // enable automatic tracking for 5 cookies

string lcHtml = loHttp.GetUrl("");

Most of those property settings are optional but just about everything in the class is accessible with simple strings. AddPostKey() automates the creation of UrlEncoded strings. Several different POST modes are supported including UrlEncoded (0), Multi-Part (2) and raw XML (4) POSTs.

The GetUrl() method has several different signatures. You can pass in an optional WebRequest object that is preconfigured so if you need to set one of the to override some properties that are not exposed you can override them there. For example:

HttpWebRequest oReq = (HttpWebRequest) WebRequest.Create(lcUrl);
oReq.Expires = -1

wwHttp Request = new wwHttp();

string lcHTML = Request.GetUrl(oReq);

wwHttp also exposes Error and ErrorMsg properties that can be used to quickly check for error conditions:

lcHtml = Request.GetUrl(lcUrl);

if (Request.Error)
  this.txtResult.Text = lcHtml;

Explict error retrieval is the default, but you can use the ThrowExceptions property to have the class pass forward exceptions to your code.

You might also like...



Why not write for us? Or you could submit an event or a user group in your area. Alternatively just tell us what you think!

Our tools

We've got automatic conversion tools to convert C# to VB.NET, VB.NET to C#. Also you can compress javascript and compress css and generate sql connection strings.

“A computer is a stupid machine with the ability to do incredibly smart things, while computer programmers are smart people with the ability to do incredibly stupid things. They are, in short, a perfect match” - Bill Bryson