Library tutorials & articles

Retrieving HTTP content in .NET

Wrapping it up

By now you're probably getting an idea of the power that is provided by the HttpWebRequest object. While using these objects is a straight forward process it does require a fair amount of code and , using these objects required a fair amount of code and knowledge of a number of classes.

Since I use HTTP access in just about every application I create, I decided to create a wrapper class called wwHttp that simplifies the whole process quite a bit (the class is included in the code download for this article). Rather than creating two separate Request and Response objects the single class handles in a single object with simple string properties. The class handles setting up POST variables for you, creating any authentication and proxy settings from strings rather than objects, manages cookie and provides an optional simplified error handler that sets properties instead of throwing exceptions. It does this while allowing access to the base objects – you can pass in a WebRequest object and retrieve a reference to both the Request and Response objects, so you can get the best of both worlds with simplicity without giving up any of the features of the framework classes. The class also provides several overloaded methods for returning strings, streams and running output to file. The class can be set up to fire events at buffer retrieval points when data is downloaded to provide feed back in a GUI application.

Start by adding the namespace:

using Westwind.Tools.Http;

Using the class you can retrieve content from a site as simply as this:

wwHttp loHttp = new wwHttp();

loHttp.Username = "ricks";
loHttp.Password = "password";

loHttp.ProxyAddress = "http://proxy-server.hawaii.rr.com:8080";

loHttp.AddPostKey("Name","Rick Strahl");
loHttp.AddPostKey("Company","West Wind Technologies");
loHttp.HandleCookies = true; // enable automatic tracking for 5 cookies

string lcHtml = loHttp.GetUrl("http://www.west-wind.com/TestPage.wwd");

Most of those property settings are optional but just about everything in the class is accessible with simple strings. AddPostKey() automates the creation of UrlEncoded strings. Several different POST modes are supported including UrlEncoded (0), Multi-Part (2) and raw XML (4) POSTs.

The GetUrl() method has several different signatures. You can pass in an optional WebRequest object that is preconfigured so if you need to set one of the to override some properties that are not exposed you can override them there. For example:

HttpWebRequest oReq = (HttpWebRequest) WebRequest.Create(lcUrl);
oReq.Expires = -1
oReq.Headers.Add("SoapAction","http://west-wind.com/soap#Method");

wwHttp Request = new wwHttp();

string lcHTML = Request.GetUrl(oReq);

wwHttp also exposes Error and ErrorMsg properties that can be used to quickly check for error conditions:

lcHtml = Request.GetUrl(lcUrl);

if (Request.Error)
  MessageBox.Show(Request.ErrorMsg);
Else
  this.txtResult.Text = lcHtml;

Explict error retrieval is the default, but you can use the ThrowExceptions property to have the class pass forward exceptions to your code.

Comments

  1. 10 Jul 2007 at 15:29
      if (oReqCookie.Name ==
            oRespCookie.Name)  {
            oReqCookie.Value =
                   oRespCookie.Name;

    Do you mean?








     if (oReqCookie.Name ==
            oRespCookie.Name)  {
            oReqCookie.Value =
                   oRespCookie.Value;




  2. 12 Mar 2007 at 18:57

    You have a paragraph that states:

    "The most common use of delegates is an eventhandler, which uses the delegate to fire events. When the event publisher fires the event method, the delegate that is assigned to handle the event is called and you're event subscriber object then can simply handle the event by implementing a method in your class."

    Unfortunately, for someone trying desparately to learn C#, this sounds like:

    "blah blah blah blah blah blah"

    I'm sorry but could you publish a REAL SIMPLE example and explanation of asynch calls in Asp.net/C#.  I have been all over the web looking for one and they all assume you already know C# so all the delegate calling callback calling method calling delegate calling calling callingcallingcalling.................has just finally gotten on my nerves.

  3. 29 Dec 2006 at 22:27
    I'm just new to c# coming from VB.NET.  This is a great example!  Thanks
  4. 10 May 2006 at 04:30

    hi

    i have similar problem like you (in a previous time) with in using httwebrequest to login in a website... especially how to handle event clicking ex: onkeypress="checklogin()"

    i hope you can help me to solve the problem..

     

    thnx b4

  5. 18 Oct 2005 at 02:51

    hi
    I use httpwebrequest to login into a website (using Networkcrendential). When i got that page i need to click a link. How can we achieve this (axwebbrowser ?) and it contains frames. And also after clicking that menu in second page i have to click a Javascript button. Is there anyway to achieve this.


    Rajesh

  6. 25 Mar 2005 at 00:48
    Your article is very good and your source code is a good start.  It has some cool features but is lacking some good programming practices.  Most noteable is the lack of finally blocks used.  You should try to insure that the Close() method on the WebRequest and WebResponse objects are always closed before the object reference goes out of scope.  You can ensure that happens by placing your code in a try/finally block.

    Also, you should consider having your class implement the IDisposable interface.  And in that implementation clean up, close, and release the resources used by your private variables.
  7. 01 Jan 1999 at 00:00

    This thread is for discussions of Retrieving HTTP content in .NET.

Leave a comment

Sign in or Join us (it's free).

Rick Strahl
AddThis

Related discussion

Related podcasts

  • More jQuery in ASP.NET

    In this episode Chris Brandsma, Rick Strahl, Dave Ward, Bertrand Le Roy, and Scott Koon conclude their discussion of Microsoft's jQuery in ASP.NET announcement1.This episode of the Alt.NET Podcast is brought to you by LLBLGen Pro, the most mature O/R mapper and code generator out there.Are ...

Events coming up

  • Nov 18

    15 Minutes of Fame

    Dresher, United States

    This is a yearly tradition. We select 10 of the favorite speakers from monthly meetings, code camps, and hands on labs. Each one does a 15 minute talk on their favorite .NET technology. This is our 10th anniversary so we plan a gala event with special prizes and refreshments.

We'd love to hear what you think! Submit ideas or give us feedback