Since there was just a posting on
HttpWebRequest and since it just so happens that I've been having fun with that very class today, I though I'd add my two cents regarding this wicked cool type.
I have a collection of URLs, a couple of hundred for example, and need to query each URL to determine if the site still exists and, if so, when it was last modified. Doing this synchronously could take a couple of minutes depending on bandwidth and traffic.
However, by scanning the URLs asynchronously, you can build a much more responsive and user-friendly application.
There are three important points that need to be integrated to completely solve this:
- Scan the list in a new thread so your UI stays responsive.
HttpWebRequest.BeginGetResponse()to initiate an asynchronous request.
ThreadPool.RegisterWaitForSingleObject()to register a timeout delegate for unresponsive Web requests.
Yo! Hands off the Thread, Man!
Thread t = new Thread(new ThreadStart(ScanSites));
t.IsBackground = true;
As you might guess, the ScanSites method (a custom method shown below) will run under a thread separate from the
Windows.Forms UI. The user will be able to interact with the application without noticing the background process chugging along (hopefully).
Reach Out and Touch Someone
A Web request begins its life fairly mundane. You first need to create a new request. The concrete
HttpWebRequest.Create() method returns an abstract
WebRequest object. You can modify its properties before calling
BeginGetResponse() method is typical of many other asychronous kick-off routines: it requires a pointer to a callback routine and a user-defined argument.
private void ScanSites ()
// for each URL in the collection...
WebRequest request = HttpWebRequest.Create(uri);
request.Method = "HEAD";
// RequestState is a custom class to pass info
RequestState state = new RequestState(request,data);
IAsyncResult result = request.BeginGetResponse(
// PLACEHOLDER: See below...
private void UpdateItem (IAsyncResult result)
// grab the custom state object
RequestState state = (RequestState)result.AsyncState;
WebRequest request = (WebRequest)state.request;
// get the Response
HttpWebResponse response =
// process the response...
So far, so good. But what's up with the PLACEHOLDER comment? Read on my friend. Read on...
Careful with that Axe Eugene!
WebRequest class has a Timeout property, it is ignored when using asynchronous requests. So we need to set up our own timer to keep an eye on lengthy HTTP calls. If a call takes too long, we should jump in and abort it, probably marking the URL as suspicious (for example, the site is down or no longer exists).
Here's an example. Replace the PLACEHOLDER comment above with the following call. Then add the
ScanTimeoutCallback routine somewhere in your class.
(30* 1000), // 30 second timeout
private static void ScanTimeoutCallback (
object state, bool timedOut)
RequestState reqState = (RequestState)state;
if (reqState != null)
Not too hard, ay? We've covered all three points with pretty straighforward code. Multi-threaded programming in
Windows.Form is trivial, but required for a good user experience. Sending asynchronous HTTP requests is equally trivial; just remember to construct a custom state object containing all the relevant information you may need within the
AsyncCallback. And, finally, remember to abort requests that refuse to complete. A little code goes a long way.