The .NET FileSystemWatcher class

This article was originally published in VSJ, which is now part of Developer Fusion.
FileSystemWatcher is a class in .NET that makes it straightforward to watch for changes in the filesystem, and take some action based on the changes. This suggests that it should be possible to use it as the basis for all sorts of interesting applications.

A few that spring to mind include:

  • Automatically processing or indexing newly created content
  • Automatically updating a web site to incorporate new pages (e.g. adding them to an index page or product list)
  • Automatically updating a database with information on files in particular locations
  • Creating a UI that works like explorer and updates automatically when the filesystem is updated by another application
  • Allowing collaborative working by detecting and acting appropriately when another user updates a file
Here I will describe the functionality in FileSystemWatcher, and give some examples of what it can be used to enable. I will also identify the challenges that limit its usefulness, and present some workarounds.

Photo library application

To introduce FileSystemWatcher, I’d like to work through a small sample application. At the company I work for we have a photo library. Each photo is stored in two folders at different resolutions, one for presentations and the other for printing. Persuading people to add pictures to the library correctly has proved impossible, so at the moment they are all added manually by one person, who ensures they are saved at appropriate resolutions. Even then, mistakes are still made. This process is illustrated in Figure 1.

Figure 1
Figure 1: The manual process

Using FileSystemWatcher and the .NET Image class, this process can be automated. We’ll create a command line application for simplicity, but the same code could be used in a Windows Service (or in a Forms application, perhaps with a status display). This service could run on a server and implement the process seamlessly in the background, and while write access to the library could be disabled for normal users, the account running the service would have it enabled. This would prevent users circumventing the correct procedure!

All we need to do is detect the arrival of a file in the “Submit” folder, create copies of it at the appropriate resolutions in the “Presentation” and “Print” folders, and remove the original from the submission folder. To detect the file arriving in “Submit”, we first create a FileSystemWatcher:

static void Main(string[] args)
	{
	FileSystemWatcher f =
		new FileSystemWatcher(
		@”C:\PhotoLibrary\Submit”,
		“*.jpg” );
The constructor overload we’ve used takes the path to monitor and a filter string to select the files on that path to be monitored. There are various properties on the FileSystemWatcher that can be used to modify its behaviour, the most important of which is IncludeSubdirectories. This is false by default, which is fine for this application. The next thing we need to do is hook up to the events that we wish to receive. Table 1 shows the events supported by the object.

Table 1: Events supported by FileSystemWatcher

Event When it occurs
Changed When a file or directory has its content or attributes modified
Created When a new file is created, or a file is moved into a monitored directory
Deleted When a file is deleted or moved out of a monitored directory
Renamed When a file is renamed
Error When an error occurs in the FileSystemWatcher object (e.g. the internal buffer overflows)
Disposed When the FileSystemWatcher object is disposed

For this application the Created event is the only one that is of interest, so we need to hook up an event handler. Once we’ve done this, we must set the EnableRaisingEvents property to true to tell the object that we’ve configured it, hooked up our event handlers, and are ready to receive events:

	f.Created += new
		FileSystemEventHandler(
		f_Created);
	f.EnableRaisingEvents = true;
Now as far as our Main function is concerned, that’s the job done. We just need to wait for the events to arrive, and provide a way for the user to quit the application:
	Console.WriteLine(“Waiting...
		Hit return to quit” );
	Console.Read();
	f.Dispose();
}
The event handler is where all the work gets done. The first thing it must do is open the image file:
Image i = Image.FromFile(e.FullPath);
Next we use a routine to save the file with specified resolution to the appropriate locations:
SaveImageMaxRes(
	@”C:\PhotoLibrary\Presentation\”
	+ e.Name, i, new Size(100, 100));
SaveImageMaxRes(
	@”C:\PhotoLibrary\Print\”
	+ e.Name, i, new Size(200, 200));
Finally, we must Dispose the image object (the .NET framework keeps the file open until the Image object is disposed) and delete the original file. We emit a message to the console too, so that we can see what’s going on:
i.Dispose();
File.Delete( e.FullPath );
Console.WriteLine(“Completed: “
	+ e.Name);
The complete source code for the Main function and event handler is shown below:
static void Main(string[] args)
{
	FileSystemWatcher f = new FileSystemWatcher(@”C:\PhotoLibrary\Submit”,
		“*.jpg”);
	f.Created+=new FileSystemEventHandler(f_Created);
	f.EnableRaisingEvents = true;
	Console.WriteLine(“Waiting... Hit return to quit.”);
	Console.Read();
	f.Dispose();
}

private static void f_Created(object sender, FileSystemEventArgs e)
{
	Image i = Image.FromFile(e.FullPath);
	SaveImageMaxRes(@”C:\PhotoLibrary\Presentation\” + e.Name, i,
		new Size(100, 100));
	SaveImageMaxRes(@”C:\PhotoLibrary\Print\” + e.Name, i, new Size(200, 200));
	i.Dispose();
	File.Delete(e.FullPath);
	Console.WriteLine(“Completed: “ + e.Name);
}
The code for the SaveImageMaxRes function is in shown below, for the sake of completeness:
static void SaveImageMaxRes(String Path, Image ImageObject, Size MaxSize)
{
	// Calculate the correct sizes for the output image
	Size OutputSize = ImageObject.Size;
	float AspectImage = (float)OutputSize.Height/(float)OutputSize.Width;
	float AspectMax = (float)MaxSize.Height/(float)MaxSize.Width;
	if (AspectMax < AspectImage)
	{
		// Image is TALLER
		OutputSize.Height = MaxSize.Height;
		OutputSize.Width = (int)((float)MaxSize.Height/AspectImage);
	}
	else
	{
		// Image is WIDER
		OutputSize.Width = MaxSize.Width;
		OutputSize.Height = (int)((float)MaxSize.Width * AspectImage);
	}

	// Never scale an image UP in size
	if ((OutputSize.Width > ImageObject.Width) ||
	(OutputSize.Height > ImageObject.Height))
	{
		OutputSize = ImageObject.Size;
	}

	// Create the render canvas
	Bitmap b = new Bitmap(OutputSize.Width, OutputSize.Height, 
		PixelFormat.Format32bppArgb);
	Graphics g = Graphics.FromImage(b);

	// Render
	g.DrawImage(ImageObject, new Rectangle(new Point(0, 0), OutputSize));

	// Output the result as a JPEG
	b.Save(Path, ImageFormat.Jpeg);

	// Tidy Up
	g.Dispose();
	b.Dispose();
}
To test this application, you must first ensure that you have created all the directories that are hard coded (in a real application of course these folders would be configured through UI or a App.Config file). Next you just need to fire up the application, and copy a .JPG image file into the “Submit” folder. Within a very short time, it will disappear, and versions with maximum size of 100 x 100 and 200 x 200 will appear in the “Presentation” and “Print” folders respectively (these sizes are unrealistic, but should ensure that you can see what’s going on). The command window where the application is running will look something like Figure_2.

Figure 2
Figure 2: The command window

This application is a simple example showing how FileSystemWatcher can enable utilities and applications that would have taken a lot of code to develop under Win32. I wrote this application in a single code file with 84 lines – and half of those are whitespace or comments.

Interpreting the notifications received from the FileSystemWatcher is pretty straightforward for an application like this, but some operations may result in more than one notification. For example, if you are monitoring a directory tree and a file is moved from one folder to another, you can expect to see a Deleted notification on the source file, a Created on the destination file, and Changed notification on both the source and destination directories. This can make figuring out what events represent which “user” operation quite complicated.

Change notifications and synchronisation

In many applications it would be useful to use change notifications to keep some other representation of the filesystem synchronised with the filesystem itself.

The most obvious example of a representation of the filesystem would be the visual one – the explorer type view. There are plenty of applications where it is desirable to display lists or trees of files in the application style rather than explorer style, and for these views to update just like explorer does. There are other applications where you might want to keep an HTML page or database updated with information relating to files such as product description pages or photos. It may even be useful to store information about files in a database, and have a server application update the file location in the database when the user moves the file in explorer.

Look at the first snippets of code in Table 2, showing the intuitive approaches to synchronising an in-memory list of files (this.Files) with the filesystem.

Table 2: Synchronising an in-memory list of files

(1) Get File Listing First (2) Enable Events First
FileSystemWatcher f;
DirectoryInfo d;
// Initialise f and d
this.Files = d.GetFiles();
// File "Foo" deleted
f.EnableRaisingEvents = true;
FileSystemWatcher f;
DirectoryInfo d;
// Initialise f and d
f.EnableRaisingEvents = true;
// File "Foo" deleted
this.Files = d.GetFiles();	
// File “Bar” deleted

This list could be databound to a UI control, or used to update an XML database, HTML file, or even a SQL Server database.

The first code snippet will never detect the deletion of the file “Foo” because the change notification is never raised – events have not yet been enabled when the trigger happens. The second snippet shows the obvious way to correct this mistake, but is also flawed. FileSystemWatcher event handlers are called on threads from the System Thread Pool. This means that the event handler for the deletion of file “Foo” could execute before the this.Files property was populated, leaving it with nothing to update! The mechanism for fixing this depends in part on whether the application is a UI application or not.

If this is a UI application, then the event handler calls can be marshalled to the UI thread, as long as the FileSystemWatcher was itself created on the UI thread. To enable this functionality, the SynchronizationObject property must be set to reference the relevant Windows form. Since in this case both the initialisation code and the event handler run on the UI thread, we can guarantee that this.Files will be populated before the event handler runs. FileSystemWatcher appears in the “Components” palette of the Windows Forms editor, and inserting it from there automatically sets SynchronizationObject.

In cases where the application isn’t Windows Forms based (e.g. a Windows Service), we will have to ensure that the initialisation code and event handlers run sequentially on our own.

Even though these methods guarantee that this.Files will have been initialised, there is no way of being sure of whether it contains a reference to the deleted files or not. In the case shown above, the event handler for the deletion of file “Foo” would not find it in the list to be removed, but the handler for file “Bar” will. This means that update logic must take account of the fact that it doesn’t know whether updates are already reflected in the data to be synchronised or not. One possible response to this would be to simply refresh by calling d.GetFiles() again on each change notification as shown here:

void Init()
{
	this.f.Deleted += new
		FileSystemEventHandler(
		f_Deleted);
	lock(this.Files)
	{
		this.f.EnableRaisingEvents
			= true;
		this.Files = d.GetFiles();
	}
}

private void f_Deleted(object sender,
	FileSystemEventArgs e)
{
	lock(this.Files)
	{
		this.Files = d.GetFiles();
		// Update UI, Database etc here
	}
}
This solution uses a Monitor implicitly through the C# lock keyword. The Monitor is a managed synchronisation primitive that prevents access to the specified object from more than one thread simultaneously. It works much like a mutex, but the C# Mutex class is a wrapper for a Win32 mutex, and therefore requires transitions out of managed code (and in fact kernel mode). Monitor is therefore more efficient when there is no need to share the object between managed and unmanaged code.

The solution shown above will keep the this.Files up to date, but if the file list was very large, or was a tree structure representing a large number of directories, doing a complete refresh could be very inefficient. It may also have negative side effects such as causing display flicker as controls are redrawn, or losing information in a database. In many applications it will therefore be necessary to write code to apply the change manually, e.g. by deleting just the relevant item from the tree control or database. This will tend to result in complex code, and anywhere there is complexity there is scope for bugs to creep in.

Under the hood

FileSystemWatcher works using the underlying ReadDirectoryChangesW API from Win32 (see sidebar sidebar – “Nobody’s Perfect”). Change information is stored in buffer memory, and the size of that buffer can be configured using the InternalBufferSize property of FileSystemWatcher. The buffer size should usually be set in multiples of 4k (4096 bytes) to match the OS page size. Increasing the size of the buffer should be done only where absolutely necessary, since the buffer is non-paged memory, which is a scarce OS resource. If you have buffer overflow problems in your application (i.e. you receive Error events from the FileSystemWatcher) then the NotifyFilter property can be used to restrict the types of data recorded in the buffer, and ensuring IncludeSubdirectories is false if the application permits also restricts data size. Note that the filter constructor parameter (or Filter property) has no effect on the quantity of data sent to the buffer.

In addition to potential overflows caused by inadequate buffer size, the application developer should consider how many notifications their application is likely to receive. In the photo library application above, only a few photos are added to the library each week, but if you want to design an application that monitors the whole of a network drive shared amongst hundreds of users, you could face hundreds or even thousands of notifications each minute. You must ensure that your application can cope with the volume of notifications!

The ReadDirectoryChangesW API is used by Windows Explorer to monitor for directory changes, and is designed not to create huge quantities of network traffic. As such, it uses notifications of change rather than polling to identify when changes occur, and so can safely be used across a network.

Conclusions

In the FileSystemWatcher class Microsoft has provided us with a straightforward means of accessing file and directory change information. I have shown that this enables simple server style applications that detect files being moved or changed, and also enables UI applications to synchronise with the filesystem. In the case where applications wish to synchronise with the filesystem in a non-UI context, things are just a little more complex, as developers are forced to implement their own synchronisation scheme.

It turns out, however, that or applications such as keeping a user interface up to date, or synchronising database information with the filesystem, the code needed to interpret information from the FileSystemWatcher correctly will be complex. In an ideal world the System.IO namespace would support a mechanism for automatically keeping UI up to date with filesystem information, perhaps through data binding. It would also be nice to have a mechanism that allowed you to snapshot a view of a file or directory at the point that the EnableRaisingEvents was turned on, so that your own code knew that changes received subsequently applied to the returned data. Finally, it would be great if the class supported synchronisation in non-UI cases, perhaps by using Monitor itself! With the addition of these features, I believe that the usefulness of FileSystemWatcher would be substantially improved._


Ian Stevenson has been developing Windows software professionally for almost 10 years, in areas ranging from WDM device drivers through to rapid-prototyping of enterprise systems. Ian currently works as a consultant for The Generics Group and can be contacted at [email protected].

Nobody’s perfect

While researching this article I came across a couple of bugs in the implementation of ReadDirectoryChangesW which must also affect FileSystemWatcher.

The first is documented in the Microsoft Knowledge Base (KB) article Q245214, and relates to Windows NT and 2000. In some circumstances the file change detection will fail if files change more than once in an hour, as a result of an OS optimisation. This can mean that notifications may be delayed, or even lost entirely in some circumstances.

The second is documented in the KB as Q290601 and points out that the file name in a notification message can be either the short or long name. In the case of file deletion in particular you will not be able to translate to the other form as the file is already gone! In the case where your application stores a memory or database representation of the filesystem it must therefore cache both long and short names to ensure that consistency can be maintained.

You might also like...

Comments

Contribute

Why not write for us? Or you could submit an event or a user group in your area. Alternatively just tell us what you think!

Our tools

We've got automatic conversion tools to convert C# to VB.NET, VB.NET to C#. Also you can compress javascript and compress css and generate sql connection strings.

“C++: an octopus made by nailing extra legs onto a dog.” - Steve Taylor