Best practices for test-driven development using Nunit

TDD is something of a recent phenomenon in the .NET community; the Java camp has been doing this for a lot longer. This is because the first implementation of a TDD framework (JUnit) was implemented in and targeted at Java. The .NET community has now caught the TDD bug (no pun intended), primarily due to the emergence of NUnit, a framework for TDD in .NET.

NUnit appeared just over two years ago (February 2002), and has rapidly gained a substantial following in the .NET community. NUnit is open-source code, and a superbly elegant implementation of a unit testing framework. The smart folks who redesigned the original implementation chose to use .NET custom attributes in order to adorn unit-testing semantics on test modules. NUnit has an active developer community, and is hosted at Sourceforge.

So what exactly are NUnit and TDD used for? Let’s examine the motivations leading to TDD in the first instance.

The dreaded test harness

Remember those test harnesses we have all written at some point, full of buttons and text boxes, controls jammed in any way possible, as long as they can exercise a given module? NUnit deals them a death blow: TDD is a much better way to achieve the same end.

Test harnesses are problematic in that they require maintenance and extension, which demands effort in terms of both construction and execution. You have to deal with widgets and the code behind them, and when running your test harness you have to worry about clicking all the right buttons to perform meaningful tests. This is time consuming, and doesn’t provide a sufficient degree of regression testing. Regression testing strives to validate that new code does not break existing code. In the scope of a single unit, this implies that you would conceptually have to click all the buttons on your test harness in order to validate that you had not introduced a defect while coding new functionality.

The test harnesses you will need to produce following the adoption of TDD will be more of the “reference implementation” or “demo” variety. Because you are freed of the requirement to test your code using the same harness, what would previously have been a test harness becomes more akin to reference implementation. However, I recommend ignoring the production of reference implementations and test harnesses, and consider the unit tests as the definitive resource for such things.

What is Unit Testing?

“The main purpose of contracts is to help us build better software by organising the communication between software elements through specifying, as precisely as possible, the mutual obligations and benefits that are involved in those communications. The specifications are called contracts.”
Bertrand Meyer
(interviewed by Bill Venners)

Unit testing is a way of effectively verifying that software contracts have not been broken. Unit testing achieves this by exercising the public interface of a module (e.g. a class) with test cases that evaluate expected results to determine a pass or fail. Unit testing frameworks such as NUnit support this activity by providing the infrastructure to quickly and easily validate software contracts by running the unit tests written against them. Furthermore, unit testing should also ensure that exception cases are handled with aplomb. If sufficiently comprehensive, unit testing is a very powerful development tool.

What is Test Driven Development?

TDD is a process pattern for constructing iterations of development projects that employ unit testing. It is a core tenet of “Agile Development”, specifically the Extreme Programming (XP) camp. When undertaking TDD, the API for the modules under development will ideally have been specified to a reasonable degree, for reasons which follow. Note that this somewhat conflicts with the principle of “Emergent Design”, another core tenet of the Agile/XP community.

The process pattern is roughly as follows:

1. Design the unit tests
2. Code/Generate the class and interface skeletons
3. Write the unit tests
4. Run the tests (all should fail!)
5. Code the minimum required to pass each test
6. Iterate

Design the Unit Tests
Unit tests should be written from a white-box perspective. This means that the unit’s external interface and inner workings are considered in the context of designing tests against it. Each public method may yield several unit tests, depending on the method’s signature. For example, consider the following method signature:

public void Foo(int x, string y) {...}

Imagine that this method body has an if/else statement, a looping construct and throws a couple of exceptions, perhaps if the x parameter is negative or the y parameter is empty.

At face value, this method yields several test cases. Test cases should be designed for various values of x, various values of y (including null and “”) and all exception cases. These tests should be comprehensive in their ranges of values, including being above, below and on thresholds. Exception cases should be given equal weighting as happy-day scenario code: your unit tests should not be biased toward testing standard behaviours.

Code or generate code skeletons
At this point, you should create the infrastructure for the core modules, including files, projects and individual units. Declare the classes, interfaces and methods as skeletons. This means that all the routines contain no code whatsoever. This is done at this point to support the next activity.

Write the unit tests
Writing the unit tests is facilitated through the use of code completion in the IDE. If you follow the suggested order of activities, the requisite class and interface skeletons will enable you to quickly author the tests (using code insight) that were identified during the test design phase.

Decouple your tests!
It’s really important when writing unit tests that each one is totally autonomous. Don’t write tests that rely on other tests in order to pass, or else you will get false positives when running them in isolation. Unit tests are just classes themselves, so if you want to share code, simply factor it out into another method. As an example, ‘factory’ methods that create and initialise instances for testing purposes frequently appear in unit testing code. NUnit (like most other unit testing frameworks) provides a flexible infrastructure for the setup of test code that facilitates decoupling tests from each other.

Run the tests
Now that you have written the code skeletons and have some unit tests to run, you are ready to code in earnest. All unit tests run at this point should fail, or else something is really amiss! Now you are ready to begin the “real” coding of your core entities. A later section of this article discusses some recommendations surrounding how best to actually run the unit tests.

Pass each test successively
During this phase, you should strive to write the least amount of code possible to pass each test. Strive for simplicity. This phase is often best accomplished with a coding partner, a practice known as Pair Programming. This practice is another fundamental tenet of the XP camp. See the resources section for pointers in this direction.

Iterate
After having completed unit testing a given module, move onto the next one, following the same process pattern. The only change following completion of the next module is that both sets of unit tests should be run prior to integrating the new code with the main development stream. The more modules that are developed, the more important this becomes.

Ideals and reality

The process pattern for Test-Driven Development is ideal in nature. The reality is of course that sometimes this optimal ordering cannot or is not followed for whatever reason. This does not spell failure for the project and does not detract from the effectiveness of unit testing in many regards. Understand that the process pattern is an ideal that you should strive toward and that if it is not always possible to follow it then so be it: you will still reap significant benefits. The most important point will be to have a suite of unit tests that comprehensively test your system.

It can sometimes help to retroactively set up unit tests, for example prior to embarking on a significant refactoring of a system. The point is that unit testing is not necessarily just for new development; it can be a boon in other development scenarios.

Using NUnit effectively

One option when running unit tests involves using the NUnit GUI. The GUI loads your “test” assembly and reflects on the types within it in order to build a view of the tests within the assembly. You have the option to select and deselect tests and (most importantly) run them. But personally I very rarely use it.

NUnit ships with a command-line interface that is more powerful than its GUI counterpart. The idea is that you have the nunit-console.exe application load your unit test class library and run the tests within it when you invoke Run from the IDE. When you do so, a Command Prompt appear displaying the results of the tests.

Because it is not normally possible to run class libraries, you have to do some minor configuration to the project properties in order to tell Visual Studio that you want nunit-console.exe to host it. You need to do the following, assuming you have added a new Class Library project for the purpose of testing to your solution:

1. Add a reference to nunit-framework.dll
2. Open up the project properties to Configuration Properties | Debugging:
   1. Set Debug Mode from “Project” to “Program”
   2. Set the Start Application to nunit-console.exe
   3. Set up the command-line arguments (see below)
   4. Set the working directory to the base directory of the project (you can simply copy it from Common Properties | General)
3. Declare the using namespace (NUnit.Framework) in the test class

Here are my preferred command-line arguments (as entered into Configuration Properties | Debugging | Command-line arguments):

Foo.csproj /nologo /wait
/transform:..\..\Console-Transform.xsl

The first item is the name of the test project, the second squashes the logo, the third keeps the Command Prompt up after having run the test (it waits for carriage-return) and finally, the /transform switch enables invoking an XSL transformation.

Test result transformation

The /transform switch enables an XSL transformation file with which to transform the XML-ised results produced by NUnit for display on the console window. The main reason I used my own and not the supplied one is that by default, only total elapsed time is displayed for all tests. I prefer to see timing information for each test, plus I wanted a couple of other minor enhancements, so I authored my own stylesheet. (If you’d like a copy of it, drop me a note and I’ll send it on). I encourage you however to author your own, especially if you haven’t done any XSL before: the XML that NUnit emits is trivial to work with and you can use the default stylesheet supplied as a starting point. Besides, coding XSL brings joy to the heart: it’s really enjoyable, almost as much fun as recompiling the Linux kernel.

Console-window configuration

Another convenience when running unit tests in this manner is to adjust the properties of the command-line window that comes up following a test run. It only has to be done once per system.

First, move and size the Command Prompt window to your liking. Then right-click the title bar, select Properties and override the Layout options by unselecting “Let system position window”. I also like to set my screen text to yellow, which I find more legible than ghost-white. Finally, select “Save properties for future windows with same title” when you click OK – this will prevent you from having to do this ever again.

100% code coverage

Unit testing should ideally be performed in tandem with a code coverage tool of some sort. Your unit tests should achieve 100% code coverage. If you are embarking on new development, I encourage you to incorporate unit testing into your development process. Notice that part of the regular code reviews is to verify that unit tests written against a given module achieve this.

It is not always possible to achieve 100% code coverage because unit tests only have the same access privileges as any other client. That is, the public interface to a class does not provide sufficient access to internal members to fully and completely test a class. So, I’m looking forward to working with C# 2.0, which will support partial classes.

Partial classes enable a class to be split across multiple source files. This may be a boon to unit testing – I haven’t yet tried it – because it may prove to be an elegant way to inject test code straight into a production class (gasp! horror!). This way we get around the interface problem because we are able to rigorously test private methods in addition to the public interface. The DEBUG compiler directive is our friend in this regard: we certainly don’t want to ship production code with a load of unit tests!

Poor man’s performance testing

Unit tests serve as a primitive form of performance testing. During the code/test cycle, it is possible to introduce a code fragment that has an adverse effect on overall performance. Finding these rogue code fragments as early as possible in the development process is a real bonus.

Recently, one of my unit tests performed one million particular operations on a custom data structure I was developing. That particular unit test originally took about 4 seconds to run but I inadvertently wrote some code that caused the execution time to jump to around 12 seconds. After some minor refactoring, I was able to drop this down to an execution time of around 4.5 seconds: much better. This highlights the ability for test-driven development to flag problems that occur as soon as they do, even with respect to performance monitoring.

A code base that has a rich set of unit tests written against it has benefits beyond those considered in isolation. These are primarily the ability to support refactoring code and integrate new code into the development stream with greater confidence.

Extreme refactoring

In the agile community, refactoring code is promoted as a “daily activity”. I personally do not agree with this. Refactoring is sometimes necessary, but it should not be performed for its own sake. The term, “refactoring” is sometimes a Dilbertesque euphemism for rewriting something that wasn’t designed properly in the first place. Another problem with daily refactoring is that it assumes that the unit tests written against a refactored code base are sufficient to guarantee its functional integrity, which is definitely not always a safe assumption to make.

The following adage sums this up nicely:

“If it ain’t broke, don’t fix it”
Anonymous

That Anonymous sure knew what she/he was talking about!

Anyway, when refactoring does become a necessity, a rich set of unit tests is invaluable. This cannot be emphasised enough. During refactoring, the unit tests may be continually re-run to give a reasonable degree of assurance that the integrity of the code is not violated, that existing contracts are not broken. For more information about refactoring, consult the Resources listed at the end of this article.

Facilitating early and continual integration

The Agile community advocates early and continual integration of code into the main development stream. Research has shown that the earlier that code is integrated into a system, the earlier that bugs will be found. The earlier that bugs are found, the cheaper and easier they are to fix, so this is a good practice.

Consider the class diagram shown in Figure 1.

Figure 1: Two separate systems that share a class

It illustrates two systems, Foo and Bar, that share a class (B). Suppose a developer working with class C in the Bar system refactors class B to satisfy a changed requirement. The developer was conscientious and updated the unit tests for class B and is happy with the modifications because the unit tests for both classes C and class B succeed. The developer checks the code in for both classes.

Unfortunately, the modifications break the Foo system which also uses class B. This can happen even if the public interface of class B was left unchanged. Had the developer been able to run the unit tests for all of class B’s dependent classes, a happier scenario would have resulted.

The solution? A solution!

NUnit provides the facility to group tests and run them in batches. Run the test solution as part of a formal reintegration process pattern.

Developers should not be relied on to run all unit tests prior to reintegrating their code. Strive to bootstrap projects early in the development process, and include automated retrieval of the entire code base to a dedicated machine, a complete rebuild of the code base and execution of the entire suite of unit tests. This way, integration problems are found as early as realistically possible. Failures should be emailed to the entire development team, but don’t bother wasting valuable developer time with emails of successful runs.

For large development shops, running all the unit tests in all the systems prior to reintegration may not be feasible in production. In this case, formally track dependencies between classes and unit tests in order to be able to determine what tests need to be run after refactoring a given unit.

Bona fide requirements management tools can help with this task, but there is another cool way to do this using UML. The idea is that you model dependencies between classes and systems in such a manner that you are able to programmatically determine what systems’ unit tests need to be run. This could be achieved by exporting the dependency model as XMI and running it through a stylesheet to generate a script to invoke the unit tests. Although I have not tried this particular technique, our team has successfully implemented far more ambitious things with XSL transformation of XMI with great success. If you are a large enough development shop, this may be worth the effort.

Conclusion

Unit testing is a powerful way to construct code and NUnit provides a superb framework to this end. High quality unit tests can significantly augment the quality of production code and can generate a tremendous amount of confidence in the integrity of tested code. Probably most significantly, programmers love to write code but hate to test it: having them write unit tests achieves both._

---

Edward Garson is a Senior Consultant with Dunstan Thomas, a company that transforms software development at a grassroots level through technology evangelism, architecture, design and process best practices. Edward specialises in OOA/D, UML, XML and enjoys both .NET and J2EE platforms. He may be reached at [email protected].

Resources NUnit Agile related web sites: agilealliance.org, agilemanifesto.org Bertrand Meyer interview Fowler, Martin – Refactoring: Improving the Design of Existing Code McCaffrey, James – Perform Code Coverage Analysis with .NET to Ensure Thorough Application Testing (MSDN article)