In this episode Federico and Matthew talk about their experience testing installers. Taking from their experience testing the ASP.NET MVC installer, they share tips and advice for those faced with the challenge of testing the very first experience of many products: the installer.
News:
Installer Testing
- Before somebody gets to use a product, they have to get it into their machines. It is the first experience with a product.
- When building an installer test plan start by brainstorming 4 general areas:
- Pre-requisites. How does the installer check for requirements? Does it provide enough information if requirements are not met? Does it fail as soon as possible?
- Installation: What files are copies and where? Registry changes? GAC changes? Folders created? IIS changes? Does it need to check the web?
- Uninstall: Is everything rolled back completely? Is anything is supposed to be left behind?
- Machine state: OS, bitness, other applications running, hardware, disk space, network access, permissions.
- Based on these variables, start building combinations of scenarios.
- Use additional heuristics for identifying and discussing special cases:
- Interruptions: Machine goes into hibernation, unplug network cable, user cancels the installer.
- Full circle: Snapshot the state of the machine before installing, make sure it is the same after uninstalling.
- Side by Side: Previous versions of the software installed, multiple versions installed.
- Reinstall/Repair: Reinstall multiple times, break the application and repair it.
- Temp folder: what is written in the temp folder.
- Unintended changes: Additional files in placed installation directory.
- Partial installs: some of files that are meant to be installed are already present, some of the files that are meant to be removed are not there.
In our experience, installer testing is closer to scripted testing than exploratory testing.
Comments