Writing tests as specifications with Expect

Have you ever thought that your tests old precious information about how your system works ? have you ever considered a specification document as a costly piece of text that gets obsolete at light speed once your system is deployed ?

Expect is a .Net OSS tool for writing specifications in a natural language and run those specifications as tests on your system. In other words, it is a tool for executable specifications.

With Expect, everything is integrated in Visual Studio 2010 or 2008 to ease and speed up the process. Specifications are run through the unit test framework of your choice (for the moment MsTest and NUnit are supported). It means that Expect could integrate nicely in your development platform. The tool also provide reporting capabilities to generate a word document from your specifications.

You could find Expect on codeplex here : http://expect.codeplex.com

No more “bla bla”, let’s dive into some (basic) example…

If you want to follow this as a tutorial, you will have to download and setup the tool. This small tutorial will use MsTest for running the specification, and it could be done with both VS2008 and VS2010.

At first, we need a VS project to host our specifications. Create a new unit test project in Visual Studio, and add a reference to Expect.Core.dll (default path is c:\program files\expect\expect framework).

When you’re done, right click on your project and select “Add new item”. You must have an item called “Expect specification”. Select it and give a name to your specification.

Clear the content of the file and write (or copy/paste) the following :

My grandmother loves |apple pies| but not |ice creams|

I know, this is a dummy sentence but it will suit our needs…

Save your changes and build the project.

If you look at the unit tests in your solution (on the main menu go to “/ Test / Windows / Test view”) you will see that someone magically created a unit test. Expect as generated two C# files attached to your specification. The one with the extension “.expect.cs” old the unit tests and will be regenerated. The other one is called the “container”. It will contains some of the fixtures that bridges the specification and the system under test.

If you run this test, it will give an inconclusive result because we don’t provide a way to interpret the specification. If you look a the test result details, you will find a trace of what Expect tries to resolve the specification. It tries to find a class or a method called “MyGrandmotherLovesButNot”.

We could create class or method by hand but at a large scale it would not be practical. Right click on the sentence and select “Expect / Container –> create method”. Expect generate the appropriate method in the container. If you run the test again, the test will fail because the created method throws an exception.

Put a breakpoint in the method “MyGrandmotherLovesButNot” and run the test in debug. If you inspect the parameters, you will see that they are filled with the values “apple pies” and “ice creams” that we wrote in the specification. More interestingly, put a breakpoint in the “.expect” file on the sentence and run the test in debug…

If you clear the method body and run the test again, it will succeed.

That’s all for now. Expect as a lot more features. The fixtures could be done with classes instead of methods, they could be formatted as tables or free fluent sentences…

You could discover more features with the sample project at codeplex. This sample project is also the source for the documentation file, so you will also be able to play with specification document generation.

At this stage, any feedback is appreciated. To get a great tool we need to know what you find interesting or not, what is missing, …

Do not hesitate to give your feedback on the mailing list here.