What’s next ?

It was a few weeks ago that I published Expect on Codeplex. You were a few hundreds of people downloading it (a little more than 700 actually) but I still have no feedback on the tool.

I will keep going on improving the tool, but I would be very glad to hear what you think about it.

For example, what is your most desired feature in such a tool ? What do you like or dislike ?

My plan for the next major feature is to allow for using shared “steps” between specifications (a group of fixtures). I think it will be very useful when you use Expect to drive UI tests. I’m just still wondering about how would the steps behave in document generation. Do they render separately or is it better to inline the steps in the multiple specifications ? I’m more for the second option now but what do you think ?

Posted in Expect | Leave a comment

Expect Beta 5

Another new Beta of Expect is available on Codeplex. The main goal for this version was to improve the tooling in Visual Studio. Unfortunately, some of the programming model have changed between VS2008 and VS2010, and I have time to develop all features twice. Consequently, the IDE improvements are only for VS 2010.

image

This screen capture illustrate two major elements of this new Beta : Smart-tags and Error squiggles.

You could now use smart tags to trigger actions on Expect elements. Smart-tags are the small rectangles that you see at the beginning of an element in VS. You could expand them by clicking on this small rectangle or using the shortcut Ctrl + ‘.’

Error squiggles are the “wavy” red lines that VS use to show you that something is wrong. In Expect, it means that the fixture is not resolved yet.

Hopefully, there are some features that you will be able to use also in VS2008.

For example, you could now combine setupfrom and setup instructions. It is possible that this new features make your VS project not compiling. In this case, you will just have to run the custom tool used by Expect to generate the unit test. You could do this by saving your Expect files, or right click on them in solution explorer and click on “Run custom tool”.

Another small feature is the ability to list the styles in your document template file. It makes it a lot easier to setup your documentation generation. To use this, right click on your Expect project. If your configuration file contains report information, you should have the following menus:

image

By clicking on ‘”List template’s styles”, Expect will write the list of styles in the Output window of Visual Studio

image

The list contains the inner name and id of each styles for paragraphs and tables. You could use either name or id in Expect report section in the configuration file.

Another change in the documents are the rendering of ObjectFeedFixture. They are now rendered as tables in the document. ObjectFeedFixture was introduced in Beta 4 and allow to set properties for an object in the context.

I will post very soon a walkthrough for document generation but in the meantime you could look at Expect samples and documentation.

kick it on DotNetKicks.com
Shout it

Posted in Expect | 2 Comments

Expect Beta 3 is out

Expect Beta 3 as just been published. This new release include some bug fixes and cool new features.

As usual, the documentation is generated from the sample and is up to date. The sample is the best place to check how Expect work.

Adding images to reports
You do this by adding a special description prefix to the line, for example, if you type

#img images/myimage.jpg

The particularity is that you have to put the image in your VS project as an embedded resource. The goal is to keep the dll autonomous for report generation. You will also enjoy intellisense when adding images ;-)

Adding variables to the report
For now, the only variables supported are the properties “AssemblyVersion” and “AssemblyFileVersion” from your spec assembly. The variables must be declared as Word document properties and added in your template as fields. A future post will cover report generation but you could get the sample if you want look at this.

Descriptions formatting
You now have a wiki-like syntax that you could use to format description text. You could also define custom blocs to apply a Word style to specific sections of your descriptions (like notes, comments, sample code, etc).

Intellisense on fixture base classes
Now intellisense work for fixture methods that you define on your base class (base container, fixture base class, etc). The base class must be part of the project and the methods decorated with the attribute “FixtureHint”.

Feel free to leave feedback on the mailing list or comment on this blog. We really need feedback to improve the tool…

Posted in Expect | Leave a comment

Writing functionnal UI tests with WatiN and Expect

One kind of functionnal tests that you could do with Expect is UI testing. Expect come with an extension for WatiN that you could use to write web UI specs/test.

This post will use MsTest to run the specs but you could use NUnit as well.

First download and install Expect framework. Once you are done, launch Visual Studio and create a new Test project. Add references to Expect.Core.dll, Expect.Watin.dll, and WatiN.Core.dll (it should reside in “program file\Expect” or “program file (x86)\Expect”.

Right click on the project and add a new application configuration file.

Put the following into the configuration file :

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="expect" type="Expect.Core.ExpectConfigurationSection, Expect.Core"/>
    <section name="expect.watin" type="Expect.WatiN.ExpectWatiNSection, Expect.WatiN"/>
  </configSections>
  <expect>
    <!-- this section define behaviour for code generation -->
    <!-- set unitTestProvider="NUnit" to generate tests for nunit -->
    <codegen unitTestProvider="MsTest" containerbaseclass="Expect.WatiN.WatiNSpecificationContainer" />
  </expect>
  <expect.watin rooturl="http://www.google.com" browser="IE" />
</configuration>

With Expect, when you write your text specification, it generates a class that wraps unit tests that corresponds to your specification. This configuration tells Expect to use a different base class for the generated unit tests. The WatiN base class contains base logic for initializing WatiN.

To have executable specifications, we will have to bridge the text specification to the code or application we want to test with fixtures. With WatiN, you write page classes that wrap the actions you will have to make on your application’s pages. The Expect extension provide a base class for pages that is also a fixture base class.

Right click on your test project and click on “Add > New Item”. Select “Expect specification” and name it “Google search”.
Delete the content of the file and write the following:

When I am on Google home page
if I search for |Expect framework|
I must have a link to |http://expect.codeplex.com|

Right click on the first line and select “Expect > Fixture -> Create actor fixture class”. Replace the inheritance from “ActorFixture” to “ExpectPage”. The “ExpectPage class” is both a base class for fixtures and WatiN page.

Add the following properties to the class:

[FindBy(Name = "q")]
public TextField SearchCriteria;

[FindBy(Name = "btnG")]
public Button SearchButton;

Those properties are pure WatiN’s page elements that wrap properties to UI elements.

Replace the content of the method “IfISearchFor” with :

SearchCriteria.TypeText(param_for_ExpectFramework);
SearchButton.Click();

I think there is no need to comment this code ;-)

And to the method “IMustHaveALinkTo”, replace the content with :

Thread.Sleep(400);
var elt = Context.Browser().Span("search");
elt.WaitUntilExists();
Assert.IsTrue(elt.Links.Any(l => l.Url.StartsWith(param_for_Httpexpectcodeplexcom)));

This code search for the link in the page.

Now if you look at the Test view in Visual Studio, you must be able to run the specification.
If you want to play a little, you could also put a breakpoint in the “.expect” file on the line “I must have a link to…” and run the test in Debug mode. You will discover the pleasure of debugging specifications ! You could also play with the parameters.
If you have a little fun in the “.expect” file you will see that you have intellisense on the elements, the fixture class if you write a new fixture (just add a blank line), or the actions available in your fixture.

Just in case, this is the complete class :

using System.Linq;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using WatiN.Core;
using Expect.Core;
using System.Threading;

namespace Expect.WatiN.Sample.Pages
{
    [FixtureHint("When I am on Google home page")]
    public class WhenIAmOnGoogleHomePage : ExpectPage
    {
        [FindBy(Name = "q")]
        public TextField SearchCriteria;

        [FindBy(Name = "btnG")]
        public Button SearchButton;

        [FixtureHint("if I search for |*|")]
        public void IfISearchFor(string searchtoken)
        {
            SearchCriteria.TypeText(searchtoken);
            SearchButton.Click();
        }

        [FixtureHint("I must have a link to |*|")]
        public void IMustHaveALinkTo(string targeturl)
        {
            Thread.Sleep(400);
            var elt = Context.Browser().Span("search");
            elt.WaitUntilExists();
            Assert.IsTrue(elt.Links.Any(l => l.Url.StartsWith(targeturl)));
        }
    }
}

You could download the sample on codeplex, along with other Expect samples.
With Expect, you could generate a Word document from your text specifications, it will be a good subject for the next post !
In the meantime, you could go to the samples and documentation to have a look…

kick it on DotNetKicks.com
Shout it

Posted in Expect, WatiN | 1 Comment

Expect Beta 2

The main goal of Expect Framework is to write tests with a lot of emphasis on the functionnal aspects. One of the way to use it is wrapping UI testing framework like WatiN or White to have a set of specifications executed on your system.

The Beta 2 of Expect introduce few but important new features :
A WatiN extension
This extension is a separate dll that helps writing WatiN tests as executable specs (more on this on the next post)

The ability to define a custom base class for specification containers
This feature helps reducing infrastructure code that you want to share among all your specs. For example, it is used in the WatiN extension to old the logic of initializing and freeing WatiN for every specification.
Defining a base class is as easy as adding a small line in your configuration file :

<codegen unitTestProvider="MsTest" containerbaseclass="Expect.WatiN.WatiNSpecificationContainer" />

Intellisense for fixtures
This feature makes writing your specs a lot faster by providing intellisense for fixture methods defined in your containers, as well as fixture classes, and even ActorFixture methods.

Posted in Expect | Leave a comment

Writing tests as specifications with Expect

Have you ever thought that your tests old precious information about how your system works ? have you ever considered a specification document as a costly piece of text that gets obsolete at light speed once your system is deployed ?

Expect is a .Net OSS tool for writing specifications in a natural language and run those specifications as tests on your system. In other words, it is a tool for executable specifications.

With Expect, everything is integrated in Visual Studio 2010 or 2008 to ease and speed up the process. Specifications are run through the unit test framework of your choice (for the moment MsTest and NUnit are supported). It means that Expect could integrate nicely in your development platform. The tool also provide reporting capabilities to generate a word document from your specifications.

You could find Expect on codeplex here : http://expect.codeplex.com

No more “bla bla”, let’s dive into some (basic) example…

If you want to follow this as a tutorial, you will have to download and setup the tool. This small tutorial will use MsTest for running the specification, and it could be done with both VS2008 and VS2010.

At first, we need a VS project to host our specifications. Create a new unit test project in Visual Studio, and add a reference to Expect.Core.dll (default path is c:\program files\expect\expect framework).

When you’re done, right click on your project and select “Add new item”. You must have an item called “Expect specification”. Select it and give a name to your specification.

Clear the content of the file and write (or copy/paste) the following :

My grandmother loves |apple pies| but not |ice creams|

I know, this is a dummy sentence but it will suit our needs…

Save your changes and build the project.

If you look at the unit tests in your solution (on the main menu go to “/ Test / Windows / Test view”) you will see that someone magically created a unit test. Expect as generated two C# files attached to your specification. The one with the extension “.expect.cs” old the unit tests and will be regenerated. The other one is called the “container”. It will contains some of the fixtures that bridges the specification and the system under test.

If you run this test, it will give an inconclusive result because we don’t provide a way to interpret the specification. If you look a the test result details, you will find a trace of what Expect tries to resolve the specification. It tries to find a class or a method called “MyGrandmotherLovesButNot”.

We could create class or method by hand but at a large scale it would not be practical. Right click on the sentence and select “Expect / Container –> create method”. Expect generate the appropriate method in the container. If you run the test again, the test will fail because the created method throws an exception.

Put a breakpoint in the method “MyGrandmotherLovesButNot” and run the test in debug. If you inspect the parameters, you will see that they are filled with the values “apple pies” and “ice creams” that we wrote in the specification. More interestingly, put a breakpoint in the “.expect” file on the sentence and run the test in debug…

If you clear the method body and run the test again, it will succeed.

That’s all for now. Expect as a lot more features. The fixtures could be done with classes instead of methods, they could be formatted as tables or free fluent sentences…

You could discover more features with the sample project at codeplex. This sample project is also the source for the documentation file, so you will also be able to play with specification document generation.

At this stage, any feedback is appreciated. To get a great tool we need to know what you find interesting or not, what is missing, …

Do not hesitate to give your feedback on the mailing list here.

kick it on DotNetKicks.com
Shout it

Posted in Expect | 8 Comments