Project White: Automated UI Testing

After using WaTiN, I have been thinking about UI Testing for WinForms, if it’s possible and if it’s even worth it. On the MbUnit mailing list I posted some syntax for an approach to WinForms and I had some good ideas, I brought up the subject again at Alt.Net.UK and while people have had success using WaTiN, they didn’t seem that interested in WinForm testing. I know others had been talking about WPF Testing during the day and problems with it.

As it happens, I read on Jeremy Miller’s blog that Thoughtworks have released ‘Project White’ which is a UI Testing framework for WPF, WinForms, Win32 and SWT (Java) and works based on Microsoft’s UIAutomation library and windows messages. Sounds promising so I decided to take a closer look, this post just discusses me playing around with the framework and a simple form to get an understanding of how it works.

Firstly, I created a standard Windows Forms application with just a single form. First test – does it display?

The form looks like this:

Using White and MbUnit, the test looks like this:

private const string path = @”……White_HelloWorldbinDebugWhite_HelloWorld.exe”;
#1 [Test]
public void ApplicationLaunch_NoArgs_Form1Displayed()
{
Application application = Application.Launch(path); #2
Window window = application.GetWindow(“Form1”, InitializeOption.NoCache); #3
Assert.IsNotNull(window);
Assert.IsTrue(window.DisplayState == DisplayState.Restored); #4

application.Kill(); #5
}

#1 We need to define the path to our executable. This is fine if you know your always going to be building into the same folder (both test and live assemblies), bit difficult when you have separate output directories.
#2 I then use White to execute the exe
#3 Once the application has launched, we get the form displayed as an object. This works based on the form’s title – in this case, Form1
#4 I then check the Window state to see if it has been displayed
#5 Finally, I close the application.

That’s a very basic test. Let’s add some functionality and explore the framework in more depth. What happens if the framework cannot find the form?

[Test]
[ExpectedException(typeof(Core.UIItems.UIActionException))]
public void ApplicationLaunch_NoArgs_Form2NotDisplayed()
{
Application application = Application.Launch(path);
Window window = application.GetWindow(“Form2”, InitializeOption.NoCache);
application.Kill();
}

White will attempt to find a window called Form2, if the timeout expires it throws the UIActionException. This is the same if it cannot find a control on the form.

To make this more interesting, I created an additional form with some buttons and labels.

The first button has a simple action, when you click it the text of the button changes to be Hello World!!. We can then create a test for this as follows:

[Test]
public void ButtonClickable_btnClick1_ChangesText()
{
Application application = Application.Launch(path);
Window window = application.GetWindow(“White Hello World”, InitializeOption.NoCache);

Button button = window.Get(“btnClick1”); #1
//Moves the mouse to click the button

button.Click(); #2
Assert.AreEqual(“Hello World!!”, button.Name); #3
}

#1 Using the Get method, we can give it the type and name of the control we want to access.
#2 We can then call the Click method which will move the mouse cursor over to the button and click it.
#3 We can then verify that the action was correctly performed, in this case the Name has changed.

Sometimes, we want to be more flexible than referring to the object by it’s name. As such, we can use the SearchCriteria object which allows us to access the control in different ways, for example by it’s text:

SearchCriteria searchCriteria = SearchCriteria.ByText(“Click Me!”);
Button button = window.Get

(searchCriteria);

One problem I did encounter was with unhandled exceptions. With one of my buttons, when clicked it will cause an exception to be thrown.

The test looks like this:

[Test]
//Doesn’t work? No way to get the exception…I guess the test would fail so you would reproduce manually.
public void ClickButton_ThrowsException()
{
Application application = Application.Launch(path);
Window window = application.GetWindow(“White Hello World”, InitializeOption.NoCache);

Button button = window.Get

(“throws”);
button.Click();
Assert.AreEqual(“Hello World!!”, button.Name);
}

The test fails because of the assertion, sadly it doesn’t report back saying that an exception was thrown which I would have liked.

failed: Equal assertion failed: [[Hello World!!]]!=[[Throws Exception]]

Another problem which I wanted to see if the framework would handle was with message boxes. When clicking btnMsg, a message box is displayed on screen. Using the framework, we can use the MessageBox() method, giving it the title of the message box in order to get a Window object (would have preferred this to be a MessageBox object). We can then treat it just like a normal window which is nice.

Button button = window.Get

(“btnMsg”);
button.Click();
Window messageBox = window.MessageBox(“”);
messageBox.Close();

One last simple test I wanted to perform was how to verify that when a button is pressed, that a label is updated correctly.

[Test]
public void ClickButton_btnLabelText_ChangesLabel()
{
Application application = Application.Launch(path);
Window window = application.GetWindow(“White Hello World”, InitializeOption.NoCache);

Button button = window.Get

(“btnLabelText”);
button.Click();
Label label = window.Get(“lblText”); #1
Assert.AreEqual(“Updated”, label.Text);
application.Kill();
}

#1 We can access a label in the same way we access another other control, such as a button. No problems at all. There are other objects for difference controls available.

One thing about all of these tests is that they aren’t very readable, we have a lot of noise about getting access to the objects which is making it harder to read what the test is actually testing. One quick way to improve readability is to move the launch and kill calls into Setup and Teardown methods.

private Application _app = null;
[SetUp]
public void Setup()
{
_app = Application.Launch(path);
}

[TearDown]
public void Teardown()
{
_app.Kill();
}

But it can be cleaned up more, one technique I’ve discussed before with WaTiN is to create a wrapper around the UI and test against that to make our tests more readable and less fragile.

I’ve taken the code from the last test and refactored it into a wrapper. The wrapper looks like this:

public class Test2 #1
{
private Application _host = null;
private Window _main = null;
public Test2(Application host) #2
{
_host = host;
_main = _host.GetWindow(“White Hello World”, InitializeOption.NoCache);
}

public Window Main
{
get { return _main; }
}

public Button btnLabelText #3
{
get { return Main.Get

(“btnLabelText”); } #4
}

public Label lblText
{
get { return Main.Get(“lblText”); }
}
}

#1 This is the same name as the Form in the code under test for readability
#2 Pass in the application so we can gain access to the running exe
#3 This could be called anything for readability
#4 Use the same code to access the button and return it to the calling test.

Using this wrapper and the Setup/Teardown, the same test as before would be this:

[Test]
public void ClickButton_btnLabelText_ChangesLabel_ImprovedSyntax()
{
Test2 form = new Test2(_app);
form.btnLabelText.Click();

Assert.AreEqual(“Updated”, form.lblText);
}

This could be reduced more by moving the form creation into Setup as well. By using this simply step, we have made the tests more much readable. The other advantage is that if the name of a button changes, we only have to change the wrapper and not all of the dependent tests.

One final problem you might encounter is finding out what each control on the form is actually called. The .Net 3.0Windows SDK includes a tool called UISpy. This allows you to see all the properties on a running application, as such all the information required for use with White.

Another good tool is HawkEye (Updated Link).

In summary, I’m really impressed with the framework. There are a few missing features, but hopefully they can be added over time, the fact that the framework does multiple different UI technologies is great, means you don’t have to worry about what to use to write your UI tests or if your UI deals with the different technologies. Can’t wait to use it on a real project…

Keep an eye out for a post how to use this with WPF and more advanced scenarios.

Project White Homepage – http://www.codeplex.com/white

Code Sample – http://blog.benhall.me.uk/Code/ProjectWhite/White_HelloWorld.zip

Technorati Tags: TDD,Testing,MbUnit,Project White,WinForms,WPF,altnetuk