Project White: Automated UI Testing

After using WaTiN, I have been thinking about UI Testing for WinForms, if it’s possible and if it’s even worth it. On the MbUnit mailing list I posted some syntax for an approach to WinForms and I had some good ideas, I brought up the subject again at Alt.Net.UK and while people have had success using WaTiN, they didn’t seem that interested in WinForm testing. I know others had been talking about WPF Testing during the day and problems with it.

As it happens, I read on Jeremy Miller’s blog that Thoughtworks have released ‘Project White’ which is a UI Testing framework for WPF, WinForms, Win32 and SWT (Java) and works based on Microsoft’s UIAutomation library and windows messages. Sounds promising so I decided to take a closer look, this post just discusses me playing around with the framework and a simple form to get an understanding of how it works.

Firstly, I created a standard Windows Forms application with just a single form. First test – does it display?

The form looks like this:

image

Using White and MbUnit, the test looks like this:

private const string path = @”……White_HelloWorldbinDebugWhite_HelloWorld.exe”;
#1 [Test]
public void ApplicationLaunch_NoArgs_Form1Displayed()
{
Application application = Application.Launch(path); #2
Window window = application.GetWindow(“Form1″, InitializeOption.NoCache); #3
Assert.IsNotNull(window);
Assert.IsTrue(window.DisplayState == DisplayState.Restored); #4

application.Kill(); #5
}

#1 We need to define the path to our executable. This is fine if you know your always going to be building into the same folder (both test and live assemblies), bit difficult when you have separate output directories.
#2 I then use White to execute the exe
#3 Once the application has launched, we get the form displayed as an object. This works based on the form’s title – in this case, Form1
#4 I then check the Window state to see if it has been displayed
#5 Finally, I close the application.

That’s a very basic test. Let’s add some functionality and explore the framework in more depth. What happens if the framework cannot find the form?

[Test]
[ExpectedException(typeof(Core.UIItems.UIActionException))]
public void ApplicationLaunch_NoArgs_Form2NotDisplayed()
{
Application application = Application.Launch(path);
Window window = application.GetWindow(“Form2″, InitializeOption.NoCache);
application.Kill();
}

White will attempt to find a window called Form2, if the timeout expires it throws the UIActionException. This is the same if it cannot find a control on the form.

To make this more interesting, I created an additional form with some buttons and labels.

image

The first button has a simple action, when you click it the text of the button changes to be Hello World!!. We can then create a test for this as follows:

[Test]
public void ButtonClickable_btnClick1_ChangesText()
{
Application application = Application.Launch(path);
Window window = application.GetWindow(“White Hello World”, InitializeOption.NoCache);



Button button = window.Get

24 thoughts on “Project White: Automated UI Testing”

  1. I liked you idea of looking at the WinForm exceptions as exception on operation performed on a control. I have added it to my list of things.

    There is something coming up in white very soon which would support screen object pattern. It is similar to the wrapper that you have mentioned here. For every
    logical window in your application you can have a screen class. All the operations that you would perform on that screen would go in that class.

    e.g. LoginScreen class containing userId, password, submitButton, cancelButton fields. Having a Login as a method on it.

  2. At the first glance: I think White is for desktop apps what Selenium is for web apps. It uses a higher level of abstraction when testing than NUnitForms does, that is why the same code could be used to test different things (WinForms, SWT…). I think they both have their place in testing, depending on what exactly you want to achieve.

  3. I have looked at it before however I didn’t take to it. I like the API of White (What is the reason for xTester in NUnitforms?) and the level it aims at.

    For the work I am doing, I feel White could be a better fit, however if I get chance I will revisit NUnitForms after using White on a production project and see how the two compare

  4. Where did you get UISpy?
    I can’t find it anywhere.
    It’s supposed to have been issued with the .Net 3.0 SDK a long time ago, but even though I think I’ve got everything installed up to date, I can’t find UISpy.exe anywhere, not even as a downloadable…

  5. I found UISpy difficult to find, if I remember correctly it is included in the Microsoft® Windows® Software Development Kit for Windows Vista™ and .NET Framework 3.0 Runtime Components (Windows SDK for .Net 3.0)

    http://www.microsoft.com/downloads/info.aspx?na=40&p=3&SrcDisplayLang=en&SrcCategoryId=&SrcFamilyId=10cc340b-f857-4a14-83f5-25634c3bf043&u=http%3a%2f%2fgo.microsoft.com%2ffwlink%2f%3fLinkId%3d74726

    There is also one for 3.5 now but I haven’t tried it.
    http://www.microsoft.com/downloads/details.aspx?FamilyId=F26B1AA4-741A-433A-9BE5-FA919850BDBF&displaylang=en

    Hope this helps.

    Ben

  6. You raise the question “is it worth it?” at the beginning of the post, and I’d like to elaborate on that.
    Since GUI changes so much, more than API (or controller interface), the cost of maintaining the tests rises. Apart from the technical ability to build the tests and its obvious merit, well, is it worth it?

  7. Re the comments about using NunitForms… I have used it for several years. The main reason I will not continue to use it seems to be a “dead” project.

    The most recent release is May 2006. And, that release did not include the source. I encountered a problem or two that I wanted to debug and fix. But, without the source, that was impossible.

    I also have not had any email from the NunitForms mailing list in over a year.

    As far as I am concerned that’s three strikes with NUnitForms at bat…

  8. an ou help me in getting the URL from Mozilla firefox. I’ve written something like this:
    Process[] myProcess = Process.GetProcesses();
    Process firefox = new Process();
    AutomationElement aeBrowser;
    System.Windows.Automation.Condition conLocation = new PropertyCondition(AutomationElement.NameProperty, “Location”);
    foreach (Process p in myProcess)
    {
    if (p.ProcessName == “firefox”)
    {
    firefox = p;
    Core.Application a = Core.Application.Attach(firefox);
    SearchCriteria sc = SearchCriteria.All.AndByText(firefox.MainWindowTitle);
    SearchCriteria sc2 = SearchCriteria.ByControlType(ControlType.ComboBox).AndByText(“Location”);
    Window w = a.GetWindow(sc, InitializeOption.NoCache);
    ComboBox cb = (ComboBox)w.Get(sc2);
    break;
    }
    }

  9. Hi
    thanks for sharing this awesome piece of work. just tried it to automate testing of a small .Net app and it was quite refreshing :)

    BTW, could I use this for testing Java UI? could you point me to some sample please? TIA

  10. As I have mentioned in the documentation, you can use white to automate Java SWT/(possibly AWT, never tested) applications. It would not work for swing applications though.
    Vivek

  11. Anyone ever tried to use White with a Silverlight 2 app? The software that I´m testing will change to this plataform soon and I need a solution for gui testing it =/

  12. About Exceptions.
    I’m struggling a little bit with that also. I’m in a project that uses white although this problem isn’t specific of white.
    It seems that A Windows Forms application treats all unhandled exceptions inside the application by default if a debugger isn’t attached. So even if you surrounded all the code with a try/catch block for unhandled exceptions you wouldn’t be able to catch it. Again this is if no debugger is attached, i.e., if you run the application in Visual Studio exceptions will be caught, but if you run them normally they won’t and that dialog will appear.
    The only way I found to surpass this problem is by adding this code to the main method of the application you are going to test Application.SetUnhandledExceptionMode
    (UnhandledExceptionMode.ThrowException);
    before Application.Run.
    This really sucks because if you are testing a 3rd party application with no source code you won’t be able to add this.
    Any ideas?

  13. When I tried the white it was good and nice actually, but after that i face a lot of problems to recognize the DevExpress controls, so i was wondering is it my wrong that i can’t use it in the right way or it is the abilities of the tool.

    Please help…

  14. Hello,
    Could you, please, tell me, what GUI(or nonGUI) Tests-Runner i must use to run White tests.
    I use NUnit to run nonGUI unit test. But,unfortunately, i can’t use it to run White tests.

    Thank.

  15. Hello,

    I would like to know how can we write the same code for NUnit
    I was not able to get the application class in C#.

    Can any one help me in this???

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>