When using IronRuby to test C# applications we are still faced with similar issues as with C# – the different is how we can handle them. For example, to stub the HttpRequestBase in C#, we could use Rhino Mocks as follows.

var stubbedHttpRequest = MockRepository.GenerateStub<HttpRequestBase>().Stub(x=>x.ApplicationPath).Return(“~/test”);

I do really like this syntax and think it works for C#.  However, if we are looking to use Ruby and a dynamic language we have the potential to be more inventive.

IronRuby has an excellent framework called Caricature which allows you to fake CLR objects. For example, here we are stubbing the HttpRequestBase from MVC.

require 'caricature'
include Caricature
stubHttpRequest = isolate System::Web::HttpRequestBase     
stubHttpRequest.when_receiving(:application_path).return("~/test")    
stubHttpRequest 

However, this got me thinking. With Ruby being dynamic, how could we take advantage when defining fakes? For example, what about the following syntax:

stubHttpRequest = stub 'System::Web::HttpRequestBase
                          .ApplicationPath.returns("~/test") &&
                          .FilePath.returns("")'

This would stub two properties, ApplicationPath and FilePath to return “~/test” and an empty string respectively. If we wanted to handle method calls and arguments, we could have the following:

stubHttpRequest = stub 'System::Web::HttpRequestBase
                          .SomeMethodCall("WithArgument").returns(SomeObject.new) && 
                          .SomeOtherMethod(*).returns(-1)'

Here we stub two methods, one stubs with a particular argument (must be the string “WithArgument”) while the other matches on any argument.

My aim is to reduce the ceremony associated with the act of stubbing and instead focus on the true intent of the defined behaviour.

Note: Imagine the ‘refactoring’ problem has been solved, and changing the method names would also update the tests.

If we look at other languages, for example Javascript’s jqMock and Ruby’s NotAMock are using a similar syntax to C#.

var alertMock = new jqMock.Mock(window, "alert");
alertMock.modify().args("hello world new!").returnValue();

I think it is time to start looking beyond the existing syntax and reveal our true intent. What do you think?

Labels: Ruby, Testing

Recently I have been paying close attention to MEF, the Managed Extensibility Framework. MEF is an extremely powerful framework aimed at making parts (internal or external extensibility points) more discoverable.

While I was looking at this, it dawn on me. Unit testing frameworks main role is identifying and executing methods. If that is their main function – how could you use MEF to identify and execute those methods? The result of this pondering is MEFUnit. Like with xUnit.GWT this is located on GitHub and has been mentioned on twitter once or twice.

The Tests

The tests look very similar to any other kind of unit testing framework you might have used. The attribute [Mef] identifies which methods should be executed, and like with any other framework they can pass (no exception), fail (assertion exception) and be skipped (skipped exception).

    [Mef]
    public void Test3()
    {
    }
}

Fundamentally, this is the main concept of most unit testing frameworks. Yes, some have parameterized tests and other such features which are great, but many people got by with just this. When you run the tests, you get the following output.

Executing Tests
Executing Test1... Test Failed
Executing Test2... Skipped
Executing Test3... Passed

Key implementations points of MEFUnit

But how are these methods actually turned into unit tests? The main point in the above example is the MEF attribute. This is simply a custom ExportAttribute. I could have wrote:

[Export("Mef", typeof(Action)]
public void Test() {}

However, I feel creating a custom attribute improves the readability and usability of my framework for the user. It also means if I need to change the contract I can do it without effecting my dependent exports. The second important fact is that the exports are of type Action.  By storing the method references as Action I can executed them anywhere in my code. This is the trick which makes this all possible. It means I can execute each test as shown and report the result to the console.

        try
        {
            mef();
            Console.WriteLine("Passed");
        }
        catch (AssertException ex)
        {
            Console.WriteLine(ex.Message);
        }
        catch (SkipException)
        {
            Console.WriteLine("Skipped");
        }
    }
}

The Mefs collection is populated via the MEF framework and the use of an Import attribute. Within the class, the property looks like this: 

[MefTests]
public IEnumerable<Action> Mefs { get; set; }

As with the Export attribute, I wanted to hide the actual contract I used to import the Action methods. This allowed me to hide the fact I was using AllowRecomposition, enabling tests to be dynamically identify when new assemblies are loaded.

public MefTests() : base("Mef", typeof(Action))
{
    AllowRecomposition = true;
}

When you combine this with the Compose method which hooks together the core parts of MEF you have the core concept of a unit testing framework.

I have to admit, I’m not expecting anyone to actually use this. However, I think it shows some of the interesting capabilities which MEF brings to the table.

Labels: MEF, Testing

I had a interesting comment on my previous post about .Net fault injection. ‘Losing Side’ asked if this would work for simulating other faults such as timeouts. It’s a good point and one I didn’t think about yesterday, but there are other faults which are interesting when testing the application. Performance is one of those areas, creating performance problems, such as slow disk IO or a slow server is difficult if you don’t have the setup in place, and even then not always possible. How can you effectively, repeatability test for a slow hard drive (and using a virtual machine doesn’t count). Tools such as ANTS Profiler will help tell you where the problems are, but only if you can reproduce the problem.

First demo of the day, I ask the question – how can you simulate a slow write process when using StreamWriter?  Based on my previous post, I've changed the method to this:

private static void MethodFails()
{
    Console.WriteLine("Writing to a file @ " + DateTime.Now);

    string path = Path.GetTempFileName();
    StreamWriter sw = new StreamWriter(path);
    sw.WriteLine("This is a test @ " + DateTime.Now);
    sw.WriteLine("This is a test @ " + DateTime.Now);
    sw.WriteLine("This is a test @ " + DateTime.Now);
    sw.WriteLine("This is a test @ " + DateTime.Now);
    sw.Close();
    Console.WriteLine("Done");

    foreach (var s in File.ReadAllLines(path))
        Console.WriteLine(s);
}

Running my console application normally, I get the following output, notice no delays between each write:

Writing to a file @ 15/11/2008 13:23:48
Done
This is a test @ 15/11/2008 13:23:48
This is a test @ 15/11/2008 13:23:48
This is a test @ 15/11/2008 13:23:48
This is a test @ 15/11/2008 13:23:48

Running this using my injector, I have a five second delay (which I set, easily could have been a random number) between each of my writes to the file.

I now get the same five second delay each time this happens, creating a scenario repeatable.

Disappointingly, I tried to use the SQLConnection object, but I couldn’t get this to work. I don’t know if its a limitation or a bug.  Still a lot more work to do until its even remotely useable, but I'm finding the concepts interesting.

Labels: C#, FaultInjection, Testing

I’ve just completed my first proof of concept which I'm very excited about and I just had to write a quick blog about it – plus it breaks my blogging silence.

I find fault injection an interesting concept, I'll explore it in more detail in later posts, but the aim is to insert exceptions at various points in order to test certain behaviours and how the application copes when an error occurs. However, raising exceptions is not a easy task, you have to create the scenario and environment in order for the exception to occur. With my fault injection application, you don’t need to create the scenario, you simply say when you want the exception to occur. For example, I would be able to throw an IOException when System.IO.File.ReadAllLines() is called without actually having to create the scenario for the exception to be raised – saving me time and effort but also allowing me to test more scenarios and error handling.

Tonight, I created a very simple concept to throw exceptions on method calls. I’ve got a very simple console application which calls two methods – nothing special about this.  When MethodFails() is called, my injector will step in and throw a MethodAccessException (could be any exception) which will be caught by the console application with the details of the exception being outputted.

using System;

namespace ConsoleApplication2
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Should output 1 message then fail");
            try
            {
                MethodA();
                MethodFails();
            }
            catch (Exception e)
            {
                Console.WriteLine("Exception caught");
                Console.WriteLine("Type: " + e.GetType());
                Console.WriteLine("Message: " + e.Message);
            }
        }

        private static void MethodA()
        {
            Console.WriteLine("This should work");
        }

        private static void MethodFails()
        {
            Console.WriteLine("this should have failed");
        }
    }
}

The output of the execution is the following information:

Should output 1 message then fail
This should work
Exception caught
Type: System.MethodAccessException
Message: You attempted to call ConsoleApplication2.Program.MethodFails, this has been blocked. Goodbye

Technical details to come at a later point, it’s getting late in the UK. However, I would love to hear your comments on this idea.

Labels: FaultInjection, Testing

Recently, Red Gate sent me on the ISEB Foundation Testing certification course. I had heard a lot of reports about the certification from fellow testers, but the course was only two days, in house and I had some free time so I thought why not – I might learn something.

Now I have my mark, I feel I should ‘share’ my experience and my view on the certification.

Theory and Terminology

I understand that this is a foundation course, however it still includes a lot of terminology which accounts for a large percentage of the course. The aim is to  have a consistent terminology across the industry and make sure everyone is reading from the same page. All of the terminology used is listed in the glossary. Personally, knowing the correct terminology isn’t actually that useful in the real world, for example knowing what the term ‘failure’ means is useless if you cannot identity it. 

In general, the course was focused more on the theory side of testing, which was mainly based on theory in the perfect world where you have a fully testable spec. The advice was that if the spec wasn’t testable, alert the business that you can’t do any work until it is testable – I'm not sure how many companies would welcome this demanding stance. As a result of focusing on specification driven testing, it didn’t go into much depth about Exploratory Testing, apart from define the term, which I feel is much more important and generally more applicable.

Ignorant of new development processes

Most of the testing process discussion was based around the V-Model. The V-Model is great, its a visual presentation of a sequence of steps,  on the left you have tasks, while on the right you have verification steps. For example, when you receive your detailed testable specification, you create your acceptance tests to be used to verify the system. However, it starts to get a bit murky when you haven’t got the stages down the left, or when your doing Test Driven Development (according to the model, you design, code, test).

While the V-Model doesn’t define what development methodology it applies to, it sits perfectly in the waterfall development lifecycle. I would have liked to see more of a focus on agile development, mentioning iterative and Rational Unified Process (RUP) doesn’t count.

We should be encouraging agile processes, how testers fit into the process and how they can work more closely with BA and Devs to ensure testable requirements and testable stories.

Agile is not just a fad, a new-age way which will never catch on! It should be taking a much more positive stance in our teaching.

“Just learn it, don’t argue with it”

For me, this is the single biggest problem with certification.  Even if you feel, or know, it is wrong – don’t argue, just learn it so you can pass and have your name on a piece of paper. This is encouraging the MCSE 2000 style certifications, where all you need is a brain dump of all the terminology and sample questions to be able to pass.

This is not a very effective way to teach, and it is definitely not an effective way to learn. Generally with software development, I have learnt by having in-depth discussions about the topic in question, the rights and wrongs, best practices and alternative approaches in order to gain a good understanding.  Someone saying that this is the only way you can do something is not very constructive.

However, you do just need to ‘learn’ the material in order to pass the exam. During my scrum master training, we had really good discussions about how scrum works in the real world. By using the material as a guide, and not the be all and end all, and not having to worry about passing an exam, we was able to dig deep into certain areas and have a discussion, as a result taking much more away from the two days.

“Principle 6: Standardized tasks and processes are the foundation for continuous improvement and employee empowerment” (The Toyota Way)

Maybe we (I) have this all wrong? Maybe the aim of testing certification isn’t to teach you the latest and great techniques, but to provide you with a set of standardised tasks and processes to use as a foundation – it is after all, a foundation certification. I’m currently reading The Toyota Way and this is similar to principle 6, have standardised tasks and processes to allow for improvement instead of reinventing the wheel each time.  It would make more sense.

If this is the case, then where is the continuous improvement and updating of the course material to take into account new processes, tools and best practices. By standardising these new ideas, we could improve them to create new best practices improving the industry in general. While the content is updated, is appears to be very static in terms of ideas.

The future for testing certifications?

What is the future for testing certification? From the numbers taking the examination, it looks like testing certification is here to stay. I think there are two initial approaches to improve the certification.  The first is that the foundation course doesn’t have an exam and instead follows a similar approach to the certified scrum master training to allow for discussion and sharing of ideas. With no exam, there is not as much red tape, there is no need for writing and marking papers allowing the content to be updated with more flexibility. The course could be changed to include the new ideas, sharing the best practices and improving the industry in general.

In terms of training, it would be great to see a similar course aimed at testers, which developers have with “Nothing but .Net” from Jean-Paul Boodhoo. A serious deep dive into different testing techniques, tools and approaches. Along side this, conferences have their role to play. This year, DeveloperDay in the UK has a number of different testing based sessions, all of which are real-world ‘take back to your office and use’ subjects. A number of testing conferences I have seen are more focused on the academic side and papers on testing, while interesting do not apply to improving your work today.

I wonder if the practitioner exam is any better?

Labels: Testing

Alan Page, a Microsoft Tester, recently posted about GUI Automation and I just wanted to provide my view. Alan's main comment was:

"For 95% of all software applications, automating the GUI is a waste of time."

Bold statement, but one I have to agree with - to a point.

Personally, I think we should automate UI’s, however, we need to be very clear and careful about what we automate and the tools we use. The most important part of UI testing is how you structure your tests and know your aim of the tests.

Generally, there are two types of automated testing tools. The first is the ‘record and playback’ approach, using tools such as QTP (HP QuickTest Professional). These tools have been around for a long time, and from my point of view has generally given automated GUI testing a bad name. 

QTP records all the users interaction with the application (Web or Desktop) and scripts it out to scripting language built on top of VBScript file for playback. 

Sadly the application doesn’t support sharing of steps which means you have to repeat the same actions within each test. If this same repeated action changes, you need to change delete and recreate all of your existing tests – something which is really frustrating when you are just about to release but need to change the name of a button.  This causes a lot of waste.

The other approach is more programmatic, where you program against an object model which interacts with the SUT (System Under Test). For example, with Watin, the html ‘<a />’ turns into Ie.Link object. I’ve spoken before about WatiN and Project White, two of the talks I suggested for DeveloperDay are based on these topics.

As mentioned in the blog posts, these frameworks allow you to interact in a more domain driven way. They allow you to write the test in a reusable way using your application terminology making them much more readable and maintainable. However, from my experience its still relatively slow to start creating the tests.  Once they are created, it fine, but identitying the GUI controls and how to interact with them can take a while.

Are these tests worth writing? Or are they just a waste?

That really depends on your aim. Both approaches will get your good test coverage. The programmatic approach will produce much more maintainable tests which results in less waste.

One approach which I have recently been thinking about is having a set of core smoke tests, created using Watin\White, focusing on core functionality of the application. These tests are just to ensure that the happy scenario of the application works, if someone picks up the latest build will they be able to use it? Or when it launches, is the first thing a user sees an unhandled exception?

For example, with Windows Live Writer, the test would create a new post, write some text, click publish and then verify the blog was posted. If that breaks, I really want to know, I want to know as soon as possible. I guess this is the 5% of UI tests.

The rest of the GUI (95% general waste) is automated using a similar approach to an API\business logic. You are still automating the UI, but you are not interacting directly with the UI, this is covered by manual test cases, exploratory testing and generally making sure the application is useable – something which you can only do manually. I want to cover this in more details in a later post, but the CompositeWPF (Prism) project from Patterns and Practices is a great start. 

Together, this should provide you with confidence that the application works as you expect and want in the most maintainable way.  If you can’t test the 95% using the correct patterns, then you might need to look at using Watin\White for more than just core functionality.

As I said, this is just one approach, which has advantages and disadvantages.

UPDATE: I think I might have been wrong with my percentages, in fact ignore the percentages. There should be a bigger ratio of tests using White\Watin, but they should only focus on core functionality.

My main point was that these tests shouldn’t attempt to cover all possible inputs and outputs, their aim should be to give you confidence in your application and that it will work on a given platform. In order to have this confidence, you might want more tests, which is fine as long as you are structuring your tests correctly and using frameworks such as Watin.

Labels: Project White, Testing, WatiN

It's always interesting to see how the different unit testing frameworks approach automated testing, each framework has it's own style, approach and feature set with everyone having their own view on which framework everyone should use. One of the more (you could say most) popular .Net framework is NUnit and over the past few months Charlie has been working on v2.5 so I thought it was about time I took a look.

Parameterised Tests (Documentation can be found here Parameterized Tests)

Similar to XUnit and MbUnit, NUnit 2.5 includes the concept of parameterised tests. RowTest was included in 2.4.7, however additional attributes have now been included.

ValuesAttribute

The first new attribute allows for combinational style testing, this is where two sets of inputs are combined to create each possible test combination. In certain situations useful - always good to have in the toolbox, however generally I wouldn't need to use this on a day-to-day basis. MbUnit has had this for a while, however I prefer the syntax of this approach, which I think is a similar syntax to MbUnit V3.

To implement this, you add the ValuesAttribute, which takes an object array for the test values, before the method parameter.

[Test]
public void ValuesAttributeTest([Values(1,2,3, 20)] int v, [Values(9,8,7)] int v2)
{
    Assert.IsTrue(v < v2);
}

When NUnit loads this up, they create a combination of the test values, as a result we have 12 tests created. If we had another parameter of the test, for example with 3 possible values, then we would have another possible combination and we would have 36 tests. As a result, you can very quickly build up a good selection of test cases.

RangeAttribute

Similar to the ValuesAttribute, however instead of you defining the values, you define the range of possible values which NUnit then creates tests for. You define the range (min, max) and NUnit does the rest. The values are then combined in a combinational style.

[Test]
public void RangeAttributeTest([Range(1,100)] int v, [Range(1,100)] int v2)
{
    Assert.IsTrue(v < v2);
}

The test creates produces then looks something like this:

RandomAttribute

Similar to the two previous attributes, Random produces a set of random input. You can either give it a count of the inputs to use, as I have done with the first attribute which produces 10 random values, while in the second parameter I provide the min and max range for the random values together with the amount of data to produce.

[Test]
public void RandomAttributeTest([Random(10)] int v, [Random(1, 50, 10)] int v2)
{
    Assert.IsTrue(v < v2);
}

However, I'm not sure how useful testing against random data would actually be?

TestCaseAttribute

This attribute is similar to RowTest, however as part of the TestCase you can set your own TestName which I think is actually really useful.

[TestCase("Test1", TestName = "This is a great!", Description = "My first test")]
[TestCase("Test2")]
[TestCase("")]
public void TestCaseAttributeTest(string test)
{
    Console.WriteLine(test);
    Assert.IsNotEmpty(test);
}

The TestCase is then reflected In the GUI and report. The first two tests use the test method name plus their parameter inputs, while the third uses the TestName property we set on the TestCaseAttribute.

Also as part of the TestCaseAtribute, you are able to provide the Result.  Generally, the expected result is passed in as a parameter and verified within the test itself. With this approach you have the result as part of the TestCase itself, with the test returning it's result which NUnit then compares to decide if the test has passed. This does make more sense as the parameters of the test as then really test inputs with the test output being verified as an output, a small difference however it is more logical.

[TestCase("Test1", TestName = "This is a great!", Description = "My first test", Result = 5)]
public int TestCaseAttributeTest(string test)
{
     Console.WriteLine(test);
     return test.Length;
}

This approach to testing has a lot of potential use cases and opportunities as it really increases test case (code) reuse and test maintenance.

TestCaseFactoryAttribute

Similar to MbUnit's FactoryAttribute, you define a class which yield returns each TestCaseData object. It has the advantage of using a fluent interface to define all the different properties, such as results and the test name.  While I have hardcoded the 4 test cases here, there is nothing stopping you dynamically creating these.

[Test,Factory(typeof(MyFactoryClass))]
public int FactoryAttributeTest(int a, int b)
{
    Console.WriteLine(a / b);
    return a/b;
}

public class MyFactoryClass
{
    [TestCaseFactory(typeof(int), typeof(int))]
    public static IEnumerable TestCases
    {
        get
        {
            yield return new TestCaseData(12, 3).Returns(4);
            yield return new TestCaseData(12, 2).Returns(6);
            yield return new TestCaseData(12, 4).Returns(3);
            yield return new TestCaseData(0, 0)
                              .Throws(typeof(DivideByZeroException))
                              .WithName("DivideByZero")
                              .WithDescription("An exception is expected");
        }
    }
}

The TestCases then look like this:

New Assertions

Assert.Throws<T>

Simple, straight forward. Does the method call throw can exception?

[Test]
public void ThrowException()
{
    Assert.Throws<Exception>(throwEx);
}

public void throwEx()
{
    throw new Exception("Test");
}

Assert.DoesNotThrow

Simpler approach, Assert that the method does not throw any exceptions when it is called.

[Test]
public void doesNotThrowExTest()
{
    Assert.DoesNotThrow(doesNotThrowEx);
}

public void doesNotThrowEx()
{}

Others

RequiredAddinAttribute

This is extremely useful if you are writing your own NUnit addins, as it is very important that the addin is available before you attempt to execute your tests. After adding attributes about the addins you require,

[assembly: RequiredAddin("MyTestFixtureAddin")]
[assembly: RequiredAddin("MyTestAddin")]
[assembly: RequiredAddin("MyDecoratorAddin")]

If they are not available and you execute the test, it will fail but will actually tell you the reason why.

PlatformAttribute

Finally, there is a new PlatformAttribute. This attribute defines which platforms the test is allowed to run on, really useful if you have issues with a certain target platform, or if you have a set of tests only targeting a platform, such as mono.

[Test]
[Platform(Exclude="Win98,WinME", Reason = "This only works on Windows 2000 and above")]
public void OnlyWorksWith2000OrHigher()
{
    Assert.IsTrue(Environment.OSVersion.Version.Major > 5);
}

[Test]
[Platform(Include = "Win98,WinME")]
public void BugOnWin98()
{
    Assert.IsTrue(Environment.OSVersion.Version.Major > 5);
}

The list of possible platforms can be found here: http://nunit.org/index.php?p=platform&r=2.5

When the test is executed on an invalid platform, the test is skipped.

This post has summarised all of the interesting bits of the new Alpha release. In future releases, I expect to see some changes going forward before the final 2.5 release. It's nice to see NUnit taking the different approaches from the different frameworks and incorporating them with their own approach, hopefully v3 will have some really interesting concepts. The main feature missing at the moment are data driven test attributes, such as pulling test cases from CSV or SQL Server, but a framework has an additional extension point to make this attribute easier to write.  After this I think most of the missing features would be more suitable in NUnit 3.0.

Release Notes: http://nunit.com/index.php?p=releaseNotes&r=2.5
Download: http://nunit.com/index.php?p=download

Labels: NUnit, TDD, Testing

During the development lifecycle, there are many different automated tests which should be wrote, each style of test has different priorities and different levels of maintenance required. For example, TDD helps with designing the code your just about to write, they are small and focused. As such, when your code changes, your tests change too. Where as acceptance tests have a much longer life-time, there role is to ensure the application works and nothing is regressed as developers continue to make changes. These tests need to be maintainable enough to cope with application changes, readable enough to identity the story \ feature they are testing with reusable\flexible sections to help readability and maintainability.

In an attempt for my tests to meet these goals, recently I've been using delegates in certain scenarios to improve my test code. Delegates are great for reusing sections of code, it allows you to have static sections of code but be flexible to cope with minor changes and different scenarios the tests might cover.

For example, below is an acceptance test which must connect to a server based on a project file which has been saved to disk.  Remember, acceptance tests should cover the system in an end-to-end fashion.

In order to test the application, the code needs to do a lot.  It needs to create and configure a project, save it to disk, create the service and finally it gets to the point of what it is testing - connecting to the service. This code, while it is flexible it is not very maintainable. If we need to change something related to the project or how we connect to the service we would have to change a lot of code to take this into account.

public class Project_Execute_Tests_Standard
{
    [Fact]
    public void Project_Can_Login_And_Can_Connect()
    {
        string path = System.IO.Path.GetRandomFileName();

        Service service = null;
        try
        {
            Project project = new Project();
            project.Server = "Test";
            project.Username = "Admin";
            project.Password = "Password";
            project.Save(path);

            service = new Service();
            service.Start();

            service.LoadProject(path);

            bool connect = service.Connect();
            Assert.True(connect);
        }
        finally
        {
            if(service != null && service.Started)
                service.Stop();
            File.Delete(path);
        }
    }
}

The first logical step would be to use helper methods.  We can extract the creation of the project and gaining access to the service into separate methods which we can reuse throughout our test code. However, these methods aren't very flexible, if we need to add some more configuration to the project we would have to create a different method and as such losing some of our maintainability.

[Fact]
public void Project_Can_Login_And_Can_Connect()
{
    string path = string.Empty;
    Service service = null;
    try
    {
        path = CreateProject();

        service = GetService(path);
        bool connect = service.Connect();
        Assert.True(connect);
    }
    finally
    {
        if (service != null && service.Started)
            service.Stop();
        File.Delete(path);
    }
}

This is why I like delegates. You have the advantage of helper methods, but the flexibility of having the code inline. Below I've created a helper class, this abstracts away from my actual implementation and manages the state, such as the file path for the project. Within the delegate for the project, I'm setting all the details based on the requirements of the test and different configurations, however the rest is abstracted away.

public class Project_Execute_Tests_Delegate
{
    [Fact]
    public void Project_Can_Login_And_Can_Connect()
    {
        ServiceTester tester = new ServiceTester();
        tester.CreateProject(delegate(Project project)
                                        {
                                            project.Server = "Test";
                                            project.Username = "Admin";
                                            project.Password = "Password";
                                        });

        bool connect = tester.ConnectToService();
        Assert.True(connect);
    }
}

While I was writing this example, DevExpress popped up and told me it can shorten my delegate to a lambda expression. As a result, the delegate could be this.

tester.CreateProject(p =>
                            {
                                p.Server = "Test";
                                p.Username = "Admin";
                                p.Password = "Password";
                            });

Within my ServiceTester helper class, my CreateProject method looks like this:

public delegate void ProjectDelegate (Project project);
public void CreateProject(ProjectDelegate ProjectSettings)
{
     ProjectPath = System.IO.Path.GetRandomFileName();
     Project project = new Project();

     ProjectSettings(project);
     project.Save(ProjectPath);
}

Given the right scenario, I think this could really improve your test code.

Download complete code sample: http://blog.benhall.me.uk/Code/Test/TestCodeDelegates.txt

Labels: TDD, Testing

As many of you are aware, I currently work at Red Gate software as a Test Engineer. I have a strong interest in both code and test side and happy to discuss either at length. I joined Red Gate as a tester because I strongly believe Red Gate are doing something amazing, producing great useable software and I felt I could make a difference (for the better) as a tester. However, after six months of being a tester I have to ask the question - in the community, where are all of the testers? Developers are easy to find, they have massive conferences (PDC, TechEd) down to small user groups (NxtGenUG), I have been a member of NxtGenUG for almost 2 years since they first started in Coventry and I attended TechEd Europe but where are the testers at these types of events? Or am I just missing something?

I know recruiters have been asking this question for a while, but from a community point of view - where are all the testers? Where are the conversations happening? There must be a conversation happening about how we can improve software testing, how testers fit into the project structure and take advantage of new development technologies. Alt.Net has really promoted thinking differently about software development, but how many testers are involved in that kind of discussion? People such as Roy Osherove provides his experience and is very passionate about unit testing, but who is leading the way when it comes to functional\acceptance\integration testing? Where are people discussing how we can take advantage of "design for testability" from the testers point of view - not just developers?

Visual Studio 10 (Rosario) is apparently very focused on testers and how the visual studio ecosystem (VSTS) can support testers equally as well as software developers, but without a good public conversation happening how is it going to happen and not just let it fall by the waste side leaving them to focus on other existing parts. In November 2005, The Register wrote about Microsoft wanting to turn testers into 'Rock Stars', but since then nothing has happened.  A Microsoft Tester Centre opened on MSDN in November 2007 (two years after The Register article) but its extremely light on content!  It's not even a feature developer centre? In fact, the link is under "Servers and Enterprise Development" within the Developers Centre! How can Microsoft expect to make a major stand and push testing if they don't even have the testing centre under the right category?

Internally, Microsoft seem to have a really good setup with Software Design Engineer in Test (SDET) who I would class as 'Test Developers', but at heart still testers (also in terms of this post), while some of those such as Steve Rowe are blogging and talking about testing the reach externally still doesn't seem to be enough.

This is my call to action, If there are any testers (or Test Developers) reading this, we should talk!! Especially those in the UK, I would love to hear from you! We should have a chat about the testing community and testing in general.

Please either leave a comment or email Blog {at} BenHall {dot} me {dot} uk. On the other side - developers - what do you think?

Labels: Testing

While executing my integration tests today, I was greeted with the excellent Assertion.Failed dialog which is displayed when using System.Diagnostics.  This dialog caused a problem on the build server as it was waiting for someone to click Ignore - that someone never came!

If you haven't saw the Debug.Assert code before, it basically looks like this:

[Test]
public void DebugAssert()
{
    System.Diagnostics.Debug.Assert(false);
}

When the Assert fails, the dialog is displayed.

The solution, add an App.Config file to your test suite.  Within the XML will clear the trace listeners, now when assert is called nothing happens because nothing is listening for it and the code continues. While executing unit tests this is fine - we just don't want the dialog being displayed.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <system.diagnostics>
        <trace>
            <listeners>
                <clear/>
            </listeners>
        </trace>
        <assert assertuienabled="false" />
    </system.diagnostics>
</configuration>

If you was interested in the debug.assert messages, you could have hooked up a different listener to write out the message to the console or a file.  This is what TestDriven.NET does, if an assertion is hit it writes the message to the output window.

Labels: TDD, Testing

Recently, Ayende released Rhino Mocks 3.5 Beta which includes a new Arrange, Act and Assert syntax style. This is a new syntax for creating your stubs and mocks. To take a deeper look into this new syntax, I'm going to migrate an old example I created for my article Beginning to Mock with Rhino Mocks and MbUnit - Part 2 which is a good read if you are new to the world of mock objects. As a quick refresher, the test I created was ensuring that the PaymentProcessor correctly communicated with the PaymentProcessingObject (in the system, this actually talked to PayPal).

[Test]
public void TakePaymentViaPaymentProcessorUsingMockService()
{
    MockRepository mocks = new MockRepository();
    IPaymentProcessing mockProxy = mocks.CreateMock<IPaymentProcessing>();

    using (mocks.Record())
    {
        Expect.Call(mockProxy.TakePayment(1, 1, 10.0)).IgnoreArguments()
                             .Return(true);
    }

    using (mocks.Playback())
    {
        PaymentProcessor pp = new PaymentProcessor(mockProxy);
        bool result = pp.TakePayment(1, 1, 10.0);
        Assert.IsTrue(result);
    }
}

The code this tested was:

public interface IPaymentProcessing
{ bool TakePayment(int paymentId, int customerId, double amount); }

internal class PaymentProcessor
{
    internal IPaymentProcessing wsProxy;

    public PaymentProcessor(IPaymentProcessing proxy)
    {
        wsProxy = proxy;
    }

    public bool TakePayment(int paymentId, int customerId, double amount)
    {
        return wsProxy.TakePayment(paymentId, customerId, amount);
    }
}

This is a very generic example, but demonstrates Rhino Mocks using the Record (Setup the mock object) and Playback (Test and use) approach to creating mock objects. Now we have our classic test, let's look at the changes for 3.5

CreateMock is Obsolete

The first change is that CreateMock has been marked as obsolete. Instead, we should be using StrictMock, this makes a lot more sense as it better defines the type of mock being created.

IPaymentProcessing mockProxy = mocks.StrictMock<IPaymentProcessing>();

But this still uses the same Record\Playback model.

AAA Syntax - Arrange, Act, Assert

This is where Rhino Mocks 3.5 is really interesting - for me at least.  We can now express our mock object is a much cleaner fashion, taking advantage of .Net 3.5 extension methods and lambda questions. Below, is the same test as above but using the Mock object.

[Test]
public void GenerateMock_TakePaymentViaPaymentProcessorUsingMockService()
{
    IPaymentProcessing mockProxy = MockRepository.GenerateMock<IPaymentProcessing>();  #1

    mockProxy.Expect(x => x.TakePayment(1, 1, 10.0))                                                                #2
                    .Constraints(Is.Equal(1), Is.Equal(1), Is.Equal(10.0)) 
                    .Return(true);

    PaymentProcessor pp = new PaymentProcessor(mockProxy);
    bool result = pp.TakePayment(1, 1, 10.0);
    Assert.IsTrue(result);

    mockProxy.VerifyAllExpectations();                                                                                       #3
}

#1  Here we tell Rhino Mocks to create us a mock object of type IPaymentProcessing.
#2  Next we define our mock. Here, we are saying we expect TakePayment to be called, we then add some constraints about what the parameters passed in much be, finally defining that true be returned when this is called.
#3  Finally, we verify the exceptions we set in #2 where correct.

I find this new approach to be much easier to read, explain and write. The first time I tried this, I actually mistaken Constraints for Return and as such the following exception was thrown.

mockProxy.Expect(x => x.TakePayment(1, 1, 10.0)).Constraints(Is.Equal(true));  = failed: System.InvalidOperationException : The number of constraints is not the same as the number of the method's parameters!

Just be aware of these new constraints on the parameters. 

But, not only can we use this for creating mocks, but we can also create Stubs.

[Test]
public void GenerateStub_TakePaymentViaPaymentProcessorUsingMockService()
{
    IPaymentProcessing stubProxy = MockRepository.GenerateStub<IPaymentProcessing>();  #1
    stubProxy.Stub(action => action.TakePayment(1, 1, 10.0)).Return(true);                           #2

    PaymentProcessor pp = new PaymentProcessor(stubProxy);
    bool result = pp.TakePayment(1, 1, 10.0);
    Assert.IsTrue(result);
}

#1  Generate the stub
#2  Define the stub 

How cool is that!! Two lines of code to create our stub!

[Test]
public void GenerateStub_AssertWasCalled_TakePaymentViaPaymentProcessorUsingMockService()
{
    IPaymentProcessing stubProxy = MockRepository.GenerateStub<IPaymentProcessing>();
    stubProxy.Stub(action => action.TakePayment(1, 1, 10.0)).Return(true);

    PaymentProcessor pp = new PaymentProcessor(stubProxy);
    bool result = pp.TakePayment(1, 1, 10.0);
    Assert.IsTrue(result);

    stubProxy.AssertWasCalled(x => x.TakePayment(1, 1, 10.00));                                                  #1
}

#1  With stubs, we can also verify that the method was called. Notice this AssertWasCalled is an extension method Rhino Mocks has added to the interface.  gain, this is helping with the readability and demonstrates an excellent use of Extension Methods.

If AssertWasCalled failed, then the following exception would be thrown and the test would fail.

Expected that IPaymentProcessing.TakePayment(1, 1, 10); would be called, but is was it was not found on the actual calls made on the mocked object  

I think the new AAA syntax is really cool. I used it to explain mock objects for my NxtGenUG Coventry session and upgrading my mock objects was a simple task and made the tests much easier to read. Really looking forward to the final release.

Download Examples Used: http://blog.benhall.me.uk/Code/RhinoMocks/RhinoMocks35Beta.zip

More examples of new syntax: http://www.ayende.com/Blog/archive/2008/05/16/Rhino-Mocks--Arrange-Act-Assert-Syntax.aspx

Download: Rhino Mocks 3.5

Labels: Rhino Mocks, TDD, Testing

I've been waiting for today for too long! Peli has announced that Pex 0.5 has been released and is available to download today! In this post, I am just going to have a very quick look at the release.  Expect a lot more posts around this framework, best subscribe now so you don't miss any.

Pex is a Microsoft research project which generates tests that cover all possible inputs. It does this by a mixture of static\dynamic analysis while taking advantage of the CLR Profiler to analyse your .Net code to find all the possible paths. Pex can then creates a series of test inputs for a test YOU wrote, you write the test which Pex then uses to find all the possible routes. You can use these tests in order to find all the different possible combinations for your application, find the areas where their are problems and have Pex suggest possible fixes.

Copied from site:

Pex comes with a Visual Studio Add-in which only works with Visual Studio 2008 Professional (or higher).
Pex also works from the command line without any Visual Studio on the machine.

Downloads

v0.5.30516.0, 05/21/2008, download pex.30516.0.msi (x86), release notes

The Installer:

Very simple msi - just click next a few times..

Taking a look at the samples

The team has spent a long term putting together some documentation and samples on how to use Pex which is great (especially for a research project). The samples are installed as part of the main framework and are in a zip file linked off the Microsoft Pex start menu item. Within the zip there is a VS2008 solution, with a project Samples.Pex which appears to contain samples for a lot of the functionality (without researching more I can't tell what).

Within the samples, the first test I picked at random is the LuhnAlgorithmTestClass. This was of interest because I had a similar test scenario for Red Gate SQL Data Generator as that has a Credit Card Number Generator for the Luhn Algorithm.

The test itself, based on MSTEST, looks very standard. At the class level, we add a PexClass attribute and give the type of class we are testing against. The MaxRuns defines how many runs will be tried during the exploration.

[TestClass]
[PexClass(typeof(LuhnAlgorithm), MaxRuns = 100)]
public partial class LuhnAlgorithmTestClass

The test itself, we add a PexMethod and most importantly a parameter for the input to use. Pex will then use this test to generate more exploratory tests and use the parameter to inject the value to test against. 

[PexMethod]
public void CharactersAreNotValid(string input)
{ 
   string number = input + 'a'; 
   bool result = LuhnAlgorithm.Validate(number); 
   Assert.IsFalse(result);
}

Pex includes a Visual Studio plugin, I can write click and go Pex It!

This should run all of my unit tests, but after executing Pex It! I got an error ClrMonitorFail (-667) - "Exit code returned when the ExtendedReflection profiler encountered a fatal error." within the Pex Results dialog. I've logged this with the team, if I get a solution I will post it online.

Luckily, Pex comes with a console application so the post isn't over! The command line is simple Pex with the assembly containing the Pex tests. The following command executes Pex for me:

D:\Users\Ben Hall\Desktop\samples\Pex\Pex\bin\Debug>pex Samples.Pex.dll 

After executing this, we get a lot of information wrote to the console application and a nice HTML report outputted.  The console information looks like this:

Microsoft Pex v0.5.30516.0 -- http://research.microsoft.com/pex
Copyright (c) Microsoft Corporation 2007-2008. All rights reserved.

instrumenting... launched Pex 0.5.30516.0 x86 Edition on .net v2.0.50727
[reports] report path: reports\Samples.Pex.80522.232227.pex
00:00:00.0> starting execution
  00:00:00.6> reflecting tests
  00:00:04.9> Samples.Pex
    00:00:05.0> LuhnAlgorithmTestClass
      00:00:06.5> CharactersAreNotValid(String)
        [test] (run 1) CharactersAreNotValidString_20080522_232253_000
        [test] (run 4) CharactersAreNotValidString_20080522_232256_001
        [test] (run 6) CharactersAreNotValidString_20080522_232256_002
        [execution] imprecision at Samples.Pex.Implementations.LuhnAlgorithm.Validate, offset 0x55
        [execution]     9 runs (88.89 % unique paths), 9/14 blocks covered
        [test] (run 10) CharactersAreNotValidString_20080522_232256_003
        [test] (run 27) CharactersAreNotValidString_20080522_232257_004
        [test] (run 28) CharactersAreNotValidString_20080522_232257_005
        [test] (run 41) CharactersAreNotValidString_20080522_232258_006
        [test] (run 45) CharactersAreNotValidString_20080522_232259_007
        [test] (run 87) CharactersAreNotValidString_20080522_232301_008
        <boundary> maxruns - 100
        <boundary> MaxRuns, 1 times
        [coverage] 11/14 block (78.57 %)
<snip>
00:07:43.6> [finished] 269 generated tests (51 failures), 00:07:43.6011870
    -- 0 critical errors, 0 errors, 4 warnings
[reports] generating reports...
[reports] html report: reports\samples.pex.80522.232227.pex\pex.html 
  EXPLORATION SUCCESS

This is showing that Pex is working against the LuhnAlgorithmTestClass.CharactersAreNotValid(string input) test (as shown above), and is generating multiple different tests based on that initial test.

With the report looking like this:

The report contains a lot of information about the execution. Clicking the name of the test class takes you to some more information about what was executed. 

Clicking the parameter values link will display all of the values used for the different tests.  Each parameter sends the code down a different route.

Another link which interested me is the Coverage link for the LuhnAlgorithmTestClass. Clicking on this, you get a really nice Code Coverage report page.

Putting this a different way, by writing a single test with a parameter, Pex can create us a series of different tests to execute all of the possible routes in the method. This is great! As a developer, I can focus on creating a test which can exercise the method, but don't have to worry about all of those edge cases as Pex will help find those.

Hello World Pex'ed

Hopefully you are still with me, now I just wanted to create a very quick Hello World application.  The class I have developed is this HelloWorld class, it has three different paths different different values for each.

public class HelloWorld
{
    public string GetHelloWorld(int ID)
    {
        if (ID == 1)
            return "Hi";

        if (ID == 2)
            return "Hello";

        if ((ID != 1) || (ID != 2) && (ID % 2 == 0))
            return "Hello World";

        return string.Empty;
    }
}

The related test is this HelloWorld(int id).  Pex will input the required parameters as a argument, and then execute the test. We want to ensure Hello World is returned.

[TestFixture]
[PexClass(typeof(HelloWorld))]
public class HelloWorldTests
{
    [PexMethod]
    public void HelloWorld(int id)
    {
        HelloWorld h = new HelloWorld();
        string helloWorld = h.GetHelloWorld(id);
        Assert.AreEqual("Hello World", helloWorld);
    }
}

On the HTML report, it displays three different parameters which caused it to go down different routes.  For 0, it worked as expected. For 1 and 2, the test failed.

To prove this, I wrote a classic test which worked successfully. 

[Test]
public void HelloWorld()
{
    HelloWorld h = new HelloWorld();
    string helloWorld = h.GetHelloWorld(0);
    Assert.AreEqual("Hello World", helloWorld);
}

What has Pex done for us? Well, it has identified one working parameter and two parameters which will cause the test to fail.

Summary

This is a huge framework and I have only just scratched a very small surface area. I haven't even touched the more advance parts (I will need to read the documentation for that).  I will be posting more, writing more and generally seeing what this framework is actually all about. Still not 100% convinced about this framework, I'm saving that until I see more.  Not being able to use it against a commercial application might limit my venture.

Labels: Pex, TDD, Testing

An article I have wrote about Extending NUnit has just been published on the Simple Talk website. In the article I discuss how to extend the framework to support your own custom attributes, the article includes sample code about how I built attributes such as SqlServerDataSource and ExecuteSql for use when writing your test code.

You can read the article here : Testing Times Ahead- Extending NUnit

-------------------------------------------------

Simple-Talk is an online technical journal and community hub for working SQL Server and .NET developers and administrators, as well as enthusiasts and those taking their first steps with these Microsoft technologies. Simple-Talk is a growing site, currently with 150,000 subscribers, who each receive a twice-monthly newsletter providing highlights from the Simple-Talk website along with updates and announcements concerning Red Gate tools

Labels: NUnit, Red Gate, TDD, Testing

I've been wanting to write about this for a while however haven't really had the chance.  While prepping for a NxtGenUG session I came across the TestSuite attribute.  By using this, you can build up your tests to execute dynamically and provide the parameters when the tests are loaded into the runner and framework.  This allows for some great flexibility and extendibility, especially if you don't know all of the possible test values at design-time and want values to be easily added (adding a row to the database, or dropping another file into a directory).

To take advantage of this, you need to use the TestSuiteFixtureAttribute at the top of your class.

[TestSuiteFixture]
public class TestSuiteTests

You can then create the test methods you want to be executed in order to test your system.  Two important points, they must take a parameter of type Object and return an Object - the rest is up to you. In the first test, I just write out the ToString value to the console,  in the second I check to see if the value is between 0 and 5.

public Object MethodUnderTest(object testData)
{
    Console.WriteLine("MethodUnderTest Executed");
    StringAssert.Contains(testData.ToString(), "Parameter");
    return null;
}

public Object AnotherMethodUnderTest(object testData)
{
    Console.WriteLine("AnotherTest");
    string testDataString = testData.ToString();

    int testExecution = Convert.ToInt32(testDataString[testDataString.Length -1].ToString());
    Assert.Between(testExecution, 0, 5);

    return null;
}

The final stage is to create the method which will programmatically create all of your tests and test parameters.

The method below is marked with the TestSuiteAttribute so the framework knows this can create tests.  Inside the method, we create a new TestSuite object with a name to identify it, we can then add new items into the collection which are then returned to the framework and executed.  The parameters to add are:

1. Name of test
2. Method to call (the test itself) as a TestDelegate object
3. Any parameters for the test

[TestSuite]
public ITestSuite Suite()
{
    TestSuite suite = new TestSuite("NewSimpleSuite");

    for (int i = 0; i < 10; i++)
    {
        suite.Add("MethodUnderTest" + i, new TestDelegate(this.MethodUnderTest), "Parameter" + i);
        suite.Add("AnotherMethodUnderTest" + i, new TestDelegate(this.AnotherMethodUnderTest), "Parameter" + i);
    }

    return suite;
}

The above code creates 20 tests, 10 for each test method. In the UI, the tests are all loaded correctly, as you can see each one is named correctly based on our code.

Very powerful attribute! But use the power wisely...

Download code sample here - http://blog.benhall.me.uk/Code/MbUnit/TestSuiteTests.cs.txt

Labels: MbUnit, TDD, Testing

After using WaTiN, I have been thinking about UI Testing for WinForms, if it's possible and if it's even worth it. On the MbUnit mailing list I posted some syntax for an approach to WinForms and I had some good ideas, I brought up the subject again at Alt.Net.UK and while people have had success using WaTiN, they didn't seem that interested in WinForm testing. I know others had been talking about WPF Testing during the day and problems with it.

As it happens, I read on Jeremy Miller's blog that Thoughtworks have released 'Project White' which is a UI Testing framework for WPF, WinForms, Win32 and SWT (Java) and works based on Microsoft's UIAutomation library and windows messages. Sounds promising so I decided to take a closer look, this post just discusses me playing around with the framework and a simple form to get an understanding of how it works.

Firstly, I created a standard Windows Forms application with just a single form. First test - does it display?

The form looks like this:

Using White and MbUnit, the test looks like this:

private const string path = @"..\..\..\White_HelloWorld\bin\Debug\White_HelloWorld.exe"; #1 [Test] public void ApplicationLaunch_NoArgs_Form1Displayed() { Application application = Application.Launch(path); #2 Window window = application.GetWindow("Form1", InitializeOption.NoCache); #3 Assert.IsNotNull(window); Assert.IsTrue(window.DisplayState == DisplayState.Restored); #4 application.Kill(); #5 }

#1 We need to define the path to our executable. This is fine if you know your always going to be building into the same folder (both test and live assemblies), bit difficult when you have separate output directories. #2 I then use White to execute the exe #3 Once the application has launched, we get the form displayed as an object. This works based on the form's title - in this case, Form1 #4 I then check the Window state to see if it has been displayed #5 Finally, I close the application.

That's a very basic test. Let's add some functionality and explore the framework in more depth. What happens if the framework cannot find the form?

[Test] [ExpectedException(typeof(Core.UIItems.UIActionException))] public void ApplicationLaunch_NoArgs_Form2NotDisplayed() { Application application = Application.Launch(path); Window window = application.GetWindow("Form2", InitializeOption.NoCache); application.Kill(); }

White will attempt to find a window called Form2, if the timeout expires it throws the UIActionException. This is the same if it cannot find a control on the form.

To make this more interesting, I created an additional form with some buttons and labels.

The first button has a simple action, when you click it the text of the button changes to be Hello World!!. We can then create a test for this as follows:

[Test] public void ButtonClickable_btnClick1_ChangesText() { Application application = Application.Launch(path); Window window = application.GetWindow("White Hello World", InitializeOption.NoCache);

Button button = window.Get<Button>("btnClick1"); #1 //Moves the mouse to click the button

button.Click(); #2 Assert.AreEqual("Hello World!!", button.Name); #3 }

#1 Using the Get method, we can give it the type and name of the control we want to access. #2 We can then call the Click method which will move the mouse cursor over to the button and click it. #3 We can then verify that the action was correctly performed, in this case the Name has changed.

Sometimes, we want to be more flexible than referring to the object by it's name. As such, we can use the SearchCriteria object which allows us to access the control in different ways, for example by it's text:

SearchCriteria searchCriteria = SearchCriteria.ByText("Click Me!"); Button button = window.Get<Button>(searchCriteria);

One problem I did encounter was with unhandled exceptions. With one of my buttons, when clicked it will cause an exception to be thrown.

The test looks like this:

[Test] //Doesn't work? No way to get the exception...I guess the test would fail so you would reproduce manually. public void ClickButton_ThrowsException() { Application application = Application.Launch(path); Window window = application.GetWindow("White Hello World", InitializeOption.NoCache);

Button button = window.Get<Button>("throws"); button.Click(); Assert.AreEqual("Hello World!!", button.Name); }

The test fails because of the assertion, sadly it doesn't report back saying that an exception was thrown which I would have liked.

failed: Equal assertion failed: [[Hello World!!]]!=[[Throws Exception]]

Another problem which I wanted to see if the framework would handle was with message boxes. When clicking btnMsg, a message box is displayed on screen. Using the framework, we can use the MessageBox() method, giving it the title of the message box in order to get a Window object (would have preferred this to be a MessageBox object). We can then treat it just like a normal window which is nice.

Button button = window.Get<Button>("btnMsg"); button.Click(); Window messageBox = window.MessageBox(""); messageBox.Close();

One last simple test I wanted to perform was how to verify that when a button is pressed, that a label is updated correctly.

[Test] public void ClickButton_btnLabelText_ChangesLabel() { Application application = Application.Launch(path); Window window = application.GetWindow("White Hello World", InitializeOption.NoCache);

Button button = window.Get<Button>("btnLabelText"); button.Click(); Label label = window.Get<Label>("lblText"); #1 Assert.AreEqual("Updated", label.Text); application.Kill(); }

#1 We can access a label in the same way we access another other control, such as a button. No problems at all. There are other objects for difference controls available.

One thing about all of these tests is that they aren't very readable, we have a lot of noise about getting access to the objects which is making it harder to read what the test is actually testing. One quick way to improve readability is to move the launch and kill calls into Setup and Teardown methods.

private Application _app = null; [SetUp] public void Setup() { _app = Application.Launch(path); }

[TearDown] public void Teardown() { _app.Kill(); }

But it can be cleaned up more, one technique I've discussed before with WaTiN is to create a wrapper around the UI and test against that to make our tests more readable and less fragile.

I've taken the code from the last test and refactored it into a wrapper. The wrapper looks like this:

public class Test2 #1 { private Application _host = null; private Window _main = null; public Test2(Application host) #2 { _host = host; _main = _host.GetWindow("White Hello World", InitializeOption.NoCache); }

public Window Main { get { return _main; } }

public Button btnLabelText #3 { get { return Main.Get<Button>("btnLabelText"); } #4 }

public Label lblText { get { return Main.Get<Label>("lblText"); } } }

#1 This is the same name as the Form in the code under test for readability #2 Pass in the application so we can gain access to the running exe #3 This could be called anything for readability #4 Use the same code to access the button and return it to the calling test.

Using this wrapper and the Setup/Teardown, the same test as before would be this:

[Test] public void ClickButton_btnLabelText_ChangesLabel_ImprovedSyntax() { Test2 form = new Test2(_app); form.btnLabelText.Click();

Assert.AreEqual("Updated", form.lblText); }

This could be reduced more by moving the form creation into Setup as well. By using this simply step, we have made the tests more much readable. The other advantage is that if the name of a button changes, we only have to change the wrapper and not all of the dependent tests.

One final problem you might encounter is finding out what each control on the form is actually called. The .Net 3.0\Windows SDK includes a tool called UISpy. This allows you to see all the properties on a running application, as such all the information required for use with White.

Another good tool is HawkEye (Updated Link).

In summary, I'm really impressed with the framework. There are a few missing features, but hopefully they can be added over time, the fact that the framework does multiple different UI technologies is great, means you don't have to worry about what to use to write your UI tests or if your UI deals with the different technologies. Can't wait to use it on a real project...

Keep an eye out for a post how to use this with WPF and more advanced scenarios.

Project White Homepage - http://www.codeplex.com/white

Code Sample - http://blog.benhall.me.uk/Code/ProjectWhite/White_HelloWorld.zip

Labels: Project White, TDD, Testing