Category: MbUnit

I’ve been wanting to write about this for a while however haven’t really had the chance.  While prepping for a NxtGenUG session I came across the TestSuite attribute.  By using this, you can build up your tests to execute dynamically and provide the parameters when the tests are loaded into the runner and framework.  This allows for some great flexibility and extendibility, especially if you don’t know all of the possible test values at design-time and want values to be easily added (adding a row to the database, or dropping another file into a directory).

To take advantage of this, you need to use the TestSuiteFixtureAttribute at the top of your class.

[TestSuiteFixture]
public class TestSuiteTests

You can then create the test methods you want to be executed in order to test your system.  Two important points, they must take a parameter of type Object and return an Object – the rest is up to you. In the first test, I just write out the ToString value to the console,  in the second I check to see if the value is between 0 and 5.

public Object MethodUnderTest(object testData)
{
    Console.WriteLine(“MethodUnderTest Executed”);
    StringAssert.Contains(testData.ToString(), “Parameter”);
    return null;
}

public Object AnotherMethodUnderTest(object testData)
{
    Console.WriteLine(“AnotherTest”);
    string testDataString = testData.ToString();

    int testExecution = Convert.ToInt32(testDataString[testDataString.Length -1].ToString());
    Assert.Between(testExecution, 0, 5);

    return null;
}

The final stage is to create the method which will programmatically create all of your tests and test parameters.

The method below is marked with the TestSuiteAttribute so the framework knows this can create tests.  Inside the method, we create a new TestSuite object with a name to identify it, we can then add new items into the collection which are then returned to the framework and executed.  The parameters to add are:

1. Name of test
2. Method to call (the test itself) as a TestDelegate object
3. Any parameters for the test

[TestSuite]
public ITestSuite Suite()
{
    TestSuite suite = new TestSuite(“NewSimpleSuite”);

    for (int i = 0; i < 10; i++)
    {
        suite.Add(“MethodUnderTest” + i, new TestDelegate(this.MethodUnderTest), “Parameter” + i);
        suite.Add(“AnotherMethodUnderTest” + i, new TestDelegate(this.AnotherMethodUnderTest), “Parameter” + i);
    }

    return suite;
}

The above code creates 20 tests, 10 for each test method. In the UI, the tests are all loaded correctly, as you can see each one is named correctly based on our code.

Very powerful attribute! But use the power wisely…

Download code sample here – http://blog.benhall.me.uk/Code/MbUnit/TestSuiteTests.cs.txt

Continuing in my writing about MbUnit, this post will focus on the TypeFixtureAttribute. The TypeFixture allows a set of tests to focus on testing a certain type of class (generally an interface) and to provide objects of that type to be tested against.

This method is really useful if you have a collection of objects which all have similar behaviour which requires testing. Instead of having to duplicate your test code, you can define the types to be tested with instances of the tests being passed into the test being executed.

In my fake system being tested, I have a collection of objects which all implement a INotNull interface. I need to test to make sure that the implementation of INotNull works correctly, that not nulls are returned when GetNotNullValues() is called. My ‘Production’ code is this:

public interface INotNull
{
IEnumerable GetNotNullValues();
}

public class TypeA : INotNull
{
public override string ToString()
{
return “TypeA”;
}

public IEnumerable GetNotNullValues()
{
yield return “TypeA”;
}
}

public class TypeB : INotNull
{
public override string ToString()
{
return “TypeB”;
}

public IEnumerable GetNotNullValues()
{
yield return 1;
yield return 2;
yield return 3;
yield return 4;
}
}

public class TypeC : INotNull
{
public override string ToString()
{
return “TypeC”;
}

public IEnumerable GetNotNullValues()
{
yield return new object();
yield return null;
yield return new object();
}
}

The code looks fine, apart from TypeC which returns a null. In a standard unit testing framework, I would have to create multiple test fixtures for each type in the system and duplicate all of the tests. TypeFixture makes our life easier by allowing us to define a single fixture and test all the objects we want. Let’s have a look at the test code.

[TypeFixture(typeof(INotNull), “Tests the INotNull interface.”)] #1
public class INotNullTests

Instead of defining a TestFixture at the top of our class, we define a TypeFixture and give it the typeof object we will be testing. We then need to tell it all the objects to use.

[Provider(typeof(TypeA))] #1
public TypeA ProvideTypeA()
{
return new TypeA(); #2
}

#1 We use the ProviderAttribute and provide the type of object we are returning as a parameter.

#2 We then create a new instance of the object and return it to the caller.

We can use the Provider attribute on as many methods as required to cover all of our objects being tested. The other two objects being tested are TypeB and TypeC.

[Provider(typeof(TypeB))]
public TypeB ProvideTypeB()
{
return new TypeB();
}

[Provider(typeof(TypeC))]
public TypeC ProvideTypeC()
{
return new TypeC();
}

Now we have that in place, we need to write out tests. The test is a standard test, however accepts the type (as defined in the TypeFixture) as a parameter.

[Test]
public void TestINotNullValuesNotNull(INotNull instance)
{
Console.WriteLine(instance.ToString());

foreach (object o in instance.GetNotNullValues())
{
Assert.IsNotNull(o, “Instance: {0}. Object was null: {1}”, instance, o);
}
}

We can then use that object within our test to verify it works correctly. This test method will be called once per object defined by a provider attribute. In this case, three tests are executed with TypeC test failing.

Adding more tests to the fixture is very quick and easy and with the TypeFixture will result in it being used to test multiple objects. Adding another implementation of the interface is easy as we define another method allowing it to be picked up by all the existing tests.

In this post, I want to talk about a little know feature of MbUnit which is the CombinationalTestAttribute.  This was originally described on Peli’s blog, but I thought I would revisit the subject.

The combinational test attribute allows us to do pair-wise testing.  This is where we take two inputs (A and B) and test them against each other ({A1, B1}, {A1, B2}, {A1, B3}).

When implementing this system, the OUT (Object Under Test) needs to implement an interface.  In this case a ICount

public interface ICount
{
    int Count(string s);
}

We then implement a X counter which will the number of X’s which appear in a string.

public class XCounter : ICount
{
    public int Count(string s)
    {
        int count = 0;
        foreach (char c in s)
            if (c == ‘x’)
                count++;
        return count;
    }
}

For the purposes of demo’ing, a BadCounter is also implemented.

public class BadCounter : ICount
{
    public int Count(string s)
    {
        return 2;
    }
}

So that would be our production code which we want to test.  Using just standard TestFixture and Test attributes, our tests would have to look something like this:

[TestFixture]
public class ICountTests_Orginal
{
    [Test]
    public void BadCounter_Count_NoX()
    {
        ICount counter = new BadCounter();
        int result = counter.Count(“”);
        Assert.AreEqual(0, result);
    }

    [Test]
    public void BadCounter_Count_TwoX()
    {
        ICount counter = new BadCounter();
        int result = counter.Count(“XX”);
        Assert.AreEqual(2, result);
    }

    [Test]
    public void BadCounter_Count_XSpaceYSpaceZ()
    {
        ICount counter = new BadCounter();
        int result = counter.Count(“X Y Z”);
        Assert.AreEqual(1, result);
    }

    [Test]
    public void XCounter_Count_NoX()
    {
        ICount counter = new XCounter();
        int result = counter.Count(“”);
        Assert.AreEqual(0, result);
    }

    [Test]
    public void XCounter_Count_TwoX()
    {
        ICount counter = new XCounter();
        int result = counter.Count(“XX”);
        Assert.AreEqual(2, result);
    }

    [Test]
    public void XCounter_Count_XSpaceYSpaceZ()
    {
        ICount counter = new XCounter();
        int result = counter.Count(“X Y Z”);
        Assert.AreEqual(1, result);
    }
}

This is only testing the two objects (imagine 100s) with three test cases (imagine 1000s).  This would very quickly become unmanageable and a maintenance nightmare. We can make this much more effective using Combinational.

First part of configuring this is to setup a Factory for the interface.  This will produce all the objects we want to be tested.

[Factory(typeof(ICount))]
public IEnumerable Counters()
{
    yield return new BadCounter();
    yield return new XCounter();
}

Next, we create a factory of test cases.

[Factory(typeof(CombinationTestItem))]
public IEnumerable Strings()
{
    yield return new CombinationTestItem(“”, 0);
    yield return new CombinationTestItem(“aaa”, 0);
    yield return new CombinationTestItem(“x”, 1);
    yield return new CombinationTestItem(“xa”, 1);
    yield return new CombinationTestItem(“xax”, 2);
    yield return new CombinationTestItem(“X x”, 2);
    yield return new CombinationTestItem(“X X”, 2);
}

I’ve created a helper class within my code called CombinationTestItem.  This is used just to store the input and expected output so I can access them within my test. This could store as much or as little as you want, only restriction is that the Factory must return IEnumerable.

public class CombinationTestItem
{
    public CombinationTestItem(string value, int xcount) { Input = value; ExpectedOutput = xcount; }
    public string Input;
    public int ExpectedOutput;
}

The last part is to actually create the test.

[CombinatorialTest]                       #1
public void CountIsExact([UsingFactories(“Counters”)] ICount counter,       #2
                         [UsingFactories(“Strings”)] CombinationTestItem test)            #3
{
    Assert.AreEqual(test.ExpectedOutput, counter.Count(test.Input));            #4
}

#1 Instead of using [Test] we use [CombinatorialTest]

#2 The first parameter of the test is the Factory of Counters we created.  We use the [UsingFactories] attribute to help the framework understand which factory to pull the data from

#3 Like with #2, we do the same for the test data factory.

#4  Finally, we call the count method on the counter object passed into the test with the value we set on our CombinationTestItem object.  We then verify that it matches the related ExpectedOutput.

When it comes to executing the test cases the framework creates a test for each ICount with each CombinationTestItem test case. The result is 14 test cases (Two ICount objects * 7 CombinationTestItem test cases).

Adding additional objects or test cases is simply a case of adding an additional line of code.

My own point about this attribute is that I have never found a real world situation where I have required this.  In theory (like above) it sounds great, but I’m but sure if it would actually solve any of my integration testing problems – I can’t imagine it would solve any unit testing problems. If this does solve a problem for you, it would be great to hear about it.

Last weekend I decided it would be great fun to do a simple screencast about getting started with MbUnit.  It’s just a very quick, create project, add reference, execute screencast but hopefully it will be useful if this is your first time to a unit testing framework and you want to get started. It also gave me chance to play around with Camtasia.

Well, it wasn’t as much fun as I first thought. I had a number of attempts to find out how to actually create a screencast and I think I fall into most pitfalls along the way (but its all experience right?), that can be another post.

The outcome was that I managed to created a screencast on Getting Started with MbUnit.

Download: Screencast_GettingStartedWithMbUnit.wmv

Length: 8 Mins 56 seconds

Size: 8.71MB

Now I have a better understanding of what to do, I might create a few more! Next ones will be better…