Introduction to XUnit

While MbUnit is my unit testing framework of choice, I’ve heard some good reports about XUnit so I decided to take a closer look.  XUnit is a unit testing framework for the .Net platform, created by Brad Wilson and James Newkirk (One of the original developers of NUnit, both currently working for Microsoft) and is available to download from http://www.codeplex.com/xunit.  The framework itself is built using .Net Framework 2.0, doesn’t require any installation (XCopy) which makes it great for storing in source control and includes a TestDriven.NET runner, ReSharper runner and a console runner for executing the tests.

This framework takes a major departure from the other frameworks currently around. The major change is the naming of the attributes which you add to your test methods. Using MbUnitNUnit, your test fixture would look something like:

[TestFixture]
public class MbUnitAccountTests
{
    [TestFixtureSetUp]
    public void TestFixtureSetup()
    {
        Console.WriteLine(“We are currently in the constructor”);
    }

    [Test]
    public void MyFirstTest()
    {
        Console.WriteLine(“We are currently in my first test – well done!”);
        Assert.IsTrue(true);
    }

    [Test]
    public void MyFirstFailingTest()
    {
        Console.WriteLine(“We are currently in my first failing test – not so well done!”);
        Assert.IsTrue(false);
    }

    [TestFixtureTearDown]
    public void TestFixtureTeardown()
    {
        Console.WriteLine(“We are currently disposing.”);
    }
}

However, using XUnit your tests would look something like this:

public class AccountTests : IDisposable   #1
{
    public AccountTests()                          #2
    {
        Console.WriteLine(“We are currently in the constructor”);
    }

    [Fact]                                                   #3
    public void MyFirstTest()
    {
        Console.WriteLine(“We are currently in my first test – well done!”);
        Assert.True(true);
    }

    [Fact]
    public void MyFirstFailingTest()
    {
        Console.WriteLine(“We are currently in my first failing test – not so well done!”);
        Assert.True(false);
    }

    public void Dispose()                           #4
    {
        Console.WriteLine(“We are currently disposing.”);
    }
}

#1 – The first change is that we don’t need to add a TestFixture attribute to our class. Also, XUnit doesn’t have any Test or TestFixture Setup and Teardown attributes, however it does allow you to take advantage of the constructor and the Dispose method (if it implements IDisposable) so you can configure anything before the tests start executing.  You can’t have methods executing before and after each test separately.  The aim of this is to improve test isolation and to make the tests more readable.

#2 – As mentioned above, we use the constructor for the test class to setup anything

#3 – Before, we would say that a method would be a Test.  With XUnit they have taken a different approach and says that TDD is about example driven development and not actually testing (which I agree with to a point) as we use the methods to express intent of the class. By using FactAttribute, we are saying that this is true for this example.

#4 – As mentioned, we use Dispose to clean up anything once the tests have executed.

If we execute the above test cases using the ReSharper plugin, the output would be like below. If you look closely at the output, you can see we are constructing and disposing the test object for EACH TEST. MbUnitNUnit creates the object once and calls the test methods on the same object. Having an object per test improves the test isolation.

image

XUnit does however support a type of Test Setup – just not as we know it. In the code below, we create SomeFixture with a method.  We then use the IUseFixture interface to inject the fixture into the test class. Within the SetFixture method (which is on the interface), we can call any method on the SomeFixture object.

public class SomeFixture
{
    public void SomeMethod()
    {
        Console.WriteLine(“SomeFixture.SomeMethod()”);
    }
}

public class AccountTests : IUseFixture, IDisposable
{
    public void SetFixture(SomeFixture data)
    {
        Console.Write(“SetFixture? t “);
        data.SomeMethod();
    }
//…Your Tests
}

Now if we execute the tests, you can see the method is being called and outputted without us writing any additional code.

image

Moving on, the framework comes a good selection of assertions.  The assertions included are:

  • Contains and Contains
  • DoesNotContain and DoesNotContain
  • Empty
  • Equal
  • False
  • InRange
  • IsNotType and IsNotType
  • IsType
  • NotEmpty
  • NotEqual
  • NotInRange
  • NotNull
  • NotSame
  • Null
  • Same
  • Throws and Throws
  • True

The framework takes full advantage of generics.  Most of them are pretty standard, but two I want to pay special attention to is InRange and Throws.

Firstly, InRange is an interesting method and something which I have really wanted in the past.  The method takes three arguments, the actual value being tested and then the min/max value which the value should be in.  This checks that the actual value is between the two values.  This is useful when you know it should be in an range but you don’t care what the value actually is.

The first is a very simply test which tests to see if myValue is inbetween the min and max values.

[Fact]
public void InRangeInt()
{
    int myValue = 100;
    int min = 1;
    int max = 10;
    Assert.InRange(myValue, min, max);
}

This fails (of course…). The output from TD.net is as follows:

TestCase ‘IntroToXUnit.Tests.AccountTests.InRangeInt’ failed: Assert.InRange() Failure
Range:  (1 – 10)
Actual: 100
    E:UsersBen HallDocumentsVisual Studio 2008ProjectsIntroToXUnitIntroToXUnitClass1.cs(49,0): at IntroToXUnit.Tests.AccountTests.InRangeInt()

This also works with other objects, for example, below does the same thing but with dates.

[Fact]
public void InRangeDateTime()
{
    Assert.InRange(new DateTime(2009, 1, 1), new DateTime(2000, 1, 1), new DateTime(2008, 1, 6));
}

As such, the failing message is this:

TestCase ‘IntroToXUnit.Tests.AccountTests.InRangeDateTime’
failed: Assert.InRange() Failure
Range:  (01/01/2000 00:00:00 – 06/01/2008 00:00:00)
Actual: 01/01/2009 00:00:00
    E:UsersBen HallDocumentsVisual Studio 2008ProjectsIntroToXUnitIntroToXUnitClass1.cs(55,0): at IntroToXUnit.Tests.AccountTests.InRangeDateTime()

The method also has a parameter which accepts a IComparer so you can customize how the checking is done.

The Throws is another departure from traditional unit testing frameworks.  Instead of saying you expect a particular exception to be thrown from somewhere in the test case, you actually define the line of code that you expect to throw the exception – again, improving readability as we know the exact call to expect the exception to be raised from.  As an example, we are testing the following method. 

public void Withdraw(decimal amount)
{
    if(amount <= 0)
        throw new ArgumentException();

    Balance += amount;
}

Using XUnit, the test we would write would look something like below.  When we call a.Withdraw(0), we wrap the call inside a delegate and pass it as a parameter to the Assert.Throws class.  We then define the type of exception we expect to be thrown as the type for the method, in this case ArgumentException.

[Fact]
public void Withdraw_Amount0_ThrowsArgumentException()
{
    Account a = new Account();
    a.Balance = 1000;

    Assert.Throws(delegate { a.Withdraw(0);});
}

This passes successfully.  If we call a different method (one which doesn’t throw the exception).

[Fact]
public void Withdraw_Amount0_ThrowsArgumentException_Fails()
{
    Account a = new Account();
    a.Balance = 1000;

    Assert.Throws(delegate { a.WithdrawWithOutException(0); });
}

Then the test fails with the following error:

TestCase ‘IntroToXUnit.Tests.AccountTests.Withdraw_Amount0_ThrowsArgumentException_Fails’
failed: Assert.Throws() Failure
Expected: System.ArgumentException
Actual:   (No exception was thrown)
    E:UsersBen HallDocumentsVisual Studio 2008ProjectsIntroToXUnitIntroToXUnitClass1.cs(76,0): at IntroToXUnit.Tests.AccountTests.Withdraw_Amount0_ThrowsArgumentException_Fails()

The final thing I want to talk about is other common scenarios, such as Ignore and Timeout for tests.  Both Skip and Timeout are parameters on the Fact attribute. If you want a test to be skipped, then you set the Skip attribute and give a reason.

[Fact(Skip = “Takes too long”)]
public void SkippedTest()
{
    //Never executed.
}

When the test is executed, the following will be outputted.

TestCase ‘IntroToXUnit.Tests.AccountTests.SkippedTest’ not executed: Takes too long

0 passed, 0 failed, 1 skipped, took 2.18 seconds.

Timeout is set in a similar fashion, but you give it the timeout in milliseconds.  Below, we want the test to timeout after 1 seconds.

[Fact(Timeout = 1000)]
public void TimeoutTest()
{
    Thread.Sleep(5000);
}

Once the timeout is reached, it fails and the output is as follows.

TestCase ‘IntroToXUnit.Tests.AccountTests.TimeoutTest’ failed: Test execution time exceeded: 1000ms

0 passed, 1 failed, 0 skipped, took 4.15 seconds.

In summary, that has been a quick summary of XUnit and what it can offer. The framework has some really interesting concepts, but I don’t know if its ready for a full production test suite just yet.  It will be interesting to see what Brad and James does once 1.0 is released.

Technorati Tags: , ,

0 thoughts on “Introduction to XUnit

  1. W.Meints

    Great explaination of the xUnit framework and the philosophy behind it. I do have some questions however.

    Currently I use [SetUp] and [TearDown] methods to activate a WCF servicehost and clean the host up after the test is done. I understand this is easily done by the IUseFixture interface. Can I use this interface too for teardown?

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>