Introduction to XUnit.net Extensions

Following on from my previous post on XUnit I decided to look at the Extensions library associated with the XUnit.net framework.  The XUnit Extensions project is available from http://www.codeplex.com/xunitext and is a separate assembly containing additional attributes (and I guess additional assertions later) and contains some really cool features.

To use the extensions, you need to reference both xunit.dll and xunitext.dll. Currently, there are six main extension attributes included within the assembly.

Theory – Support for Data Driven Tests

The first extensions I am going to look at are a set of extensions categorised as Theories. These theories allow external data to be used as the test data when executing. The extensions assembly has support for the following data providers:

1. ExcelData
2. InlineData
3. OleDbData
4. PropertyData
5. SqlServerData

To start with, I will look at the most basic – InlineData.  The InlineDataAttribute allows us to specify data to use within the test, executing the test each time for each attribute – similar to MbUnit’s RowTest.  Below is a test which verifies that the length of a string is as expected.

[Theory]
[InlineData(“hello”, 5)]
[InlineData(“hello world”, 11)]
[InlineData(“failing”, 0)]
public void Theory_InLine(string value, int expectedLength)
{
    Console.WriteLine(“String:” + value + ” With expected length of ” + expectedLength);
    Assert.Equal(expectedLength, value.Length);
}

Using TestDriven.NET, each attribute causes a separate test to be executed with the output shown below.

String:hello With expected length of 5
String:hello world With expected length of 11
String:failing With expected length of 0
TestCase ‘IntroToXUnitExtensions.Ext.Theory_InLine(“failing”, 0)’
failed: Assert.Equal() Failure
Expected: 0
Actual:   7
    E:UsersBen HallDocumentsVisual Studio 2008ProjectsIntroToXUnitIntroToXUnitExtensionsClass1.cs(20,0): at IntroToXUnitExtensions.Ext.Theory_InLine(String value, Int32 expectedLength)

2 passed, 1 failed, 0 skipped, took 1.96 seconds

This is great for reducing the amount of duplicate test code and really helps readability as if you want to add additional test cases you just include another attribute.  Sadly, the ReSharper plugin just thinks this is a single test – instead of three separate tests like TD.net does.  Having it just as a single test makes it difficult to identify the failing test.

Moving on, next is the PropertyData attribute.  Property Data allows you to specify a property which returns an IEnumerable.  Below is my property which returns the same data as in the above test. I’ve hard coded the values, but this could some from another source to meet your own requirements.

public static IEnumerable MyTestData
{
  get
  {
       yield return new object[] { “hello”, 5 };
       yield return new object[] { “hello world”, 11 };
       yield return new object[] { “failing”, 0 };
  }
}

With the test, you define a PropertyDataAttribute passing in the name of the Property as a string argument.  The test itself then should accept the same parameters as being returned within the object array (in this case, a string and int).  The code to implement this is as below.

[Theory]
[PropertyData(“MyTestData”)]
public static void Theory_Property(string value, int expectedLength)
{
    Console.WriteLine(“String:” + value + ” With expected length of ” + expectedLength);
    Assert.Equal(expectedLength, value.Length);
}

When it hits the failing test, TD.net outputs the following.  I think this might be a bug as the output isn’t as clean as Inline. ReSharper still threats this as a single test.

String:hello With expected length of 5
String:hello world With expected length of 11
String:failing With expected length of 0
TestCase ‘M:IntroToXUnitExtensions.Ext.Theory_Property(System.String,System.Int32)’
failed: Exception has been thrown by the target of an invocation.
    System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> Xunit.EqualException: Assert.Equal() Failure
    Expected: 0
    Actual:   7
    E:UsersBen HallDocumentsVisual Studio 2008ProjectsIntroToXUnitIntroToXUnitExtensionsClass1.cs(39,0): at IntroToXUnitExtensions.Ext.Theory_Property(String value, Int32 expectedLength)
       — End of inner exception stack trace —
…….

Some ‘limitations’, not sure if limitations is the right word – more rules of usage.  The property must be within the same class as the test being executed and the property must return IEnumerable>.  However the advantage of this is what we can reuse the property in multiple different tests instead of redefining the test cases for each test like with Inline.

The next set of attributes pull data from an external location.  The first is ExcelData, this allows us to use a Excel spreadsheet as the source of the data for the tests. By using an excel spreadsheet, we can easily edit and maintain the test data separate to the actual code but it also removes any dependency on having a database setup to hold out test data.  The test with the attribute would look something like below.  You specify the Excel file to use and then a query which will be used to pull the data, the TestData name is actually a Named Range in Excel so the test knows what data to use.

[Theory, ExcelData(@”ExcelTestData.xls”, “select * from TestData”)]
public void Theory_Excel(string value, int expectedLength)
{
    Console.WriteLine(“String:” + value + ” With expected length of ” + expectedLength);
    Assert.Equal(expectedLength, value.Length);
}

However, the first time I executed this test I got an exception which was something like:

“The Microsoft Jet database engine could not find the object…”

The reason for this was because I hadn’t included the ExcelTestData.xls file into the project and set Copy to Output to Copy always.

The file ExcelTestData contains the following, note you must include a header (otherwise your first test case is ignored):

To set the data as a named range, right the block of data (including header), right click and select Name a range

If you want to edit or delete an existing name, then you need to use the Name Manager under the Formulas tab

Important note – it must be a Excel 2003 format – not the 2007 format.

Next is the SqlServerDataAttribute. This allows us to use data within a Sql Server Database and use a SELECT statement to return the test data we want to use within our test code.  This is great when combined with a tool such as Red Gate’s SQL Data Generator.

The SqlServerDataAttribute accepts either Server, Database, SelectStatement or Server,Database,Username,Password,SelectStatement.

[Theory]
[SqlServerData(“(local)”, “IntroToXUnit”, “SELECT teststring, expectedLength FROM TestDataTable”)]
public void Theory_SqlServer(string value, int expectedLength)
{
    Console.WriteLine(“String:” + value + ” With expected length of ” + expectedLength);
    Assert.Equal(expectedLength, value.Length);
}

One important note, the amount of columns returned from the select statement must match the parameters of the test.

Finally, OleDBDataAttribute works in a similar fashion to SqlServerDataAttribute however allows you to connect to other data sources apart from SqlServer.  The attribute is:

[OleDbDataAttribute(string connectionString, string SelectStatement);

AutoRollback

Similar to MbUnit’s Rollback feature, the extensions library has an AutoRollback attribute. By adding this attribute to your test, the entire test will be wrapped within a transaction so once the test has finished (passed or failed) the transaction will be rolled back meaning any changes will be undone.  For example, if we delete the entire customers table, once the test has finished the changes will be rolled back – leaving the database in the same state as before the test.  This is very important for test isolation as we want each test to be independent of any other tests, when it comes to interacting with the database we especially don’t want any tests making permanent changes to the database which is used by other tests.

[Fact]
[AutoRollback]
public void AutoRollback()
{
    string connectionString = “Data Source=BIGBLUE;Initial Catalog=Northwind;Integrated Security=True”;

    using (SqlConnection connection = new SqlConnection(connectionString))
    {
        connection.Open();
        //SqlCommand command = new SqlCommand(“DELETE FROM Customers”, connection);
        //command.ExecuteNonQuery();

        SqlCommand command2 = new SqlCommand(“SELECT CustomerID FROM Customers”, connection);
        SqlDataReader reader = command2.ExecuteReader();
        try
        {
            while (reader.Read())
            {
                Console.WriteLine(String.Format(“{0}”, reader[0]));
            }
        }
        finally
        {
            // Always call Close when done reading.
            reader.Close();
        }
    }
}

The Rollback attribute also includes additional properties, IsolationLevel, TransactionScopeOption and a Timeout. 

FreezeClock

Unit testing anything to do with the system clock is always difficult!  The problem is that you don’t know what time your tests will be executed, and you definitely don’t want to limit them to only running at one extract moment.  In my article Beginning to Mock with Rhino Mocks and MbUnit – Part 1: ASP Alliance I spoke about mocking the date time to get around this.

When I first heard about this I was intrigued how they managed to freeze the clock, turns out it wasn’t so special – but still useful.  What the extensions framework provides is a Clock class which is a wrapper around the .Net DateTime class.  It properties three properties to access the current DateTime – Now, Today, UtcNow.

In your production code, you use this Clock class instead of DateTime to offer all of your date functionality. When dealing with dates, you would do something like below (of course, instead of outputting to the console you would do something real like query the database).

public class ProductionCode
{
    public void ProcessSomethingInvolvingTime()
    {
        Thread.Sleep(1500);
        Console.WriteLine(Clock.Now);
    }
}

To test this we would then add the FreezeClock attribute, which automatically freezes the Clock object for us. We then use the same Clock.Now method to access the now frozen time.

[Fact, FreezeClock]
public void FrozenProductionCode()
{
    DateTime expected = Clock.Now;
    Console.WriteLine(expected);

    ProductionCode pc = new ProductionCode();
    pc.ProcessSomethingInvolvingTime();
}

Outputting both to the console, we verify that the two dates are actually the same – impossible with the standard DateTime.

07/01/2008 00:48:26
07/01/2008 00:48:26

1 passed, 0 failed, 0 skipped, took 3.56 seconds.

However, I really really don’t like the fact that I would have to use the Clock class and reference xunitext within my production code.  Just gives me a nasty feeling and I can’t imagine this is how Brad and James wanted it to be used.  Instead, I would prefer to use a type of Dependency Injection (well Date Injection), passing in a DateTime object into the method under test.

public void ProcessSomethingInvolvingTimeDI(DateTime now)
{
    Thread.Sleep(1500);
    Console.WriteLine(now);
}

Our test now looks like this.  Instead of letting the method obtain the time itself we give it the DateTime object (which we got from Clock.Now) to use.

[Fact, FreezeClock]
public void FrozenProductionCodeDI()
{
    DateTime expected = Clock.Now;
    Console.WriteLine(expected);

    ProductionCode pc = new ProductionCode();
    pc.ProcessSomethingInvolvingTimeDI(expected);
}

When we run the test, the date time is still the same and still frozen. However, if we are doing this why not just use the DateTime object itself?

[Fact]
public void DateTimeProductionCodeDI()
{
    DateTime expected = DateTime.Now;
    Console.WriteLine(expected);

    ProductionCode pc = new ProductionCode();
    pc.ProcessSomethingInvolvingTimeDI(expected);
}

I think its something useful, but not sure if I like it.

AssumeIdentity

Assume Identify allows us to change the Current Principal (the user currently executing the assembly).  This is used for various things such as Role management and within CAS (Code Access Security).

[Fact]
public void AssumeIdentityWithoutAttribute()
{
    Console.WriteLine(“Username of Thread:” + Thread.CurrentPrincipal.Identity.Name);
}

What is outputted is nothing.

“Username of Thread: “

If we use the AssumeIdentityAttribute, we can pass in a RoleName for which the user executing the test will be a remember of.

[Fact, AssumeIdentity(“RoleName”)]
public void AssumeIdentityWithAttribute()
{
    Console.WriteLine(“Username of Thread:” + Thread.CurrentPrincipal.Identity.Name);
    Assert.True(Thread.CurrentPrincipal.IsInRole(“RoleName”));
}

If we run this test, what is outputted is the name of XUnit and IsInRole returns true.

“Username of Thread:xUnit”

If a method we was testing involved CAS as below, the user exectuing the test would have to be a remember of that role – not always the case for build servers.

[PrincipalPermission(SecurityAction.Demand, Role = “Administrators”)]
public void AdminOnly()
{
    Console.WriteLine(“Executing….”);
}

With AssumeIdentity, the test can correctly execute the method.

[Fact, AssumeIdentity(“Administrators”)]
public void AssumeIdentityCAS()
{
    ProductionCode p =new ProductionCode();
    p.AdminOnly();
}

Trace

Adding Trace attribute to your tests causes the test name to be wrote out to the Trace listener before and after the test is being executed. Useful for debugging if your tests have a habit of stopped mid-execution.

RunWithNUnit

Finally, the RunWithNUnit attribute. This method actually lives in the xunitext.nunit.dll assembly.  To check that I wasn’t going made, I created a separate project referencing NUnit 2.4, XUnit and XUnitExt.NUnit.  I then use the XUnit console to make sure I’m executing under XUnit.

The code should look something like this:

[XunitExt.NUnit.RunWithNUnit]
[NUnit.Framework.TestFixture]
public class OldNUnitTests
{
     [NUnit.Framework.Test]
     public void RunWithNUnit()
     {
          NUnit.Framework.Assert.AreEqual(“Running with NUnit”, “Running with NUnit”);
     }
}

However, I couldn’t get this to work correctly as I kept receiving an “error: Exception has been thrown by the target of an invocation. in TargetInvocationException” exception.  I posted on the forum, but at the time of posting no reply.

RunWithNUnit throws exception

In summary, I hope you have found this post useful when getting started using XUnit.net and their extensions. Any feedback/comments then please leave a comment on the post.