I've been wanting to write about this for a while however haven't really had the chance.  While prepping for a NxtGenUG session I came across the TestSuite attribute.  By using this, you can build up your tests to execute dynamically and provide the parameters when the tests are loaded into the runner and framework.  This allows for some great flexibility and extendibility, especially if you don't know all of the possible test values at design-time and want values to be easily added (adding a row to the database, or dropping another file into a directory).

To take advantage of this, you need to use the TestSuiteFixtureAttribute at the top of your class.

[TestSuiteFixture]
public class TestSuiteTests

You can then create the test methods you want to be executed in order to test your system.  Two important points, they must take a parameter of type Object and return an Object - the rest is up to you. In the first test, I just write out the ToString value to the console,  in the second I check to see if the value is between 0 and 5.

public Object MethodUnderTest(object testData)
{
    Console.WriteLine("MethodUnderTest Executed");
    StringAssert.Contains(testData.ToString(), "Parameter");
    return null;
}

public Object AnotherMethodUnderTest(object testData)
{
    Console.WriteLine("AnotherTest");
    string testDataString = testData.ToString();

    int testExecution = Convert.ToInt32(testDataString[testDataString.Length -1].ToString());
    Assert.Between(testExecution, 0, 5);

    return null;
}

The final stage is to create the method which will programmatically create all of your tests and test parameters.

The method below is marked with the TestSuiteAttribute so the framework knows this can create tests.  Inside the method, we create a new TestSuite object with a name to identify it, we can then add new items into the collection which are then returned to the framework and executed.  The parameters to add are:

1. Name of test
2. Method to call (the test itself) as a TestDelegate object
3. Any parameters for the test

[TestSuite]
public ITestSuite Suite()
{
    TestSuite suite = new TestSuite("NewSimpleSuite");

    for (int i = 0; i < 10; i++)
    {
        suite.Add("MethodUnderTest" + i, new TestDelegate(this.MethodUnderTest), "Parameter" + i);
        suite.Add("AnotherMethodUnderTest" + i, new TestDelegate(this.AnotherMethodUnderTest), "Parameter" + i);
    }

    return suite;
}

The above code creates 20 tests, 10 for each test method. In the UI, the tests are all loaded correctly, as you can see each one is named correctly based on our code.

Very powerful attribute! But use the power wisely...

Download code sample here - http://blog.benhall.me.uk/Code/MbUnit/TestSuiteTests.cs.txt

Labels: MbUnit, TDD, Testing

Just an early heads up, I will be doing a Test Driven Development talk at NxtGenUG Coventry on Monday May 19th.

My plan for the talk is to start with nothing, and build up a application using a test driven development approach.  Unlikely it will be a feature complete SubText level application, but my aim is to demonstrate how to get started with TDD with some real application features, how to take advantage of mocking and of course some of the excellent MbUnit features...

Talk Abstract

Red, Green, Refactor!

Starting to unit test your first project is difficult, where to start? What to test? How do you even get started? In this session, Ben starts from scratch and implements an ASP.net 2.0 application using test driven development techniques. The application will have to deal with real world situations such as databases, web services and even some users! Ben will demonstrate how to design the application for testability and how unit testing and mock frameworks can make your life easier and your tests less fragile. At the end, will the tests go green?

Register:  http://www.nxtgenug.net/ViewEvent.aspx?EventID=112

Labels: MbUnit, NxtGenUG, TDD

Continuing in my writing about MbUnit, this post will focus on the TypeFixtureAttribute. The TypeFixture allows a set of tests to focus on testing a certain type of class (generally an interface) and to provide objects of that type to be tested against.

This method is really useful if you have a collection of objects which all have similar behaviour which requires testing. Instead of having to duplicate your test code, you can define the types to be tested with instances of the tests being passed into the test being executed.

In my fake system being tested, I have a collection of objects which all implement a INotNull interface. I need to test to make sure that the implementation of INotNull works correctly, that not nulls are returned when GetNotNullValues() is called. My 'Production' code is this:

public interface INotNull { IEnumerable GetNotNullValues(); }

public class TypeA : INotNull { public override string ToString() { return "TypeA"; }

public IEnumerable GetNotNullValues() { yield return "TypeA"; } }

public class TypeB : INotNull { public override string ToString() { return "TypeB"; }

public IEnumerable GetNotNullValues() { yield return 1; yield return 2; yield return 3; yield return 4; } }

public class TypeC : INotNull { public override string ToString() { return "TypeC"; }

public IEnumerable GetNotNullValues() { yield return new object(); yield return null; yield return new object(); } }

The code looks fine, apart from TypeC which returns a null. In a standard unit testing framework, I would have to create multiple test fixtures for each type in the system and duplicate all of the tests. TypeFixture makes our life easier by allowing us to define a single fixture and test all the objects we want. Let's have a look at the test code.

[TypeFixture(typeof(INotNull), "Tests the INotNull interface.")] #1 public class INotNullTests

Instead of defining a TestFixture at the top of our class, we define a TypeFixture and give it the typeof object we will be testing. We then need to tell it all the objects to use.

[Provider(typeof(TypeA))] #1 public TypeA ProvideTypeA() { return new TypeA(); #2 }

#1 We use the ProviderAttribute and provide the type of object we are returning as a parameter.

#2 We then create a new instance of the object and return it to the caller.

We can use the Provider attribute on as many methods as required to cover all of our objects being tested. The other two objects being tested are TypeB and TypeC.

[Provider(typeof(TypeB))] public TypeB ProvideTypeB() { return new TypeB(); }

[Provider(typeof(TypeC))] public TypeC ProvideTypeC() { return new TypeC(); }

Now we have that in place, we need to write out tests. The test is a standard test, however accepts the type (as defined in the TypeFixture) as a parameter.

[Test] public void TestINotNullValuesNotNull(INotNull instance) { Console.WriteLine(instance.ToString());

foreach (object o in instance.GetNotNullValues()) { Assert.IsNotNull(o, "Instance: {0}. Object was null: {1}", instance, o); } }

We can then use that object within our test to verify it works correctly. This test method will be called once per object defined by a provider attribute. In this case, three tests are executed with TypeC test failing.

Adding more tests to the fixture is very quick and easy and with the TypeFixture will result in it being used to test multiple objects. Adding another implementation of the interface is easy as we define another method allowing it to be picked up by all the existing tests.

Labels: MbUnit, TDD, Testing

In this post, I want to talk about a little know feature of MbUnit which is the CombinationalTestAttribute.  This was originally described on Peli's blog, but I thought I would revisit the subject.

The combinational test attribute allows us to do pair-wise testing.  This is where we take two inputs (A and B) and test them against each other ({A1, B1}, {A1, B2}, {A1, B3}).

When implementing this system, the OUT (Object Under Test) needs to implement an interface.  In this case a ICount

public interface ICount
{
    int Count(string s);
}

We then implement a X counter which will the number of X's which appear in a string.

public class XCounter : ICount
{
    public int Count(string s)
    {
        int count = 0;
        foreach (char c in s)
            if (c == 'x')
                count++;
        return count;
    }
}

For the purposes of demo'ing, a BadCounter is also implemented.

public class BadCounter : ICount
{
    public int Count(string s)
    {
        return 2;
    }
}

So that would be our production code which we want to test.  Using just standard TestFixture and Test attributes, our tests would have to look something like this:

[TestFixture]
public class ICountTests_Orginal
{
    [Test]
    public void BadCounter_Count_NoX()
    {
        ICount counter = new BadCounter();
        int result = counter.Count("");
        Assert.AreEqual(0, result);
    }

    [Test]
    public void BadCounter_Count_TwoX()
    {
        ICount counter = new BadCounter();
        int result = counter.Count("XX");
        Assert.AreEqual(2, result);
    }

    [Test]
    public void BadCounter_Count_XSpaceYSpaceZ()
    {
        ICount counter = new BadCounter();
        int result = counter.Count("X Y Z");
        Assert.AreEqual(1, result);
    }

    [Test]
    public void XCounter_Count_NoX()
    {
        ICount counter = new XCounter();
        int result = counter.Count("");
        Assert.AreEqual(0, result);
    }

    [Test]
    public void XCounter_Count_TwoX()
    {
        ICount counter = new XCounter();
        int result = counter.Count("XX");
        Assert.AreEqual(2, result);
    }

    [Test]
    public void XCounter_Count_XSpaceYSpaceZ()
    {
        ICount counter = new XCounter();
        int result = counter.Count("X Y Z");
        Assert.AreEqual(1, result);
    }
}

This is only testing the two objects (imagine 100s) with three test cases (imagine 1000s).  This would very quickly become unmanageable and a maintenance nightmare. We can make this much more effective using Combinational.

First part of configuring this is to setup a Factory for the interface.  This will produce all the objects we want to be tested.

[Factory(typeof(ICount))]
public IEnumerable Counters()
{
    yield return new BadCounter();
    yield return new XCounter();
}

Next, we create a factory of test cases.

[Factory(typeof(CombinationTestItem))]
public IEnumerable Strings()
{
    yield return new CombinationTestItem("", 0);
    yield return new CombinationTestItem("aaa", 0);
    yield return new CombinationTestItem("x", 1);
    yield return new CombinationTestItem("xa", 1);
    yield return new CombinationTestItem("xax", 2);
    yield return new CombinationTestItem("X x", 2);
    yield return new CombinationTestItem("X X", 2);
}

I've created a helper class within my code called CombinationTestItem.  This is used just to store the input and expected output so I can access them within my test. This could store as much or as little as you want, only restriction is that the Factory must return IEnumerable.

public class CombinationTestItem
{
    public CombinationTestItem(string value, int xcount) { Input = value; ExpectedOutput = xcount; }
    public string Input;
    public int ExpectedOutput;
}

The last part is to actually create the test.

[CombinatorialTest]                       #1
public void CountIsExact([UsingFactories("Counters")] ICount counter,       #2
                         [UsingFactories("Strings")] CombinationTestItem test)            #3
{
    Assert.AreEqual(test.ExpectedOutput, counter.Count(test.Input));            #4
}

#1 Instead of using [Test] we use [CombinatorialTest]

#2 The first parameter of the test is the Factory of Counters we created.  We use the [UsingFactories] attribute to help the framework understand which factory to pull the data from

#3 Like with #2, we do the same for the test data factory.

#4  Finally, we call the count method on the counter object passed into the test with the value we set on our CombinationTestItem object.  We then verify that it matches the related ExpectedOutput.

When it comes to executing the test cases the framework creates a test for each ICount with each CombinationTestItem test case. The result is 14 test cases (Two ICount objects * 7 CombinationTestItem test cases).

Adding additional objects or test cases is simply a case of adding an additional line of code.

My own point about this attribute is that I have never found a real world situation where I have required this.  In theory (like above) it sounds great, but I'm but sure if it would actually solve any of my integration testing problems - I can't imagine it would solve any unit testing problems. If this does solve a problem for you, it would be great to hear about it.

Labels: MbUnit, TDD, Testing

Yesterday I received feedback from my DDD6 talk on Testing applications with MbUnit.  Attendees are asked to leave feedback on the sessions they attended and rate the speaker in four categories; overall, knowledge, presentation and content.  My averages for these where:

Some of the comments where:

"Was expecting more Gallio stuff, even just more of an introduction to the concepts.. As it was I already knew a lot of"

Fully agree that this was a problem, it's a shame more of Gallio wasn't ready at that point. You can read more about it here and here.

"This guys did a great job. Well-prepared, well-executed and full of useful information."

"Voice projection was an issue at the start, I think this got better as it went on. Some of the content was rushed through, this was probably due to a large subject being tackled in depth in only an hour."

"The speaker clearly knew his stuff. As a result I will be taking a look at MbUnit. A first time presenter, and I'm sure there were a few nerves - did well. "

On a personal note, I think it went OK - but there was/is definitely room for improvement. Thanks for all the feedback, I will take it into account when doing future talks.

Labels: DDD6, MbUnit

Personally, I use TestDriven.NET to launch the MbUnit GUI to execute my tests.  However, not everyone has that installed.  One alterative approach is to use the External Tools option within Visual Studio.

To create this setting, select the Tools option and select External Tools.

You can then Add a new tool and point it at either the GUI or Cons application.

The most important settings are the variables to use. First, you need to set the Initial Directory to be the $(BinDir),  this will be where your built assemblies end up and generally is \debug\bin. Next, you need to set what arguments to pass into the MbUnit GUI. $(TargetName)$(TargetExt) will provide you with the assembly name with its extension on the end.  This allows MbUnit to find it in the bin directory and load it successfully.

Just make sure you have your test project selected before selecting the MbUnit option from the menu.

You should find this also works with NUnit and XUnit runners.

Labels: MbUnit

One of the interesting features of MbUnit 2.4 is the Rollback attribute.  I've spoke about this in passing in some of my previous posts, but I thought I would drive a little deep into what is going on.

The Rollback attribute can be added to any test method, when the test has finished (either passed or failed) any changes made to the database is rolled back - even if it has transactions inside.  This means that your tests are more isolated

Rollback uses Enterprise services and COM+ and is based on the .Net 1.1 implementation. Rollback2 uses TransactionScope from System.Transactions which was included in .Net 2.0.  If your project is using .Net 2.0, Rollback2 would be the preferred attribute to use.   An example of how this could be used is:

[Test]
[RollBack]
public void GetCategoryByName_NameOfValidCategory_ReturnCategoryObject()
{
    InsertCategory();

    Category c = ProductController.GetCategory("Microsoft Software");

    Assert.AreEqual("Microsoft Software", c.Title);
    Assert.AreEqual("All the latest Microsoft releases.", c.Description);
}

One question I was asked at DDD was if Rollback supports other databases or if its just SQL Server. The answer is, I still haven't been able to find a definitive answer. In theory it should work, but I have read some forum comments about errors when using TransactionScope. If I can get everything setup, I'll write another post with my findings.

While looking around about this feature I came across a RestoreDatabase attribute.  Had to have a play!

Turns out, it allows a backup to be restored before the test is executed (if you didn't get it from the name).  It gets all the information based on the SqlRestoreInfo attribute at the test fixture level and uses that to store the database before the test is executed.

[TestFixture]
[SqlRestoreInfo("Data Source=BIGBLUE;Initial Catalog=Master;Integrated Security=True", "Northwind", "E:\\Northwind.bak")]
public class Class1
{
    string connectionString = "Data Source=BIGBLUE;Initial Catalog=Northwind;Integrated Security=True";

    [Test]
    [RestoreDatabaseFirst]
    public void GetOrders()
    {
        //string queryString = "DELETE FROM dbo.[Order Details];";  If we execute this in the first run the table is empty.
        string queryString = "SELECT * FROM dbo.[Order Details];";  If we execute this in the second run, the data is restored.
        string queryString2 = "SELECT * FROM dbo.[Order Details];";
        using (SqlConnection connection = new SqlConnection(connectionString))
        {
            SqlCommand command = new SqlCommand(queryString, connection);
            SqlCommand command2 = new SqlCommand(queryString2, connection);
            connection.Open();
            command.ExecuteNonQuery();
            SqlDataReader reader = command.ExecuteReader();
            try
            {
                while (reader.Read())
                {
                    Console.WriteLine(String.Format("{0}, {1}",
                        reader[0], reader[1]));
                }
            }
            finally
            {
                // Always call Close when done reading.
                reader.Close();
            }
        }
    }

It's a neat idea and good for integration testing, but I wouldn't recommend it for unit testing as the additional time to restore the database would soon mount up, expect improvements to testing with databases in MbUnit v3.

Labels: MbUnit

Last weekend I decided it would be great fun to do a simple screencast about getting started with MbUnit.  It's just a very quick, create project, add reference, execute screencast but hopefully it will be useful if this is your first time to a unit testing framework and you want to get started. It also gave me chance to play around with Camtasia.

Well, it wasn't as much fun as I first thought. I had a number of attempts to find out how to actually create a screencast and I think I fall into most pitfalls along the way (but its all experience right?), that can be another post.

The outcome was that I managed to created a screencast on Getting Started with MbUnit.

Download: Screencast_GettingStartedWithMbUnit.wmv

Length: 8 Mins 56 seconds

Size: 8.71MB

Now I have a better understanding of what to do, I might create a few more! Next ones will be better...

Labels: MbUnit, TDD, Testing

One of the cool items I didn't mentioned in my announcement post is that we now have the ability to integrate screenshots from a WatiN test into the reports.

Here we have a simple test which searches for MbUnit on Google and then asserts to check that NUnit is mentioned on the page (this fails).

[Test]
public void DemoCaptureOnFailure()
{
    using (Log.BeginSection("Go to Google, enter MbUnit as a search term and click I'm Feeling Lucky"))
    {
        ie.GoTo("http://www.google.com");

        ie.TextField(Find.ByName("q")).TypeText("MbUnit");
        ie.Button(Find.ByName("btnI")).Click();
    }

    // Of course this is ridiculous, we'll be on the MbUnit homepage...
    Assert.IsTrue(ie.ContainsText("NUnit"), "Expected to find NUnit on the page.");
}

Normally, it would be hard to find out what was actually on the webpage at that point to see why the test decided to fail. In the tear down method, there is a check if the test passed or failed, if it failed then it takes a snapshot and includes it in the report.

[TearDown]
public void DisposeBrowser()
{
    if (Context.CurrentContext.Outcome == TestOutcome.Failed)
        Snapshot("Final screen when failure occurred.", LogStreamNames.Failures);

    if (ie != null)
        ie.Dispose();
}

When you view the report, on the test that failed you can see that the screenshot of the webpage has been included.  This will make debugging the error a lot easier, making the tests more useful.

I think this is a great feature, well done to Jeff and the team for implementing this.

Labels: MbUnit, WatiN

Today, I am going to be looking at WatiN, which stands for Web Application Testing in .Net, and seeing how well it actually works with an ASP.net web application.

WatiN is a type of record and playback framework to allow for automating the UI testing for web applications.  It uses IE and allows you to interact with the page, for example clicking buttons and typing in text.

A simple example from their documentation shows how to search for WatiN on google and assert you got the correct results back.

[Test]
public void SearchForWatiNOnGoogle()
{
using (IE ie = new IE("http://www.google.com"))
{
  ie.TextField(Find.ByName("q")).TypeText("WatiN");
  ie.Button(Find.ByName("btnG")).Click();
  Assert.IsTrue(ie.ContainsText("WatiN"));
}
}

You can use any test framework to get started, with MbUnit you simply write the tests as normal, but in your test fixture attribute you need to set the ApartmentState to STA. ApartmentState is part of .Net which states how the thread should execute. STA means that the thread will create and enter a single threaded apartment, everything will just be kept on the single thread. 

[TestFixture(ApartmentState = ApartmentState.STA)]

If you don't have this set, when you run your tests you will receive the following exception:

Message: The CurrentThread needs to have it's ApartmentState set to ApartmentState.STA to be able to automate Internet Explorer.

Type: System.Threading.ThreadStateException
Source: WatiN.Core
TargetSite: Void CheckThreadApartmentStateIsSTA()

Now we have running tests, we can start to look at testing a 'real' application. Below is the screenshot of the application under test (AUT).

Entering information would cause the page to look like this:

Very simple.  To test this worked correctly, we would want to simulate what the user is doing and to verify it worked the same way a person would, in this case that it displayed the search term typed into the text box.

First we need to find the text box (called searchText) by it's name and enter a string.  We then find the button (by its value - the text displayed to the user and not the ID, the button ID is in fact Button1) and click it.  We then verify the response.

[Test]
public void WatiNSearchText_SearchWatiN_TextDisplayedOnScreen()
{
    IE ie = new IE("http://localhost:49992/WatiNSite/");

    // Find the search text field and type Watin in it.
    ie.TextField(Find.ByName("searchText")).TypeText("WatiN");

    // Click the search button.
    ie.Button(Find.ByValue("Search")).Click();

    //Verify it contains search term.
    bool result = ie.Text.Contains("WatiN");
    Assert.IsTrue(result);
}

The URL points to the ASP.net development server for the solution, with a static port set.

That works with simply html, but what happens when ASP.net is pain with the naming. When using a master page with ASP.net, you place all of the pages content within a ContentPlaceHolder. When ASP.net renders this, to ensure naming of elements does not clash it prefixes the IDs.

Without using the placeholder, the textbox html is:

<input name="searchText" type="text" id="searchText" />

However, after using master pages it becomes:

<input name="ctl00$ContentPlaceHolder1$searchText" type="text" id="ctl00_ContentPlaceHolder1_searchText" />

Not great! All the elements are referred as strings, by moving to master pages all of our tests would break. WatiN will report this as:

Message: Could not find a 'INPUT (text password textarea hidden) or TEXTAREA' tag containing attribute name with value 'searchText'

One solution has been provided by James Avery in his WatiN Test Pattern post.  Here, he basically says that each page in the website should have a adapter in the test code which we code against which in turn calls the page (this is not mocking the page).  It means that changes to the website, such as naming, requires only changing the adapter class to match.  Following his pattern, the test would be:

[Test]
public void WatiNSearchText_SearchWatiN_TextDisplayedOnScreen2()
{
    Default page = new Default("http://localhost:49992/WatiNSite/");

    page.SearchText.TypeText("WatiN");

    page.SearchButton.Click();

    bool result = page.Text.Contains("WatiN");
    Assert.IsTrue(result);
}

Its a lot easier to read as everything is referred to as the page and you can easily imagine what is happening.  The adapter looking like this:

public class Default : IE
{
    public TextField SearchText
    {
        get { return TextField(Find.ByName("searchText")); }
    }

    public Button SearchButton
    {
        get { return Button(Find.ByValue("Search")); }
    }

    public Default(string url) : base(url)
    {

    }
}

We could even take this a step on and abstract the searching into its own method which we call in the test.

The test would then be:

[Test]
public void WatiNSearchText_SearchWatiN_TextDisplayedOnScreen3()
{
    Default page = new Default("http://localhost:49992/WatiNSite/");

    page.SearchFor("WatiN");

    bool result = page.Text.Contains("WatiN");
    Assert.IsTrue(result);
}

This does solve some of the problems as it makes the tests less dependent on the application code.  However, the ASP.net naming is still getting in the way. If we add another container or the element is inside a custom control we are still going to have a problem.  The answer is RegEx!

In our adapter, I change the way we find the element to use a method RegExName. Now its Find - ByName - RegExName - ElementName.

public TextField SearchText
{
    get { return TextField(Find.ByName(RegExName("searchText"))); }
}

Our RegExName method just takes in the elementname and return a regex which will ignore all of the ASP.net prefixes.

private Regex RegExName(string elementName)
{
    return new Regex(".*" + elementName + "$");
}

By using an adapter and RegEx we can abstract away from the real implementation and make our tests a lot more beneficial.  The tests are quite easy to write as long as you keep things focused. By arranging your tests as above, you should have a lot more long term success.

But having an automated test run for the UI will never replace a human tester as there are other scenarios to test for, cross browser, usability, wording which a test framework cannot test.

Labels: MbUnit, TDD, Testing, WatiN

I'm pleased to say that MbUnit v3 Alpha 1 has been released.  This is our first release and contains some really cool concepts, however don't rush and migrate all of your unit tests just yet. If you want to know more about what we have planned in the future take a look at our Roadmap.

Within Alpha 1, there are a few core features which I want to pick out.

Gallio Lcarus

As we are building on top of the Gallio Automation Platform, we take advantage of their UI.

Gallio has the ability to execute MbUnit v2.4, MbUnit v3, NUnit and XUnit tests all within the same user interface.  In this screenshot we have all four frameworks executing as a single test run.

Over time the GUI will only improve and we have got some great plans going forward.  I think its already an improvement over the GUI today.

Gallio Echo

Like with the UI, we also have a new command line.  Again, this has the ability to execute tests from the four different frameworks. The command below simply loads in all the test assemblies, and outputs a html report to \Reports.

E:\Source Control\MbUnit\v3\build\bin>Gallio.Echo.exe "E:\Users\Ben Hall\Desktop\Gallio Test Resources\MbUnit\Gallio.TestResources.MbUnit.dll" "E:\Users\Ben Hall\Desktop\Gallio Test Resources\MbUnit2\Gallio.TestResources.MbUnit2.dll" "E:\Users\Ben Hall\Desktop\Gallio Test Resources\NUnit\Gallio.TestResources.NUnit.dll" "E:\Users\Ben Hall\Desktop\Gallio Test Resources\XUnit\Gallio.TestResources.Xunit.dll" /rd:"E:\Users\Ben Hall\Desktop\Gallio Test Resources\Reports" /rt:Html

When executing, failing tests are displayed in red.

At the end, you will be shown a summary to the test run.

Reports

Go go with this, Gallio has a new report design.  I really like these, think they look much better than previously.  The report gives you all the information about the test run, such as assemblies loaded, statistics and summary.

Then it goes into detail on each of the test run, highlighting failures.

This is really useful and provides a great way to keep track of each test run. The new report actually makes it really simply to see what is going on.  Notice, that in the above screenshot, we have the xml documentation for the method inline with the test report.  Very cool! :)

Nice work Jeff and team.  I haven't spoke about what's new within MbUnit v3.  Well its still very early days for the framework itself as Gallio was originally MbUnit v3 until decided to release it as a separate framework.  As such, the framework hasn't had some much attention, however from the release notes we have completed the following work:

- Dogfood.
- Initial draft of the attribute extensibility model.
- Supports [Test] and [TestFixture] attributes.
- Assertions ported from MbUnit v2 in the interim.

More information:

www.mbunit.com

Jeff's post - MbUnit v3 Alpha 1 Release

Andrew's post - MbUnit v3 and Gallio alpha 1

Download: http://mb-unit.googlecode.com/files/MbUnit%20v3%20Alpha%201%20Setup.exe

Release Notes: http://mb-unit.googlecode.com/files/MbUnit%20v3%20Alpha%201%20Release%20Notes.txt

Labels: Gallio, MbUnit

What an amazing day!! Having enjoyed DDD3, DDD4 and DDD5 yesterday I was given the chance to speak at DDD6! I felt honoured that I was given the chance and really enjoyed walking around in my green DDD t-shirt. My session was one of the last in the day but I managed to pull in a room full of attendees (about 50) which was amazing and thank you to everyone who attended. The day goes so much quicker when your speaking!

My session kind of went to plan, words came out my mouth and I finished on time but I did forget some points and think I could have explained things better (I think my demo scenario could have been better). But it was great experience and I will take the improvements onboard for when I do a similar session at NxtGenUG Cambridge. Paul Lockwood who did a CCNet talk also attended my session - Read his thoughts @ http://dotnetworkaholic.com/ddd6-uk-code-camp/

As for the rest of the day, I attended some great sessions. My favourite session of the day (apart from mine) was Why do I need an Inversion of Control Container? by Mike Hadlow, good content and well presented. After the day, I went for one or two well deserved beers and the geek dinner which was great and spoke to some really nice people. Cheers to Zi for organising that.

Demo: TestingApplicationsWithMbUnit.zip

Slides: TestingYourApplicationsWith Mbunit.pptx.zip (UPDATE: Download link now works. Sorry)

Finally, don't forget to fill out your feedback - http://www.developerday.co.uk/ddd/feedback.asp. Can't wait for the next conference!

Labels: DDD, DDD6, MbUnit

"All problems in computer science can be solved by another level of indirection" Butler Lampson (Or Abstraction as some people refer to it as)

Over the last few posts, I have been covering how to unit test various topics, in this post I will cover Linq to SQL. I've been thinking about this for a while, but never had chance to post about it. Various people have spoken about this before, but I thought I would give my view.  I'm going to discuss how to unit test systems involving Linq to SQL, how to mock Linq to SQL, how this will come into play with the new ASP.net MVC Framework and how TypeMock might have the answer.

Scenario

For this post, I decided to implement a small part of a shopping cart.  You can return a list of categories, a list of products for each category and the product itself.

Unit Testing Linq to SQL

In my previous post, I discussed how to unit test Linq to XML and mentioned how it really isn't that different to any other system.  Linq to SQL is similar in the fact that you can easily ignore the underneath implementation (Linq) when unit testing, however I'm going to focus more on how to decide the system and how it all should hang together.

Like in any Linq to SQL implementation you will need a data context.  While you could unit test this implementation, I always prefer to test it as part of a higher layer as its generated code to give the tests more context and more meaningful for the system. The most important part of the system will be how we actually access the DataContext, retrieve the data and make sure that it can be effectively unit tested.

In my system I will have my HowToUnitTestDataContext generated by the Linq to SQL Designer.  I will then have a ProductController which will talk to my DataContext and return the generated object. Unlike with SubSonic, we can return the designer generated objects as they are just POCO (Plain Old C# Objects).  I will then have a set of unit tests targeting the ProductController.  We are not doing any mocking at this point and all of our unit tests will hit the database.

First test, create a data context and ensure it has a connection string.  This ensures everything is setup correctly.

[Test]
public void CreateDataContext_ConnectionString_ReturnsDataContextObject()
{
    HowToUnitTestDataContext db = ProductController.CreateDataContext();
    Assert.IsNotNull(db);
    Assert.IsNotNull(db.Connection);
}

All the method does is initialise a data context and return it to the caller. The next test requests a list of all the categories from the database.  However, because we haven't populated anything yet it should just return an empty list.

[Test]
public void GetCategories_NoContent_ReturnEmptyList()
{
    List<Category> categories = ProductController.GetCategories();
    Assert.IsNotNull(categories);
    Assert.AreEqual(0, categories.Count);
}

We now have a base structure in place and can start filling in the details. The following test first inserts a category into the database (code is in the solution which you can download at the bottom of the post).  It then requests all the categories from the database and ensures that what we returned was correct. We then use the MbUnit RollBack feature to ensure the database is left in the same state as it was before the test. The rollback works perfectly with Linq to SQL!

[Test]
[RollBack]
public void GetCategories_SimpleContent_ReturnPopulatedList()
{
    InsertCategory();

    List<Category> categories = ProductController.GetCategories();
    Assert.IsNotNull(categories);
    Assert.AreEqual(1, categories.Count);
    Assert.AreEqual("Microsoft Software", categories[0].Title);
}

The code for GetCategories() is a simply Linq to SQL statement which returns a generic list.

public static List<Category> GetCategories()
{
    HowToUnitTestDataContext db = CreateDataContext();
    var query = from c in db.Categories
                       select c;
    return query.ToList();
}

The next important test is the one which returns product information.  Here, we use a tested GetCategory method to return a particular category. We then insert a temp product into the database for that category, meaning that we now have a known database state to work with. The test then simply verifies that when given a category we can return all the products in the database for it.

[Test]
[RollBack]
public void GetProductsForCateogry_ValidCategoryWithProduct_PopulatedList()
{
    InsertCategory();

    Category c = ProductController.GetCategory("Microsoft Software");
    InsertProduct(c);

    List<Product> products = ProductController.GetProductsForCategory(c);
    Assert.AreEqual(1, products.Count);
    Assert.AreEqual("Visual Studio 2008", products[0].Title);
}

The implementation of this method is a little bit more complex as it joins a ProductCategories table to return the products within the particular category.

public static List<Product> GetProductsForCategory(Category c)
{
    HowToUnitTestDataContext db = CreateDataContext();
    var query = from p in db.Products
                       join pc in db.ProductCategories on p.id equals pc.ProductID
                       where pc.CategoryID == c.id
                       select p;

    return query.ToList();
}

The final method is to return a particular product based on its ID.  It works in a similar fashion to the previous methods.

[Test]
[RollBack]
public void GetProductByID_ValidProductID_ReturnProductID()
{
    InsertCategory();

    Category c = ProductController.GetCategory("Microsoft Software");
    InsertProduct(c);

    List<Product> products = ProductController.GetProductsForCategory(c);
    Assert.AreEqual(1, products.Count);

    Product p = ProductController.GetProduct(products[0].id);
    Assert.AreEqual(p.Title, products[0].Title);
}

In the implementation we then just return a single product using a lambda expression.

public static Product GetProduct(int productID)
{
    HowToUnitTestDataContext db = CreateDataContext();

    Product product = db.Products.Single(p => p.id == productID);
    return product;
}

That pretty much covers unit testing basic Linq to SQL. Other parts of the system, such as the business layer or UI layer, can then talk directly to the ProductController to return all the information. However, this doesn't offer anything if you want to mock out Linq to SQL.

Mocking Linq to SQL

Unit testing Linq to SQL isn't very difficult, however mocking Linq to SQL is a different beast. As with SubSonic, the best approach to take is to abstract away from your database. In this case, I am going to add an additional layer in between my ProductController and my DataContext called LinqProductRepository which can then be mocked. 

My first set of tests are focused on testing the LinqProductRepository which talks to my DataContext and as such my database. The tests are very similar to the above tests for ProductController. I always test this against the database to ensure that it will work effectively when its in production/live, with mock objects you can never have the same level of confidence.

LinqProductRepository m_ProductRepository = new LinqProductRepository();
[Test]
[RollBack]
public void GetCategoryByName_NameOfValidCategory_ReturnCategoryObject()
{
    InsertCategory();

    Category c = m_ProductRepository.GetCategory("Microsoft Software");

    Assert.AreEqual("Microsoft Software", c.Title);
    Assert.AreEqual("All the latest Microsoft releases.", c.Description);
}

To give you an idea of the implementation, the GetCategory method looks like this:

public Category GetCategory(string categoryTitle)
{
    using (HowToUnitTestDataContext db = CreateDataContext())
    {
        Category category = db.Categories.Single(c => c.Title == categoryTitle);
        return category;
    }
}

In order to make the ProductRepository mockable it is required to implement an interface.  The interface is very simply, as shown:

public interface IProductRepository
{
    List<Category> GetCategories();
    Category GetCategory(string categoryTitle);
    List<Product> GetProductsForCategory(Category c);
    Product GetProduct(int productID);
}

We now have a fully implemented and tested ProductRepository so we can create the ProductController. To start with, in my ProductControllerTests I setup the variables and the [Setup] method for each test.  This ensures that we have our MockRepository (via RhinoMocks) to hand, a copy of our mocked IProductRepository together with our stub category and product.  These two objects are simple well known objects (for the system) which we will return from our mocked methods.  I'm using parameter injection to set the mocked repository on the ProductController which will be used during the tests. By using a parameter we can have a default implementation for our production code but a way for our test code to injection the mock object.

MockRepository m_Mocks;
IProductRepository m_ProductRepos;
Category m_MockedCategory;
Product m_MockedProduct;

[SetUp]
public void Setup()
{
    m_Mocks = new MockRepository();
    m_ProductRepos = m_Mocks.CreateMock<IProductRepository>();
    ProductController.ProductRepository = m_ProductRepos;

    m_MockedCategory = MockCategory();
    m_MockedProduct = MockProduct();
}

We can then write our unit tests based on this information which will be similar to our previous units tests as they are implementing the same requirements. Within this test, we setup our expected return for the method GetCategories on our ProductRepository, this simply uses C# 3.0 Collection Initialises to create a list with one item, the stub category. We can then execute our test/asserts against the ProductController to view it is all linked correctly and working as expected.

[Test]
public void GetCategories_SimpleContent_ReturnPopulatedList()
{
    using (m_Mocks.Record())
    {
        Expect.Call(m_ProductRepos.GetCategories()).Return(new List<Category> { m_MockedCategory });
    }

    using (m_Mocks.Playback())
    {
        List<Category> categories = ProductController.GetCategories();
        Assert.IsNotNull(categories);
        Assert.AreEqual(1, categories.Count);
        Assert.AreEqual("Microsoft Software", categories[0].Title);
    }
}

The ProductController simply passes the call onto our ProductRepository, which is set using a parameter in our test Setup.

private static IProductRepository m_ProductRep = new LinqProductRepository();

public static IProductRepository ProductRepository
{
    get { return m_ProductRep; }
    set { m_ProductRep = value; }
}

public static List<Category> GetCategories()
{
    return ProductRepository.GetCategories();
}

This allows us to use the ProductController with a mock object. ProductController could be/do anything and by going via the LinqProductRepository we can use mock objects to save us accessing the database.  In our real system, we would use the real tested LinqProductRepository object.

ASP.net MVC and Linq to SQL

Recently there has been a lot of buzz around the new ASP.net MVC framework the ASP.net team are releasing (CTP soon).  If you haven't read about this, Scott Guthrie has done a great post on the subject.

However, Scott Hanselman did a demo of the MVC framework at DevConnections and has posted the source code online here - DevConnections And PNPSummit MVC Demos Source Code. Demo 3 and 4 is about TDD and uses mock objects and Linq to SQL.

The layout is as follows:

The BlogController talks to a IPostRepository object which is either LinqPostRepository or TestPostRepository. They use a Stub object instead of a mock object, but the architecture is the same as my system. I would just like to say I didn't just copy them :)

From this demo code, it looks like the way to mock your data access layer will be to use this approach.

Mocking the DataContext

The thing which first struck me about this was why couldn't we just mock the DataContext itself instead of messing around with Repositories and Controllers. The first answer was that it didn't implement an interface, but that's a simple fix.  The second problem is that the data context returns everything as System.Data.Linq.Table<>, and this cannot be mocked. It does implement an ITable interface, however you cannot cast another object implementing ITable to Table.

Ayende, creator of Rhino Mocks, wrote a post called Awkward Testability for Linq for SQL which covers why it cannot be mocked.  Its a shame the Linq team didn't think about TDD and Mocking the repository as it would have made a big difference to the system design. Maybe this is a lesson for anyone creating an API at the moment - think about testability! I think the ASP.net team have realised this.

Mocking with TypeMock

That all said, it looks like TypeMock might be able to provide an answer.  TypeMock is a mocking framework, however is extremely powerful as it can mock any object within your system, it doesn't care about the implementation or if there is an interface available. It can simply mock anything.  I will be looking at TypeMock more over the next few weeks, but before then visit their site. It's not free (30 day trail available), but if you need to do this type of mocking then it really is your only solution and it can greatly increase your test coverage and your test suite as a whole.

You can read the initial ideas over on their blog - Eli Lopian’s Blog (TypeMock) » Blog Archive » Mocking Linq - Preview.  Going in depth on TypeMock and Linq deserves its own post, however I couldn't resist posting some code now.

Below, we have a test which mocks out querying a simple list.  Here, we have a list of customers and a dummydata collection. Our actual Linq query is against the m_CustomerList, however TypeMock does some 'magic' under the covers and tells the CLR that the query should just return the dummyData. As such, instead of just returning the one customer record, we are returning the two dummy data records. How cool!!!

public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }
    public string City { get; set; }
}

[Test]
[VerifyMocks]
public void MockList()
{
    List<Customer> m_CustomerList = new List<Customer> {
        new Customer{ Id = 1, Name="Dave", City="Sarasota" },
        new Customer{ Id = 2, Name="John", City="Tampa" },
        new Customer{ Id = 3, Name="Abe", City="Miami" }
    };

    var dummyData = new[] {new {Name="fake",City="typemock"},
               new {Name="another",City="typemock"}};
    using (RecordExpectations recorder = RecorderManager.StartRecording())
    {
        // Mock Linq
        var queryResult = from c in m_CustomerList
                          where c.City == "Sarasota"
                          select new { c.Name, c.City };
        // return fake results
        recorder.Return(dummyData);
    }

    var actual =  from c in m_CustomerList
                  where c.City == "Sarasota"
                  select new { c.Name, c.City };
    Assert.AreEqual(2, actual.Count());
}

But it gets better!  Given our original ProductController (no repositories, mocks, fakes, stubs) we can tell TypeMock that for this statement, always return the dummyData. As such, the ProductController never hits the database.

[Test]
[VerifyMocks]
public void MockDataContext()
{
    Category category = new Category();
    category.id = 1;
    category.Title = "Test";
    category.Description = "Testing";

    List<Category> dummyData = new List<Category> { category };
    using (RecordExpectations r = RecorderManager.StartRecording())
    {
        // Mock Linq
        List<Category> mocked = ProductController.GetCategories();
        // return fake results
        r.Return(dummyData);
    }

    List<Category> categories = ProductController.GetCategories();
    Assert.AreEqual(1, categories.Count);
    Assert.AreEqual("Test", categories[0].Title);
    Assert.AreEqual("Testing", categories[0].Description);
}

Feel free to download the solution and debug the methods to see it for yourself, I think its cool.

Summary

In summary, I hope you have found this post useful and interesting. I've covered how to unit test your Linq to SQL code and how you go about mocking Linq to SQL. I then finished by giving you a quick look at what TypeMock can offer. It is possible to mock Linq to SQL, however it's not as simple as it could be. If you have any questions, please feel free to contact me.

Download Solutions - TestingLinqToSQL.zip | TestingLinqToSQLMock.zip | TestingLinqToSQLTypeMock

Labels: ASP.net MVC, Linq, MbUnit, TDD, Testing

Now that I'm back from TechEd, it's time to continue my series of "How To Unit Test" posts.  In this post, I've decided to have a quick look into Linq to XML and how that could be unit tested.  I've decided to pick the nice standard sample for Linq to XML - A Rss Reader :).  This also follows on nicely from my previous post on How To Unit Test Using SubSonic and Rhino Mocks. 

Unit testing Linq to Xml is no great big secret. The fact that you are performing Linq queries under the covers doesn't really affect your unit tests, but it's still quite interesting.

To start with, I'm going to be using the BBC News feed as my sample xml feed to query. My first test is to return an RssChannel object which had a url and title.  Note, I'm not implementing all the properties as that would get very boring very quickly.

[Test]
public void GetRssChannel_StandardRss_ReturnDataRssChannel()
{
    RssProcessor rss = new RssProcessor();
    RssChannel chan = rss.GetRssChannel(TestHelper.url);
    Assert.AreEqual(TestHelper.url, chan.Url);
    Assert.Contains(chan.Title, "BBC");
}

The implement on the method then simple queries a XDocument, creates my RssChannel object and returns it to the caller.

public RssChannel GetRssChannel(string url)
{
    XDocument x_Feed = XDocument.Load(url);
    var feeds = from f in m_Feed.Elements("rss").Elements("channel").Take(1)
                       select new RssChannel
                       {
                           Title = f.Element("title").Value,
                           Url = m_Url
                       };

    return feeds.First();
}

The next method does a similar function but returns a RSSItem for the position in the document (Linq to XML is great for doing queries like this).

[Test]
public void GetRssItem_RssFeed_ReturnsASingleRSSItem()
{
    RssProcessor rss = new RssProcessor();
    RssItem item = rss.GetRssItem(TestHelper.url, 1);
    Assert.IsNotNull(item);
    Assert.IsNotNull(item.Title);
}

The query would be:

var feeds = from f in m_Feed.Elements("rss").Elements("channel").Elements("item").Skip(itemPosition - 1).Take(1)
            select new RssItem
            {
                Title = f.Element("title").Value
            };

Instantly we can see from the tests that we can do a bit of refactoring.  Firstly, we can move the creation of the RssProcessor into a [Setup] method.  We can also give the Url for the feed as a parameter in the constructor.  This makes our test code cleaner as there is less duplication, but also more readable as we are focusing more on the intent of the tests.

The complete solution can be downloaded at the end of the post, however there is one more test which I want to refer to.

The following returns an RssFeed object which has a reference to the RssChannel and a List<RssItem>.

[Test]
public void GetRssFeed_EntireFeed_ReturnsChannelAndItems()
{
    RssFeed feed = m_Rss.GetFeed();
    Assert.IsNotNull(feed.Channel);
    Assert.Contains(feed.Channel.Title, "BBC");
    Assert.GreaterThan(feed.Items.Count,0);
}

The test works fine, however based on the implementation on the existing tests, each method loads their own Rss feed/XDocument. This is great for isolation, but doesn't work well when you put it all together as loading a single feed will cause it to actually be loaded three times - that simply doesn't scale.

The solution was to have a LoadFeed method on the RssProcessor which would be called before GetFeed().  This could also allow for a single object to be used for returning multiple feeds.  Note for future: My first test should really have been for this method to make sure I can actually receive an object to work with before querying it.

m_Rss.LoadFeed(TestHelper.url);

Having this single method as the way to load and return the Xml makes mocking very easy. Following the approaches in previous posts, I created myself an IFeedGetter object, with a property to injection a mock if required on my RssProcessor.  The load method then simply passes it's call onto the IFeedGetter object.

public XDocument LoadFeed(string url)
{
    m_Url = url;
    m_Feed = m_FeedGetter.GetXDocument(url);
    return m_Feed;
}

I can then create a stub feed getter to be used during unit testing.

public class StubFeedGetter : IFeedGetter
{
    public System.Xml.Linq.XDocument GetXDocument(string url)
    {
        return XDocument.Load(System.Xml.XmlReader.Create(new System.IO.StringReader(GetFakeXml())));
    }
}

So that's it,  we have now unit tested our complete Rss implementation using mocks and Linq to XML!  If you need support in getting this to work, please give me a shout.

Download Solution: HowToUnitTest_MockLinq-ToXml.zip

Labels: Linq, MbUnit, TDD, Testing

Following on from my previous post, I wanted to implement another feature in the blog application. The feature is to pick out the Technorati tags from a post and pull relates posts from the Technorati API.  I've already blogged about how to process the Technorati API.

To implement this feature, the first thing i'm going to have implement is being able to return a list of all the tags for a post.

I found unit testing RegEx really difficult.  I'm not great at RegEx as I only rarely have a requirement to do it and this is where unit testing becomes difficult.  Its easy to unit test code you have wrote before but unit testing code you haven't done in a while takes a little more effort.  I feel the reason is because you are thinking about the design in your head to try and figure out how it all fits together.  During this implementation, I found the best thing to do was simply just start writing simple code and spike RegEx.  To spike RegEx I wrote a simple test which I thought should pass successfully based on my previous.

[Test]
public void RegExMatchesString()
{
    Regex r = new Regex("<a href=\http://technorati.com/tags/(.+?)\ rel=\"tag\">(.+?)</a>");
    MatchCollection collection =
        r.Matches("<a href=\"http://technorati.com/tags/TestCode\" rel=\"tag\">Test Code</a>");

    Assert.AreEqual(1, collection.Count, "Collection");

    foreach (Match match in collection)
    {
        Assert.IsTrue(match.Success);
        Assert.AreEqual(2, match.Captures.Count, "Captures");
        Assert.AreEqual("TestCode", match.Captures[0].Value, "One");
        Assert.AreEqual("Test Code", match.Captures[1].Value, "Two");
    }
}

This fails, so I knew I must have done something wrong. I then jumped quickly into the debugger, and found instead of using the Captures collection, I wanted the Groups collection. I retested it and found Groups actually includes both the two matches plus the whole thing. The whole thing is at 0 index in the collection so I had to move everything alone one.  A couple of minutes later I had my regex code working.  There's nothing like seeing a test go green!  The actual implementation is below.

[Test]
public void Spike_RegExMatchesString()
{
    Regex r = new Regex("<a href=\http://technorati.com/tags/(.+?)\ rel=\"tag\">(.+?)</a>");
    MatchCollection collection =
        r.Matches("<a href=\"http://technorati.com/tags/TestCode\" rel=\"tag\">Test Code</a>");

    Assert.AreEqual(1, collection.Count, "Collection");

    foreach (Match match in collection)
    {
        Assert.IsTrue(match.Success);
        Assert.AreEqual(3, match.Groups.Count, "Captures");
        Assert.AreEqual("TestCode", match.Groups[1].Value, "One");
        Assert.AreEqual("Test Code", match.Groups[2].Value, "Two");
    }
}

Based on the code completed during the spike, I can quickly test and implement the code by refactoring out the important parts into actual unit tested methods.  Spending that bit extra time to look into the spike made the actual implementation a lot quicker and easier.

[Test]
public void GetMatchCollectionForHtml()
{
    string post = "<a href=\"http://technorati.com/tags/TestCode\" rel=\"tag\">Test Code</a>";
    MatchCollection collection = Technorati.GetMatches(post);
    Assert.AreEqual(1, collection.Count, "Collection");
}

[Test]
public void GetListFromMatches()
{
    string post = "<a href=\"http://technorati.com/tags/TestCode\" rel=\"tag\">Test Code</a>";
    MatchCollection collection = Technorati.GetMatches(post);

    List<string> tags = Technorati.GetTagList(collection);
    Assert.AreEqual(1, tags.Count);
    Assert.AreEqual("TestCode", tags[0]);
}

Once they work, I write a test to ensure it works together with 'real' data.

[Test]
public void GetTagListFromPostText()
{
    RssFeed rss = TestHelper.CreateMockRSS(TestHelper.url);
    RssItemCollection items = rss.Channels[0].Items;

    string post = items[0].Description;
    List<string> tags = Technorati.GetTagListFromPost(post);
    Assert.AreEqual(3, tags.Count);
    Assert.AreEqual("MbUnit", tags[0]);
    Assert.AreEqual("TDD", tags[1]);
    Assert.AreEqual("Testing", tags[2]);
}

Great, that's our tag list completed.  We now just need to pull the posts from Technorati.

My previous post wasn't very testable, the method was doing a lot.  It was pulling data, processing the feeds and returning a list. This version has been separated into a lot smaller methods to make unit testing easier and more effective.  The first test is to make sure we can return a list of XmlNodes from the API.

[Test]
public void GetXmlFromRss()
{
    int limit = 10;
    string technoratiTerm = "MbUnit";

    XmlNodeList xmlNodes = Technorati.GetXmlNodeList(Technorati.GetUrl(technoratiTerm));
    Assert.AreEqual(limit, xmlNodes.Count);
}

We then want to make a Blog object from one of the XmlNodes in the list.

[Test]
public void GivenXmlNodeCreateBlog()
{
    string technoratiTerm = "MbUnit";

    XmlNodeList xmlNodes = Technorati.GetXmlNodeList(Technorati.GetUrl(technoratiTerm));
    Blog blog = Technorati.CreateBlogPost(xmlNodes[0]);
    Assert.AreEqual(blog.Title, xmlNodes[0]["title"].InnerText);
}

After that has been implemented, we can can a list of Blog objects from a list of nodes (it simply calls the above method multiple times).

[Test]
public void GetBlogPostListTerm()
{
    string technoratiTerm = "MbUnit";

    XmlNodeList xmlNodes = Technorati.GetXmlNodeList(Technorati.GetUrl(technoratiTerm));
    List<Blog> blogs = Technorati.CreateBlogPostList(xmlNodes);
    Assert.AreEqual(10, blogs.Count);
}

Finally, we can get a list of blog posts for a list of strings (again, calling the above method multiple times).

[Test]
public void GetBlogPostListForListOfTerms()
{
    List<string> terms = new List<string>(3);
    terms.Add("MbUnit");
    terms.Add("TDD");
    terms.Add("Testing");

    List<Blog> blogs = Technorati.CreateBlogPostList(terms);
    Assert.AreEqual(30, blogs.Count);
}

The only problem with these tests is that they are hitting the Technorati API for each test.  It would be much better if we could abstract away from the API and use a stub object instead. A stub object allows us to replace the implementation however are they not as cleaver as a mock object and more limited, but for this simple task it will fit the purpose as using a full mock framework isn't really required and similar the better really.

To do this, I'm going to inject the stub object the same way I would a mock object by using a static method on the Technorati object.  Within my test code, I use TechnoratiStub (Implemented within my test assembly, no test code inside the actual assembly under test).  In the real system I use TechnoratiAccessor.

Technorati.TechnoratiAPI = new TechnoratiStub();

private static ITechnoratiAccessor dataStore = new TechnoratiAccessor();
public static ITechnoratiAccessor TechnoratiAPI
{
    get { return dataStore; }
    set { dataStore = value; }
}

I then move the implementation of hitting the API and returning the XmlNodeList to the TechnoratiAccessor.

public interface ITechnoratiAccessor
{
    XmlNodeList GetXmlNodeList(string technoratiUrl);
}

public class TechnoratiAccessor : ITechnoratiAccessor
{
    public XmlNodeList GetXmlNodeList(string technoratiUrl)
    {
        XmlDocument xmlResultsSet = new XmlDocument();
        xmlResultsSet.Load(new XmlTextReader(technoratiUrl));
        XmlNodeList xmlResponse = xmlResultsSet.SelectNodes("/tapi/document/item");
        return xmlResponse;
    }
}

Then, in my Technorati object, I just return whatever the TechnoratiAPI gives me (be it the real implementation or the stub).

public static XmlNodeList GetXmlNodeList(string technoratiUrl)
{
    return TechnoratiAPI.GetXmlNodeList(technoratiUrl);
}

All of our tests still pass, so we can now implement the stub.  In all the tests I setup the property to point to the Stub and then the stub just loads a valid XML response I have cached in a string to a XmlDocument and return the XmlNodeList. Very quick and easy.

public class TechnoratiStub : ITechnoratiAccessor
{
    public XmlNodeList GetXmlNodeList(string technoratiUrl)
    {
        #region Xml
        string cachedXmlResponse = @"Too long to insert here";
        #endregion

        XmlDocument xmlDoc = new XmlDocument();
        xmlDoc.LoadXml(cachedXmlResponse);
        return xmlDoc.SelectNodes("/tapi/document/item");
    }
}

Now when all of our tests execute, they use this Stub object instead of the real API.  Hope you find this useful.

Downloads: TechnoratiTests.cs  ||  Technorati.cs  (These include all of the discussed in this post).

Labels: MbUnit, TDD, Testing

"All problems in computer science can be solved by another level of indirection" Butler Lampson (Or Abstraction as some people refer to it as). 

This is one of the most used quotes within computer science and when it comes to talking about mock objects I think it is particularly true. 

As discussed in Part 1 and Part 2 of my articles on Rhino Mocks, attempting to mock part of the .Net framework is not worth the effort and it is better if you create a wrapper around the framework implementation and then mock the wrapper (adding another layer of abstraction).

In this post, I will outline one possible approach to using mock objects and Rhino Mocks with SubSonic.  There are other approaches, mainly focusing around MVC, however for this approach I want to keep it focusing on using SubSonic as a DAL without discussing MVC.

Scenario

In this post, I'm going to implement a rss aggregator/roller.  It will check blog posts, save any new posts into a database and then the application will pull the blog posts out of the database. 

The flow of data will be this:

RSS > Post Manager > Database > Post Manager ||  Application.

For the moment, I'm not interested in the UI, I will be building the API which could easily be used by an application.  The reason why I picked a blog aggregator is because it provides a interesting TDD/Unit Testing challenge.  Not only are we dealing with databases, which as we found out is difficult, but we are also pulling data from a 3rd party source via the internet.

Implementation

Let's start implementing the solution.  For a RSS Aggregator, the most important thing really is to take a RSS Feed and process the contents.  Because we are using mocks, we can mock out the external RSS feed and just use a known valid response, at runtime this will be replaced with code which makes a call to the web server.  The most important thing to remember is to mock at the external boundary as its at this point we lose control over the execution.

Important considerations for the implementation

Before we start, there are some important considerations about the implementation which we should think about before writing tests/code.  One of the main things to remember is to keep everything isolated and focus on the task in hand, we need to make sure our methods and tests aren't going off and breaking their boundaries.  When we are testing the Post Manager for obtaining the RSS Feeds, we don't want to be hitting the database to see if it saves correctly as well. Next, we want to be sure we are mocking the correct object in the system.  There is no point mocking the Rss object if all it does is hold data, we want to be mocking the object directly related to the OUT (Object Under Test) which has it's own set of related tests to ensure it works correctly.  The objects we generally want to mock out are objects which interact with external dependences, in this case the RssFeed on the web and the database.

Finally, we want to make sure that our tests are running independently of each other and cleaning up after themselves if they make any permanent charges.

With that said, The first requirement for the system is be able to get the xml for a url.

[Test]
public void GetXmlFromURL()
{
    RssFeed rss = PostManager.GetRssFeed(http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml);
    Assert.AreEqual(TestHelper.url, rss.Url);
}

The test simply says we want to access a static method on the PostManager called GetRssFeed and by giving it a url, we can return an RssFeed object. For my RSS objects, I will be using RSS.Net to save me implementing and testing the feed parsing (why build it when its already done).  I actually found this framework very easy to use and completely open source so you can modify it as you see fit (I did to make RssFeed.Url not read-only).  To implement the method on the PostManager, we call a method on the Rss.Net object.

public Rss.RssFeed GetRssFeed(string url)
{
    return RssFeed.Read(url);
}

Now we have a working implementation, it would be nice if we could mock obtaing the RSS instead of having to hit the rss feed every time we wanted to test the method.

[Test]
public void MockGettingRssFeed()
{
    //This is what we want our mock object to return to the calling code.
    RssFeed rss = TestHelper.CreateMockRSS(TestHelper.url);
    MockRepository mocks = new MockRepository();
    IFeedGetter feedGetter = mocks.CreateMock<IFeedGetter>();

    using (mocks.Record()) //Setting up what we expect the mock object to do.
    {
        Expect.Call(feedGetter.GetFeed(TestHelper.url)).Return(rss);
    }

    using (mocks.Playback()) //Performing the test
    {
        PostManager.FeedGetter = feedGetter;
        RssFeed actualRss = PostManager.GetRssFeed(TestHelper.url);

        Assert.GreaterThan(rss.Channels.Count, 0);
        Assert.AreEqual(TestHelper.url, rss.Channels[0].Link.AbsoluteUri);
    }
}

In order for obtaining the RSS feed to be mocked out, I moved the GetFeed method into a IFeedGetter interface (implemented as FeedManager) and added a FeedGetter property to the PostManager object.  This allows me to use the default object (FeedManager) in the real system without changing the code, however during unit testing (or in the future) I can replace the default implementation with my own - in this case a mock IFeedGetter.

private static IFeedGetter feedGetter = new FeedManager();
public static IFeedGetter FeedGetter
{
    get { return feedGetter; }
    set { feedGetter = value; }
}

In order to keep the test code cleaner, I extracted the creation of the mock RssFeed object into a TestHelper method, this can then be called from multiple locations within my test code without having to manually create an object each time.  There is also a unit test associated with this method to make sure it works correctly.

Our implementation of FeedGetter and PostManager is complete.  Using Rss.Net allowed us to cut the implementation time.  Moving on, we need to be able to save this RssFeed object into the database. 

If you are not aware, an RSS is made up of general information about the feed (RssChannel) and then a number of items (RssItem).  To save the feed, the first thing we need to do is save the RssChannel object.

[Test]
public void InsertFeedDetailsIntoDatabase()
{
    using (mocks.Record())
    {
        Expect.Call(database.InsertFeedDetails(rssFeed.Channels[0])).Return(1);
    }

    using (mocks.Playback())
    {
        int recordID = DatabaseManager.InsertFeedDetails(rssFeed.Channels[0]);

        Assert.GreaterThan(recordID, 0);
    }
}

This method simple says that a method on the database object should accept an RssChannel object and return a 1 if its inserted correctly.  We them access this method via a DatabaseManager object.  For the test to be shorter, I moved a lot of the code into the [Setup] for the test fixture.

RssFeed rssFeed;
MockRepository mocks;
IDataStore database;

[SetUp]
public void TestSetup()
{
    rssFeed = TestHelper.CreateMockRSS(TestHelper.url);
    mocks = new MockRepository();
    database = mocks.CreateMock<IDataStore>();
    DatabaseManager.DataStore = database;
}

Notice I'm setting the DataStore of the DatabaseManager to the IDataStore mock object via a property. One thing which I want to test for is to make sure that if the property hasn't been setup correctly, then the correct exception is thrown.  This just saves any nasty surprises later.

[Test]
[ExpectedArgumentNullException]
public void ThrowExceptionOnSaveFeedIfNoDataSource()
{
    DatabaseManager.DataStore = null;
    DatabaseManager.SaveFeed(rssFeed);
}

Moving on, the next item to implement is saving an actual RssItem.   The test just makes sure that the InsertFeedItem is called twice, once for each RssItem in the collection.

[Test]
public void InsertFeedItemsIntoDatabase()
{
    int channelId = 1;
    using (mocks.Record())
    {
        Expect.Call(database.InsertFeedItem(channelId, rssFeed.Channels[0].Items[0])).Return(1);
        Expect.Call(database.InsertFeedItem(channelId, rssFeed.Channels[0].Items[1])).Return(2);
    }

    using (mocks.Playback())
    {
        bool result = DatabaseManager.InsertFeedItems(channelId, rssFeed.Channels[0].Items);
        Assert.IsTrue(result);
    }
}

Now we have both parts to save the feed in place, we can write another method which tests we can do both.

[Test]
public void InsertFeedIntoDatabase()
{
    int channelID = 1;

    using (mocks.Record())
    {
        Expect.Call(database.InsertFeedDetails(rssFeed.Channels[0])).Return(channelID);
        Expect.Call(database.InsertFeedItem(channelID, rssFeed.Channels[0].Items[0])).Return(1);
        Expect.Call(database.InsertFeedItem(channelID, rssFeed.Channels[0].Items[1])).Return(2);
    }

    using (mocks.Playback())
    {
        bool result = DatabaseManager.SaveFeed(rssFeed);
        Assert.IsTrue(result);
    }
}

Now our DatabaseManager is setup correctly with the methods we want to expose to the public which meet the requirements, we can move onto implementing our IDataSource which will actually store the information.  For this, we are using SubSonic to save all of our Rss information into the database.

The first problem we face is converting our Rss.Net objects into the ActiveRecord SubSonic objects for the database, a method is required to covert between the two objects. Inside of SubSonicTests.cs, we have a test to make sure this is done correctly on our dataSource object.

[Test]
public void ConvertFromRssChannelToSubSonicRssChannel()
{
    RssChannel rssChannel = rss.Channels[0];
    //Returns a SubSonic object.
    MockSubSonic.DAL.RssChannel subSonicChannelObject = dataSource.ConvertRssChannelObject(rssChannel);

    Assert.AreEqual(rssChannel.Title, subSonicChannelObject.Title);
    Assert.AreEqual(TestHelper.url, rssChannel.Link.AbsoluteUri);
    Assert.AreEqual(rssChannel.Link.AbsoluteUri, subSonicChannelObject.Uri);
    Assert.AreEqual(rssChannel.Description, subSonicChannelObject.Description);
    Assert.AreEqual(rssChannel.PubDate, subSonicChannelObject.PubDate);
    Assert.IsTrue(subSonicChannelObject.IsNew); //it shouldn't save it.
}

We can then implement our solution to save the details into the database, first the details:

[Test]
public void InsertRssChannelIntoDatabase()
{
    int channelId = dataSource.InsertFeedDetails(rss.Channels[0]);
    Assert.AreEqual(channelId, 1);
}

Then the RssItems.

[Test]
public void InsertRssItemIntoDatabase()
{
    int channelId = dataSource.InsertFeedDetails(rss.Channels[0]);
    Assert.GreaterEqualThan(channelId, 0);

    int itemId = dataSource.InsertFeedItem(channelId, rss.Channels[0].Items[0]);
    Assert.AreEqual(itemId, 1);
}

We are not using mock objects at this point because we have refactored to the lowest reasonable point.  Any more and it gets too confusing and becomes pointless. We have cut a lot of the database access out by making it mockable in the first place, having these methods hit the database isn't going to be a great problem.  So we don't get any problems in the future, we need to make sure we clean up after ourselves.

[SetUp]
public void TestSetup()
{
    rss = TestHelper.CreateMockRSS(TestHelper.url);
    dataSource = new SubSonicDataStore();
    ExecuteSql("TRUNCATE TABLE RssChannel");
    ExecuteSql("TRUNCATE TABLE RssItem");
}

[TearDown]
public void TestTearDown()
{
    rss = null;
    dataSource = null;
    ExecuteSql("TRUNCATE TABLE RssChannel");
    ExecuteSql("TRUNCATE TABLE RssItem");
}

This is the bases for the rest of the implementation of the database.  We use SubSonic within the DataSource without any tricks or mock objects to ensure its working correctly, however in any later which isn't interacting directly with the database, we are using mock objects.  The other part of the system which I will cover is getting information from the database.

The test below is to ensure that our DataManager object can return RssItems for a Channel based on the ChannelID.

[Test]
public void GetFeedItemsForChannelFromDatabase()
{
    int channelId = 1;
    using (mocks.Record())
    {
        Expect.Call(database.GetRssItemsForChannel(channelId)).Return(rssFeed.Channels[0].Items);
    }

    using (mocks.Playback())
    {
        RssItemCollection rssFeeds = DatabaseManager.GetRssFeedsForChannel(channelId);
        Assert.IsNotNull(rssFeeds);
        Assert.AreEqual(2, rssFeeds.Count);
    }
}

The SubSonic test for this requirement is below.  To start with we are inserting the channel and two feeds from our mock object into the database so we can actually return something.  We then obtain the collection from our database and to ensure all the feeds we got are correct we are verifying them within a foreach loop against our known TestHelper Rss Mock object which we inserted.

[Test]
public void GetRssChannelFeeds()
{
    int channelId = dataSource.InsertFeedDetails(rss.Channels[0]);
    dataSource.InsertFeedItem(channelId, rss.Channels[0].Items[0]);
    dataSource.InsertFeedItem(channelId, rss.Channels[0].Items[1]);

    RssItemCollection rssColl = dataSource.GetRssItemsForChannel(channelId);
    Assert.AreEqual(2, rssColl.Count);

    int i = 0;
    foreach (RssItem item in rssColl)
    {
        Assert.AreEqual(rss.Channels[0].Items[i].Title, item.Title);
        Assert.AreEqual(rss.Channels[0].Items[i].PubDate.ToString(), item.PubDate.ToString());
        Assert.AreEqual(rss.Channels[0].Items[i].Link.AbsoluteUri, item.Link.AbsoluteUri);
        Assert.AreEqual(rss.Channels[0].Items[i].Description, item.Description);
        Assert.AreEqual(rss.Channels[0].Items[i].Author, item.Author);
        i++;
    }
}

The advantage to inserting all the information at the start of the test is that we know what is in the database and what to expect when its returned.  If we relied on the contents of the database, then other tests could change the values or someone could come along and clean out the database - our tests would then fail as what we expected wasn't what was actually their. This makes debugging the failure even more difficult as we have to investigate why it failed as it might not be because of our implementation but conflicting test runs.

If your interested in how the rest of the system is implemented, please download the project solution at the bottom of the post.

Passing Tests

After we have implemented our API, we have a number of tests which should all pass green.

The reason we haven't got 100% code coverage is because one or two lines of error handling I inserted which wasn't tested against (for example, if a user couldn't be found).  In a real system, we would need to test against these situations.  I have exclude the code generated by SubSonic and the RSS.Net project from the report, as that's not directly related to the system and doesn't provide a good indication of our unit test coverage.

To actually make sure it all works correctly, I created a simple Console Application which calls all the methods.

static void Main(string[] args)
{
    Console.WriteLine("Welcome UserA to MockSubSonic");
    User u = new User("UserA", "PassA", "User@TestMail.com");
    DatabaseManager.SaveUser(u);

    Console.WriteLine("Saving Feed");
    DatabaseManager.SaveFeed(RssFeed.Read("http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml"));

    Console.WriteLine("Subscribing");
    bool result = DatabaseManager.SubscribeUserToFeed(u.Name, "http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml");

    if(result)
        Console.WriteLine("You successfully subscribed to http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml");
    else
        Console.WriteLine("You failed to subscribed to http://newsrss.bbc.co.uk/rss/newsonline_uk_edition/front_page/rss.xml. Sorry");

    RssFeedCollection feeds = DatabaseManager.GetUsersRssFeedCollection(u.Name);
    Console.WriteLine("You are now subscribed to: " + feeds.Count);
    Console.WriteLine("First one is : " + feeds[0].Channels[0]);

    Console.ReadLine();
}

Another way of doing this would have been a more Integrated style test using MbUnit.  It's the kind of the same as the tests we have wrote before, however it touches more/all parts of the system to verify it is working correctly together.

Summary

I hope you have found this post useful and interesting if you are looking at SubSonic and mock objects.  The important thing to remember when mocking SubSonic is to wrap the SubSonic object inside your own access layer (in this case SubSonicDataStore : IDataStore).

Feedback/comments welcome!

Resources:

Code - Unit Tests, API, Application, SQL zip

SubSonic

Rhino Mocks

MbUnit

Rss.Net

Labels: MbUnit, Rhino Mocks, SubSonic, TDD, Testing