I’ve just completed my first proof of concept which I’m very excited about and I just had to write a quick blog about it – plus it breaks my blogging silence.
I find fault injection an interesting concept, I’ll explore it in more detail in later posts, but the aim is to insert exceptions at various points in order to test certain behaviours and how the application copes when an error occurs. However, raising exceptions is not a easy task, you have to create the scenario and environment in order for the exception to occur. With my fault injection application, you don’t need to create the scenario, you simply say when you want the exception to occur. For example, I would be able to throw an IOException when System.IO.File.ReadAllLines() is called without actually having to create the scenario for the exception to be raised – saving me time and effort but also allowing me to test more scenarios and error handling.
Tonight, I created a very simple concept to throw exceptions on method calls. I’ve got a very simple console application which calls two methods – nothing special about this. When MethodFails() is called, my injector will step in and throw a MethodAccessException (could be any exception) which will be caught by the console application with the details of the exception being outputted.
using System;
namespace ConsoleApplication2
{
class Program
{
static void Main(string[] args)
{
Console.WriteLine(“Should output 1 message then fail”);
try
{
MethodA();
MethodFails();
}
catch (Exception e)
{
Console.WriteLine(“Exception caught”);
Console.WriteLine(“Type: ” + e.GetType());
Console.WriteLine(“Message: ” + e.Message);
}
}
private static void MethodA()
{
Console.WriteLine(“This should work”);
}
private static void MethodFails()
{
Console.WriteLine(“this should have failed”);
}
}
}
The output of the execution is the following information:
Should output 1 message then fail
This should work
Exception caught
Type: System.MethodAccessException
Message: You attempted to call ConsoleApplication2.Program.MethodFails, this has been blocked. Goodbye
Technical details to come at a later point, it’s getting late in the UK. However, I would love to hear your comments on this idea.
Fault injection is very important, specially when you have code that tries to recover from exceptions. This code is usually very had to test with unit tests.
Anytime you do a file operation, read from the network, there’s a number of potential failure (timeout, disk full, network disconnected), fault injection ensures that your code _also_ works in production.
Problem with fault injection is that careless injection leads to false negatives.
This does sound quite exciting. Can you also handle slowness, the other major source of faults?
@Peli – You are correct, one I didn’t really think about last night. For example, calling MethodFails wouldn’t have caused an exception (Doh!), maybe I should have had it write to a filestream – but you get the general idea.
Careless injection does lead to false negatives. Hopefully people would be using it to target certain use-cases and known exceptions – such as network timeout.
@LosingSide – It wasn’t in my initial use-case, but yes the framework can cope with this. Given this method call, delay it by 5 seconds. Not sure how it would deal with remoting etc, but its definitely an interesting usecase. Thanks.
Ben,
Can you elaborate on the difference between your tool and the ability to “inject” faults using a standard mocking framework?
Hi Lior,
Its about different levels of interaction. While a tool like TypeMock would allow you to do fault injection, but it does mean you have to write a integration test around the block of code which isn’t always the approach you want to take. Especially around end-to-end and exploratory testing which is where my approach differs.
It would be interesting of we could configure (in a config file)a list of exceptions for any line of code and able to change it dynamically too.
i.e. if you want an exception file not found to be thrown on MethodFails() you specify
DoException(FNF);
MethodFails();
i,e the exception specified in DoException is throwed on the next line executed.
Dont know whether its a good design or idea but it will really look good.
But hats of to Ben for the concept