Announcing NTestDataBuilder library

Further to the blog post I posted yesterday about the approach to generating test fixture data for automated tests that my team has adopted I have open sourced a base class and other supporting code infrastructure to quickly, tersely and maintainably generate fixture data in a consistent way.

I have released this library to GitHub and NuGet under the name NTestDataBuilder.

The GitHub page has a bunch of code snippets that will get you started quickly. Read my previous blog post to get a feel for how I use the code and in particular how I combine the approach with the Object Mother pattern for best effect.


Test Data Generation the right way: Object Mother + Test Data Builders + NSubstitute + NBuilder

Generating test data for automated testing is an area that I have noticed can get very, very tedious when your application gets more and more complex. More importantly, it’s an area that I have noticed results in hard-to-maintain test logic. If you are a proponent of treating test code with the same importance of production code like I am, then you will want to refactor your test logic to make it more simple, maintainable and DRY.

In my last two projects I have been working with my team to come up with a good solution for this problem and I’m pretty stoked with what we’ve got so far. While I certainly had my hand in the end result, I certainly can’t claim all of the credit – far from it! Big props to my team mates Matt Kocaj and Poya Manouchehri for significant contributions to this.

Newing up objects everywhere

If you organically grow your automated test suite without thinking about how you are generating your test data then it’s likely you will end up with a lot of tests that have arrange sections full of object instantiation, and these sections are likely to get more and more complex as your object model gets complex (what if you need to have a customer that has 3 orders, each with 3 products?).

If you are diligent with refactoring your tests then you will likely start pulling out common instantiations to private methods (or similar), but even then you are likely to notice the following patterns start to emerge:

  • If you need to change the constructor of one of your classes there will be many, many instantiations using that constructor across your test project and so that refactor becomes a pain (and you are likely to do hacks like add default values that you wouldn’t otherwise when adding new parameters to make it easier).
  • There is likely to be a lot of repetition across different test classes in the objects that are created.
  • Building lists is really verbose and will likely feature a lot of repetition / hard-to-read code.
    • You can bring in libraries like NBuilder to make this easier and more readable.
    • You can’t use the best parts of NBuilder if you are not using public property setters so you can enforce domain invariants.
  • Your approach to generating data will be largely inconsistent across test classes – some will have private methods, some will do it inline, some might delegate to factories or helper classes etc. – this makes the code hard to understand and maintain.
  • It’s not immediately clear which of the parameters you are passing into the constructor are there to meet the requirements of the call and which are there because the test dictates it – this means the intent of your test is somewhat obscured

Object Mother

A good approach to start to address the problem is to implement the Object Mother pattern. This allows you to quickly and tersely create pre-canned objects and give them a descriptive name. Now instead of having complex chains of object instantiations you have one line of code (e.g. ObjectMother.CustomerWith3FilledOrders), which is a lot more descriptive and your tests become simpler as a result. This also helps ensure that your approach for data generation is consistent, and thus more maintainable.

If you just use Object Mother though, you will likely observe the following problems:

  • While your constructor calls are likely to be in one file now, there is still potentially a lot of calls so it’s still tricky to make constructor changes.
  • As soon as you have a small tweak needed for the data returned there is a tendency to simply create a new Object Mother property.
  • Because of this, the Object Mother class can quickly get unwieldy and a nightmare to maintain and understand – it becomes a god object of sorts.

Test Data Builder

An approach that seems to have stemmed from the Object Mother pattern is the application of the builder pattern to result in the Test Data Builder pattern. This involves creating a builder class responsible for creating a particular type of object via a fluent interface that keeps track of the desired state of the object being built before being asked to actually build the object.

This provides a number of advantages:

  • Your constructor calls will be in your test project once only making them easy to change.
  • You have a readable, discoverable, fluent interface to generate your objects making your tests easy to read/write/maintain.
  • You are consistent in how you generate your test data.
  • It works with objects that you don’t have public property setters for since there is an explicit build action, which can invoke the constructor with the parameters you have set.
  • It’s flexible enough to give you the ability to express and perform actions to your object after it’s constructed – e.g. specify that an order is paid (which might be a Pay() method rather than a constructor parameter; you wouldn’t expect to be able to create an order in the paid state!).
  • Because of this your builder objects provide a blueprint / documentation for how to interact with your domain objects.
  • The test data builder can contain sensible defaults for your object instantiation and thus your test only needs to specify the values that need to be arranged thus making the intent of your test very clear
  • We have added a method to our builders (.AsProxy()) that allows the object returned to be an NSubstitute substitute (a proxy / mock object if you aren’t familiar with that library) rather than a real object (with the public properties set to automatically return the values specified in the builder). See below for a code snippet.
    • This means that when we need substitutes to check whether certain methods are called or to ensure that certain method calls return predetermined values we can generate those substitutes consistently with how we generate the real objects.
    • This is very powerful and improves understandability and maintainability of the tests.
    • In general we try and avoid using substitutes because it means that it’s possible to get objects that violate domain invariants, however, on occasion it’s very useful to properly unit test a domain action when it interacts with other domain objects

Here is a (somewhat contrived) example of the .AsProxy() method described above.

public void Given10YearMembership_WhenCalculatingDiscount_ThenApply15PercentLongMembershipDiscount() {
    var member = MemberBuilder.AsProxy().Build();

    var discount = _discountCalculator.CalculateDiscountFor(member, new DateTimeProvider());

    Assert.That(discount, Is.EqualTo(0.15));

Combining NBuilder and the Test Data Builders

Another cool thing we have done is to combine our test data builders with NBuilder to allow for terse, expressive generation of lists of objects while still supporting objects that don’t have public property setters. We do this by generating a list of builders with NBuilder and then iterating that list to build each builder and get the actual object. Here is an example:

var members = Builder.CreateListOfSize(4)
    .TheFirst(1).With(b => b.WithFirstName("Rob"))
    .TheNext(2).With(b => b.WithFirstName("Poya"))
    .TheNext(1).With(b => b.WithFirstName("Matt"))
    .Select(b => b.Build());

We have made the above code even terser, by adding some extension methods to allow for this:

var members = MemberBuilder.CreateList(4)
    .TheFirst(1).With(b => b.WithFirstName("Rob"))
    .TheNext(2).With(b => b.WithFirstName("Poya"))
    .TheNext(1).With(b => b.WithFirstName("Matt"))

Adding back Object Mother

One of the really nice things about the Object Mother pattern is that you reduce a lot of repetition on your tests by re-using pre-canned objects. It also means the tests are a lot terser since you just specify the name of the pre-canned object that you want and assuming that name describes the pre-canned object well then your test is very readable.

What we have done is combined the maintainability and flexibility that is afforded by the Test Data Builder pattern with the terseness afforded by the Object Mother by using Object Mothers that return Test Data Builders. This overcomes all of the disadvantages noted above with the Object Mother pattern.

In order to make it easy to find objects we also use a static partial class for the Object Mother – this is one of the few really nice uses I’ve seen for partial classes. Basically, we have an ObjectMother folder / namespace in our test project that contains a file for every entity we are generating that includes a static nested class within the root ObjectMother class. This ensures that all test data is gotten by first typing “ObjectMother.” – this makes it really consistent and easily accessible, while not having an unmaintainable mess of a god object.

For instance we might have (simple example):

// Helpers\ObjectMother\Customers.cs
using MyProject.Tests.Helpers.Builders;

namespace MyProject.Tests.Helpers.ObjectMother
    public static partial class ObjectMother
        public static class Customers
            public static CustomerBuilder Robert
                get { return new CustomerBuilder().WithFirstName("Robert"); }
            public static CustomerBuilder Matt
                get { return new CustomerBuilder().WithFirstName("Matt"); }

If our test requires any small tweaks to the predefined object then you don’t need to necessarily add another property to your object mother – you can use the methods on the builder, e.g. ObjectMother.Customers.Robert.WithLastName("Moore").Build();

We generally stick to using properties rather than methods in the object mothers since the builders mean that it’s not necessary to pass any parameters to get your pre-canned object out, but occasionally we do have some that have methods when there is something meaningful to configure that makes sense in the object mother e.g. ObjectMother.Members.MemberWithXProducts(int numProducts).

Making the Test Data Builders terse

One argument against this approach, particularly when you first start out your test project (or indeed if you decide to refactor it to use this technique after the fact) is that you are writing a lot of code to get it all up and running. We decided to avoid that by using a number of techniques:

  • We set any reasonable defaults in our builder constructor so new CustomerBuilder().Build() will give a reasonable object (unless for that particular type of object there are any properties that make sense to always have to specify, in which case we don’t add a default for that property).
  • We have created a base class that allows us to set/get the property values into a dictionary using a lambda expression that identifies the property whose value is being set/retrieved – this reduces the code in the builder by eliminating the need for most (sometimes there is still a need to keep state in the builder where you are storing something not expressed by one of the properties) of the private variables in the builder.
  • We only add fluent methods for the properties we are actually using in our tests that point in time – this means we don’t have dead methods lying around in the builders and initially they are very terse.
  • We have a base class that defines a lot of the common infrastructure behind defining a builder (including the ability to return a proxy object and the ability to create a list of builders using NBuilder – I’ve open sourced the code as NTestDataBuilder).

This generally means that we can be up and running with an object mother and builder for a new entity within a minute and it’s well worth doing it from the start with all our objects to keep consistency in the codebase and ensure all tests are maintainable, terse and readable from the start.

Here is an example of what I mean from the NTestDataBuilder library I have released:

class CustomerBuilder : TestDataBuilder<Customer, CustomerBuilder>
    public CustomerBuilder()

    public CustomerBuilder WithFirstName(string firstName)
        Set(x => x.FirstName, firstName);
        return this;

    public CustomerBuilder WithLastName(string lastName)
        Set(x => x.LastName, lastName);
        return this;

    public CustomerBuilder WhoJoinedIn(int yearJoined)
        Set(x => x.YearJoined, yearJoined);
        return this;

    protected override Customer BuildObject()
        return new Customer(
            Get(x => x.FirstName),
            Get(x => x.LastName),
            Get(x => x.YearJoined)

Explaining seemingly weird NSubstitute behaviour

I think that NSubstitute is an absolutely amazing library! I’ve used a few different mocking libraries and NSubstitute is hands down the best one. Not only is the syntax beautiful and terse, but it’s the simplest mocking library you could ever use. Don’t know the difference between a mock and a stub? WHO CARES! It’s not something you should need to know. I couldn’t explain it better than the NSubstitute homepage:

Mock, stub, fake, spy, test double? Strict or loose? Nah, just substitute for the type you need!

NSubstitute is designed for Arrange-Act-Assert (AAA) testing, so you just need to arrange how it should work, then assert it received the calls you expected once you’re done. Because you’ve got more important code to write than whether you need a mock or a stub.

99 times out of 100 you will have nothing but pleasure with NSubstitute, but like all things it’s not perfect. When I first started using NSubstitute I didn’t realise how it was implemented and just assumed it used some sort of crazy voodoo magic. More recently, I’ve actually contributed to NSubstitute to fix a bug I was experiencing. After doing so, I understand a lot more now how it actually works. It turns out it’s not magic, but it is very clever.

One of the things that can trip you up about the way it’s implemented is due to the fact that .Returns and Arg.Any/Is are based on a static stack and thus sometimes if you do things like set up a received check or return value for a  non-virtual method or set up a return value using the result of a substituted call then what the code looks like it should do doesn’t actually happen. When this does happen, if you don’t understand the implementation of what’s going on then this can be incredibly difficult to diagnose and you will generally spend a few confused hours.

This recently happened for some of the devs on my team and I wanted to try and explain to them what was happening so I came up with a gist that shows some examples for .Returns of the effect the static stack implementation has. I haven’t done any for Arg.Is/Any at this stage.

If you come across problems with NSubstitute or you need to explain to other people this behaviour feel free to use my gist. It uses NUnit and all of the tests in the gist passed when I ran it (against NSubstitute 1.4.3).

AutofacContrib.NSubstitute v3 release

I’d like to announce a new release for the AutofacContrib.NSubstitute library. The library was in need of a bit of love because it forced you to use a really old NSubstitute version.

I’ve made the following changes:

  • Deprecated the AutoMock class in favour of AutoSubstitute (in keeping with the theme of the NSubstitute library).
  • Added a new constructor that allows you to make modifications to the ContainerBuilder before the IContainer is built.
  • Dependency on the latest version of NSubstitute and removed the hard restriction on the NSubstitute and Autofac version numbers so the package is compatible with the latest versions of those libraries.
  • Moved the source to Github
  • Added some documentation and an example class showing some usage examples


Testing Url Helper within controllers in ASP.NET MVC

I recently had some code like the following in a controller that I wanted to test. The Facebook Connect API, despite being fairly awesome, is tied to the URL of the page so whenever you change the URL of your page then you will lose all your comments. This code was to get the initial URL of the page and store it in the database so that URL was used from then on:

var url = ControllerContext.RequestContext.HttpContext.Request.Url;
vm.FacebookCommentUrl = string.Format("https://{0}{1}", url.Authority, Url.Action("Detail", new { Id = "000", Controller = ArticleType.ToString() })).Replace("000", "{0}");

It’s easy to mock Request.Url within the ControllerContext; something like MVCContrib.TestHelper will mock it out for you with a few lines of code. Personally, I use the following code in conjunction with the AutoMock library to do it:

    public static class AutoMockContainer
        public static AutoMock Create()
            var autoMock = new AutoMock();

            var httpContext = Substitute.For<HttpContextBase>();

            var server = Substitute.For<HttpServerUtilityBase>();

            var request = Substitute.For<HttpRequestBase>();
            var parameters = new NameValueCollection();
            var formParameters = new NameValueCollection();
            var qsParameters = new NameValueCollection();
            var headers = new NameValueCollection();
            headers.Add("Host", "localhost");
            request.Url.Returns(new Uri("http://localhost/"));
            request.Cookies.Returns(new HttpCookieCollection());
            request.ServerVariables.Returns(new NameValueCollection());

            var response = Substitute.For<HttpResponseBase>();
            response.Cookies.Returns(new HttpCookieCollection());
            response.ApplyAppPathModifier(Arg.Any<string>()).Returns(a => a.Arg<string>());

            var routeData = new RouteData();

            var requestContext = Substitute.For<RequestContext>();
            requestContext.RouteData = routeData;
            requestContext.HttpContext = httpContext;

            var actionExecutingContext = Substitute.For<ActionExecutingContext>();
            actionExecutingContext.RequestContext = requestContext;

            var actionExecutedContext = Substitute.For<ActionExecutedContext>();
            actionExecutedContext.RequestContext = requestContext;

            var controller = Substitute.For<ControllerBase>();

            var controllerContext = Substitute.For<ControllerContext>();
            controllerContext.HttpContext = httpContext;
            controllerContext.RouteData = routeData;
            controllerContext.RequestContext = requestContext;
            controllerContext.Controller = controller;
            controller.ControllerContext = controllerContext;

            var iView = Substitute.For<IView>();

            var viewDataDictionary = new ViewDataDictionary();

            var iViewDataContainer = Substitute.For<IViewDataContainer>();

            var textWriter = Substitute.For<TextWriter>();

            var viewContext = new ViewContext(controllerContext, iView, viewDataDictionary, new TempDataDictionary(), textWriter)
                HttpContext = httpContext,
                RouteData = routeData,
                RequestContext = requestContext,
                Controller = controller

            return autoMock;

        public static T GetController<T>(this AutoMock autoMock) where T : Controller
            var routes = new RouteCollection();
            var controller = autoMock.Resolve<T>();
            controller.ControllerContext = autoMock.Resolve<ControllerContext>();
            controller.Url = new UrlHelper(autoMock.Resolve<RequestContext>(), routes);
            return controller;

Above is the current version of the code that I use, but I change it every now and then when there is some other part of MVC that I need to mock, but haven’t yet. In this particular instance I was having trouble figuring out how to get Url.Action to return the actual URL – it was returning an empty string. I was registering my routes by calling the static method in my MvcApplication class, but that wasn’t enough. It took some Googling and perseverance to finally find the right set of steps to get it working (the one that really threw me was having to mock ApplyAppPathModifier in the Response object!):

  1. Register your routes in the route table
  2. Mock Request.Url (new Uri(“http://someurl/”))
  3. Mock Request.AppRelativeCurrentExecutionFilePath (“~/”)
  4. Mock Request.ApplicationPath (“/”)
  5. Mock Request.ServerVariables (new NameValueCollection())
  6. Mock Response.ApplyAppPathModifier(…) (return what is passed into it!)
  7. Set the UrlHelper in your controller to: new UrlHelper(A mock for request context with the above mocked, A route collection which has your applications routes registered into it);

Quite a strange an bizarre set of steps, but it’s what’s required to get around the core implementation within ASP.NET MVC. Maybe they will make it easier to test in the future…

With the above code snippet, I can do the following two lines of code to automatically get a controller with everything mocked correctly (not to mention any interfaces in it’s constructor automatically set to an NSubstitute Mock!):

var autoMock = AutoMockContainer.Create();
var controller = autoMock.GetController<HomeController>();

Testing code that uses Autofac + DependencyResolver in ASP.NET MVC

Sometimes it’s necessary to use DependencyResolver.Current in your MVC code where there is code that can’t be simply injected using your dependency injection framework. MVC provides a lot of extensibility points for things like model binding, model validation, and action filter, controller and view instantiation that mean that 90% of the time you can simply use your DI framework. This is made especially easy by frameworks like Autofac that provide easy MVC integration.

Use Case: Filter Provider

The main circumstance I’ve needed to use DependencyResolver.Current is in a filter provider (when trying to get precise control over globally applied filters and their execution order). I have tried passing through functors from Autofac that return the things I’m after, but you get Autofac resolution errors because it can’t tell what scope you are calling them from and if they are bound to, say, the HTTP lifetime then it can’t guarantee that the objects will be disposed correctly. There is possibly a way to inject something to correctly resolve the right Autofac lifetime scope, but I haven’t looked into it further.

An example such filter provider might be:

    public class MyAppFilterProvider : IFilterProvider
        private readonly Environment _currentEnvironment;

        public MyAppFilterProvider(Environment currentEnvironment)
            // Injected from Autofac
            _currentEnvironment = currentEnvironment;

        public IEnumerable GetFilters(ControllerContext controllerContext, ActionDescriptor actionDescriptor)

            // Ensure all pages other than the homepage are HTTPS in production
            if (_currentEnvironment == Environment.Prod && !(actionDescriptor.ActionName == "Index" && controllerContext.Controller.GetType() == typeof(StaticPageController)))
                yield return new Filter(new RequireHttpsAttribute(), FilterScope.First, -5);

            // Resolve the "current user" filter including any dependencies it may have
            // This could do something like, say put a reference to the current user from the database into the ViewBag so it can be accessed by the controller and the view
            var currentUserFilter = DependencyResolver.Current.GetService();
            yield return new Filter(currentUserFilter, FilterScope.First, -1);

Testing when there are calls to Dependency Resolver

Because we make use of DependencyResolver we need to be able to test it. This is trickier than it seems when using Autofac since if you resolve a service from the dependency resolver when there is no HttpContext then it will fail with “System.InvalidOperationException : The request lifetime scope cannot be created because the HttpContext is not available”. This problem may well affect other DI frameworks as well.

There are two options to combat this:

  • Create a custom class to resolve dependencies for that/those specific test/s and set it using DependencyResolver.SetResolver(new MyTestDependencyResolver()), but this could become tedious to maintain and complicates your test
  • Simulate a HTTP context by using HttpSimulator, it’s easy to install via NuGet (thanks Matt): Install-Package HttpSimulator

When testing I use an Autofac container that automatically resolves interfaces as NSubstitute Substitutes (there are similar packages for other mocking frameworks, check out Autofac.Contrib). The following code is an example of how you can set up a test:

    public class ClassThatUsesDependencyResolver
        public ClassThatUsesDependencyResolver()
            var service = DependencyResolver.Current.GetService();

    class DependencyResolverTests
        private AutoMock _autoMock;
        private HttpSimulator _httpSimulator;

        public void Setup()
            _autoMock = new AutoMock();
            DependencyResolver.SetResolver(new AutofacDependencyResolver(_autoMock.Container));
            _httpSimulator = new HttpSimulator().SimulateRequest();

        public void Teardown()

        public void Test()
            new ClassThatUsesDependencyResolver();

One thing to note is that I used DependencyResolver.Current in the test rather than _autoMock.Resolve<…>() because the scoping that Autofac has is such that they resolve as different objects (which can be a bit confusing).

Testing the filter provider

Given I’ve shown an example filter provider above I figure I should demonstrate some of the tests that I have done in the past. There are two main things to test as far as I see (which you test for or indeed if you test for both (or don’t test it at all) depends on your situation – be pragmatic):

  • Test the order that the filters get applied is as you expect (you don’t want database calls for the current user before passive (e.g. single-sign-on) authentication is processed for instance…
  • Test that in certain circumstances a filter is applied (or not!)

An example showing both for the above filter provider is given below:

    class MyAppFilterProviderShould
        private AutoMock _autoMock;
        private HttpSimulator _httpSimulator;
        private ControllerContext _controllerContext;
        private ActionDescriptor _actionDescriptor;

        public void Setup()
            _autoMock = new AutoMock();
            DependencyResolver.SetResolver(new AutofacDependencyResolver(_autoMock.Container));
            _httpSimulator = new HttpSimulator().SimulateRequest();
            _controllerContext = Substitute.For<ControllerContext>();
            _actionDescriptor = Substitute.For<ActionDescriptor>();

        public void Teardown()

        private IEnumerable<Filter> GetFilters(Environment environment)
            var filterProvider = new MyAppFilterProvider(environment);
            return filterProvider.GetFilters(_controllerContext, _actionDescriptor);

        public void Resolve_filters_in_correct_order()
            var filters = GetFilters(Environment.Prod).ToList().OrderBy(f => f.Scope).OrderBy(f => f.Order).ToList();
            var index = 0;
            Assert.That(filters[index++].Instance, Is.TypeOf<RequireHttpsAttribute>()); // Require Https should always be run first
            Assert.That(filters[index++].Instance, Is.TypeOf<CurrentUserFilter>());
            Assert.That(filters, Has.Count.EqualTo(index)); // This ensures we update the test when more filters are added

        public void Not_require_https_when_not_in_production([Values(Environment.Dev, Environment.CI, Environment.Test)] Environment environment)
            Assert.That(GetFilters(environment).Select(f => f.Instance), Has.None.TypeOf<RequireHttpsAttribute>());

        public void Not_require_https_for_homepage_in_production()
            _controllerContext.Controller = _autoMock.Resolve<StaticPageController>();
            Assert.That(GetFilters(Environment.Prod).Select(f => f.Instance), Has.None.TypeOf<RequireHttpsAttribute>());

Update (23/03/2013)

It might not be necessary to ever use DependencyResolver.Current any more due to a new technique I’ve found.