Yesterday I started working on my first professional usage of ASP.NET Web API and as part of that created a handy base class to perform full-stack unit and/or integration testing of controller actions in-process. It was based on some stuff I found online, but put my own flair on so I thought I would share it here in case anyone else found it useful. While I’m at it I thought I’d briefly outline my testing strategy for most MVC applications I write.

ASP.NET MVC Testing

When I’m testing an ASP.NET MVC application I have had good mileage with covering the following (this is a combination of what you might consider to be integration and unit tests):

• Routes: Typically you aren’t going to change the URLs in your application (because then you would be dealing with potential search engine optimisation and bookmarking issues) so if they do change it’s a good indication that (in an MVC application) you’ve added a new route definition that has broken some existing definitions. I will typically test the URLs for all controller actions (including mapping route values to action input parameters) and test that calling Url.Route with the same parameters results in the same route being generated. This kind of testing has saved my bacon many times in the past and it is quick and effective to write the tests whenever adding a new controller action using the code I use (a version of MVCContrib.TestHelper that I modified to use NSubstitute). It’s particularly handy when you have areas in your application because they have a nasty tendency of breaking your routes.
• Controllers: If any particular controller action is more than a few lines of code (and thus it isn’t bleedingly obviously that it’s either correct or not correct just by looking at it), then like all complex business logic I try and unit test the controllers. I do this using the Fluent MVC Testing library that I created because it’s terse and easy to write these tests.
• Controller Instantiation: As previously blogged.
• Filters, Filter Providers and Model Binders: I will typically unit test these, for an example see my previous post about unit testing filter providers.
• UI Testing: If the circumstances of the project allow for it then well-placed full-stack tests provide a lot of extra confidence. My fellow TestStacker Mehdi Khalili has a brilliant slide deck on this.
• Database Layer: I perform tests against individual queries / commands / repository methods by doing a full integration test against a real database and wrapping the whole thing in a Transaction Scope so I can roll-back any changes in the test teardown. This ensures the test database always remains clean (read: empty) and each test can work in isolation.
• Database migrations: I like to include one test that brings the test database to version 0 and then back up to the latest version so that if you ever need to use your down migrations then you have confidence they work (as well as continuing confidence that all of your up migrations work).
• Business logic / services / domain logic: I will always unit test this
• Infrastructure code: Where practical I try and unit test this. Sometimes this code is tested as a result of other tests e.g. the database tests and controller instantiation tests

If there are any particular types of testing that you would like me to do a more detailed blog post feel free to add a comment below.

ASP.NET Web API

The tests above have a nice mixture of unit and integrations tests and I find that they combine to provide a good level of confidence and coverage as the codebase evolves. I have only really played around with Web API in my spare time so far so the following recommendations are likely to evolve over time, but this is my current gut feel about this kind of testing.

Firstly, there are a few good posts I came across that give a broad idea of the different ways in which you can test Web API:

• Unit Test ASP.NET Web API
• Testing routes in ASP.NET Web API
• ASP.Net Web Api Integration Tests With Self Hosting

And then my initial approach / thoughts:

• Routes:The other day I stumbled across a library someone had created to do route testing for ASP.NET Web API and while I’m not a fan of the syntax they created (vs MVCContrib.TestHelper style syntax) it’s a good start in this direction. For the moment I’m thinking (as you will see) that I can take care of route testing implicitly. Furthermore, at least for my current project, the number of routes I’m dealing with doesn’t necessitate route testing. If the routes that you deal with are complex enough and or large enough in number then unit testing routes will likely provide a lot of value.
  Controllers: There is less need for a library like Fluent MVC Testing with Web API since most of your controllers will simply return the data that they queried directly and rely on formatters to give the correct response. This makes unit testing the controllers really simple. As for route testing the initial approach I’ve settled on will cover this testing anyway.
• Controller Instantiation: For the same reason this is valuable for ASP.NET MVC projects I think this is valuable (and just as easy to test). In my current project I haven’t bothered creating this yet since I’m only dealing with one controller with a couple of dependencies so I don’t have any code for this.
• Filters, etc.: I think there is value in unit testing these things in the same way as there is for MVC applications. In this instance the approach I’ve settled on tests these as well because it’s a full stack test. This is fine for simple filters / formatters etc., but if you have complex ones then I highly recommend unit testing them as well.
• UI Testing: This one is interesting, because at first thought there is no UI to test so this isn’t applicable. However, the way I see it, the equivalent in API tests are testing from the viewpoint of the client/consumer e.g. if I make a get request to this URL then I expect that object in JSON format. If you are producing an API according to a specification then this is the viewpoint the specification is written in and depending on how you do things, the acceptance criteria for the work you are doing will also be in terms of this. For these reasons, I think in some ways, while it is integration testing, testing the full stack from the viewpoint of a client provides a lot of bang for buck. As I will explain further below, this is the approach I have decided to initially take. I should note that the traditional pain points of UI testing for MVC applications (cross-browser differences, really slow, out of process so you can’t easily mock things, etc.) can easily be mitigated when doing this testing, which is why I find it a suitable approach in lieu of some of the other testing I might normally do.
• Database / business logic / infrastructure: None of this changes.

The approach I’ve taken for Web API testing

This is heavily based on the ASP.Net Web Api Integration Tests With Self Hosting post I linked to above. The problems I had with the code shown in that post was:

• The route was defined in the test rather than re-using the routes you define in your application
• The test itself is defined within a lamdba
• There was no abstraction of the code to set up the server
• The base URL was specified multiple times
• The test was really verbose (apart from not abstracting out the server it had a stream reader and WebRequest.Create etc.)

In the end I created a base class (using NUnit, but the same applies to any unit testing framework) that abstracted away the creation of the server as well as the HTTP call. This resulted in tests that were dead simple and very terse (for the moment there is only a get request, but you can easily add other methods and headers etc.). I also exposed the dependency injection so that I could insert mocks to control whether I want to unit test the controller action or integration test it with real dependencies. In this case it uses Autofac, but again the same applies for any DI framework.

Another thing to note is that I’ve baked in support for detecting if Fiddler is currently running and proxying requests through Fiddler in that instance. That way you can seemlessly move your test between your local computer (with or without Fiddler running) and your CI server without changing any code. The fact it proxies through Fiddler makes it really easy to debug exactly what is going on too.

Two other things to note are that you need to change PingController to a controller (or any class really) in the assembly that contains your API controllers and that this assumes you have created a static method inside the RouteConfig class called RegisterApiRoutes that defines the routes for your API.

using System;
using System.Net.Http;
using System.Web.Http.SelfHost;
using Autofac;
using Autofac.Integration.WebApi;
using NUnit.Framework;
using WebApiTesting.App_Start;
using WebApiTesting.Controllers;
namespace WebApiTesting.Tests.TestHelpers
{
    class WebApiTestBase
    {
        private HttpSelfHostServer _webServer;
        protected IContainer Container { get; private set; }
        protected ContainerBuilder ContainerBuilder { get; set; }
        protected Uri BaseUri { get; private set; }
        private bool _fiddlerActive = true;
        [TestFixtureSetUp]
        public void TestFixtureSetup()
        {
            var client = new HttpClient();
            try
            {
                client.GetAsync("http://ipv4.fiddler/").Wait();
            }
            catch (Exception)
            {
                _fiddlerActive = false;
            }
        }
        [SetUp]
        public virtual void Setup()
        {
            BaseUri = new Uri("http://localhost:3000");
            var config = new HttpSelfHostConfiguration(BaseUri);
            ContainerBuilder = ContainerBuilder ?? new ContainerBuilder();
            ContainerBuilder.RegisterApiControllers(typeof(PingController).Assembly);
            Container = ContainerBuilder.Build();
            RouteConfig.RegisterApiRoutes(config);
            config.DependencyResolver = new AutofacWebApiDependencyResolver(Container);
            _webServer = new HttpSelfHostServer(config);
            _webServer.OpenAsync().Wait();
        }
        protected HttpResponse PerformGetTo(string url)
        {
            var client = new HttpClient();
            var response = client.GetAsync((_fiddlerActive ? "http://ipv4.fiddler:3000/" : BaseUri.ToString()) + url).Result;
            var content = response.Content.ReadAsStringAsync().Result;
            return new HttpResponse { Content = content, Response = response };
        }
        [TearDown]
        public void Teardown()
        {
            _webServer.CloseAsync().Wait();
            _webServer.Dispose();
            Container.Dispose();
            ContainerBuilder = null;
            Container = null;
        }
    }
    class HttpResponse
    {
        public string Content { get; set; }
        public HttpResponseMessage Response { get; set; }
    }
}

So, how exactly do you use this class? Great question, but first a bit of context. My example will be for a endpoint at /api/ping that takes a GET request and returns the UTC system time in ISO format within an object with a property “Timestamp” along with a 200 OK if the database is up and running or a blank response with a 500 error if the database is not up and running.

Here is my test class:

using System;
using System.Net;
using Autofac;
using NSubstitute;
using NUnit.Framework;
using WebApiTesting.Infrastructure;
using WebApiTesting.Tests.TestHelpers;
namespace WebApiTesting.Tests.Controllers
{
    internal class PingControllerTests : WebApiTestBase
    {
        private IDateTimeProvider _dateTimeProvider;
        private IPingRepository _pingRepository;
        [SetUp]
        public override void Setup()
        {
            ContainerBuilder = new ContainerBuilder();
            _dateTimeProvider = Substitute.For<IDateTimeProvider>();
            _pingRepository = Substitute.For<IPingRepository>();
            ContainerBuilder.Register(c => _dateTimeProvider);
            ContainerBuilder.Register(c => _pingRepository);
            base.Setup();
        }
        [Test]
        public void GivenTheDatabaseIsUpAndRunning_WhenGetToPing_ReturnCurrentTimestampAnd200Ok()
        {
            _dateTimeProvider.Now().Returns(new DateTimeOffset(new DateTime(2012, 11, 4, 12, 20, 6), TimeSpan.Zero));
            var response = PerformGetTo("api/ping");
            Assert.That(response.Response.StatusCode, Is.EqualTo(HttpStatusCode.OK));
            Assert.That(response.Content, Is.EqualTo(@"{""Timestamp"":""2012-11-04 12:20:06""}"));
        }
        [Test]
        public void GivenTheDatabaseIsNotRunning_WhenGetToPing_Return500ErrorWithNoContent()
        {
            _pingRepository.When(r => r.CheckDatabase()).Do(a => { throw new Exception(); });
            var response = PerformGetTo("api/ping");
            Assert.That(response.Response.StatusCode, Is.EqualTo(HttpStatusCode.InternalServerError));
            Assert.That(response.Content, Is.EqualTo(string.Empty));
        }
    }
}

There are a few things to note:

• To register the listener to the port you need to run Visual Studio (and your CI test runner) as admin otherwise you will get a could not register port error.
• I used port 3000, but if you are running as admin and get an error saying that port can’t be registered it then you might have something else on that port so feel free to change.
• Unfortunately I couldn’t make use of AutoSubstitute, but unfortunately that broke the HTTP server (I imagine because it was giving mocks for things that the server needed real stuff for). The downside of this is that I have to keep track of the dependencies of the controller within the test making the test more verbose.
• Because I need the container builder before starting up the web server I have to create the container builder in the setup method and register everything that I need to mock for any of the tests there, rather than setting up what needs to be mocked in each individual test.
• I am setting up and shutting down the web server for each test to isolate them, but if there are a lot of tests this might be too slow and attaching start up to fixture setup or even test assembly set up might be a better option.
• As mentioned above for now there is only a PerformGetTo method, but it would be easy to support the other things needed.

I’ve uploaded the example source code to Github. Feel free to fork it and play around. If you have any suggestions or improvements feel free to add a comment :)

Enjoy!

Read More

I think that NSubstitute is an absolutely amazing library! I’ve used a few different mocking libraries and NSubstitute is hands down the best one. Not only is the syntax beautiful and terse, but it’s the simplest mocking library you could ever use. Don’t know the difference between a mock and a stub? WHO CARES! It’s not something you should need to know. I couldn’t explain it better than the NSubstitute homepage:

Mock, stub, fake, spy, test double? Strict or loose? Nah, just substitute for the type you need!

NSubstitute is designed for Arrange-Act-Assert (AAA) testing, so you just need to arrange how it should work, then assert it received the calls you expected once you’re done. Because you’ve got more important code to write than whether you need a mock or a stub.

99 times out of 100 you will have nothing but pleasure with NSubstitute, but like all things it’s not perfect. When I first started using NSubstitute I didn’t realise how it was implemented and just assumed it used some sort of crazy voodoo magic. More recently, I’ve actually contributed to NSubstitute to fix a bug I was experiencing. After doing so, I understand a lot more now how it actually works. It turns out it’s not magic, but it is very clever.

One of the things that can trip you up about the way it’s implemented is due to the fact that .Returns and Arg.Any/Is are based on a static stack and thus sometimes if you do things like set up a received check or return value for a  non-virtual method or set up a return value using the result of a substituted call then what the code looks like it should do doesn’t actually happen. When this does happen, if you don’t understand the implementation of what’s going on then this can be incredibly difficult to diagnose and you will generally spend a few confused hours.

This recently happened for some of the devs on my team and I wanted to try and explain to them what was happening so I came up with a gist that shows some examples for .Returns of the effect the static stack implementation has. I haven’t done any for Arg.Is/Any at this stage.

If you come across problems with NSubstitute or you need to explain to other people this behaviour feel free to use my gist. It uses NUnit and all of the tests in the gist passed when I ran it (against NSubstitute 1.4.3).

Read More

I’d like to announce that I’m joining TestStack and my FluentMCVTesting library will become a TestStack library.

What is TestStack?

I think it can be summed up nicely by a post from one of the core TestStack team, Michael Whelan, in the blog post he wrote. tl;dr: TestStack is a collection of open source projects which mainly focus on making the left side of Marrick’s Agile Testing Quadrant easier!

Why am I joining TestStack?

I think that the libraries in TestStack are awesome and the core team are a group of people I trust and respect. It’s a pleasure to be invited into the core team and I think it’s really great for the FluentMVCTesting library to be present alongside the other awesome testing libraries.

What will happen to FluentMVCTesting?

The nuget package and namespace is changing to have a TestStack prefix for consistency with other TestStack libraries, but other than that nothing much - the library is the same library, but it’s more likely to have additions made to it and hopefully more people using it.

If you update the existing FluentMVCTesting package then it will include the new package as a dependency and wrap around the new classes, but mark the old classes as deprecated. This should provide a smooth transition to existing users of the package as well as letting them know there is a new package out there.

I’m also dropping the strong signing support for now to keep consistency with the other TestStack libraries. We will add that in due course to all the libraries if we feel there is enough demand for it.

Read More

One of the things I find frustrating with ASP.NET MVC out of the box is creating forms. That’s not to say I find it more frustrating than other frameworks or libraries though; on the contrary I think it’s painful almost always. I think ASP.NET MVC has an amazingly powerful and useful capacity to handle forms. However, out of the box I find it fiddly and annoying to output the HTML associated with the form in a consistent and maintainable way across a whole website.

The other problem occurs when you want to change the layout of the form. For instance, imagine I create a proof-of-concept application using Twitter Bootstrap to get up and running with something that looks nice quickly. I may well want to scrap the application because it turns out to not be popular or useful. On the other hand, if it became really popular then I might want to spend some more time and effort customising the look-and-feel. At that point I will more than likely want to move away from the HTML template that Twitter Bootstrap imposes on you to use a more semantic markup (e.g. definition lists).

Enter Chameleon Forms

For the last 2 years, while I was working at Curtin University I was lucky enough to be able to use a library that we developed for tersely and maintainably developing forms that conformed to the forms standard that we had at Curtin. We had the advantage that the library was able to be specific to the Curtin standard for forms and didn’t have to be very generic. We still ensured that the library was very flexible though, because we want to make sure we could use it for most of the use cases we came across and still be able to break out into straight HTML for the weird edge cases.

ChameleonForms is my attempt to use the learnings and concepts from the previous forms library I was using and apply them to a new, open source library that I can use on my personal and professional projects to ease the pain of creating forms and improve the maintainability of them.

I am implementing this library alongside Matt Davies, who I often work closely with.

This blog post is announcing the initial release of this library, which contains basic functionality only. There is more information about the library, an example of how to use it and the roadmap we have for it at the Github page.

If you want to install this initial version from NuGet then simply use:

Install-Package ChameleonForms

Enjoy!

Read More

Earlier in the year I was developing some business logic to integrate social (in this case Facebook) sign-in with a website’s existing “local account”. At the time I created a set of acceptance criteria in conjunction with the product owner in order to work through the rather complex logic that results (as you will see there are quite a few permutations and it gets quite complex to express this understandably).

I recently started working on specifying some work that involves social sign-in integration and it occurred to me that the previous work I had done is probably quite relevant and reusable in other situations. To that end I’ve included the acceptance criteria below in case it’s helpful for anyone.

Note: Don’t hesitate to adjust it according to your needs; some of it is specific to the implementation that I did earlier in the year. I would say it’s worth using it as a base to consider some of the possibilities when linking your website’s login with social sign-in.

    GIVEN a user is not logged in
    AND doesn't have a "local account"
    WHEN the user authenticates their facebook account
    THEN a new "local account" will be created for them linked to their facebook account
    AND the user will be logged into the new "local account"
     
     
    GIVEN a user is logged in
    AND has not linked a facebook account
    WHEN the user authenticates their facebook account
    THEN the user will be prompted for their password to link their facebook account
     
     
    GIVEN a user is logged in
    AND has linked their facebook account
    WHEN the user authenticates their facebook account
    THEN they will be redirected to the Home page
     
     
    GIVEN a user is not logged in
    AND has linked a facebook account
    WHEN the user authenticates their facebook account
    THEN the user is logged into their "local account"
     
     
    GIVEN a user has linked a facebook account
    AND a second user is logged in
    WHEN the second user tries to authenticate that facebook account
    THEN the second user will be displayed a message indicating that facebook account is linked to another account
     
     
    GIVEN a user is logged in
    AND has a facebook email that matches the email of a non-Facebook-linked "local account"
    WHEN the user authenticates their facebook account
    THEN the user will be displayed a message indicating that email address is used by another user
     
     
    GIVEN a user is not logged in
    AND has a facebook email that matches the email of a non-Facebook-linked "local account"
    WHEN the user authenticates their facebook account
    THEN the user will be prompted for their password to link their facebook account and log in
     
     
    GIVEN a user is logged in
    AND has not linked a facebook account
    AND they have authenticated their facebook account
    WHEN the user submits their password
    THEN the user's facebook account will be linked to their "local account"
     
     
    GIVEN a user is not logged in
    AND has a facebook email that matches the email of a non-Facebook-linked "local account"
    AND they have authenticated their facebook account
    WHEN the user submits their password
    THEN the user's facebook account will be linked to their "local account"
    AND they will be logged in

Read More