Azure Web Sites: Paid shared hosting, custom domain names and continuous deployment

There was a set of announcements about Azure Web Sites today, which give some insight into how the feature is developing and the focus areas for Microsoft with this particular service.

The Announcements

In short:

  • A new paid shared hosting model to supplement the free one
  • Based on what Scott said in his post it looks like you can have an unlimited number of web sites in reserved mode (with the initial release it was 10)
    • I suspect this also applies to the paid shared hosting, but regardless at a point you would upgrade to reserved anyway since it’s not a per-site cost so it becomes cheaper than a whole heap of shared hosting sites
  • You can now use CNames against the (paid) shared hosting model
  • You can now use A records against (paid) shared hosting or against reserved hosting!
  • Support for doing continuous deployment from a Github or Codeplex account
    • This means you don’t need to push your changes to azure explicitly and can instead use azure as a bit of a CI server (although you can’t ensure tests pass first, but the TFS integration allows for that!)
    • This mimics similar functionality in App Harbor
    • There isn’t support for private keys yet so public repos only, but Scott did say that it will be supported soon 🙂

Note: today’s release supports establishing connections with public GitHub/CodePlex repositories.  Support for private repositories will be enabled in a few weeks.

  • Support for multiple branches, which can be used to provide test and production deployments from the same repository (pretty cool!)

SSL

The thing I’ve been waiting for the most with Azure Web Sites is the ability to use SSL with your custom domain name. Whilst it was unfortunately not announced today mention was made of it; more specifically:

We will also in the future enable SNI based SSL as a built-in feature with shared mode web-sites (this functionality isn’t supported with today’s release – but will be coming later this year to both the shared and reserved tiers).

This is particularly exciting because it’s the main reason I haven’t used Web Sites much yet!

Commentary

The announcement of the new paid shared hosting model is really interesting, but it makes a lot of sense. It does two things:

  • Provides an accessible service for people that need a custom domain name and more than 165MB/day of bandwidth, but can’t afford a small instance
  • Provides a monetisation strategy for Microsoft for people in the above situation that places them on-par with most standard hosting companies ($113.88/year), but of course the service they are providing and the performance is really great so it’s something that could really take off. That can only be a good thing; the more popular Azure is the more it can grow! Bring it on!

The continuous deployment announcement is cool, but not surprising; it’s more of the same stuff that has been coming out in releases and blog posts by the awesome people that are currently driving a lot of this stuff. It’s great that Microsoft is supporting Github as a first class citizen rather than just their own platform (Codeplex).

Interestingly for me is that yet again there is a set of features in Web Sites that aren’t exposed in Web Roles either. It’s still unclear whether Web Sites will eventually replace Web Roles. I don’t think that’s likely to happen at least in the short term because there is too many things that are different about them and all the documentation clearly places them as filling separate needs (Web Roles is for when you need more control / advanced use cases).

For instance, I don’t ever see them giving RDP for Web Sites given it’s multi-tenanted and your web site could be hosted on one of a whole heap of random servers at any point in time and that can (probably) change over time too.

The whole point is Web Sites is simple and abstracts all that complexity away from you. For that reason Web Sites is well targeted at less technically inclined people and thus is more accessible to more people (all of the tutorials and presentations and marketing is around how easy it is to get up and running with minimal code and configuration!) – a great thing!

So, given that Web Roles probably isn’t going anywhere I’m surprised that they haven’t released A Record and better deployment support to Web Roles.

Windows Azure High-level diagram

I recently stumbled across the Windows Azure Solution Cookbook by Clint Edmonson from Microsoft. It’s a pretty awesome set of reference architectures for solving a few different scenarios that Windows Azure is a natural fit for solving.

One of the things I noticed is that on the lead-in page there is what I can only describe as the most amazing high-level diagram of Azure that I’ve been able to find to date. It wasn’t quite up to date though, having been done before the release of Web Sites and Mobile Servies. I recently asked Clint if he could do an updated version via the magic of Twitter and he has come through.

If you need to give any presentations about Azure or frankly, just want to see an example of all the different (and many!) components that make up Azure these days and how they fit into each other than take a look!

Azure Architecture Overview

TeamCity deployment pipeline (part 1: structure)

TeamCity (in particular version 7 onwards) makes the creation of continuous delivery pipelines fairly easy. There are a number of different approaches that can be used though and there isn’t much documentation on how to do it.

This post outlines the set up that I have used for continuous delivery and also the techniques I have used to make it quick and easy to get up and running with new applications and to maintain the build configurations over multiple applications.

While this post is .NET focussed, the concepts here apply to any type of deployment pipeline.

Maintainable, large-scale continuous delivery with TeamCity series

This post is part of a blog series jointly written by myself and Matt Davies called Maintainable, large-scale continuous delivery with TeamCity:

  1. Intro
  2. TeamCity deployment pipeline
  3. Deploying Web Applications
    • MsDeploy (onprem and Azure Web Sites)
    • OctopusDeploy (nuget)
    • Git push (Windows Azure Web Sites)
  4. Deploying Windows Services
    • MsDeploy
    • OctopusDeploy
    • Git push (Windows Azure Web Sites Web Jobs)
  5. Deploying Windows Azure Cloud Services
    • OctopusDeploy
    • PowerShell
  6. How to choose your deployment technology

Designing the pipeline

When designing the pipeline we used at Curtin we wanted the following flow for a standard web application:

  1. The CI server automatically pulls every source code push
  2. When there is a new push the solution is compiled
  3. If the solution compiles then all automated tests (unit or integration – we didn’t have a need to distinguish between them for any of our projects as none of them took more than a few minutes to run) will be run (including code coverage analysis)
  4. If all the tests pass then the web application will be automatically deployed to a development web farm (on-premise)
  5. A button can be clicked to deploy the last successful development deployment to a UAT web farm (either on-premise or in Azure)
  6. A UAT deployment can be marked as tested and a change record number or other relevant note attached to that deployment to indicate approval to go to production
  7. A button can be clicked to deploy the latest UAT deployment that was marked as tested to production and this button is available to a different subset of people that can trigger UAT deployments

Two other requirements were that there is a way to visualise the deployment pipeline for a particular set of changes and also that there was an ability to revert a production deployment by deploying the last successful deployment if something went wrong with the current one. Ideally, each of the deployments should take no more than a minute.

The final product

The following screenshot illustrates the final result for one of the projects I was working on:

Continuous delivery pipeline in TeamCity dashboard

Some things to notice are:

  • The continuous integration step ensures the solution compiles and that any tests run; it also checks code coverage while running the tests – see below for the options I use
  • I use separate build configurations for each logical step in the pipeline so that I can create dependencies between them and use templates (see below for more information)
    • This means you will probably need to either buy an enterprise TeamCity license if you are putting more than two or three projects on your CI server (or spin up new servers for each two or three projects!)
  • I prefix each build configuration name with a number that illustrates what step it is relative the the other build configurations so they are ordered correctly
  • I postfix each build configuration with an annotation that indicates whether it’s a step that will be automatically triggered or that needs to be manually triggered for it to run (by clicking the corresponding “Run…” button)
    • I wouldn’t have bothered with this if TeamCity had a way to hide Run buttons for various build steps.
    • You will note that the production deployments have some additional instructions as explained below. This keeps consistency that the postfix between the “[” and “]”are user instructions
    • In retrospect, for consistency I should have made the production deployment say “[Manual; Last pinned prod package]”
  • The production deployment is in a separate project altogether
    • As stated above – one of my requirements that a different set of users were to have access to perform production deployments
    • At this stage TeamCity doesn’t have the ability to give different permissions on a build configuration level – only on a project level, which effectively forced me to have a separate project to support this
    • This was a bit of a pain and complicates things, so if you don’t have that requirement then I’d say keep it all in one project
  • I have split up the package step to be separate from the deployment step
    • In this case I am talking about MSDeploy packages and deployment, but a similar concept might apply for other build and deployment scenarios
    • The main reason for this is for consistency with the production package, which had to be separated from the deployment for the reasons explained below under “Production deployments”
  • In this instance the pipeline also had a NuGet publish, which is completely optional, but in this case was needed because part of the project (business layer entities) was shared with a separate project and using NuGet allowed us to share the latest version of the common classes with ease

Convention over Configuration

One of the main concepts that I employ to ensure that the TeamCity setup is as maintainable as possible for a large number of projects is convention over configuration. This requires consistency between projects in order for them to work and as I have said previously, I think this is really important for all projects to have anyway.

These conventions allowed me to make assumptions in my build configuration templates (explained below) and thus make them generically apply to all projects with little or no effort.

The main conventions I used are:

  • The name of the project in TeamCity is {projectname}
    • This drives most of the other conventions
  • The name of the source code repository is ssh://git@server/{projectname}.git
    • This allowed me to use the same VCS root for all projects
  • The code is in the master branch (we were using Git)
    • As above
  • The solution file is at /{projectname}.sln
    • This allowed me to have the same Continuous Integration build configuration template for all projects
  • The main (usually web) project is at /{projectname}/{projectname}.csproj
    • This allowed me to use the same Web Deploy package build configuration template for all projects
  • The IIS Site name of the web application will be {projectname} for all environments
    • As above
  • The main project assembly name is {basenamespace}.{projectname}
    • In our case {basenamespace} was Curtin
    • This allowed me to automatically include that assembly for code coverage statistics in the shared Continuous Integration build configuration template
  • Any test projects end in .Tests and built a dll ending in .Tests in the binRelease folder of the test project after compilation
    • This allowed me to automatically find and run all test assemblies in the shared Continuous Integration build configuration template

Where necessary, I provided ways to configure differences in these conventions for exceptional circumstances, but for consistency and simplicity it’s obviously best to stick to just the conventions wherever possible. For instance the project name for the production project wasn’t {projectname} because I had to use a different project and project names are unique in TeamCity. This meant I needed a way to specify a different project name, but keep the project name as the default. I explain how I did this in the Build Parameters section below.

Build Configuration templates

TeamCity gives you the ability to specify a majority of a build configuration via a shared build configuration template. That way you can inherit that template from multiple build configurations and make changes to the template that will propagate through to all inherited configurations. This is the key way in which I was able to make the TeamCity setup maintainable. The screenshot below shows the build configuration templates that we used.

Build Configuration Templates

Unfortunately, at this stage there is no way to define triggers or dependencies within the templates so some of the configuration needs to be set up each time as explained below in the transition sections.

The configuration steps for each of the templates will be explained in the subsequent posts in this series apart from the Continuous Integration template, which is explained below. One of the things that is shared by the build configuration template is the VCS root so I needed to define a common Git root (as I mentioned above). The configuration steps for that are outlined below.

Build Parameters

One of the truly excellent things about TeamCity build configuration templates are how they handle build parameters.

Build parameters in combination with build configuration templates are really powerful because:

  • You can use build parameters in pretty much any text entry field through the configuration; including the VCS root!
    • This is what allows for the convention over configuration approach I explained above (the project name, along with a whole heap of other values, is available as build parameters)
  • You can define build parameters as part of the template that have no value and thus require you to specify a value before a build configuration instance can be run
    • This allows you to create required parameters that must be specified, but don’t have a sensible default
    • When there are parameters that aren’t required and don’t have a sensible default I set their value in the build configuration template to an empty string
  • You can define build parameters as part of the template that have a default value
  • You can overwrite any default value from a template within a build configuration
  • You can delete any overwritten value in a build configuration to get back the default value from the template
  • You can set a build configuration value as being a password, which means that you can’t see what the value is after it’s been entered (instead it will say %secure:teamcity.password.{parametername}%)
  • Whenever a password parameter is referenced from within a build log or similar it will display as ***** i.e. the password is stored securely and is never disclosed
    • This is really handy for automated deployments e.g. where service account credentials need to be specified, but you don’t want developers to know the credentials
  • You can reference another parameter as the value for a parameter
    • This allows you to define a common set of values in the template that can be selected from in the actual build configuration without having to re-specify the actual value. This is really important from a maintainability point of view because things like server names and usernames can easily change
    • When referencing a parameter that is a password it is still obscured when included in logs 🙂
  • You can reference another parameter in the middle of a string or even reference multiple other parameter values within a single parameter
    • This allows you to specify a parameter in the template that references a parameter that won’t be specified until an actual build configuration is created, which in turn can reference a parameter from the template.
    • When the web deploy post in this series is released you will be able to see an example of what I mean.
  • This is how I managed to achieve the flexible project name with a default of the TeamCity project name as mentioned above
    • In the template there is a variable called env.ProjectName that is then used everywhere else and the default value in the build configuration template is %system.teamcity.projectName%
    • Thus the default is the project name, but you have the flexibility to override that value in specific build configurations
    • Annoyingly, I had to specify this in all of the build configuration templates because there is no way to have a hierarchy of templates at this time
  • There are three types of build parameters: system properties, environment variables and configuration parameters
    • System properties are defined by TeamCity as well as some environment variables
    • You can specify both configuration parameters and environment variables in the build parameters page
    • I created a convention that configuration parameters would only be used to specify common values in the templates and I would only reference environment variables in the build configuration
    • That way I was able to create a consistency around the fact that only build parameters that were edited within an actual build configuration were environment variables (which in turn may or may not reference a configuration parameter specified in the template)
    • I think this makes it easier and less confusing to consistently edit and create the build configurations

Snapshot Dependencies

I make extensive use of snapshot dependencies on every build configuration. While not all of the configurations need the source code (since some of them use artifact dependencies instead) using snapshot dependencies ensures that the build chain is respected correctly and also provides a list of pending commits that haven’t been run for that configuration (which is really handy to let you know what state everything is in at a glance).

The downside of using snapshot dependencies though is that when you trigger a particular build configuration it will automatically trigger every preceding configuration in the chain as well. That means that if you run, say, the UAT deployment and a new source code push was just made then that will get included in the UAT deployment even if you weren’t ready to test it. In practice, I found this rarely if ever happened, but I can imagine that for a large and / or distributed team it could do so watch out for it.

What would be great to combat this was if TeamCity had an option for snapshot dependencies similar to artifact dependencies where you can choose the last successful build without triggering a new build.

Shared VCS root

The configuration for the shared Git root we used is detailed in the below screenshots. We literally used this for every build configuration as it was flexible enough to meet our needs for every project.

Git VCS Root Configuration
Git VCS Root Configuration 2

You will note that the branch name is specified as a build parameter. I used the technique I described above to give this a default value of master, but allow it to be overwritten for specific build configurations where necessary (sometimes we spun up experimental build configurations against branches).

Continuous Integration step configuration

A lot of what I do here is covered by the posts I referenced in the first post of the series apart from using the relevant environment variables as defined in the build configuration parameters. Consequently, I’ve simply included a few basic screenshots below that cover the bulk of it:

Continuous Integration Build Configuration Template 1 Continuous Integration Build Configuration Template 2 Continuous Integration Build Configuration Template - Build Step 1 Continuous Integration Build Configuration Template - Build Step 2 Continuous Integration Build Configuration Template - Build Step 2 (part 2) Continuous Integration Build Configuration Template - Build Triggering Continuous Integration Build Configuration Template - Build Parameters

Some notes:

  • If I want to build a releasable dll e.g. for a NuGet package then I have previously used a build number of %env.MajorVersion%.%env.MinorVersion%.{0} in combination with the assembly patcher and then exposed the dlls as build artifacts (to be consumed by another build step that packages a nuget package using an artifact dependency)
    • Then whenever I want to increment the major or minor version I adjust those values in the build parameters section and the build counter value appropriately
    • With TeamCity 7 you have the ability to include a NuGet Package step, which eliminates the need to do it using artifact dependencies
    • In this case that wasn’t necessary so the build number is a lot simpler and I didn’t necessarily need to include the assembly patcher (because the dlls get rebuilt when the web deployment package is built)
  • I set MvcBuildViews to false because the MSBuild runner for compiling views runs as x86 when using “Visual Studio (sln)” in TeamCity and we couldn’t find an easy way around it and thus view compilation fails if you reference 64-bit dlls
    • We set MvcBuildViews to true when building the deployment package so any view problems do get picked up quickly
  • Be careful using such an inclusive test dll wildcard specification; if you make the mistake of referencing a test project from within another test project then it will pick up the referenced project twice and try and run all the tests from it twice
  • The coverage environment variable allows projects that have more than one assembly that needs code coverage to have those extra dependencies specified
    • If you have a single project then you don’t need to do anything because the default configuration picks up the main assembly (as specified in the conventions section above)
    • Obviously, you need to change “BaseNamespace” to whatever is appropriate for you
    • I’ve left it without a default value so you are prompted to add something to it (to ensure you don’t forget when first setting up the build configuration)
  • The screens that weren’t included were left as their default, apart from Version Control Settings, which had the shared VCS root attached

Build configuration triggering and dependencies

The following triggers and dependencies are set up on the pipeline to set up transitions between configurations. Unfortunately, this is the most tedious part of the pipeline to set up because there isn’t a way to specify the triggers as part of a build configuration template. This means you have to manually set these up every time you create a new pipeline (and remember to set them up correctly).

  • Step 1.1: Continuous Integration
    • VCS Trigger – ensures the pipeline is triggered every time there is a new source code push; I always use the default options and don’t tweak it.
  • Step 2.1: Dev package
    • The package step has a build trigger on the last successful build of Step 1 so that dev deployments automatically happen after a source code push (if the solution built and the tests passed)
    • There is a snapshot dependency on the last successful build of Step 1 as well
  • Step 2.2: Dev deployment
    • In order to link the deployment with the package there is a build trigger on the last successful build of the package
    • There is also a snapshot dependency with the package step
    • They also have an artifact dependency from the same chain so the web deployment package that was generated is copied across for deployment; there will be more details about this in the web deploy post of the series
  • Step 3.1: UAT package
    • There is no trigger for this since UAT deployments are manual
    • There is a snapshot dependency on the last successful dev deployment so triggering a UAT deployment will also trigger a dev deployment if there are new changes that haven’t already been deployed
  • Step 3.2: UAT deploy
    • This step is marked as [Manual] so the user is expected to click the Run button for this build to do a UAT deployment
    • It has a snapshot on the UAT package so it will trigger a package to be built if the user triggers this build
    • There is an artifact dependency similar to the dev deployment too
    • There is also a trigger on successful build of the UAT package just in case the user decides to click on the Run button of the package step instead of the deployment step; this ensures that these two steps are always in sync and are effectively the same step
  • Step 4.1: Production package
    • See below section on Production deployments
  • Step 5: Production deployment
    • See below section on Production deployments

Production deployments

I didn’t want a production build to accidentally deploy a “just pushed” changeset so in order to have a separation between the production deployment and the other deployments I didn’t use a snapshot dependency on the production deployment step.

This actually has a few disadvantages:

  • It means you can’t see an accurate list of pending changesets waiting for production
    • I do have the VCS root attached so it shows a pending list, which will mostly be accurate, but will be cleared when you make a production deployment so if there were changes that weren’t being deployed at that point then they won’t show up in the pending list of the next deployment
  • It’s the reason I had to split up the package and deployment steps into separate build configurations
    • This in turn added a lot of complexity to the deployment pipeline because of the extra steps as well as the extra dependency and trigger configuration that was required (as detailed above)
  • The production deployment doesn’t appear in the build chain explicitly so it’s difficult to see what build numbers a deployment corresponds too and to visualise the full chain

Consequently, if you have good visibility and control over what source control pushes occur it might be feasible to consider using a snapshot dependency for the production deployment and having the understanding that this will push all current changes to all environments at the same time. In our case this was unsuitable, hence the slightly more complex pipeline. If the ability to specify a snapshot dependency without triggering all previous configurations in the chain was present (as mentioned above) this would be the best of both worlds.

Building the production package still needs a snapshot dependency because it requires the source code to run the build. For this reason, I linked the production package to the UAT deployment via a snapshot dependency and a build trigger. This makes some semantic sense because it means that any UAT deployment that you manually trigger then becomes a release candidate.

The last piece of the puzzle is the bit that I really like. One of the options that you have when choosing an artifact dependency is to use the last pinned build. When you pin a build it is a manual step and it asks you to enter a comment. This is convenient in multiple ways:

  • It allows us to mark a given potential release candidate (e.g. a built production package) as an actual release candidate
  • This means we can actually deploy the next set of changes to UAT and start testing it without being forced to do a production deployment yet
  • This gives the product owner the flexibility to deploy whenever they want
  • It also allows us to make the manual testing that occurs on the UAT environment an explicit step in the pipeline rather than an implicit one
  • Furthermore, it allows us to meet the original requirement specified above that there could be a change record number or any other relevant notes about the production release candidate
  • It also provides a level of auditing and assurance that increases the confidence in the pipeline and the ability to “sell” the pipeline in environments where you deal with traditional enterprise change management
  • It means we can always press the Run button against production deployment confident in the knowledge that the only thing that could ever be deployed is a release candidate that was signed off in the UAT environment

Archived template project

I explained above that the most tedious part of setting up the pipeline is creating the dependencies and triggers between the different steps in the pipeline. There is a technique that I have used to ease the pain of this.

One thing that TeamCity allows you to do is to make a copy of a project. I make use of this in combination with the ability to archive projects to create one or more archived projects that form a “project template” of sorts that strings together a set of build configuration templates including the relevant dependencies and triggers.

At the very least I will have one for a project with the Continuous Integration and Dev package and deployment steps already set up. But, you might also have a few more for other common configurations e.g. full pipeline for an Azure website as well as full pipeline for an on-premise website.

Furthermore, I actually store all the build configuration templates against the base archived project for consistency so I know where to find them and they all appear in one place.

Archived Project Template with Build Configuration Templates

Web server configuration

Another aspect of the convention over configuration approach that increases consistency and maintainability is the configuration of the IIS servers in the different environments. By configuring the IIS site names, website URLs and server configurations the same it made everything so much easier.

In the non-production environments we also made use of wildcard domain names to ensure that we didn’t need to generate new DNS records or SSL certificates to get up and running in development or UAT. This meant all we had to do was literally create a new pipeline in TeamCity and we were already up and running in both those environments.

MSBuild import files

Similarly, there are certain settings and targets that were required in the .csproj files of the applications to ensure that the various MSBuild commands that were being used ran successfully. We didn’t want to have to respecify these every time we created a new project so we created a set of .targets files in a pre-specified location (c:msbuild_curtin in our case -we would check this folder out from source control so it could easily be updated; you could also use a shared network location to make it easier to keep up to date everywhere) that contained the configurations. That way we simply needed to create a single import directive in the .csproj (or .ccproj) that included the relevant .targets file and we were off and running.

The contents of these files will be outlined in the rest of the posts in this blog series.

Build numbers

One of the things that is slightly annoying by having separate build configurations is that by default they all use different build numbers so it’s difficult to see at a glance what version of the software is deployed to each environment without looking at the build chains view. As it turns out, there are a number of possible ways to copy the build number from the first build configuration to the other configurations. I never got a chance to investigate this and figure out the best approach though.

Harddrive space

One thing to keep in mind is that if you are including the deployment packages as artifacts on your builds (not to mention the build logs themselves!) the amount of harddrive space used by TeamCity can quickly add up. One of the cool things in TeamCity is that if you are logged in as an admin it will actually pop up a warning to tell you when the harddrive space is getting low. Regardless, there are options in the TeamCity admin to set up clean-up rules that will automatically clean up artifacts and build history according to a specification of your choosing.

Production database

One thing that isn’t immediately clear when using TeamCity is that by default it ships with a file-based database that isn’t recommended for production use. TeamCity can be configured to support any one of a number of the most common database engines though. I recommend that if you are using TeamCity seriously that you investigate that.

Update 7 September 2012: Rollbacks

I realised that there was one thing I forgot to address in this post, which is the requirement I mentioned above about being able to rollback a production deployment to a previous one. It’s actually quite simple to do – all you need to do is go to your production deployment build configuration, click on the Build Chains tab and inspect the chains to see which deployment was the last successful one. At that point you simply expand the chain and then click on the TeamCity trigger previous build as custom build button button to open the custom build dialog and then run it.

NHibernate.SqlAzure: Transient fault-handling NHibernate driver for use with Windows Azure SQL Database

I would like to announce the release of a software library I have written in conjunction with Matt Davies as open source via NuGetGithub: NHibernate.SqlAzure.

When using Windows Azure SQL Database (as it’s now known: what a mouthful! :P) one of the things that you need to account for is transient errors that come up as part of the fact you are using a high availability database.

This essentially involves looking at the error code returned from the database and if it’s one of a set of numbers then retrying the communication (be it a SQL command or a connection establishment) again with some sort of retry pattern. Obviously, you don’t want to just retry every query because the error that comes back might be a legitimate error.

Microsoft have helpfully released a library as part of Enterprise Library called the Transient Fault Handling Application Block that handles these transient errors as well as various retry patterns via the ReliableSqlConnection class.

While there are lots of articles out there about integrating this library with Entity Framework and Linq to SQL there isn’t really any guidance for NHibernate; only a few stack overflow posts and similar.

I’ve been working with Matt over the last few months to develop an NHibernate driver that will automatically make use of the transient connection and command retry logic within the Transient Fault Handling Application Block in an unobtrusive manner. This means that you can add one line of code to your existing applications (be it Fluent NHibernate or XML configuration) to make use of this feature. The library works fine for local connections (since none of the error codes it is looking for will usually get thrown by a non SQL Azure database (and frankly if they did then there is no harm in retrying!).

The other cool thing about this integration is that you can make use of it for other retry situations by specifying a different transient error detection strategy and extending the ReliableSql2008ClientDriver class with a custom implementation. See the Github site for more information about that.

This library was actually fairly tricky to write because of a whole heap of explicit type-casts inside ADO.NET and NHibernate so I’m pretty proud of being able to release this!

One last note – I’ve been able to prove with automated tests that the library does actually handle transient errors by continuously pausing and restarting the SQL Express service while the test is running and it works beautifully!! 🙂

Enjoy!

Windows Azure Web Sites (App Service: Web Apps) vs Web Roles

Welcome! This is a post that covers in great detail the differences between Windows Azure Web Sites (App Service: Web Apps as they are now called – this article will just refer to them by their original name) and Windows Azure Web Roles (azurewebsites.net vs cloudapp.net). It was first written a couple of days after Web Sites was first released, but I have been keeping it up to date as changes to Web Sites are announced. There is a change log of updates at the bottom of the post and I have struck out content that becomes out of date over time. There is also a good official Microsoft post that gives a similar comparison, but  not quite as comprehensive that you can also refer to.

Shameless Plug: If you are interested in using the power of Azure Web Roles with the nice deployment experience of Azure Web Sites then check out the AzureWebFarm and AzureWebFarm.OctopusDeploy libraries I work on.

Diagrammatic Overview

The Windows Azure Training Kit has a slide with a really nice overview of the three options for deploying web applications in Windows Azure (note: Virtual Machines are outside the scope of this post):

Azure Web Apps Options Overview
Options for deploying websites in Azure and comparison to on-premise hosting.

tl;dr

Azure Web Sites has the following advantages:

  • Out of the box support for ASP.NET, python, PHP, node.js and classic ASP
  • Out of the box support for near-instant Git, VSO, FTP, Web Deploy and Dropbox deployments
  • Out of the box support for configuring a VSO build that can compile code and run tests as part of the deployment
  • Instant roll-backs (when using Git or VSO deployments)
  • Free, out-of-the-box valid HTTPS when using default azurewebsites.net domain
  • Out-of-the-box manual and automated backup and restore
  • Much faster site provisioning and scaling (often takes seconds)
  • Simpler Visual Studio solution (no need for extra Cloud project)
  • One-click install of common open-source blog and CMS platforms
  • Friendlier, comprehensive in-built (read: one-click to activate) monitoring and diagnostics
  • Automatic configuration modification and config transforms on deploy
  • Multiple websites (deployed separately) on a single farm out of the box
  • Memory dumps via a REST API
  • Out-of-the-box support for remote debugging from Visual Studio
  • Ability to have staging environment with custom domain name and ability to change configuration settings at the same time
  • Always On Support to keep your application pool warmed up and (optionally) auto-heal any problems in your app pool
  • Multiple programming language support for periodic and continuous background jobs
  • Ability to view processes running the machine and their CPU and memory use from the Azure Portal and remotely kill rogue processes
  • Ability to add pre-packaged extensibility via Site Extensions
  • Ability to connect to on-premise resources securely without needing VPN via Hybrid connections
  • In-built A/B testing / canary deployment ability using Testing in Production  (TiP) / Traffic Routing feature

Azure Web Roles has the following advantages:

  • Use non-standard HTTP and/or TCP ports
  • Remote desktop into the VMs
  • Faster Windows Azure Virtual Network connection
  • Run arbitrary startup scripts with or without elevated privileges
  • Use Azure Drive to mount NTFS volumes from blob storage
  • More control over what diagnostic information you would like to collect
  • Full control to configure IIS
  • Perform complex autoscaling
  • Scale to hundreds of VM instances (max in Web Sites is currently 10)
  • Use Extra Small (low cost) and A4 (Extra Large) / A5 / A6 / A7 / A8 / A9 (intense workloads), D-series and G-series VM sizes
  • Install SSL for unlimited custom domain names for free
  • Resolve arbitrary domain names to your site without specifying them in the portal
  • Configure affinity groups and make use of upgrade and fault domains
  • Use the CDN against your web site content
  • In-role cache nodes
  • 99.95% SLA as opposed to a 99.9% SLA
  • You get more than 50GB of disk space
  • Ability to use GDI+, Report Viewer and DNS lookups in your code
  • Ability to make calls to localhost
  • Access to performance counters

Sometimes Virtual Machines are a valid choice – check out my whitepaper on deploying web applications in Azure for more information.

Conclusion

If you need enormous scale, a non-standard/complex configuration (e.g. full IIS customisation, non-standard ports, complex diagnostics or start up scripts / custom software installs), more than 50GB of disk space or RDP then you need Web Roles.

In most other scenarios Web Sites is the best choice since it’s the simplest option and is fully-managed. The Web Sites platform is being constantly innovated on with new features surfacing almost every month – Microsoft are investing a lot of resources into making sure it’s a state of the art platform. If your application grows in complexity over time and needs to migrate from Web Sites to Web Roles or Web Roles to Virtual Machines this is a relatively easy process so you should be able to start with the simplest option initially (Web Sites) with confidence you can change later if required.

Web Sites

Long-story short: Convention-over-configuration IIS-as-a-service to get you up and running quickly in Azure with a minimum of fuss (kind of like App Harbor). Also, it allows you to use a certain amount of Azure resources for free (10 websites on shared instances with limited bandwidth), which is a welcome addition and will be great for proof of concepts or just playing around with stuff!

The advantages that Web Sites seems to have over Web Roles are:

  • Out of the box support for Git (including hosted within Azure, CodePlex, Github and Bitbucket), VSOFTP and Web Deploy and Dropbox deployments, which makes automated deployments and continuous delivery dead easy (not that it’s terribly difficult using PowerShell for Web Roles), and importantly, quick.
    • Annoyingly (unless I’ve missed something), the git publishing doesn’t use private keys so you have to keep typing in the password, although from watching David Ebbo’s Youtube video on publishing from Git there must be some way to make it remember the password (I wouldn’t know as I always use private keys). [See update at end of post for instructions]
    • It would be cool if some of these features were added to Web Roles and Worker Roles; I’m not sure if this tweet from David Ebbo is any indication that might happen!
    • It seems that Microsoft have open sourced their Git-IIS deployment library, Kudu, which is cool 🙂
    • I tried to deploy a solution that had multiple web projects, but no matter what I did it kept deploying the first web project that I had deployed. I’m not sure what conventions it uses, but if you need to configure which project is deployed then I imagine you can consult the relevant Kudu documentation (edit: looks like there are some ways to address it).
    • If you use the Git publishing model (and I assume the TFS publishing model) then you don’t have to commit your NuGet package binaries as they will automatically get resolved when you push (you do have to turn on Package Restore on the solution in Visual Studio for it to do it though – edit: package restore is being deprecated). This is great because it makes your deployment much faster. It does mean that if you use packages that aren’t in the official NuGet repository (e.g. in a private repository, or using a package service) then you will need to include those ones or update the nuget config – I haven’t tried seeing what happens when you do that though yet, so try with caution.
  • Gives you the ability to roll-back to previous deployments instantly via the management portal if you are using Git (and I assume TFS) deployments.
    Redeploy previous deployments
  • You can deploy almost instantly out of the box (with the above mentioned deployment methods), rather than in 8-12 minutes or having to set up something like Accelerator for Web Roles Azure Web Farm or AzureWebFarm.OctopusDeploy.
  • You can provision a new Web Site much faster than a Web Role (seconds rather than 8+ minutes); I’m guessing Microsoft will always have spare capacity spun up and ready at any point in time – their capacity management software must be fascinating!
  • For .NET web apps: You don’t have to set up a separate project in your Visual Studio solution to deploy to Azure – you can simply deploy a normal web app solution as-is.
    • This also means you don’t have to maintain two sets of configurations and set up a storage account for diagnostics so you can be up and running much quicker.
  • Out of the box support for installing ready-to-go, open source web applications (from a list of common ones; mainly CMS’s and blogs).Create Azure Web Site with CMS
  • Built-in monitoring from the management portal (Web Roles has CPU and (optionally) memory as well now, but this also includes HTTP errors, requests and data)Web Sites has out of the box support for per-minute diagnostics now, which you can get the data for with Web Roles, but it’s all in-built into the portal for Web Sites and easier to use / consume:
    Monitoring dashboard
  • Extra diagnostics options at the click of a button:
    Azure Web Sites Diagnostics Options
  • Ability to inject configuration variables to your web.config file rather than having to package the final version with the deployment.
    • This means you don’t have to use encrypted web.config files if you want to keep production configuration information in source control so you can do automated deployments, but still prevent developers from accessing the production environment.
    • It does mean you potentially need to be more careful about who you give access to the management portal for your instances because the connection strings will appear in plain text be accessible to them.
  • If you have a web.release.config file then the XDT transformations in this file will automatically be applied before deploying your site.
  • You can add multiple host names that your site responds to (if it’s using reserved instances standard or shared tier, but not for shared free tier).
    • It appears you have to do this if you want to point a CName record at the site – given there are multiple sites being hosted on the same IIS instances this makes sense.
    • Given Web Roles allow you to point as many domain names as you want at it and it will resolve for all of them (assuming you are trying to resolve the main web site that is deployed in the Azure package rather than using a solution like Accelerator for Web Roles Azure Web Farm, which does require you to specify the host name) this isn’t really an advantage that Web Sites has over Web Roles.
    • What it does mean though is that you can host multiple sites on a single set of reserved servers out of the box (which is why Web Sites deprecated Accelerator for Web Roles in combination with near-instant deployments).
    • There is an implication if you are using shared free tier rather than reserved standard or shared tiers that you can’t point custom domain names to your web site and thus have to use the azurewebsites.net domain name (which is probably fine if you are doing a prototype or proof of concept).
    • It seems that for a given subscription in a given region you can have either free/shared or reserved Web Sites and thus any web sites that you add for a given subscription / region that has reserved Web Sites will be shared on your reserved servers.
      • I can’t confirm this explicitly, but it’s the only thing that makes sense to me after seeing the management portal and reading between the lines in the documentation.Web Site ModeShared to ReservedReserved to Shared
      • I would suggest that it will be important to keep this in mind if you want to segregate the servers that certain combinations of web sites that you deploy. You might want to do this due to their load profiles conflicting with each other. This is important unless Microsoft provides an easy way in the future to segregate your reserved instances within a single subscription or provide an easy way to migrate a site to a different subscription or region (unless I am missing something!).
  • You can use MySQL (edit: You can use this with Web Roles easily too now by adding a MySQL database using Add-ons).
  • You only need one instance to get the SLA rather than 2 instances in Web Roles
  • It supports multiple programming languages out of the box
  • It takes seconds rather than minutes to scale up and out as well as changing tier.
  • You can host your “naked” domain name (i.e. without www.) in a supported way rather than having to resort to external services, clever programmatic solutions or being careful not to nuke your deployment.
  • You can get memory dumps via a REST API via Kudu
  • You can perform remote debugging from Visual Studio
  • You can keep your application pool constantly warmed up with Always On Support
  • As part of the Always On support you can define triggers that result in your app pool recycling automatically to auto-heal any problems in your website
  • You can run periodic or continuous background jobs in multiple languages out of the box and you can deploy them as part of deploying your web application
  • You can use a staging environment and immediately switch the files to production, while changing connection strings etc. plus you can assign a custom DNS name to your staging environment rather than the random <guid>.cloudapp.net domain name for the Web Role staging slot (see the below comment from joshka though)
  • You get free, valid HTTPS out-of-the-box with the azurewebsites.net domain
  • You get out-of-the-box support for manual and automated backup and restore
  • Ability to view processes running the machine and their CPU and memory use from the Azure Portal and remotely kill rogue processes
  • Ability to add pre-packaged extensibility to your web site at the click of a button via Site Extensions
  • You can connect to on-premise resources securely without needing a VPN using Hybrid Connections
  • There is an in-built function that allows you to perform A/B testing and canary deployments called Testing in Production (TiP) / Traffic Routing

Web Roles

With all that in mind, this is what I see as the advantages of / situations you would use Web Roles (a lot of those also apply to Virtual Machines, but that is outside the scope of this post):

  • If you need to create an application that has non standard HTTP ports open (or frankly ports open for non-HTTP traffic) you can’t use Web Sites.
  • You can’t RDP into the server for debugging or other purposes when using Web Sites.
  • If you want surety about network / virtual machine isolation of your application for security reasons you may not want to use (at the very least free/shared) Web Sites. This is a bit of conjecture though because without knowing the internals of how Web Sites is implemented it’s hard to say what kind of isolation it provides. (edit: VM isolation will be the same for standard tier, see next point about network isolation)
  • You can use Windows Azure Virtual Networks to get network isolation of your Web Roles, but this can’t be used with Web Sites (edit: you can connect an Azure Web Site to a Virtual Network using a Point-To-Site VPN connection, which is limited to 100 Mbps, whereas Web Roles can be directly connected to the network)
  • If you want to deploy a worker role to perform background processing alongside your web site via a worker role then you can do it using the same instance with a Web Role (thus costing less money; also, Azure Web Farm has the ability to support execution of console applications packaged with your web application providing the same kind of benefit). (edit: Web Sites now supports Web Jobs so this point is no longer relevant)
  • You can run elevated start up scripts with a Web Role that then allow you to install any software you want or configure it in any way you desire.
  • You can mount a NTFS filesystem stored in blob storage using Azure Drive
  • You have full control over what diagnostic information is collected from a Web Role, which you may want to use for auto scaling, or customised monitoring solutions.
  • You have full control over how IIS is configured in a Web Role; note: it’s worth noting that Microsoft are consistently opening up IIS configuration options from web.config.
  • Your Web Role endpoint resolves to a static IP address assuming you don’t nuke your deployment.  edit: the fact you can have A Record support means this can be achieved with Web Sites too. edit: both Web Sites and Web Roles allow for A-record support, with Web Roles you can reserve a static IP address.
  • You can automatically scale your web site using Web Roles. You can perform much more complex auto-scaling with Web Roles – the slider-based auto-scaling in the portal is much more comprehensive and allows CPU and Azure Storage Queue based scaling and time schedules (edit: Web Sites now has the ability to scale using time schedules in the portal) and if you want even more complex auto-scaling you can use the Autoscaling Application Block (aka WASABi)

    Azure Web Roles CPU portal autoscaling
    Web Roles CPU autoscaling
    Azure Web Roles Storage Queue portal autoscaling
    Web Roles queue autoscaling

    Azure Web Sites CPU portal autoscaling
    Web Sites CPU autoscaling
  • You can scale out to a very large number of instances for extremely high load web sites (I know of a situation where at least 250 instances were used). Web Sites (at least at the moment, and with the default configuration when you first get it – it’s potentially possible to increase this) seems to have a maximum of three instances has a maximum of 6 shared or 10 reserved standard instances.
  • Furthermore, as per above screen shot and at least for the moment, Web Sites doesn’t have Extra Small (great for cost effective hosting of low load web sites – but the free and shared instances make up for this) or A4 (Extra Large) / A5 / A6 / A7 / A8 / A9 / D-series / G-series (if you need extreme vertical scale – but you probably only need this for intense Worker Role processing rather than for web sites [unless you are doing something very wrong/weird with your web application :P]).
  • At the moment you can only have SSL connections to your custom domain name by using Web Roles since there is no ability to upload Security Certificates to your Web Site instances, but this will probably be added soon. There is SSL for the azurewebsites.net domain name that you get though. edit: SSL for custom domain names in Azure Web Sites has been released, but it comes at a cost, whereas you can install SSL to your Web Role for free.
  • If you want securely managed Security Certificates for any other purpose than SSL e.g. config encryption, authentication, etc. then you will need to use Web Roles (for now at least). edit: you can now use Azure Web Sites for arbitrary certificates.
  • Apparently, during the preview there is a limit of 1GB of space per subscription that can be used with Web Sites and you can have a maximum of 10 web sites. I found this in a Stack Overflow answer from a Microsoft employee working on Azure. Free and paid shared instances have a limit of 1GB of space with free instances having a limit of 10 websites and shared instances having a limit of 100 websites (reserved standard instances have 10GB of space and a 500 website limit [I wonder if this is an IIS limit otherwise why would it be there since you have a complete VM(s) at your disposal?]). With Accelerator for Web Roles Azure Web Farm and AzureWebFarm.OctopusDeploy there is no limit on the number of sites, and depending on your role size you can certainly have more than 1 GB of storage (apart from the fact you have have more than one hosted service, each with more than 1 GB of space).
  • If you want to have any domain name you want to resolve against your web site using a CName without having to specify those domain names explicitly in the management portal then you will need to use Web Roles (see the comments section below for explanation).
  • You have the option of using VIP swap as a deployment option – this allows you to fully test the deployment you are about to make and then switch between old and new very quickly (and more usefully, vice versa if something goes wrong) (edit: Web Sites now has a staging environment with similar (but more advanced) functionality)
    • While I think this is very useful in some circumstances, particularly if the act of deploying is accompanied by separate, long-running processes that could go wrong (e.g. complex database changes that are triggered before the vip swap) I think in general if you can simplify your deployment process such that it’s not required (and simply make it really quick and easy to rollback to the last version) then that’s better (Web Sites enables this due to it’s out-of-the-box deployment options)
  • Affinity groups to allow you to ensure that Web Roles are close to storage accounts and potentially other roles within the datacentre (decreasing latency and increasing performance).
    • I suspect that Web Sites may well do this behind the scenes for the storage account you associate with the Web Site, but of course you can only do it with that storage account and not with multiple ones or other sites / roles
    • You can create linked resources, which may (I can’t verify or find any information to back this up) provide the same effect – you can do this for Azure SQL Databases and storage accounts:
      Azure Web Sites Add Linked Resource Dialog
  • Explicit upgrade and fault domains
    • Web Sites makes upgrade domains redundant given the deployment options (see note above about VIP swap) and I assume that fault domains are likely transparently implemented for you, but that could be wrong
  • In-role cache nodes
  • The cscfg file and ability to edit in the portal
    • You can edit web.config variables and connection strings within the portal for Web Sites so this makes it somewhat redundant
  • If you want to use Traffic Manager to distribute your application in multiple data centres across the globe then you can’t use Web Sites, however you can get a similar affect using an Azure Virtual Machine and IIS ARR as per http://stackoverflow.com/questions/13697863/is-it-possible-to-provision-the-same-domain-name-on-multiple-azure-websites
  • If you want to use the Content Delivery Network against your web site rather than a static Storage Account then you need a Cloud Service and can’t use Web Sites.
  • Windows Azure Web Roles has an SLA (99.95%), but Web Sites has no SLA. Web Sites has a 99.9% SLA whereas Web Roles has a 99.95% SLA.
  • Web Roles can be easily deleted and redeployed from a package in blob storage so if you only need your site functioning for part of the day you could delete the deployment when it’s not needed and have a significant cost saving.
    • Web Roles are charged by the minute – I can’t work out if Web Sites are the same or not though, but I assume so since the pricing page talks about converting the usage to Compute hour equivalents.
  • Web Roles have 64-bit app pools by default – Web Sites are 32-bit by default and in fact you have to upgrade to standard mode to get a 64-bit app pool; it is possible to downgrade the Web Role app pool to 32-bit
  • If you want to use GDI+ or report viewer then that is not currently supported in Web Sites (but does work on Web Roles) – it is in the long term plan to add support and you can keep track of progress via the MSDN forums
  • If you want to use DNS lookups in code then that isn’t possible in Azure Web Sites (sorry – I can’t find a link for this one)
  • The regions that you can deploy Web Sites to are currently restricted while this feature is in preview, but there are probably plans to roll out wider (see point 3 in the link) (edit: it’s available in all regions except South East Asia – I can’t find any reason why it’s not available in that region).
  • You can make calls to localhost on the role, which isn’t possible in Azure Web Sites since it doesn’t use a * binding in IIS – instead you have to use the domain name of the site, which will go out through the Azure load balancer
  • You can access performance counters

While reading up on this post I found out something about Web Roles that I didn’t know – apparently you can deploy a Web Role package that contains multiple web sites (by specifying their host headers), but it does mean that you have to always deploy all the websites at the same time and keep them all in the same Visual Studio solution.

Updates

Update (10 June 2012)

David Ebbo send me a message via Twitter to let me know the solution he is using to remember his Git password. Also, I found a post that describes how to point to other NuGet feeds than the official one when using package restore.

Update (28 August 2012)

I read a truly excellent post yesterday about Virtual Networks and it made the point that you can’t join a Web Site to a Virtual Network, but of course you can join a VM or Web/Worker Role. Thus I added that as another limitation above.

Update (15 September 2012)

West US Azure data centres now support Web Sites.

Update (18 September 2012)

There were a few big announcements today about Azure Web Sites and I’ve updated this post with some minor corrections accordingly.

Update (9 October 2012)

Added some rather obvious differences I originally missed thanks to the comment below from Roland.

Update (1 January 2013)

Updated to reflect the fact that East Asia is now supported for Azure Web Sites (as per this announcement) and the fact you can now have up to 6 shared and 10 reserved instances rather than the original 3 (as per this announcement).

Update (27 April 2013)

Updated to add the fact that Traffic Manager isn’t supported for Web Sites.

Update (25 May 2013)

Updated to add a note about CDN support as pointed out by Teo below.

Update (17 July 2013)

Overhaul of the post to get it up to date with recent changes to Web Sites and Web Roles as well as a general tidy up and making it more friendly (e.g. added diagram, new intro and tl;dr section and moved around some of the points that were in the wrong section).

Update (8 September 2013)

Added information about GDI+/Report Viewer as pointed out by Travis below.

Update (26 January 2014)

Added information about Web Sites staging environment, Always On Support, DNS lookups, AzureWebFarm.OctopusDeploy and In-role cache and emphasised the 10GB limit for Azure Web Sites.

Update (29 January 2014)

Added note about the web.config options for configuring IIS that are increasingly being opened up for Azure Web Sites.

Update (8 February 2014)

Added notes about A8 and A9 VM sizes, 32-bit vs 64-bit app pools and auto healing.

Update (22 October 2014)

Updated post to reflect the following:

  • Out-of-the-box HTTPS with Azure Web Sites
  • Azure Web Sites Backup
  • Azure Web Sites Processes
  • D-series and G-series VMs for Azure Web Roles
  • VPN connection to a Virtual Network for Azure Web Sites
  • South East Asia available for Azure Web Sites and TFS support is now VSO support with a build as per Josh’s comment
  • Can’t make localhost calls on Azure Web Sites
  • Site Extensions for Azure Web Sites
  • Clarified that you can reserve a static IP for Web Roles for A record support now
  • Out-of-the-box Remote Debugging with Azure Web Sites

Update (31 October 2014)

Updated post to mention Hybrid Connections support in Azure Web Sites and the fact that you can use custom certificates in Azure Web Sites now.

Update (6 April 2015)

Added some notes about App Service: Web Apps and added information about the TiP feature.

Update (3 January 2016)

Added note about access to performance counters as pointed out by Matt below.

Windows Azure announcements

Microsoft made a HUGE set of announcements to do with Windows Azure the other night (well, day if you are in the US). With this release Microsoft are demonstrating how serious they are about Azure and cloud computing, which can only lead to more innovation and exciting developments! While nothing that Microsoft have announced is particularly unique (most of it is on offer from providers like AWS and App Harbor), it really tightens up the Microsoft offering (which as a PaaS offering was already impressive). If Microsoft continue to pump development into Azure at the rate they have done then the other providers better watch out!

For anyone that saw my tweets after I read the above linked post then you will know that I am EXTREMELY excited about this release. In particular, the things that really struck me were the new portal (the old one is slow, clunky and you have to have Silverlight so no tablet or phone support if you don’t have a Windows device), the monitoring dashboard (previously you had to roll your own) and the new Websites feature.

I have spent a lot of time in the last few months playing around with speeding up Azure deployments and I wasn’t terribly happy with the solutions available. For web roles I was using Windows Azure Accelerator for Web Roles in combination with web deploys from a CI server and for worker roles I was using a PowerShell command that did an in place upgrade. This lead to deployment times of around 30 seconds and 8 minutes respectively. I am happy with 30s (although the Accelerator has a few deployment problems that I have come across that make it less robust than I would like), but 8 minutes is way too long. I actually have an idea for getting fast deployments of worker roles using a similar technique to Accelerator that I have been cooking up with my team’s Senior Developer, Matt Davies, but I’ll leave that for another post.

As Matt and I expected, the new Azure announcements have left the Accelerator project in a deprecated state. I actually think there are still valid uses of it, but I’ll leave that for another post too.

So long story short, I’m really happy that Microsoft have done a lot of work on making deployments and multi-tenant website hosting on Azure a snap because it was one of the biggest frustrations of Azure. Certainly, it was this that lead to the creation of App Harbor. I’ve had a bit of a play with App Harbor over the last 6 months, and while there are certain aspects of it that are very cool, in particular if you aren’t using a CI server, I’m not a huge fan so the Azure announcement is even more welcome news.

Type-inferenced constants in abstract, generic base classes

Today I was creating a set of worker threads that periodically pull a whole heap of data from an on-premise database and upload it to Azure for processing. One of the things that these threads needed was the table name for the on-premise database table. This was obviously different for each thread implementation.

I had a base, abstract class that took the DTO class that was being abstracted as a generic type and performed all the logic within the thread (given it was the same for each implementation bar a couple of minor details).

The first way that we implemented the constant for the on-premise table name was via a protected, abstract property. This is nice, because it forces you to implement it for each implementation meaning you can’t forget it:

    public abstract class BaseBulkLoader : IBulkLoader where T : IBulkLoadedEntity
    {
        protected abstract string TableName { get; }
        ...
    }

This is a nice way of doing it, but then I had a situation where I had a second type of bulk processor for each entity type (I needed another one that periodically send the identifier for all current records so that a comparison could be made to delete any old records). At this point a magic string would either reside in both implementations (ugh) given the table name is the same, or the better solution would be to have a constants class (or config file) that contained the table name in a single place.

Even still, it seemed really messy to me to have to reference a constant class twice, given the base class already had the entity type and thus should be able to find the table name using that type somehow. Thus, my type-inferenced constants implementation was subsequently born:

    public static class BulkProcessorConstants
    {
        public static readonly List<Tuple<Type, string>> TableNames = new List<Tuple<Type, string>>
        {
            TableName<Entity1>("ENTITY_1_TABLE"),
            TableName<Entity2>("ENTITY_2_TABLE"),
        };

        #region Helper
        private static Tuple<Type, string> TableName<TEntity>(string s) where TEntity : IBulkLoadedEntity
        {
            return new Tuple<Type, string>(typeof(TEntity), s);
        }
        #endregion
    }

    public static class BulkProcessorConstants<T> where T: IBulkLoadedEntity
    {
        public static string Table {
            get
            {
                return BulkProcessorConstants.TableNames.Single(t => t.Item1 == typeof(T)).Item2;
            }
        }
    }

I could have simply done a set of if statements or a switch on the type name in the BulkProcessorConstants<T> class, but I wanted to make it really easy to read and modify the code so it was more maintainable.

With this implementation, all that is needed to get the table name in the base class is: BulkProcessorConstants<T>.Table!

Another nice side-effect of this strategy is that it groups all the magic strings together in a constants class rather than having them lying around in all the different implementations. If the table names change depending on deployment environment it would make sense to move them out to a config file.

… A year later

It seems rather funny that it was exactly one year since I’ve done a post on my blog. Usual story of course, I started out with good intentions to regularly blog about all the cool stuff I discover along my journey, but time got the better of me. I guess they were the same intentions that I originally had to skin this blog :S.

One thing I have learnt over the last year is that prioritisation is one of the most important things you can do and abide by both personally and professionally. No matter what there will never be enough time to do all the things that you need and want to do so you just have to prioritise and get done all you can – what more can you ask of yourself. With that in mind I guess I haven’t prioritised my blog 😛

I really respect people that manage to keep up with regular blog posts as well as full-time work and other activities. I find that writing blog posts is really time consuming because the pedantic perfectionist in me strives to get every relevant little detail in there and ensure it’s all formatted correctly. Combining that with the insane number of things I seem to find myself doing and trying to get some relax time in somewhere isn’t terribly conducive. It’s a pity really because I enjoy writing posts and hopefully I contribute some useful information here and there.

So, that aside, what have I been doing for the last year. If you are interested feel free to peruse the below list, which has some of what I’ve been doing and is written in no particular order; it’s really just a brain dump ^^. There are a few posts that I have been intending on writing along the way with particularly interesting (to me at least) topics so I’ll try and write some posts over the next few days 🙂

  • Worked with out Project Management Office at work to come up with a way to use PRINCE 2 to provide high level project management to our Agile projects without impacting on the daily work that the teams perform under Scrum. Despite my early scepticism about PRINCE 2 it’s actually a really impressive and flexible project management framework and has worked well.
  • Learnt PowerShell – it’s amazing!
  • Wrote some interesting / powerful NuGet packages (not public I’m afraid) using PowerShell install scripts
  • Attended a really great conference
  • Discovered and started living and breathing (and evangelising) continuous delivery and dev ops
  • Started thinking about the concept of continuous design as presented by Mary Poppendieck at Yow
  • Created a continuous delivery pipeline for a side-project with a final prod deployment to Windows Azure controlled by the product owner at the click of a button with a 30-45s deployment time!
  • Started learning about the value of Lean thinking, in particular with operational teams
  • Started evangelising lean thinking to management and other teams at work (both software and non-software)
  • Started using Trello to organise pretty much everything (both for my team, myself personally and at work and various projects I’m working on in and out of work) – it’s AMAZING.
  • Delivered a number of interesting / technically challenging projects
  • Became a manager
  • Assisted my team to embark on the biggest project we’ve done to date
  • Joined a start-up company based in Melbourne in my spare time
  • Joined Linked in (lol; I guess it had to finally happen)
  • Gave a number of presentations
  • Became somewhat proficient in MSBuild (*shudders*) and XDT
  • Facilitated countless retrospectives including a few virtual retrospectives (ahh Trello, what would I do without you)
  • Consolidated my love for pretty much everything Jetbrains produce for .NET (in particular TeamCity 7 and ReSharper 6 are insanely good, I’ll forgive them for dotCover)
  • Met Martin Fowler and Mary and Tom Poppendieck
  • Participated in the global day of code retreat and then ran one for my team (along with a couple of Fedex days)
  • Got really frustrated with 2GB of RAM on my 3 year old computer at home after I started doing serious development on it (with the start-up) and upgraded to 6GB (soooo much better, thanks Evan!)
  • Participated on a couple of panels for my local Agile meetup group
  • Got an iPhone 4S 🙂 (my 3GS was heavily on the blink :S)
  • Took over as chairman of the young professionals committee for the local branch of the Institution of Engineering and Technology
  • Deepened my experience with Microsoft Azure and thoroughly enjoyed all the enhancements they have made – they have gone a long way since I first started in 2010!

Of course there is heaps more, but this will do for now.