Maintainable, large-scale continuous delivery with TeamCity Blog Series

I’ve been a ardent supporter of continuous delivery since I first learnt about it from a presentation by Martin Fowler and Jez Humble. At the time I loved how it encouraged less risky deployments and changed the decision of when/what to deploy from being a technical decision to being a business decision.

I personally think that embracing continuous delivery is an important intermediate step on the journey towards moving from technical agility to strategic agility.

This post was first written in August 2012, but has since been largely rewritten in February 2014 to keep it up to date.

This post forms the introduction to a blog series that is jointly written by myself and Matt Davies.

  1. Intro
  2. TeamCity deployment pipeline
  3. Deploying Web Applications
    • MsDeploy (onprem and Azure Web Sites)
    • OctopusDeploy (nuget)
    • Git push (Windows Azure Web Sites)
  4. Deploying Windows Services
    • MsDeploy
    • OctopusDeploy
    • Git push (Windows Azure Web Sites Web Jobs)
  5. Deploying Windows Azure Cloud Services
    • OctopusDeploy
    • PowerShell
  6. How to choose your deployment technology

Continuous Delivery with TeamCity

One of the key concepts in continuous delivery is the creation of a clear deployment pipeline that provides a clear set of sequential steps or stages to move software from a developer’s commits to being deployed in production.

We have been using TeamCity to develop deployment pipelines that facilitate continuous delivery for the last few years. While TeamCity is principally a continuous integration tool, it is more than adequate for creating a deployment pipeline and comes with the advantage that you are then using a single tool for build and deployment.

We have also tried combining TeamCity with other tools that are more dedicated to deployments, such as OctopusDeploy. Those tools provide better deployment focussed features such as visualisation of the versions of your application deployed to each environment. This approach does create the disadvantage of needing to rely on configuring and using two separate tools rather than just one, but can still be useful depending on your situation.

There are a number of articles that you will quickly come across in this space that give some really great advice on how to set up a continuous delivery pipeline with TeamCity and complement our blog series:

Purpose of this blog series

The purpose of this series is three-fold:

  • Document any findings that myself and Matt Davies have found from implementing continuous delivery pipelines using TeamCity that differ from the articles above;
  • Outline the techniques we have developed to specifically set up the TeamCity installation in a way that is maintainable for a large number of projects;
  • Cover the deployment of Windows Services and Azure Roles as well as IIS websites.

Our intention is to build of the work of our predecessors rather than provide a stand-alone series on how to set up continuous delivery pipelines with TeamCity.

Other options

TeamCity is by no means the only choice when it comes to creating a deployment pipeline. Feel free to explore some of the other options:

Windows Azure Web Sites (App Service: Web Apps) vs Web Roles

Welcome! This is a post that covers in great detail the differences between Windows Azure Web Sites (App Service: Web Apps as they are now called – this article will just refer to them by their original name) and Windows Azure Web Roles (azurewebsites.net vs cloudapp.net). It was first written a couple of days after Web Sites was first released, but I have been keeping it up to date as changes to Web Sites are announced. There is a change log of updates at the bottom of the post and I have struck out content that becomes out of date over time. There is also a good official Microsoft post that gives a similar comparison, but  not quite as comprehensive that you can also refer to.

Shameless Plug: If you are interested in using the power of Azure Web Roles with the nice deployment experience of Azure Web Sites then check out the AzureWebFarm and AzureWebFarm.OctopusDeploy libraries I work on.

Diagrammatic Overview

The Windows Azure Training Kit has a slide with a really nice overview of the three options for deploying web applications in Windows Azure (note: Virtual Machines are outside the scope of this post):

Azure Web Apps Options Overview
Options for deploying websites in Azure and comparison to on-premise hosting.

tl;dr

Azure Web Sites has the following advantages:

  • Out of the box support for ASP.NET, python, PHP, node.js and classic ASP
  • Out of the box support for near-instant Git, VSO, FTP, Web Deploy and Dropbox deployments
  • Out of the box support for configuring a VSO build that can compile code and run tests as part of the deployment
  • Instant roll-backs (when using Git or VSO deployments)
  • Free, out-of-the-box valid HTTPS when using default azurewebsites.net domain
  • Out-of-the-box manual and automated backup and restore
  • Much faster site provisioning and scaling (often takes seconds)
  • Simpler Visual Studio solution (no need for extra Cloud project)
  • One-click install of common open-source blog and CMS platforms
  • Friendlier, comprehensive in-built (read: one-click to activate) monitoring and diagnostics
  • Automatic configuration modification and config transforms on deploy
  • Multiple websites (deployed separately) on a single farm out of the box
  • Memory dumps via a REST API
  • Out-of-the-box support for remote debugging from Visual Studio
  • Ability to have staging environment with custom domain name and ability to change configuration settings at the same time
  • Always On Support to keep your application pool warmed up and (optionally) auto-heal any problems in your app pool
  • Multiple programming language support for periodic and continuous background jobs
  • Ability to view processes running the machine and their CPU and memory use from the Azure Portal and remotely kill rogue processes
  • Ability to add pre-packaged extensibility via Site Extensions
  • Ability to connect to on-premise resources securely without needing VPN via Hybrid connections
  • In-built A/B testing / canary deployment ability using Testing in Production  (TiP) / Traffic Routing feature

Azure Web Roles has the following advantages:

  • Use non-standard HTTP and/or TCP ports
  • Remote desktop into the VMs
  • Faster Windows Azure Virtual Network connection
  • Run arbitrary startup scripts with or without elevated privileges
  • Use Azure Drive to mount NTFS volumes from blob storage
  • More control over what diagnostic information you would like to collect
  • Full control to configure IIS
  • Perform complex autoscaling
  • Scale to hundreds of VM instances (max in Web Sites is currently 10)
  • Use Extra Small (low cost) and A4 (Extra Large) / A5 / A6 / A7 / A8 / A9 (intense workloads), D-series and G-series VM sizes
  • Install SSL for unlimited custom domain names for free
  • Resolve arbitrary domain names to your site without specifying them in the portal
  • Configure affinity groups and make use of upgrade and fault domains
  • Use the CDN against your web site content
  • In-role cache nodes
  • 99.95% SLA as opposed to a 99.9% SLA
  • You get more than 50GB of disk space
  • Ability to use GDI+, Report Viewer and DNS lookups in your code
  • Ability to make calls to localhost
  • Access to performance counters

Sometimes Virtual Machines are a valid choice – check out my whitepaper on deploying web applications in Azure for more information.

Conclusion

If you need enormous scale, a non-standard/complex configuration (e.g. full IIS customisation, non-standard ports, complex diagnostics or start up scripts / custom software installs), more than 50GB of disk space or RDP then you need Web Roles.

In most other scenarios Web Sites is the best choice since it’s the simplest option and is fully-managed. The Web Sites platform is being constantly innovated on with new features surfacing almost every month – Microsoft are investing a lot of resources into making sure it’s a state of the art platform. If your application grows in complexity over time and needs to migrate from Web Sites to Web Roles or Web Roles to Virtual Machines this is a relatively easy process so you should be able to start with the simplest option initially (Web Sites) with confidence you can change later if required.

Web Sites

Long-story short: Convention-over-configuration IIS-as-a-service to get you up and running quickly in Azure with a minimum of fuss (kind of like App Harbor). Also, it allows you to use a certain amount of Azure resources for free (10 websites on shared instances with limited bandwidth), which is a welcome addition and will be great for proof of concepts or just playing around with stuff!

The advantages that Web Sites seems to have over Web Roles are:

  • Out of the box support for Git (including hosted within Azure, CodePlex, Github and Bitbucket), VSOFTP and Web Deploy and Dropbox deployments, which makes automated deployments and continuous delivery dead easy (not that it’s terribly difficult using PowerShell for Web Roles), and importantly, quick.
    • Annoyingly (unless I’ve missed something), the git publishing doesn’t use private keys so you have to keep typing in the password, although from watching David Ebbo’s Youtube video on publishing from Git there must be some way to make it remember the password (I wouldn’t know as I always use private keys). [See update at end of post for instructions]
    • It would be cool if some of these features were added to Web Roles and Worker Roles; I’m not sure if this tweet from David Ebbo is any indication that might happen!
    • It seems that Microsoft have open sourced their Git-IIS deployment library, Kudu, which is cool 🙂
    • I tried to deploy a solution that had multiple web projects, but no matter what I did it kept deploying the first web project that I had deployed. I’m not sure what conventions it uses, but if you need to configure which project is deployed then I imagine you can consult the relevant Kudu documentation (edit: looks like there are some ways to address it).
    • If you use the Git publishing model (and I assume the TFS publishing model) then you don’t have to commit your NuGet package binaries as they will automatically get resolved when you push (you do have to turn on Package Restore on the solution in Visual Studio for it to do it though – edit: package restore is being deprecated). This is great because it makes your deployment much faster. It does mean that if you use packages that aren’t in the official NuGet repository (e.g. in a private repository, or using a package service) then you will need to include those ones or update the nuget config – I haven’t tried seeing what happens when you do that though yet, so try with caution.
  • Gives you the ability to roll-back to previous deployments instantly via the management portal if you are using Git (and I assume TFS) deployments.
    Redeploy previous deployments
  • You can deploy almost instantly out of the box (with the above mentioned deployment methods), rather than in 8-12 minutes or having to set up something like Accelerator for Web Roles Azure Web Farm or AzureWebFarm.OctopusDeploy.
  • You can provision a new Web Site much faster than a Web Role (seconds rather than 8+ minutes); I’m guessing Microsoft will always have spare capacity spun up and ready at any point in time – their capacity management software must be fascinating!
  • For .NET web apps: You don’t have to set up a separate project in your Visual Studio solution to deploy to Azure – you can simply deploy a normal web app solution as-is.
    • This also means you don’t have to maintain two sets of configurations and set up a storage account for diagnostics so you can be up and running much quicker.
  • Out of the box support for installing ready-to-go, open source web applications (from a list of common ones; mainly CMS’s and blogs).Create Azure Web Site with CMS
  • Built-in monitoring from the management portal (Web Roles has CPU and (optionally) memory as well now, but this also includes HTTP errors, requests and data)Web Sites has out of the box support for per-minute diagnostics now, which you can get the data for with Web Roles, but it’s all in-built into the portal for Web Sites and easier to use / consume:
    Monitoring dashboard
  • Extra diagnostics options at the click of a button:
    Azure Web Sites Diagnostics Options
  • Ability to inject configuration variables to your web.config file rather than having to package the final version with the deployment.
    • This means you don’t have to use encrypted web.config files if you want to keep production configuration information in source control so you can do automated deployments, but still prevent developers from accessing the production environment.
    • It does mean you potentially need to be more careful about who you give access to the management portal for your instances because the connection strings will appear in plain text be accessible to them.
  • If you have a web.release.config file then the XDT transformations in this file will automatically be applied before deploying your site.
  • You can add multiple host names that your site responds to (if it’s using reserved instances standard or shared tier, but not for shared free tier).
    • It appears you have to do this if you want to point a CName record at the site – given there are multiple sites being hosted on the same IIS instances this makes sense.
    • Given Web Roles allow you to point as many domain names as you want at it and it will resolve for all of them (assuming you are trying to resolve the main web site that is deployed in the Azure package rather than using a solution like Accelerator for Web Roles Azure Web Farm, which does require you to specify the host name) this isn’t really an advantage that Web Sites has over Web Roles.
    • What it does mean though is that you can host multiple sites on a single set of reserved servers out of the box (which is why Web Sites deprecated Accelerator for Web Roles in combination with near-instant deployments).
    • There is an implication if you are using shared free tier rather than reserved standard or shared tiers that you can’t point custom domain names to your web site and thus have to use the azurewebsites.net domain name (which is probably fine if you are doing a prototype or proof of concept).
    • It seems that for a given subscription in a given region you can have either free/shared or reserved Web Sites and thus any web sites that you add for a given subscription / region that has reserved Web Sites will be shared on your reserved servers.
      • I can’t confirm this explicitly, but it’s the only thing that makes sense to me after seeing the management portal and reading between the lines in the documentation.Web Site ModeShared to ReservedReserved to Shared
      • I would suggest that it will be important to keep this in mind if you want to segregate the servers that certain combinations of web sites that you deploy. You might want to do this due to their load profiles conflicting with each other. This is important unless Microsoft provides an easy way in the future to segregate your reserved instances within a single subscription or provide an easy way to migrate a site to a different subscription or region (unless I am missing something!).
  • You can use MySQL (edit: You can use this with Web Roles easily too now by adding a MySQL database using Add-ons).
  • You only need one instance to get the SLA rather than 2 instances in Web Roles
  • It supports multiple programming languages out of the box
  • It takes seconds rather than minutes to scale up and out as well as changing tier.
  • You can host your “naked” domain name (i.e. without www.) in a supported way rather than having to resort to external services, clever programmatic solutions or being careful not to nuke your deployment.
  • You can get memory dumps via a REST API via Kudu
  • You can perform remote debugging from Visual Studio
  • You can keep your application pool constantly warmed up with Always On Support
  • As part of the Always On support you can define triggers that result in your app pool recycling automatically to auto-heal any problems in your website
  • You can run periodic or continuous background jobs in multiple languages out of the box and you can deploy them as part of deploying your web application
  • You can use a staging environment and immediately switch the files to production, while changing connection strings etc. plus you can assign a custom DNS name to your staging environment rather than the random <guid>.cloudapp.net domain name for the Web Role staging slot (see the below comment from joshka though)
  • You get free, valid HTTPS out-of-the-box with the azurewebsites.net domain
  • You get out-of-the-box support for manual and automated backup and restore
  • Ability to view processes running the machine and their CPU and memory use from the Azure Portal and remotely kill rogue processes
  • Ability to add pre-packaged extensibility to your web site at the click of a button via Site Extensions
  • You can connect to on-premise resources securely without needing a VPN using Hybrid Connections
  • There is an in-built function that allows you to perform A/B testing and canary deployments called Testing in Production (TiP) / Traffic Routing

Web Roles

With all that in mind, this is what I see as the advantages of / situations you would use Web Roles (a lot of those also apply to Virtual Machines, but that is outside the scope of this post):

  • If you need to create an application that has non standard HTTP ports open (or frankly ports open for non-HTTP traffic) you can’t use Web Sites.
  • You can’t RDP into the server for debugging or other purposes when using Web Sites.
  • If you want surety about network / virtual machine isolation of your application for security reasons you may not want to use (at the very least free/shared) Web Sites. This is a bit of conjecture though because without knowing the internals of how Web Sites is implemented it’s hard to say what kind of isolation it provides. (edit: VM isolation will be the same for standard tier, see next point about network isolation)
  • You can use Windows Azure Virtual Networks to get network isolation of your Web Roles, but this can’t be used with Web Sites (edit: you can connect an Azure Web Site to a Virtual Network using a Point-To-Site VPN connection, which is limited to 100 Mbps, whereas Web Roles can be directly connected to the network)
  • If you want to deploy a worker role to perform background processing alongside your web site via a worker role then you can do it using the same instance with a Web Role (thus costing less money; also, Azure Web Farm has the ability to support execution of console applications packaged with your web application providing the same kind of benefit). (edit: Web Sites now supports Web Jobs so this point is no longer relevant)
  • You can run elevated start up scripts with a Web Role that then allow you to install any software you want or configure it in any way you desire.
  • You can mount a NTFS filesystem stored in blob storage using Azure Drive
  • You have full control over what diagnostic information is collected from a Web Role, which you may want to use for auto scaling, or customised monitoring solutions.
  • You have full control over how IIS is configured in a Web Role; note: it’s worth noting that Microsoft are consistently opening up IIS configuration options from web.config.
  • Your Web Role endpoint resolves to a static IP address assuming you don’t nuke your deployment.  edit: the fact you can have A Record support means this can be achieved with Web Sites too. edit: both Web Sites and Web Roles allow for A-record support, with Web Roles you can reserve a static IP address.
  • You can automatically scale your web site using Web Roles. You can perform much more complex auto-scaling with Web Roles – the slider-based auto-scaling in the portal is much more comprehensive and allows CPU and Azure Storage Queue based scaling and time schedules (edit: Web Sites now has the ability to scale using time schedules in the portal) and if you want even more complex auto-scaling you can use the Autoscaling Application Block (aka WASABi)

    Azure Web Roles CPU portal autoscaling
    Web Roles CPU autoscaling
    Azure Web Roles Storage Queue portal autoscaling
    Web Roles queue autoscaling

    Azure Web Sites CPU portal autoscaling
    Web Sites CPU autoscaling
  • You can scale out to a very large number of instances for extremely high load web sites (I know of a situation where at least 250 instances were used). Web Sites (at least at the moment, and with the default configuration when you first get it – it’s potentially possible to increase this) seems to have a maximum of three instances has a maximum of 6 shared or 10 reserved standard instances.
  • Furthermore, as per above screen shot and at least for the moment, Web Sites doesn’t have Extra Small (great for cost effective hosting of low load web sites – but the free and shared instances make up for this) or A4 (Extra Large) / A5 / A6 / A7 / A8 / A9 / D-series / G-series (if you need extreme vertical scale – but you probably only need this for intense Worker Role processing rather than for web sites [unless you are doing something very wrong/weird with your web application :P]).
  • At the moment you can only have SSL connections to your custom domain name by using Web Roles since there is no ability to upload Security Certificates to your Web Site instances, but this will probably be added soon. There is SSL for the azurewebsites.net domain name that you get though. edit: SSL for custom domain names in Azure Web Sites has been released, but it comes at a cost, whereas you can install SSL to your Web Role for free.
  • If you want securely managed Security Certificates for any other purpose than SSL e.g. config encryption, authentication, etc. then you will need to use Web Roles (for now at least). edit: you can now use Azure Web Sites for arbitrary certificates.
  • Apparently, during the preview there is a limit of 1GB of space per subscription that can be used with Web Sites and you can have a maximum of 10 web sites. I found this in a Stack Overflow answer from a Microsoft employee working on Azure. Free and paid shared instances have a limit of 1GB of space with free instances having a limit of 10 websites and shared instances having a limit of 100 websites (reserved standard instances have 10GB of space and a 500 website limit [I wonder if this is an IIS limit otherwise why would it be there since you have a complete VM(s) at your disposal?]). With Accelerator for Web Roles Azure Web Farm and AzureWebFarm.OctopusDeploy there is no limit on the number of sites, and depending on your role size you can certainly have more than 1 GB of storage (apart from the fact you have have more than one hosted service, each with more than 1 GB of space).
  • If you want to have any domain name you want to resolve against your web site using a CName without having to specify those domain names explicitly in the management portal then you will need to use Web Roles (see the comments section below for explanation).
  • You have the option of using VIP swap as a deployment option – this allows you to fully test the deployment you are about to make and then switch between old and new very quickly (and more usefully, vice versa if something goes wrong) (edit: Web Sites now has a staging environment with similar (but more advanced) functionality)
    • While I think this is very useful in some circumstances, particularly if the act of deploying is accompanied by separate, long-running processes that could go wrong (e.g. complex database changes that are triggered before the vip swap) I think in general if you can simplify your deployment process such that it’s not required (and simply make it really quick and easy to rollback to the last version) then that’s better (Web Sites enables this due to it’s out-of-the-box deployment options)
  • Affinity groups to allow you to ensure that Web Roles are close to storage accounts and potentially other roles within the datacentre (decreasing latency and increasing performance).
    • I suspect that Web Sites may well do this behind the scenes for the storage account you associate with the Web Site, but of course you can only do it with that storage account and not with multiple ones or other sites / roles
    • You can create linked resources, which may (I can’t verify or find any information to back this up) provide the same effect – you can do this for Azure SQL Databases and storage accounts:
      Azure Web Sites Add Linked Resource Dialog
  • Explicit upgrade and fault domains
    • Web Sites makes upgrade domains redundant given the deployment options (see note above about VIP swap) and I assume that fault domains are likely transparently implemented for you, but that could be wrong
  • In-role cache nodes
  • The cscfg file and ability to edit in the portal
    • You can edit web.config variables and connection strings within the portal for Web Sites so this makes it somewhat redundant
  • If you want to use Traffic Manager to distribute your application in multiple data centres across the globe then you can’t use Web Sites, however you can get a similar affect using an Azure Virtual Machine and IIS ARR as per http://stackoverflow.com/questions/13697863/is-it-possible-to-provision-the-same-domain-name-on-multiple-azure-websites
  • If you want to use the Content Delivery Network against your web site rather than a static Storage Account then you need a Cloud Service and can’t use Web Sites.
  • Windows Azure Web Roles has an SLA (99.95%), but Web Sites has no SLA. Web Sites has a 99.9% SLA whereas Web Roles has a 99.95% SLA.
  • Web Roles can be easily deleted and redeployed from a package in blob storage so if you only need your site functioning for part of the day you could delete the deployment when it’s not needed and have a significant cost saving.
    • Web Roles are charged by the minute – I can’t work out if Web Sites are the same or not though, but I assume so since the pricing page talks about converting the usage to Compute hour equivalents.
  • Web Roles have 64-bit app pools by default – Web Sites are 32-bit by default and in fact you have to upgrade to standard mode to get a 64-bit app pool; it is possible to downgrade the Web Role app pool to 32-bit
  • If you want to use GDI+ or report viewer then that is not currently supported in Web Sites (but does work on Web Roles) – it is in the long term plan to add support and you can keep track of progress via the MSDN forums
  • If you want to use DNS lookups in code then that isn’t possible in Azure Web Sites (sorry – I can’t find a link for this one)
  • The regions that you can deploy Web Sites to are currently restricted while this feature is in preview, but there are probably plans to roll out wider (see point 3 in the link) (edit: it’s available in all regions except South East Asia – I can’t find any reason why it’s not available in that region).
  • You can make calls to localhost on the role, which isn’t possible in Azure Web Sites since it doesn’t use a * binding in IIS – instead you have to use the domain name of the site, which will go out through the Azure load balancer
  • You can access performance counters

While reading up on this post I found out something about Web Roles that I didn’t know – apparently you can deploy a Web Role package that contains multiple web sites (by specifying their host headers), but it does mean that you have to always deploy all the websites at the same time and keep them all in the same Visual Studio solution.

Updates

Update (10 June 2012)

David Ebbo send me a message via Twitter to let me know the solution he is using to remember his Git password. Also, I found a post that describes how to point to other NuGet feeds than the official one when using package restore.

Update (28 August 2012)

I read a truly excellent post yesterday about Virtual Networks and it made the point that you can’t join a Web Site to a Virtual Network, but of course you can join a VM or Web/Worker Role. Thus I added that as another limitation above.

Update (15 September 2012)

West US Azure data centres now support Web Sites.

Update (18 September 2012)

There were a few big announcements today about Azure Web Sites and I’ve updated this post with some minor corrections accordingly.

Update (9 October 2012)

Added some rather obvious differences I originally missed thanks to the comment below from Roland.

Update (1 January 2013)

Updated to reflect the fact that East Asia is now supported for Azure Web Sites (as per this announcement) and the fact you can now have up to 6 shared and 10 reserved instances rather than the original 3 (as per this announcement).

Update (27 April 2013)

Updated to add the fact that Traffic Manager isn’t supported for Web Sites.

Update (25 May 2013)

Updated to add a note about CDN support as pointed out by Teo below.

Update (17 July 2013)

Overhaul of the post to get it up to date with recent changes to Web Sites and Web Roles as well as a general tidy up and making it more friendly (e.g. added diagram, new intro and tl;dr section and moved around some of the points that were in the wrong section).

Update (8 September 2013)

Added information about GDI+/Report Viewer as pointed out by Travis below.

Update (26 January 2014)

Added information about Web Sites staging environment, Always On Support, DNS lookups, AzureWebFarm.OctopusDeploy and In-role cache and emphasised the 10GB limit for Azure Web Sites.

Update (29 January 2014)

Added note about the web.config options for configuring IIS that are increasingly being opened up for Azure Web Sites.

Update (8 February 2014)

Added notes about A8 and A9 VM sizes, 32-bit vs 64-bit app pools and auto healing.

Update (22 October 2014)

Updated post to reflect the following:

  • Out-of-the-box HTTPS with Azure Web Sites
  • Azure Web Sites Backup
  • Azure Web Sites Processes
  • D-series and G-series VMs for Azure Web Roles
  • VPN connection to a Virtual Network for Azure Web Sites
  • South East Asia available for Azure Web Sites and TFS support is now VSO support with a build as per Josh’s comment
  • Can’t make localhost calls on Azure Web Sites
  • Site Extensions for Azure Web Sites
  • Clarified that you can reserve a static IP for Web Roles for A record support now
  • Out-of-the-box Remote Debugging with Azure Web Sites

Update (31 October 2014)

Updated post to mention Hybrid Connections support in Azure Web Sites and the fact that you can use custom certificates in Azure Web Sites now.

Update (6 April 2015)

Added some notes about App Service: Web Apps and added information about the TiP feature.

Update (3 January 2016)

Added note about access to performance counters as pointed out by Matt below.

Windows Azure announcements

Microsoft made a HUGE set of announcements to do with Windows Azure the other night (well, day if you are in the US). With this release Microsoft are demonstrating how serious they are about Azure and cloud computing, which can only lead to more innovation and exciting developments! While nothing that Microsoft have announced is particularly unique (most of it is on offer from providers like AWS and App Harbor), it really tightens up the Microsoft offering (which as a PaaS offering was already impressive). If Microsoft continue to pump development into Azure at the rate they have done then the other providers better watch out!

For anyone that saw my tweets after I read the above linked post then you will know that I am EXTREMELY excited about this release. In particular, the things that really struck me were the new portal (the old one is slow, clunky and you have to have Silverlight so no tablet or phone support if you don’t have a Windows device), the monitoring dashboard (previously you had to roll your own) and the new Websites feature.

I have spent a lot of time in the last few months playing around with speeding up Azure deployments and I wasn’t terribly happy with the solutions available. For web roles I was using Windows Azure Accelerator for Web Roles in combination with web deploys from a CI server and for worker roles I was using a PowerShell command that did an in place upgrade. This lead to deployment times of around 30 seconds and 8 minutes respectively. I am happy with 30s (although the Accelerator has a few deployment problems that I have come across that make it less robust than I would like), but 8 minutes is way too long. I actually have an idea for getting fast deployments of worker roles using a similar technique to Accelerator that I have been cooking up with my team’s Senior Developer, Matt Davies, but I’ll leave that for another post.

As Matt and I expected, the new Azure announcements have left the Accelerator project in a deprecated state. I actually think there are still valid uses of it, but I’ll leave that for another post too.

So long story short, I’m really happy that Microsoft have done a lot of work on making deployments and multi-tenant website hosting on Azure a snap because it was one of the biggest frustrations of Azure. Certainly, it was this that lead to the creation of App Harbor. I’ve had a bit of a play with App Harbor over the last 6 months, and while there are certain aspects of it that are very cool, in particular if you aren’t using a CI server, I’m not a huge fan so the Azure announcement is even more welcome news.

Web Deploy Package Connection String Parameterisation Problem

I recently came across a problem using web deploy packages, and more specifically with connection string parameterisation. I was generating a web deploy package using the “Package” MSBuild command and then deploying to a remote server using MSDeploy. The problem I came across was that the parameterised connection strings weren’t being replaced.

That resulted in the deployed web.config file looking something like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  ...
  <connectionStrings>
      <add name="SomeConnectionString" connectionString="$(ReplacableToken_SomeConnectionString-Web.config Connection String_0)" />
  </connectionStrings>
  ...
</configuration>

I used a variety of approaches – DeployOnBuild MSBuild command, Visual Studio GUI interface, the generated .cmd file and direct msdeploy call – to deploy the package to check whether the problem was simply that I was somehow deploying it incorrectly, but they all resulted in the same problem.

Most of the Google results that came up simply said to turn off parameterisation, but in this instance I wanted to use parameterisation. While I was using XDT transformations, I also wanted to use connection string parameterisation so I could substitute the connection strings for entries in a password safe as part of an automated deployment step.

The build output wasn’t terribly helpful, so I passed the verbose option to msdeploy and got the following interesting output:

...
[16:08:33][Step 2/2] Verbose: Parameter entry 'SomeConnectionString-Web.config Connection String/1' could not be applied anywhere.
[16:08:33][Step 2/2] Verbose: The dependency check 'DependencyCheckInUse' found no issues.
[16:08:33][Step 2/2] Verbose: The synchronization completed in 2 pass(es).
[16:08:33][Step 2/2] Total changes: 10 (0 added, 0 deleted, 10 updated, 0 parameters changed, 1004778 bytes copied)
[16:08:33][Step 2/2] Process exited with code 0

This didn’t really make a huge amount of sense and after some sanity checking to make sure parameterisation does actually work (created a new ASP.NET website and deployed it successfully) I started comparing all the differences in my .csproj configuration to a blank project. After some tedious trial and error I managed to narrow the errant configuration to the following (seemingly) innocuous line:

<BaseIntermediateOutputPath>$(ProjectDir)..obj</BaseIntermediateOutputPath>

You might be asking why I had that line in there in the first place? It is certainly a bit weird and you would normally expect it to be set to “obj”. In this case it was there so that when MvcBuildViews was next executed after building a package it didn’t choke with the following error (on the intermediate web.config files in the obj folder):

It is an error to use a section registered as allowDefinition='MachineToApplication' beyond application level.  This error can be caused by a virtual directory not being configured as an application in IIS.

When I initially came across that problem I changed the location of the obj folder since it seemed the best option compared to the other options such as running clean before all builds or deleting the obj folder, both of which would slow down the build and be tedious and annoying. Also, I didn’t want to turn MvcBuildViews off since I want assurance on the build server that the views all compiled fine.

Since this was no longer an option (up until this point all web deploys were done manually so we hadn’t actually come across this new problem!) I came up with the following configuration as a solution (props to my Senior Dev, Matt Davies, for finding the relevant StackOverflow post for the inspiration):

  <Target Name="CleanPackage" BeforeTargets="MvcBuildViews" Condition="'$(MvcBuildViews)'=='true'">
    <CallTarget Targets="CleanWebsitesPackage" />
    <CallTarget Targets="CleanWebsitesPackageTempDir" />
    <CallTarget Targets="CleanWebsitesTransformParametersFiles" />
  </Target>
  <Target Name="MvcBuildViews" AfterTargets="Build" Condition="'$(MvcBuildViews)'=='true'">
    <AspNetCompiler VirtualPath="temp" PhysicalPath="$(WebProjectOutputDir)" />
  </Target>

Basically, if you have MvcBuildViews set to true then it will run MvcBuildViews directly after the build and before running MvcBuildViews it will run the CleanWebsitesPackage, CleanWebsitesPackageTempDir and CleanWebsitesTransformParametersFiles targets which clean up the web deploy package files and thus the web.config files that mess with MvcBuildViews.

Update (17 Feb 2013)

If you are trying to use the above MSBuild with Visual Studio 2012 then it won’t work. I have blogged a solution that works in both Visual Studio 2012 and Visual Studio 2010 though.

… A year later

It seems rather funny that it was exactly one year since I’ve done a post on my blog. Usual story of course, I started out with good intentions to regularly blog about all the cool stuff I discover along my journey, but time got the better of me. I guess they were the same intentions that I originally had to skin this blog :S.

One thing I have learnt over the last year is that prioritisation is one of the most important things you can do and abide by both personally and professionally. No matter what there will never be enough time to do all the things that you need and want to do so you just have to prioritise and get done all you can – what more can you ask of yourself. With that in mind I guess I haven’t prioritised my blog 😛

I really respect people that manage to keep up with regular blog posts as well as full-time work and other activities. I find that writing blog posts is really time consuming because the pedantic perfectionist in me strives to get every relevant little detail in there and ensure it’s all formatted correctly. Combining that with the insane number of things I seem to find myself doing and trying to get some relax time in somewhere isn’t terribly conducive. It’s a pity really because I enjoy writing posts and hopefully I contribute some useful information here and there.

So, that aside, what have I been doing for the last year. If you are interested feel free to peruse the below list, which has some of what I’ve been doing and is written in no particular order; it’s really just a brain dump ^^. There are a few posts that I have been intending on writing along the way with particularly interesting (to me at least) topics so I’ll try and write some posts over the next few days 🙂

  • Worked with out Project Management Office at work to come up with a way to use PRINCE 2 to provide high level project management to our Agile projects without impacting on the daily work that the teams perform under Scrum. Despite my early scepticism about PRINCE 2 it’s actually a really impressive and flexible project management framework and has worked well.
  • Learnt PowerShell – it’s amazing!
  • Wrote some interesting / powerful NuGet packages (not public I’m afraid) using PowerShell install scripts
  • Attended a really great conference
  • Discovered and started living and breathing (and evangelising) continuous delivery and dev ops
  • Started thinking about the concept of continuous design as presented by Mary Poppendieck at Yow
  • Created a continuous delivery pipeline for a side-project with a final prod deployment to Windows Azure controlled by the product owner at the click of a button with a 30-45s deployment time!
  • Started learning about the value of Lean thinking, in particular with operational teams
  • Started evangelising lean thinking to management and other teams at work (both software and non-software)
  • Started using Trello to organise pretty much everything (both for my team, myself personally and at work and various projects I’m working on in and out of work) – it’s AMAZING.
  • Delivered a number of interesting / technically challenging projects
  • Became a manager
  • Assisted my team to embark on the biggest project we’ve done to date
  • Joined a start-up company based in Melbourne in my spare time
  • Joined Linked in (lol; I guess it had to finally happen)
  • Gave a number of presentations
  • Became somewhat proficient in MSBuild (*shudders*) and XDT
  • Facilitated countless retrospectives including a few virtual retrospectives (ahh Trello, what would I do without you)
  • Consolidated my love for pretty much everything Jetbrains produce for .NET (in particular TeamCity 7 and ReSharper 6 are insanely good, I’ll forgive them for dotCover)
  • Met Martin Fowler and Mary and Tom Poppendieck
  • Participated in the global day of code retreat and then ran one for my team (along with a couple of Fedex days)
  • Got really frustrated with 2GB of RAM on my 3 year old computer at home after I started doing serious development on it (with the start-up) and upgraded to 6GB (soooo much better, thanks Evan!)
  • Participated on a couple of panels for my local Agile meetup group
  • Got an iPhone 4S 🙂 (my 3GS was heavily on the blink :S)
  • Took over as chairman of the young professionals committee for the local branch of the Institution of Engineering and Technology
  • Deepened my experience with Microsoft Azure and thoroughly enjoyed all the enhancements they have made – they have gone a long way since I first started in 2010!

Of course there is heaps more, but this will do for now.