Breaking up software projects into small, focussed milestones

This post highlights a particular learning I’ve had over the last year about setting milestones for organising high-level goals for software projects.

Background

When I am first involved in a project there is undoubtedly a huge list of things that the product owner / client wants and it’s generally the case that there isn’t enough time or money for it all to be all completed. Let’s ignore for a moment the fact that there will be reasons why it’s not sensible to complete it all anyway since the low priority things will be much less important than high priority work on other projects or emergent requirements / user feedback for the current project.

Often, the person/people in question don’t really have a good understanding about what is realistic within a given time period (how could they when it’s hard enough for the development team?) and certainly the traditional way that software development has been executed exacerbates the issue by promising that a requirements specification will be delivered by a certain time. Despite the fact this almost never works people still sometimes expect software to be able to be delivered in this way. Thankfully, Agile software development becoming more mainstream means that I see this mentality occurring less and less.

An approach I’ve often taken in the past to combat this is to begin by getting product owners to prioritise high-level requirements with the MoSCoW system and then take the must-have’s and possibly the should-have’s and labelled that as phase 1 and continued to flesh out those requirements into user stories. This then leaves the less important requirements at the bottom of the backlog giving a clear expectation that any estimations and grooming effort will be expended on the most important features and won’t include “everything” that the person initially had in their vision (since that’s unrealistic).

This is a common-sense approach that most people seem to understand and allows for a open and transparent approach to providing a realistic estimate that can’t be misconstrued as being for everything. Note: I also use other techniques that help set clear expectations like user story mapping and inception decks.

The problem

These “phase 1” milestones have a number of issues:

  • They are arbitrary in make up and length, which results in a lack of focus and makes it easier for “scope creep” to occur
    • While scope-creep isn’t necessarily a problem if it’s being driven by prioritisation from a product owner it does tend to increase feedback cycles for user feedback and makes planning harder
    • Small variations to direction and scope tend to get hidden since they are comparatively small, however the small changes add up over time and can have a very large impact (positive and negative) on the project direction that isn’t intended
  • They tend to still be fairly long (3+ months)
    • This increases the size of estimation errors and the number and size of unknowns
    • I’ve noticed this also reduces the urgency/pace of everyone involved

A different approach

I’ve since learnt that a much better approach is to create really small, focused milestones that are named according to the goal they are trying to meet e.g. Allow all non-commercial customers who only have product X to use the new system (if you are doing a rewrite) or Let customers use feature Y (new feature to a system).

More specifically, having focused milestones:

  • Helps with team morale (everyone can see the end goal within their grasp and can rally around it)
  • Helps frame conversations with the product owner around prioritising stories to keep focus and not constantly increasing the backlog size (and by association how far away the end goal is)
  • Helps create more of a sense of urgency with everyone (dev team, ops, management etc.)
  • Helps encourage more frequent releases and thinking earlier about release strategies to real end users
  • Provides a nice check and balance against the sprint goal – is the sprint goal this sprint something that contributes towards our current milestone and in-turn are all the stories in the sprint backlog contributing to the sprint goal?

The end goal (probably not “end”; there is always room for improvement)

I don’t think that the approach I describe above is necessarily the best way of doing things. Instead I think it is a stepping stone for a number of things that are worth striving for:

Presentation: Moving from Technical Agility to Strategic Agility

I recently gave a presentation with my colleague Jess Panni to the ACS WA Conference about Agile and where we see it heading in the next 5-10 years. When offered the speaking slot the requirements were that it involved data analytics in some way and that it wasn’t the same old Agile stuff that everyone has been talking about for years, but rather something a bit different. Both of those requirements suited me because it fit in perfectly with thoughts I’d been having recently about Agile and where it is heading.

Jess and I had a lot of fun preparing the talk – it’s not often we get time to sit down and chat about process, research what the industry leaders are saying and brainstorm our own thoughts and experiences in light of that research. I’m very proud of the content that we’ve managed to assemble and the way we’ve structured it.

We paid particular care to make the slide deck useful for people – there are comprehensive notes on each slide and there are a bunch of relevant references at the end for further reading.

I’ve put the slides up on GitHub if you are interested.

TeamCity deployment pipeline (part 1: structure)

TeamCity (in particular version 7 onwards) makes the creation of continuous delivery pipelines fairly easy. There are a number of different approaches that can be used though and there isn’t much documentation on how to do it.

This post outlines the set up that I have used for continuous delivery and also the techniques I have used to make it quick and easy to get up and running with new applications and to maintain the build configurations over multiple applications.

While this post is .NET focussed, the concepts here apply to any type of deployment pipeline.

Maintainable, large-scale continuous delivery with TeamCity series

This post is part of a blog series jointly written by myself and Matt Davies called Maintainable, large-scale continuous delivery with TeamCity:

  1. Intro
  2. TeamCity deployment pipeline
  3. Deploying Web Applications
    • MsDeploy (onprem and Azure Web Sites)
    • OctopusDeploy (nuget)
    • Git push (Windows Azure Web Sites)
  4. Deploying Windows Services
    • MsDeploy
    • OctopusDeploy
    • Git push (Windows Azure Web Sites Web Jobs)
  5. Deploying Windows Azure Cloud Services
    • OctopusDeploy
    • PowerShell
  6. How to choose your deployment technology

Designing the pipeline

When designing the pipeline we used at Curtin we wanted the following flow for a standard web application:

  1. The CI server automatically pulls every source code push
  2. When there is a new push the solution is compiled
  3. If the solution compiles then all automated tests (unit or integration – we didn’t have a need to distinguish between them for any of our projects as none of them took more than a few minutes to run) will be run (including code coverage analysis)
  4. If all the tests pass then the web application will be automatically deployed to a development web farm (on-premise)
  5. A button can be clicked to deploy the last successful development deployment to a UAT web farm (either on-premise or in Azure)
  6. A UAT deployment can be marked as tested and a change record number or other relevant note attached to that deployment to indicate approval to go to production
  7. A button can be clicked to deploy the latest UAT deployment that was marked as tested to production and this button is available to a different subset of people that can trigger UAT deployments

Two other requirements were that there is a way to visualise the deployment pipeline for a particular set of changes and also that there was an ability to revert a production deployment by deploying the last successful deployment if something went wrong with the current one. Ideally, each of the deployments should take no more than a minute.

The final product

The following screenshot illustrates the final result for one of the projects I was working on:

Continuous delivery pipeline in TeamCity dashboard

Some things to notice are:

  • The continuous integration step ensures the solution compiles and that any tests run; it also checks code coverage while running the tests – see below for the options I use
  • I use separate build configurations for each logical step in the pipeline so that I can create dependencies between them and use templates (see below for more information)
    • This means you will probably need to either buy an enterprise TeamCity license if you are putting more than two or three projects on your CI server (or spin up new servers for each two or three projects!)
  • I prefix each build configuration name with a number that illustrates what step it is relative the the other build configurations so they are ordered correctly
  • I postfix each build configuration with an annotation that indicates whether it’s a step that will be automatically triggered or that needs to be manually triggered for it to run (by clicking the corresponding “Run…” button)
    • I wouldn’t have bothered with this if TeamCity had a way to hide Run buttons for various build steps.
    • You will note that the production deployments have some additional instructions as explained below. This keeps consistency that the postfix between the “[” and “]”are user instructions
    • In retrospect, for consistency I should have made the production deployment say “[Manual; Last pinned prod package]”
  • The production deployment is in a separate project altogether
    • As stated above – one of my requirements that a different set of users were to have access to perform production deployments
    • At this stage TeamCity doesn’t have the ability to give different permissions on a build configuration level – only on a project level, which effectively forced me to have a separate project to support this
    • This was a bit of a pain and complicates things, so if you don’t have that requirement then I’d say keep it all in one project
  • I have split up the package step to be separate from the deployment step
    • In this case I am talking about MSDeploy packages and deployment, but a similar concept might apply for other build and deployment scenarios
    • The main reason for this is for consistency with the production package, which had to be separated from the deployment for the reasons explained below under “Production deployments”
  • In this instance the pipeline also had a NuGet publish, which is completely optional, but in this case was needed because part of the project (business layer entities) was shared with a separate project and using NuGet allowed us to share the latest version of the common classes with ease

Convention over Configuration

One of the main concepts that I employ to ensure that the TeamCity setup is as maintainable as possible for a large number of projects is convention over configuration. This requires consistency between projects in order for them to work and as I have said previously, I think this is really important for all projects to have anyway.

These conventions allowed me to make assumptions in my build configuration templates (explained below) and thus make them generically apply to all projects with little or no effort.

The main conventions I used are:

  • The name of the project in TeamCity is {projectname}
    • This drives most of the other conventions
  • The name of the source code repository is ssh://git@server/{projectname}.git
    • This allowed me to use the same VCS root for all projects
  • The code is in the master branch (we were using Git)
    • As above
  • The solution file is at /{projectname}.sln
    • This allowed me to have the same Continuous Integration build configuration template for all projects
  • The main (usually web) project is at /{projectname}/{projectname}.csproj
    • This allowed me to use the same Web Deploy package build configuration template for all projects
  • The IIS Site name of the web application will be {projectname} for all environments
    • As above
  • The main project assembly name is {basenamespace}.{projectname}
    • In our case {basenamespace} was Curtin
    • This allowed me to automatically include that assembly for code coverage statistics in the shared Continuous Integration build configuration template
  • Any test projects end in .Tests and built a dll ending in .Tests in the binRelease folder of the test project after compilation
    • This allowed me to automatically find and run all test assemblies in the shared Continuous Integration build configuration template

Where necessary, I provided ways to configure differences in these conventions for exceptional circumstances, but for consistency and simplicity it’s obviously best to stick to just the conventions wherever possible. For instance the project name for the production project wasn’t {projectname} because I had to use a different project and project names are unique in TeamCity. This meant I needed a way to specify a different project name, but keep the project name as the default. I explain how I did this in the Build Parameters section below.

Build Configuration templates

TeamCity gives you the ability to specify a majority of a build configuration via a shared build configuration template. That way you can inherit that template from multiple build configurations and make changes to the template that will propagate through to all inherited configurations. This is the key way in which I was able to make the TeamCity setup maintainable. The screenshot below shows the build configuration templates that we used.

Build Configuration Templates

Unfortunately, at this stage there is no way to define triggers or dependencies within the templates so some of the configuration needs to be set up each time as explained below in the transition sections.

The configuration steps for each of the templates will be explained in the subsequent posts in this series apart from the Continuous Integration template, which is explained below. One of the things that is shared by the build configuration template is the VCS root so I needed to define a common Git root (as I mentioned above). The configuration steps for that are outlined below.

Build Parameters

One of the truly excellent things about TeamCity build configuration templates are how they handle build parameters.

Build parameters in combination with build configuration templates are really powerful because:

  • You can use build parameters in pretty much any text entry field through the configuration; including the VCS root!
    • This is what allows for the convention over configuration approach I explained above (the project name, along with a whole heap of other values, is available as build parameters)
  • You can define build parameters as part of the template that have no value and thus require you to specify a value before a build configuration instance can be run
    • This allows you to create required parameters that must be specified, but don’t have a sensible default
    • When there are parameters that aren’t required and don’t have a sensible default I set their value in the build configuration template to an empty string
  • You can define build parameters as part of the template that have a default value
  • You can overwrite any default value from a template within a build configuration
  • You can delete any overwritten value in a build configuration to get back the default value from the template
  • You can set a build configuration value as being a password, which means that you can’t see what the value is after it’s been entered (instead it will say %secure:teamcity.password.{parametername}%)
  • Whenever a password parameter is referenced from within a build log or similar it will display as ***** i.e. the password is stored securely and is never disclosed
    • This is really handy for automated deployments e.g. where service account credentials need to be specified, but you don’t want developers to know the credentials
  • You can reference another parameter as the value for a parameter
    • This allows you to define a common set of values in the template that can be selected from in the actual build configuration without having to re-specify the actual value. This is really important from a maintainability point of view because things like server names and usernames can easily change
    • When referencing a parameter that is a password it is still obscured when included in logs 🙂
  • You can reference another parameter in the middle of a string or even reference multiple other parameter values within a single parameter
    • This allows you to specify a parameter in the template that references a parameter that won’t be specified until an actual build configuration is created, which in turn can reference a parameter from the template.
    • When the web deploy post in this series is released you will be able to see an example of what I mean.
  • This is how I managed to achieve the flexible project name with a default of the TeamCity project name as mentioned above
    • In the template there is a variable called env.ProjectName that is then used everywhere else and the default value in the build configuration template is %system.teamcity.projectName%
    • Thus the default is the project name, but you have the flexibility to override that value in specific build configurations
    • Annoyingly, I had to specify this in all of the build configuration templates because there is no way to have a hierarchy of templates at this time
  • There are three types of build parameters: system properties, environment variables and configuration parameters
    • System properties are defined by TeamCity as well as some environment variables
    • You can specify both configuration parameters and environment variables in the build parameters page
    • I created a convention that configuration parameters would only be used to specify common values in the templates and I would only reference environment variables in the build configuration
    • That way I was able to create a consistency around the fact that only build parameters that were edited within an actual build configuration were environment variables (which in turn may or may not reference a configuration parameter specified in the template)
    • I think this makes it easier and less confusing to consistently edit and create the build configurations

Snapshot Dependencies

I make extensive use of snapshot dependencies on every build configuration. While not all of the configurations need the source code (since some of them use artifact dependencies instead) using snapshot dependencies ensures that the build chain is respected correctly and also provides a list of pending commits that haven’t been run for that configuration (which is really handy to let you know what state everything is in at a glance).

The downside of using snapshot dependencies though is that when you trigger a particular build configuration it will automatically trigger every preceding configuration in the chain as well. That means that if you run, say, the UAT deployment and a new source code push was just made then that will get included in the UAT deployment even if you weren’t ready to test it. In practice, I found this rarely if ever happened, but I can imagine that for a large and / or distributed team it could do so watch out for it.

What would be great to combat this was if TeamCity had an option for snapshot dependencies similar to artifact dependencies where you can choose the last successful build without triggering a new build.

Shared VCS root

The configuration for the shared Git root we used is detailed in the below screenshots. We literally used this for every build configuration as it was flexible enough to meet our needs for every project.

Git VCS Root Configuration
Git VCS Root Configuration 2

You will note that the branch name is specified as a build parameter. I used the technique I described above to give this a default value of master, but allow it to be overwritten for specific build configurations where necessary (sometimes we spun up experimental build configurations against branches).

Continuous Integration step configuration

A lot of what I do here is covered by the posts I referenced in the first post of the series apart from using the relevant environment variables as defined in the build configuration parameters. Consequently, I’ve simply included a few basic screenshots below that cover the bulk of it:

Continuous Integration Build Configuration Template 1 Continuous Integration Build Configuration Template 2 Continuous Integration Build Configuration Template - Build Step 1 Continuous Integration Build Configuration Template - Build Step 2 Continuous Integration Build Configuration Template - Build Step 2 (part 2) Continuous Integration Build Configuration Template - Build Triggering Continuous Integration Build Configuration Template - Build Parameters

Some notes:

  • If I want to build a releasable dll e.g. for a NuGet package then I have previously used a build number of %env.MajorVersion%.%env.MinorVersion%.{0} in combination with the assembly patcher and then exposed the dlls as build artifacts (to be consumed by another build step that packages a nuget package using an artifact dependency)
    • Then whenever I want to increment the major or minor version I adjust those values in the build parameters section and the build counter value appropriately
    • With TeamCity 7 you have the ability to include a NuGet Package step, which eliminates the need to do it using artifact dependencies
    • In this case that wasn’t necessary so the build number is a lot simpler and I didn’t necessarily need to include the assembly patcher (because the dlls get rebuilt when the web deployment package is built)
  • I set MvcBuildViews to false because the MSBuild runner for compiling views runs as x86 when using “Visual Studio (sln)” in TeamCity and we couldn’t find an easy way around it and thus view compilation fails if you reference 64-bit dlls
    • We set MvcBuildViews to true when building the deployment package so any view problems do get picked up quickly
  • Be careful using such an inclusive test dll wildcard specification; if you make the mistake of referencing a test project from within another test project then it will pick up the referenced project twice and try and run all the tests from it twice
  • The coverage environment variable allows projects that have more than one assembly that needs code coverage to have those extra dependencies specified
    • If you have a single project then you don’t need to do anything because the default configuration picks up the main assembly (as specified in the conventions section above)
    • Obviously, you need to change “BaseNamespace” to whatever is appropriate for you
    • I’ve left it without a default value so you are prompted to add something to it (to ensure you don’t forget when first setting up the build configuration)
  • The screens that weren’t included were left as their default, apart from Version Control Settings, which had the shared VCS root attached

Build configuration triggering and dependencies

The following triggers and dependencies are set up on the pipeline to set up transitions between configurations. Unfortunately, this is the most tedious part of the pipeline to set up because there isn’t a way to specify the triggers as part of a build configuration template. This means you have to manually set these up every time you create a new pipeline (and remember to set them up correctly).

  • Step 1.1: Continuous Integration
    • VCS Trigger – ensures the pipeline is triggered every time there is a new source code push; I always use the default options and don’t tweak it.
  • Step 2.1: Dev package
    • The package step has a build trigger on the last successful build of Step 1 so that dev deployments automatically happen after a source code push (if the solution built and the tests passed)
    • There is a snapshot dependency on the last successful build of Step 1 as well
  • Step 2.2: Dev deployment
    • In order to link the deployment with the package there is a build trigger on the last successful build of the package
    • There is also a snapshot dependency with the package step
    • They also have an artifact dependency from the same chain so the web deployment package that was generated is copied across for deployment; there will be more details about this in the web deploy post of the series
  • Step 3.1: UAT package
    • There is no trigger for this since UAT deployments are manual
    • There is a snapshot dependency on the last successful dev deployment so triggering a UAT deployment will also trigger a dev deployment if there are new changes that haven’t already been deployed
  • Step 3.2: UAT deploy
    • This step is marked as [Manual] so the user is expected to click the Run button for this build to do a UAT deployment
    • It has a snapshot on the UAT package so it will trigger a package to be built if the user triggers this build
    • There is an artifact dependency similar to the dev deployment too
    • There is also a trigger on successful build of the UAT package just in case the user decides to click on the Run button of the package step instead of the deployment step; this ensures that these two steps are always in sync and are effectively the same step
  • Step 4.1: Production package
    • See below section on Production deployments
  • Step 5: Production deployment
    • See below section on Production deployments

Production deployments

I didn’t want a production build to accidentally deploy a “just pushed” changeset so in order to have a separation between the production deployment and the other deployments I didn’t use a snapshot dependency on the production deployment step.

This actually has a few disadvantages:

  • It means you can’t see an accurate list of pending changesets waiting for production
    • I do have the VCS root attached so it shows a pending list, which will mostly be accurate, but will be cleared when you make a production deployment so if there were changes that weren’t being deployed at that point then they won’t show up in the pending list of the next deployment
  • It’s the reason I had to split up the package and deployment steps into separate build configurations
    • This in turn added a lot of complexity to the deployment pipeline because of the extra steps as well as the extra dependency and trigger configuration that was required (as detailed above)
  • The production deployment doesn’t appear in the build chain explicitly so it’s difficult to see what build numbers a deployment corresponds too and to visualise the full chain

Consequently, if you have good visibility and control over what source control pushes occur it might be feasible to consider using a snapshot dependency for the production deployment and having the understanding that this will push all current changes to all environments at the same time. In our case this was unsuitable, hence the slightly more complex pipeline. If the ability to specify a snapshot dependency without triggering all previous configurations in the chain was present (as mentioned above) this would be the best of both worlds.

Building the production package still needs a snapshot dependency because it requires the source code to run the build. For this reason, I linked the production package to the UAT deployment via a snapshot dependency and a build trigger. This makes some semantic sense because it means that any UAT deployment that you manually trigger then becomes a release candidate.

The last piece of the puzzle is the bit that I really like. One of the options that you have when choosing an artifact dependency is to use the last pinned build. When you pin a build it is a manual step and it asks you to enter a comment. This is convenient in multiple ways:

  • It allows us to mark a given potential release candidate (e.g. a built production package) as an actual release candidate
  • This means we can actually deploy the next set of changes to UAT and start testing it without being forced to do a production deployment yet
  • This gives the product owner the flexibility to deploy whenever they want
  • It also allows us to make the manual testing that occurs on the UAT environment an explicit step in the pipeline rather than an implicit one
  • Furthermore, it allows us to meet the original requirement specified above that there could be a change record number or any other relevant notes about the production release candidate
  • It also provides a level of auditing and assurance that increases the confidence in the pipeline and the ability to “sell” the pipeline in environments where you deal with traditional enterprise change management
  • It means we can always press the Run button against production deployment confident in the knowledge that the only thing that could ever be deployed is a release candidate that was signed off in the UAT environment

Archived template project

I explained above that the most tedious part of setting up the pipeline is creating the dependencies and triggers between the different steps in the pipeline. There is a technique that I have used to ease the pain of this.

One thing that TeamCity allows you to do is to make a copy of a project. I make use of this in combination with the ability to archive projects to create one or more archived projects that form a “project template” of sorts that strings together a set of build configuration templates including the relevant dependencies and triggers.

At the very least I will have one for a project with the Continuous Integration and Dev package and deployment steps already set up. But, you might also have a few more for other common configurations e.g. full pipeline for an Azure website as well as full pipeline for an on-premise website.

Furthermore, I actually store all the build configuration templates against the base archived project for consistency so I know where to find them and they all appear in one place.

Archived Project Template with Build Configuration Templates

Web server configuration

Another aspect of the convention over configuration approach that increases consistency and maintainability is the configuration of the IIS servers in the different environments. By configuring the IIS site names, website URLs and server configurations the same it made everything so much easier.

In the non-production environments we also made use of wildcard domain names to ensure that we didn’t need to generate new DNS records or SSL certificates to get up and running in development or UAT. This meant all we had to do was literally create a new pipeline in TeamCity and we were already up and running in both those environments.

MSBuild import files

Similarly, there are certain settings and targets that were required in the .csproj files of the applications to ensure that the various MSBuild commands that were being used ran successfully. We didn’t want to have to respecify these every time we created a new project so we created a set of .targets files in a pre-specified location (c:msbuild_curtin in our case -we would check this folder out from source control so it could easily be updated; you could also use a shared network location to make it easier to keep up to date everywhere) that contained the configurations. That way we simply needed to create a single import directive in the .csproj (or .ccproj) that included the relevant .targets file and we were off and running.

The contents of these files will be outlined in the rest of the posts in this blog series.

Build numbers

One of the things that is slightly annoying by having separate build configurations is that by default they all use different build numbers so it’s difficult to see at a glance what version of the software is deployed to each environment without looking at the build chains view. As it turns out, there are a number of possible ways to copy the build number from the first build configuration to the other configurations. I never got a chance to investigate this and figure out the best approach though.

Harddrive space

One thing to keep in mind is that if you are including the deployment packages as artifacts on your builds (not to mention the build logs themselves!) the amount of harddrive space used by TeamCity can quickly add up. One of the cool things in TeamCity is that if you are logged in as an admin it will actually pop up a warning to tell you when the harddrive space is getting low. Regardless, there are options in the TeamCity admin to set up clean-up rules that will automatically clean up artifacts and build history according to a specification of your choosing.

Production database

One thing that isn’t immediately clear when using TeamCity is that by default it ships with a file-based database that isn’t recommended for production use. TeamCity can be configured to support any one of a number of the most common database engines though. I recommend that if you are using TeamCity seriously that you investigate that.

Update 7 September 2012: Rollbacks

I realised that there was one thing I forgot to address in this post, which is the requirement I mentioned above about being able to rollback a production deployment to a previous one. It’s actually quite simple to do – all you need to do is go to your production deployment build configuration, click on the Build Chains tab and inspect the chains to see which deployment was the last successful one. At that point you simply expand the chain and then click on the TeamCity trigger previous build as custom build button button to open the custom build dialog and then run it.

Consistency == Maintainability

I think I can categorically say that the most important thing I’ve learnt about software engineering is that consistency is the most important thing in software engineering that leads to something being maintainable. It’s something I feel very strongly about.

If you have awful code or non-standard concepts in your code then as long as you apply the awful code or concepts consistently across your application then as soon as anyone that needs to maintain that code gets their head around what’s going on they can maintain the code effectively. It comes down to readability and comprehension. The quicker and easier it is for someone to read a code snippet and understand what is going on (no matter how badly it makes their eyes bleed) the easier it is for them to identify problems and introduce a fix.

I’m not saying that writing best practice code and using principles like DI, DRY and YAGNI aren’t going to make something maintainable because they do and they are critically important when you are developing software! What I am saying though is they don’t have as big an impact on maintainability as consistency. Furthermore, from a pragmatic perspective, sometimes you inherit code that wasn’t written with best practice approaches and you need to maintain it; yes you could spend a whole heap of time rewriting it, but at the end of the day as long as it’s consistent then it’s maintainable and there may well be more valuable uses of your time.

It does raise an interesting question though about refactoring. If you do have a crappy codebase you have inherited and you decide it will deliver value to do a major refactor then it’s important to ensure that you refactor in a way that leaves the codebase in a consistent state!

Maintainable, large-scale continuous delivery with TeamCity Blog Series

I’ve been a ardent supporter of continuous delivery since I first learnt about it from a presentation by Martin Fowler and Jez Humble. At the time I loved how it encouraged less risky deployments and changed the decision of when/what to deploy from being a technical decision to being a business decision.

I personally think that embracing continuous delivery is an important intermediate step on the journey towards moving from technical agility to strategic agility.

This post was first written in August 2012, but has since been largely rewritten in February 2014 to keep it up to date.

This post forms the introduction to a blog series that is jointly written by myself and Matt Davies.

  1. Intro
  2. TeamCity deployment pipeline
  3. Deploying Web Applications
    • MsDeploy (onprem and Azure Web Sites)
    • OctopusDeploy (nuget)
    • Git push (Windows Azure Web Sites)
  4. Deploying Windows Services
    • MsDeploy
    • OctopusDeploy
    • Git push (Windows Azure Web Sites Web Jobs)
  5. Deploying Windows Azure Cloud Services
    • OctopusDeploy
    • PowerShell
  6. How to choose your deployment technology

Continuous Delivery with TeamCity

One of the key concepts in continuous delivery is the creation of a clear deployment pipeline that provides a clear set of sequential steps or stages to move software from a developer’s commits to being deployed in production.

We have been using TeamCity to develop deployment pipelines that facilitate continuous delivery for the last few years. While TeamCity is principally a continuous integration tool, it is more than adequate for creating a deployment pipeline and comes with the advantage that you are then using a single tool for build and deployment.

We have also tried combining TeamCity with other tools that are more dedicated to deployments, such as OctopusDeploy. Those tools provide better deployment focussed features such as visualisation of the versions of your application deployed to each environment. This approach does create the disadvantage of needing to rely on configuring and using two separate tools rather than just one, but can still be useful depending on your situation.

There are a number of articles that you will quickly come across in this space that give some really great advice on how to set up a continuous delivery pipeline with TeamCity and complement our blog series:

Purpose of this blog series

The purpose of this series is three-fold:

  • Document any findings that myself and Matt Davies have found from implementing continuous delivery pipelines using TeamCity that differ from the articles above;
  • Outline the techniques we have developed to specifically set up the TeamCity installation in a way that is maintainable for a large number of projects;
  • Cover the deployment of Windows Services and Azure Roles as well as IIS websites.

Our intention is to build of the work of our predecessors rather than provide a stand-alone series on how to set up continuous delivery pipelines with TeamCity.

Other options

TeamCity is by no means the only choice when it comes to creating a deployment pipeline. Feel free to explore some of the other options:

My take on documentation

I usually like to keep documentation for a project within OneNote due to the fact it’s really simple and easy to use (especially for dragging in attachments and images!), has first class integration with the Office suite and can be easily synced between team members by hosting the OneNote file in SharePoint or Windows Live. Second to that, I like to store relevant documentation alongside the code in source control so it is versioned and it encourages you to keep it up to date because it is “in your face”; this would normally consist of text-based documentation such as Markdown files. Wikis are also a reasonable option.

One of the things I think is often misunderstood about Agile is a perception that if you are doing Agile then you don’t need documentation. My take on it is that you should at any point in time focus on activities that deliver the maximum value. That means that if you are spending time writing superfluous documentation that won’t provide any value then it’s a waste of time. That doesn’t mean that all documentation doesn’t provide value though; on the contrary, documentation is really important. If there is insufficient documentation for an application then how can a support team be expected to support that application? Similarly, a lack of documentation is a problem if someone is completely new to an application and don’t know where to start looking or know any of the important background information behind it. Documentation plays an important role in the maintainability of any solution.

I see documentation of a system as a combination of:

  • The code itself – thus making it important that code is written in such a way that it is as readable as possible and the intent of the code clear, it also means sections of the code that are hard to read / complex / non-standard should be annotated with comments and in the interest of keeping the code as terse and quick to read as possible leaving comments out of the self-explanatory parts of the code. As an extension of this, I only see it as necessary to add xml-doc comments to interfaces that are used across an application and any public classes in a shared library.
  • The tests – the tests should be an expression of your intent on the design of the system and thus should be able to form a part of the documentation of a system.
  • Other appropriate documentation – by appropriate I mean that the format the documentation is in is the most optimal format for the “thing” being documented. For instance, the documentation to specify a report that needs to be created should probably be a sample Excel file of the report (which is going to be a lot easier, more expressive and more understandable than a chunk of text to describe the report).

I also think it’s important to tailor documentation to the audience; the documentation doesn’t provide value unless it’s accessible and understandable to the people that need to read it.

 

… A year later

It seems rather funny that it was exactly one year since I’ve done a post on my blog. Usual story of course, I started out with good intentions to regularly blog about all the cool stuff I discover along my journey, but time got the better of me. I guess they were the same intentions that I originally had to skin this blog :S.

One thing I have learnt over the last year is that prioritisation is one of the most important things you can do and abide by both personally and professionally. No matter what there will never be enough time to do all the things that you need and want to do so you just have to prioritise and get done all you can – what more can you ask of yourself. With that in mind I guess I haven’t prioritised my blog 😛

I really respect people that manage to keep up with regular blog posts as well as full-time work and other activities. I find that writing blog posts is really time consuming because the pedantic perfectionist in me strives to get every relevant little detail in there and ensure it’s all formatted correctly. Combining that with the insane number of things I seem to find myself doing and trying to get some relax time in somewhere isn’t terribly conducive. It’s a pity really because I enjoy writing posts and hopefully I contribute some useful information here and there.

So, that aside, what have I been doing for the last year. If you are interested feel free to peruse the below list, which has some of what I’ve been doing and is written in no particular order; it’s really just a brain dump ^^. There are a few posts that I have been intending on writing along the way with particularly interesting (to me at least) topics so I’ll try and write some posts over the next few days 🙂

  • Worked with out Project Management Office at work to come up with a way to use PRINCE 2 to provide high level project management to our Agile projects without impacting on the daily work that the teams perform under Scrum. Despite my early scepticism about PRINCE 2 it’s actually a really impressive and flexible project management framework and has worked well.
  • Learnt PowerShell – it’s amazing!
  • Wrote some interesting / powerful NuGet packages (not public I’m afraid) using PowerShell install scripts
  • Attended a really great conference
  • Discovered and started living and breathing (and evangelising) continuous delivery and dev ops
  • Started thinking about the concept of continuous design as presented by Mary Poppendieck at Yow
  • Created a continuous delivery pipeline for a side-project with a final prod deployment to Windows Azure controlled by the product owner at the click of a button with a 30-45s deployment time!
  • Started learning about the value of Lean thinking, in particular with operational teams
  • Started evangelising lean thinking to management and other teams at work (both software and non-software)
  • Started using Trello to organise pretty much everything (both for my team, myself personally and at work and various projects I’m working on in and out of work) – it’s AMAZING.
  • Delivered a number of interesting / technically challenging projects
  • Became a manager
  • Assisted my team to embark on the biggest project we’ve done to date
  • Joined a start-up company based in Melbourne in my spare time
  • Joined Linked in (lol; I guess it had to finally happen)
  • Gave a number of presentations
  • Became somewhat proficient in MSBuild (*shudders*) and XDT
  • Facilitated countless retrospectives including a few virtual retrospectives (ahh Trello, what would I do without you)
  • Consolidated my love for pretty much everything Jetbrains produce for .NET (in particular TeamCity 7 and ReSharper 6 are insanely good, I’ll forgive them for dotCover)
  • Met Martin Fowler and Mary and Tom Poppendieck
  • Participated in the global day of code retreat and then ran one for my team (along with a couple of Fedex days)
  • Got really frustrated with 2GB of RAM on my 3 year old computer at home after I started doing serious development on it (with the start-up) and upgraded to 6GB (soooo much better, thanks Evan!)
  • Participated on a couple of panels for my local Agile meetup group
  • Got an iPhone 4S 🙂 (my 3GS was heavily on the blink :S)
  • Took over as chairman of the young professionals committee for the local branch of the Institution of Engineering and Technology
  • Deepened my experience with Microsoft Azure and thoroughly enjoyed all the enhancements they have made – they have gone a long way since I first started in 2010!

Of course there is heaps more, but this will do for now.