Breaking up software projects into small, focussed milestones

This post highlights a particular learning I’ve had over the last year about setting milestones for organising high-level goals for software projects.

Background

When I am first involved in a project there is undoubtedly a huge list of things that the product owner / client wants and it’s generally the case that there isn’t enough time or money for it all to be all completed. Let’s ignore for a moment the fact that there will be reasons why it’s not sensible to complete it all anyway since the low priority things will be much less important than high priority work on other projects or emergent requirements / user feedback for the current project.

Often, the person/people in question don’t really have a good understanding about what is realistic within a given time period (how could they when it’s hard enough for the development team?) and certainly the traditional way that software development has been executed exacerbates the issue by promising that a requirements specification will be delivered by a certain time. Despite the fact this almost never works people still sometimes expect software to be able to be delivered in this way. Thankfully, Agile software development becoming more mainstream means that I see this mentality occurring less and less.

An approach I’ve often taken in the past to combat this is to begin by getting product owners to prioritise high-level requirements with the MoSCoW system and then take the must-have’s and possibly the should-have’s and labelled that as phase 1 and continued to flesh out those requirements into user stories. This then leaves the less important requirements at the bottom of the backlog giving a clear expectation that any estimations and grooming effort will be expended on the most important features and won’t include “everything” that the person initially had in their vision (since that’s unrealistic).

This is a common-sense approach that most people seem to understand and allows for a open and transparent approach to providing a realistic estimate that can’t be misconstrued as being for everything. Note: I also use other techniques that help set clear expectations like user story mapping and inception decks.

The problem

These “phase 1” milestones have a number of issues:

  • They are arbitrary in make up and length, which results in a lack of focus and makes it easier for “scope creep” to occur
    • While scope-creep isn’t necessarily a problem if it’s being driven by prioritisation from a product owner it does tend to increase feedback cycles for user feedback and makes planning harder
    • Small variations to direction and scope tend to get hidden since they are comparatively small, however the small changes add up over time and can have a very large impact (positive and negative) on the project direction that isn’t intended
  • They tend to still be fairly long (3+ months)
    • This increases the size of estimation errors and the number and size of unknowns
    • I’ve noticed this also reduces the urgency/pace of everyone involved

A different approach

I’ve since learnt that a much better approach is to create really small, focused milestones that are named according to the goal they are trying to meet e.g. Allow all non-commercial customers who only have product X to use the new system (if you are doing a rewrite) or Let customers use feature Y (new feature to a system).

More specifically, having focused milestones:

  • Helps with team morale (everyone can see the end goal within their grasp and can rally around it)
  • Helps frame conversations with the product owner around prioritising stories to keep focus and not constantly increasing the backlog size (and by association how far away the end goal is)
  • Helps create more of a sense of urgency with everyone (dev team, ops, management etc.)
  • Helps encourage more frequent releases and thinking earlier about release strategies to real end users
  • Provides a nice check and balance against the sprint goal – is the sprint goal this sprint something that contributes towards our current milestone and in-turn are all the stories in the sprint backlog contributing to the sprint goal?

The end goal (probably not “end”; there is always room for improvement)

I don’t think that the approach I describe above is necessarily the best way of doing things. Instead I think it is a stepping stone for a number of things that are worth striving for:

Presentation: Moving from Technical Agility to Strategic Agility

I recently gave a presentation with my colleague Jess Panni to the ACS WA Conference about Agile and where we see it heading in the next 5-10 years. When offered the speaking slot the requirements were that it involved data analytics in some way and that it wasn’t the same old Agile stuff that everyone has been talking about for years, but rather something a bit different. Both of those requirements suited me because it fit in perfectly with thoughts I’d been having recently about Agile and where it is heading.

Jess and I had a lot of fun preparing the talk – it’s not often we get time to sit down and chat about process, research what the industry leaders are saying and brainstorm our own thoughts and experiences in light of that research. I’m very proud of the content that we’ve managed to assemble and the way we’ve structured it.

We paid particular care to make the slide deck useful for people – there are comprehensive notes on each slide and there are a bunch of relevant references at the end for further reading.

I’ve put the slides up on GitHub if you are interested.

… A year later

It seems rather funny that it was exactly one year since I’ve done a post on my blog. Usual story of course, I started out with good intentions to regularly blog about all the cool stuff I discover along my journey, but time got the better of me. I guess they were the same intentions that I originally had to skin this blog :S.

One thing I have learnt over the last year is that prioritisation is one of the most important things you can do and abide by both personally and professionally. No matter what there will never be enough time to do all the things that you need and want to do so you just have to prioritise and get done all you can – what more can you ask of yourself. With that in mind I guess I haven’t prioritised my blog 😛

I really respect people that manage to keep up with regular blog posts as well as full-time work and other activities. I find that writing blog posts is really time consuming because the pedantic perfectionist in me strives to get every relevant little detail in there and ensure it’s all formatted correctly. Combining that with the insane number of things I seem to find myself doing and trying to get some relax time in somewhere isn’t terribly conducive. It’s a pity really because I enjoy writing posts and hopefully I contribute some useful information here and there.

So, that aside, what have I been doing for the last year. If you are interested feel free to peruse the below list, which has some of what I’ve been doing and is written in no particular order; it’s really just a brain dump ^^. There are a few posts that I have been intending on writing along the way with particularly interesting (to me at least) topics so I’ll try and write some posts over the next few days 🙂

  • Worked with out Project Management Office at work to come up with a way to use PRINCE 2 to provide high level project management to our Agile projects without impacting on the daily work that the teams perform under Scrum. Despite my early scepticism about PRINCE 2 it’s actually a really impressive and flexible project management framework and has worked well.
  • Learnt PowerShell – it’s amazing!
  • Wrote some interesting / powerful NuGet packages (not public I’m afraid) using PowerShell install scripts
  • Attended a really great conference
  • Discovered and started living and breathing (and evangelising) continuous delivery and dev ops
  • Started thinking about the concept of continuous design as presented by Mary Poppendieck at Yow
  • Created a continuous delivery pipeline for a side-project with a final prod deployment to Windows Azure controlled by the product owner at the click of a button with a 30-45s deployment time!
  • Started learning about the value of Lean thinking, in particular with operational teams
  • Started evangelising lean thinking to management and other teams at work (both software and non-software)
  • Started using Trello to organise pretty much everything (both for my team, myself personally and at work and various projects I’m working on in and out of work) – it’s AMAZING.
  • Delivered a number of interesting / technically challenging projects
  • Became a manager
  • Assisted my team to embark on the biggest project we’ve done to date
  • Joined a start-up company based in Melbourne in my spare time
  • Joined Linked in (lol; I guess it had to finally happen)
  • Gave a number of presentations
  • Became somewhat proficient in MSBuild (*shudders*) and XDT
  • Facilitated countless retrospectives including a few virtual retrospectives (ahh Trello, what would I do without you)
  • Consolidated my love for pretty much everything Jetbrains produce for .NET (in particular TeamCity 7 and ReSharper 6 are insanely good, I’ll forgive them for dotCover)
  • Met Martin Fowler and Mary and Tom Poppendieck
  • Participated in the global day of code retreat and then ran one for my team (along with a couple of Fedex days)
  • Got really frustrated with 2GB of RAM on my 3 year old computer at home after I started doing serious development on it (with the start-up) and upgraded to 6GB (soooo much better, thanks Evan!)
  • Participated on a couple of panels for my local Agile meetup group
  • Got an iPhone 4S 🙂 (my 3GS was heavily on the blink :S)
  • Took over as chairman of the young professionals committee for the local branch of the Institution of Engineering and Technology
  • Deepened my experience with Microsoft Azure and thoroughly enjoyed all the enhancements they have made – they have gone a long way since I first started in 2010!

Of course there is heaps more, but this will do for now.