neil-vass.com

Deliver Sessions: Probabilistic forecasting

Poster for the meetup: an outside, evening photo of the outside of the Express Building in Manchester. At the top it says "How probabilistic forecasting helped our team" "19 Sep 2023, 6:30pm", and it has the Transform and Deliver Sessions logos.

I’ve helped organise Deliver Sessions meetup for a few years, and now I try to write a few notes after each one so I remember all the good topics they cover (like facilitation, outcomes, and working together). In September 2023, we were hosted at Transform’s office in Ancoats, where we heard a true story from Sarah Miles of using probabilistic forecasting.

The talk

“Warning: this talk contains maths. That might give some of us flashbacks to school… don’t worry, it’s not too hard and it is very useful”

There’s lots situations probabilistic forecasting can help with. Sarah’s story introduced a dramatic one: new industry regulations had introduced a must-not-miss deadline, and if her team didn’t make the needed changes before that date, all kinds of business process would have to stop. “The cost of delay would be huge”. Everyone involved was, understandably, very keen to know whether the needed work was going to get done by then.

Sarah at the front of the room with a microphone, with a big screen behind her. Screen has an image of 3 dice, and words: "Probabilistic forecasting makes uncertainties visible so we can talk about ways to make work more predictable and reduce risk"

There was still lots of detail needed on exactly what the work was, never mind how fast they’d get through it. Sarah and team used user story mapping to get a clearer agreement on what they’d need to do, and used Kanban to visualise the work and start to assess how likely they were to meet that deadline.

“When you think of Kanban, most people think of a board with columns, and work moving left to right. There’s actually quite a bit more to it.”

This is definitely an attitude I recognize! Various teams think of Kanban as “oh good we can stop all that tedious sprint planning and reviews” and call that Kanban, missing out on all the other benefits they could be getting. There’s a combination of using metrics to understand how work is progressing, and things you can try – based on what the metrics are telling you – to improve how work flows.

There’s lots of different software you could use to get started with metrics. Sarah used spreadsheets from Troy Magennis to forecast a best understanding of how likely it was they’d get everything needed done by various dates.

Screenshot of the "Throughput forecaster" spreadsheet, from the page linked to just before this image. It has lots of fields to fill in (start date, number of items, throughput, more) and a table of results, showing a range of dates from 95% likely to 0% likely.

Plugging in the numbers, they found they had a 50-55% chance of meeting that super-critical, business-defining deadline. Presenting this to stakeholders was a breakthrough moment; everyone wanted to know what they could do to help make success more certain.

Sarah toured through a range of helpful changes they made over time. Having a deadline that everyone in the business united around helped – the usual competing priorities fell away, as it was this one that mattered.

They asked: How can we give ourselves less work to do?

  • Descoped things that didn’t absolutely need to be done for that regulation. These aren’t gone forever, but anything that can be left until “after” was hugely helpful.
  • Focused on doing fewer things, not on lowering quality – don’t do a poorly-implemented version of everything, look at what you can drop or make simpler and get the remainder working very well.

They asked: How can we give ourselves more certainty?

  • Did back end engineering work to understand technical challenges
  • Set tight design timescales (we need to explore / decide this by that date)
  • Focused user research on the critical areas, to understand which things they needed to get right

They asked: How can we get more done each week?

  • Swapped team members to bring in missing skills; there were a few software teams in this area, and the typically had a mix of experience on each team to help grow and develop everyone. For this work, they shuffled all the people with the most relevant skills onto this team.
  • This was coupled with appreciation for the people on those other teams – frequently mentioning how their keeping existing services running underpinned the new regulation work, enabling Sarah’s team to focus.
  • They cut out meetings and non-value added work – doing their best to have just a few people in each session, rather than everyone who might be useful or have an interest.

They asked: How can we take care of our team?

  • They never got anyone to work into the evenings, at weekends, or through their lunchtimes; that gets counter-productive extremely quickly and is an awful way to try to squeeze more work out.
  • Used retros and team health surveys to look out for each other’s well-being.

Looking wider than the team, they asked:

  • Who do we need help from?
  • What could slow us down?
  • Who needs to make decisions, quickly?

Sharing the forecast helped people understand the importance of things. Put the percentages into weeknotes every week, along with notes on what needed to be done to improve things.

In the end, the work was all in place two weeks before the deadline, leaving time for calm improvements and a smooth release.

This approach got a huge amount of work done on that important business priority. There’s aspects of it that would be useful for most teams – more focus and better understanding of what helps get work done effectively. But if you worked like this all the time, you’d be missing out on lots of opportunities: for spotting new ideas, for developing capabilities of more people in the team. I liked Sarah’s reflections about having this approach as one of a range you can have in your toolkit.

The discussion

As well as hearing interesting talks, we want our meetups to give attendees a chance to talk to each other and find out what everyone thinks about the topic. So instead of an “any questions” at the end, we tried a different format.

Before the talk, we asked everyone to take a few notes on two prompts while they listened to the talk:

  • What sounds familiar to you? That could be about the exact tools and techniques Sarah’s used, or more general about some of the challenges her team faced.
  • What would you love to hear more about? If there’s anything from the talk you’d like to put into practice, what do you need a bit more detail on to help you?

After the talk, everyone got a good 15 minutes to discuss these or anything else the talk brought to mind.

Tables of people discussing the talk - about 30-something people in total. The room is colourfully decorated, with zigzag lines on the floor and large circles and triangles on the walls.

To finish the group discussion, each group was asked to choose the top few questions they’d most like to ask Sarah about. Sarah then came back on stage for a Q+A session. This felt more useful than the usual hands-up-after-the-talk approach: a good chance to understand what others in the room thought about the talk, and to agree what we’d all like to hear more about.

Some links

I’ve written about some forecasting techniques myself in the past, I liked hearing Sarah recommend some of the same authors:

Sarah also shared some links for learning about Kanban in general:

Follow Deliver Sessions at deliver-sessions.eventbrite.com to hear when future meetups get announced.

Posted

in

by

Neil Vass

Tags: