Scrum & KPI’s

Scrum & KPI’s

January 4, 2018 Agile 0

I’m amazed by the amount of Scrum Masters online who say that KPI’s have no place in Scrum. And even worse, the few Scrum Masters (or old-school managers) that do believe that KPI’s have a role in Scrum but use Agile-based metrics that would result in an incredibly skewed perception of the entire process. Words like velocity or story points are based on agile theory and – while having uses for the Scrum team itself – have little use in achieving real-world business objectives.

Before we start, let’s take a look at what a KPI is:

Key Performance Indicator (KPI) is a measurable value that demonstrates how effectively a company is achieving key business objectives.

So a good KPI:

  • Should be rooted in reality (because the business objective is too)
  • Should help the user make decisions
  • Should quantify factors required for reaching the chosen business objective

Let’s unpack that.

1. A KPI should be rooted in reality

Sounds obvious right? Well, not really. A lot of confusion has started because Scrum provides many different metrics that allow for decision making within the context of Scrum. These are local metrics that do not work on a higher level. For example, you cannot use a Scrum-concept of velocity to measure a real concept like effectivity.

So, in order to get metrics for this deeper level, we need to start on a fundamentally deeper level. Forget everything you know about Scrum and think about real-world tools of scientific measurements: A thermometer(heat), stopwatch(time) and others. These are tools that are rooted in reality because they return the same results when used in identical situations. There are also measurement tools which combine various factors, such as the profit-loss statement. While not scientific, it can still seen as an objective measurement of value/success. 

So, we should watch the wording when creating our KPIs so we get measurements that are:

  1. Repeatable
  2. Independant per team or researcher.

Just like how two thermometers should show identical results when placed in different rooms with the same temperature, so should our KPIs show identical results when placed in seperate teams with identical starting points. Any deviation can then be explained.

A concept of velocity is not rooted in reality because Team A could have a velocity of 20 and Team B could have a velocity of 300. It doesn’t say anything. It only means something within the team.

2. A KPI should help the user make decisions

A tool of measurement exists to help the user make a decision. Because different users need to make different decisions, the tools must be matched to their exact decision-making needs.

There are three main groups in Scrum

  • The Public – Is interested in the quality of Agile (Customers & Management)
  • The Development Team – Is interested in the quality of Kaizen
  • The Scrum Master – Is interested in the quality of Scrum

Agile: Ability to adapt to changing demands. This means discovering the customer’s need and adapt to those needs in the quickest time possible. This results in happy customers and profits (In that exact order)

Kaizen: The act of continuous improvement. This means the continuous discovery and adaptation of possible improvements to the proces in order to maximize quality, team happiness and throughput of the team. (in that exact order)

So the KPI’s we want to discover need to answer the following questions:

  • Can we quickly adapt to changing demands? (Quality of Agile)
  • Are we continuously improving? (Quality of Kaizen)
  • Is the quality of Scrum improving? (Quality of Scrum)

 3. A KPI should quantify factors required for reaching the chosen business objective

Alright, so we have identified the three groups + their needs and we’re identified the major factors for a KPI (Repeatable, independant). Let’s see if we can come up with some KPIs that clarify how close we are to reaching each objective.

KPIs Public (+ Team + Scrum Master)

OBJECTIVE: CAN WE QUICKLY ADAPT TO CHANGING DEMANDS?

  • Happiness of customers
    • Do we deliver value?
  • Perceived quality of product by customers
    • Do we deliver the highest value possible?
  • Visibilty of our team in the organisation
    • Can stakeholders communicate effectively what they value?
  • Visibilty of our product in the organisation
    • Can stakeholders easily discover the value our product offers?
  • Amount of customers (Penetration of Products)
    • Are we considered valuable in the organisation?

KPIs Team ( + Scrum Master )

OBJECTIVE: IS OUR TEAM CONTINUOUSLY IMPROVING?

  • Technical Debt Stories in Product Backlog
    • Pie chart. Are we still on a long-term value path?
  • Product Value Improvement Stories in Product Backlog
    • Pie chart. Are we still on a long-term value path?
  • Backlog Security Stories in Product Backlog
    • Pie chart. Are we still on a long-term value path?
  • Bugs found & resolved
    • Quality of DoD
  • Number of branches not covered by tests
    • Do we need to spend more time on KPIs?
  • Happiness Team
    • Is the team motivated?
  • CI/CD Deployment Frequency
    • Is our deployment process streamlined?
  • CI/CD Speed of Deployment
    • Is our deployment process efficient?
  • CI/CD Deployment Success
    • Do we need to update our coding practices?

KPIs Scrum Master

OBJECTIVE: IS THE QUALITY OF SCRUM IMPROVING?

  • Happiness Manager
    • Is your boss happy with the value you provide?
  • Open Impediments
    • Do you need to find help to improve the process?
  • Open Retrospective Items
    • Does the Dev team need help to improve the process?
  • Goal clarity within Team
    • Does the PO need help to improve the process?
  • Estimation Accuracy / Predictability / Velocity Consistency
    • How good is the team in making estimations?

Of course, the only group that is allowed to see the KPIs is the group for they’re written for. KPIs public is for everybody, KPIs team is for the team + Scrum Master and KPI Scrummaster are private. The KPI “Estimation Accuracy” can be maintained together with the PO in private in order to improve estimations and predictability.

That’s it. By taking in account the different needs of the user groups, you can create real KPIs that provide real value for the stakeholder, the team, the Scrum Master and management!