Tag Archives: agile

The Perfect Agile Tool – 12 Key Features

Learn more about transforming people, process and culture with the Real Agility Program

The Perfect Agile Tool doesn’t yet exist.  In my training and consulting work, I often have strong words to say about electronic tools.  Most of the tools out there are really bad.  Unfortunately, JIRA, the most common tool, is also the worst that I know of.  (Actually, the only tool worse than JIRA for an Agile team is MS Project – which is just plain evil).  Some Agile tools do a bit better, but most fall far short of a good physical task board (information radiator).  I am often asked to evaluate and / or partner with tool vendors to “bless” their products.  Here is what I am looking for before I will consider an outright endorsement of such a tool.

Features for a Perfect Agile Tool

This list is roughly organized in order of features which do show up in some tools to those which I have never seen or heard of in tools.

1. Skeumorphism: Cards and Wall

The tool should display the current work of an Agile team in a way that is immediately recognizable as a set of note cards or PostIt’s on a physical wall.  This includes colours, sizes, etc.  Most people will type to enter data so fonts should be chosen to mimic hand-printed letters.  Every aspect of the display should remind people of the physical analogue of the tool.

2. Live Update

As team members are using the tool, all updates that they make should be visible as immediate updates to all the other team members including typing, moving cards around, etc.  There is no off-line mode for the tool.  In fact, if the tool is not receiving live updates, it should be clearly disabled so that the team member knows there is a problem with the information they have displayed.

3. Simple or No Access Control

Most Agile methods strongly de-emphaisize or even disallow traditional roles and encourage self-organizing teams.  This means that fine-grained access control to different features of the tool should be eschewed in favour of extremely simple access control: everyone can do anything with the tool.  (It actually helps if there is no “undo” feature, just like there’s no easy way to erase Sharpie written on a note card.)

4. Infinite Zoom In/Out

When you are using cards on a wall, it is easy to see the whole wall or to get up close and see even very fine details on a single note card.  Although it does not have to be literally infinite, the wide and tight zoom levels in the tool should be at least a few orders of magnitude difference.  As well, the zoom feature should be extremely easy to use, similar perhaps to the way that Google Maps functions.  Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.

5. Touch Device Compatible

This seems like a super-obvious feature in this day and age of tablets, smart phones and touch-screen laptops.  And it would take the cards on the wall metaphor just that extra little way.  But very few tools are actually easy to use on touch devices.  Dragging cards around and pinch to zoom are the obvious aspects of this feature.  But nice finger-drawing features would also be a big plus (see below)!

6. Size Limit on Cards

For techies, this one is extremely counter-intuitive: limit the amount of information that can be stored on a “card” by the size of the card.  It shouldn’t be possible to attach documents, screen shots, and tons of meta-data to a single card.  Agile methods encourage time-boxing (e.g. Sprints), work-boxing (e.g. Work-in-Process limits), and space-boxing (e.g. team rooms).  This principle of putting boundaries around an environment should apply to the information stored on a card.  Information-boxing forces us to be succinct and to prefer face-to-face communication over written communication.  Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.

7. Minimal Meta-Data

Information-boxing also applies to meta-data.  Cards should not be associated with users in the system.  Cards should not have lots of numerical information.   Cards should not have associations with other cards such as parent-child or container-contained.  Cards should not store “state” information except in extremely limited ways.  At most, the electronic tool could store a card ID, card creation and removal time-stamps, and an association with either an Agile team or a product or project.

8. Overlapping Cards

Almost every electronic tool for Agile teams puts cards in columns.  Get rid of the columns, and allow cards to overlap.  If there is any “modal” behaviour in the tool, it would be to allow a team member to select and view a small collection of cards by de-overlapping them temporarily.  Overlapping allows the creation of visually interesting and useful relationships between cards.  Cards can be used to demarcate columns or groupings without enforcing strict in/out membership in a process step.

9. Rotatable, Foldable, Rip-able Cards

Increase the fidelity of the metaphor with physical cards on a wall.  Rotation, folding and ripping are all useful idioms for creating distinct visual cues in physical cards.  For example, one team might rotate cards 45 degrees to indicate that work is blocked on that card.  Or another team might fold a dog-ear on a card to indicate it is in-progress.  Or another team might rip cards to show they are complete.  The flexibility of physical cards needs to be replicated in the electronic environment to allow a team to create its own visual idioms.  Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.

10. Easy Sketching on Cards… Including the Back

Cards should allow free-form drawing with colours and some basic diagramming shapes (e.g. circles, squares, lines).  Don’t make it a full diagramming canvas!  Instead, allow team members to easily sketch layouts, UML, or state diagrams, or even memory aides.  The back side of the card is often the best place for more “complex” sketches, but don’t let the zoom feature allow for arbitrarily detailed drawing.  Lines need a minimum thickness to prevent excessive information storage on the cards.

11. Handwriting Recognition

With Siri and other voice-recognition systems, isn’t it time we also built in handwriting recognition?  Allowing a team member to toggle between the handwriting view and the “OCR” view would often help with understanding.  Allow it to be bi-directional so that the tool can “write” in the style of each of the team members so that text entry can be keyboard or finger/stylus.

12. Sync Between Wall and Electronic Tool

This is the most interesting feature: allow a photo of cards on a wall to be intelligently mapped to cards in an electronic tool (including creating new cards) and for the electronic tool to easily print on physical note cards for placement on a wall.  There is all sorts of complexity to this feature including image recognition and a possible hardware requirement for a printer that can handle very small paper sizes (not common!)

Key Anti-Features

These are the features that many electronic tools implement as part of being “enterprise-ready”.  I’ll be brief on these points:

No Individual Tracking – the team matters, not who does what.

No Dependency Management – teams break dependencies, tools don’t manage dependencies.

No Time Tracking – bums in seats typing doesn’t matter: “the primary measure of progress is working software” (or whatever valuable thing the team is building) – from the Agile Manifesto.

No Actuals vs. Estimates – we’re all bad at predicting the future so don’t bother with trying to get better.

No Report Generation – managers and leaders should come and see real results and interact directly with the team (also, statistics lie).

No Integration Points – this is the worst of the anti-features since it is the one that leads to the most anti-agile creeping featuritis.  Remember: “Individuals and interactions [are valued] over processes and tools” – from the Agile Manifesto.

Evaluation of Common Agile Tools

I go from “Good” to “Bad” with two special categories that are discontinuous from the normal scale: “Ideal” and “Evil”.  I think of tools as falling somewhere on this scale, but I acknowledge that these tools are evolving products and this diagram may not reflect current reality.  The scale looks like this, with a few examples put on the scale:

Perfect Agile Tool evaluation scale with examples

Plea for the Perfect Agile Tool

I still hope that some day someone will build the perfect Agile tool.  I’ve seen many of the ideal features listed above in other innovative non-Agile tools.  For example, 3M made a PostIt® Plus tool for the iPhone that does some really cool stuff.  There’s other tools that do handwriting recognition, etc.  Putting it all together in a super-user-friendly package would really get me excited.

Let me know if you think you know of a tool that gets close to the ideal – I would be happy to check it out and provide feedback / commentary!

Please share!

YouTube Video: What is Real Agility?

Learn more about transforming people, process and culture with the Real Agility Program

Many organizations are attempting to use Agile methods.  Banks, telecom companies, government agencies, and all manner of mid-size and small organizations.  Most of these attempts are limited in that they think of Agile as a solution instead of as a culture.  In this video, I explore some of the conditions for creating Real Agility.

This is the first video in a series of eleven that is oriented towards what managers need to know to create Real Agility in their organizations.  The final two videos in the series are going to be content exclusively available to subscribers to our Real Agility Newsletter.  Those final two videos are about “Dealing with Crisis” and “The Knowing-Doing Gap”.  (Our newsletter also includes other great content including interviews – we are featuring an interview with Mary and Tom Poppendieck in just a few weeks!)

Please share!

Fun Video: Bloopers from the Scrum Myths Video Series

Learn more about transforming people, process and culture with the Real Agility Program

The Scrum Myths videos have been popular and I’m very happy with people’s comments about the videos.  I’m going to be making an even more extensive new series for publication in just a few weeks.

Unbeknownst to me, the videographer, my brother Alexei Berteig, created a bloopers video from some of the many, many, many, MANY mistakes I made while making the videos.  I hope you will watch it…. but I strongly recommend taking a look at one or two of the “good” videos first.  Try these:

Scrum Myth – Excessive Preparation

Scrum Myths – Stretch Goals are Good

Now, without further ado, here is the Scrum Myths Bloopers video:

Please share!

Example of Visualizing Process Cycle Efficiency with LEGO

Learn more about transforming people, process and culture with the Real Agility Program

In-depth article here: Using Lego[sic] to capture raw data for Cycle Time and Process Cycle Efficiency.

From the article:

The typical way to collect baseline numbers for these metrics is to conduct a value stream mapping workshop that involves most or all team members for one day or longer. The client is worried about losing too much time in the workshops when the teams could be doing value-add work. Therefore, we needed a less intrusive way to collect baseline measurements. There is also the question of how accurate the PCE data will be when it is obtained through a workshop rather than by direct observation of team activity.

I came up with the idea of using Lego bricks to collect the raw data for Cycle Time and PCE. There is some impact on team member time, but hopefully it is not too intrusive. My observation is that people enjoy manipulating Lego bricks, and they don’t mind sticking the bricks on a plate to represent their work.

Please share!

Link: The Cost of Turnover on an Agile Team

Learn more about transforming people, process and culture with the Real Agility Program

Great article by Mike Griffiths: http://leadinganswers.typepad.com/leading_answers/2015/10/agile-talent-management.html

Please share!

The Daily Goal

Learn more about transforming people, process and culture with the Real Agility Program

When my Scrum team first proposed trying mobbing I wasn’t sure what to expect. No one on the team (including myself) was an expert. I reserved judgment and watched a few sprints. I stayed silent when the same team decided to skip the tasking out portion of planning and simply pulled in enough stories to fulfill their Sprint goal.

After a couple of sprints it became obvious that there were a number of pros; the team was engaged and aligned. They responded as a consistent, unified voice at Sprint Review and Retro. Planning went a lot faster as the team no longer wrote out all their tasks and didn’t have to copy all of them into Jira.

In particular, the Daily Standup’s usual agenda of “what did you do yesterday? what are you doing today? what are you doing tomorrow?” became redundant. And this had me thinking; ‘maybe the team didn’t need a stand up anymore? What’s the point?”

Then, one of the team members introduced me to the concept of the daily goal and I watched as he walked the team thru it. I thought this made a lot of sense and so did the team. We kept up the practice. Some days the team would forget about it or be too tired to bother. I noticed on those days the team wouldn’t be as clear on what they were working on or how to align on a particular challenge. I also noticed that the Stand Up the next day would be more fragmented and meander a bit. They would forget to communicate to one another and look at me as their Scrum Master expectedly, wanting me to drive it.

I started to coach the team back to their daily goal practice and reminded them this Stand Up was for them to align on the work ahead for the day. This became more important as the team divided up into mini mobs or pairs and were no longer one big mob.

I find the Daily Goal useful for a number of reasons. Instead of tasking out the work a week in advance, they decide how to approach the work on a daily basis that takes into account any day to day changes that have happened. Planning goes a lot faster now that we’re not tasking out all the work in advance. The team stays focused on a daily basis as opposed to at just Planning. They write their daily goal on their Scrum Board making it visible to anyone on the floor what their focus is for the day. Even better, the daily stand up as renewed purpose and the dialogue is more interactive.

Here are some examples of our Daily Goals:

  • Refactor Story 1
  • Coordinate with outside team members to align on test strategy
  • Complete happy path for Story 3
  • And sometimes it is as simple as “Complete story 4”

I hope to continue the practice and frankly, it’s fun! Achieving short time goals is motivating and brings a sense of accomplishment. I highly recommend it. Let me know your thoughts and if you’re trying this technique let me know how it’s working out for your team.

[EDITOR’S NOTE: Alexandra Dragon is a first-time contributor to Agile Advice – please welcome her in the comments.  And let us know if you are interested in contributing.]

Please share!

9 Agile Estimation Techniques

Learn more about transforming people, process and culture with the Real Agility Program

Many people have used a variation of Planning Poker to do Agile estimation.  Here is a reference of 9 different Agile estimation techniques for different circumstances.  I have seen all of these techniques work in practice, except one.  Try a new one each Sprint!

Planning Poker

Participants use specially-numbered playing cards to vote for an estimate of an item.  Voting repeats with discussion until all votes are unanimous.  There are lots of minor variations on Planning Poker.  Good technique to estimate a very small number of items (2 to 10).

The Bucket System

Using the same sequence as Planning Poker, a group or a team estimate items by placing them in “buckets”.  The Bucket System is a much faster Agile estimation technique than Planning Poker because there is a “divide-and-conquer” phase.  The Bucket System can also be used with larger groups than Planning Poker and with very large numbers of items to be estimated (50 to 500).


For super-fast Agile estimation, the items to be estimated are simply placed by the group in one of three categories: big, uncertain and small.  The group starts by discussing a few together, and then, like the Bucket System, uses divide-and-conquer to go through the rest of the items.

TFB / NFC / 1 (Sprint)

This Agile estimation technique is similar to Big/Uncertain/Small but puts a specific “size” into the mix, namely 1 Sprint.  The categories are “Too F-ing Big”, “No F-ing Clue” and “1” Sprint (or less).  I learned this one recently from someone in one of my CSPO classes.

Dot Voting

Dot voting is usually considered a decision-making tool, not an Agile estimation technique.  However, for estimating small numbers of items, dot voting can be a super-simple and effective technique.  Each person gets a small number of “dots” and uses them as votes to indicate the size of an item; more dots means bigger.

T-Shirt Sizes

Items are categorized into t-shirt sizes: XS, S, M, L, XL.  The sizes can, if needed, be given numerical values after the estimation is done.  This is a very informal technique, and can be used quickly with a large number of items.  Usually, the decisions about the size are based on open, collaborative discussion, possibly with the occasional vote to break a stalemate.  There is a brief description of T-Shirt Sizes here.

Affinity Mapping

Items are grouped by similarity – where similarity is some dimension that needs to be estimated.  This is usually a very physical activity and requires a relatively small number of items (20 to 50 is a pretty good range).  The groupings are then associated with numerical estimates if desired.

Ordering Protocol

Items are placed in a random order on a scale labeled simply “low” to “high”.  Each person participating takes turns making a “move”.  A move involves one of the following actions: change the position of an item by one spot lower or one spot higher, talking about an item, or passing.  If everyone passes, the ordering is done.  The Challenge, Estimate, Override and the Relative Mass Valuation methods are variations on the ordering protocol.

Divide until Maximum Size or Less

The group decides on a maximum size for items (e.g. 1 person-day of effort).  Each item is discussed to determine if it is already that size or less.  If the item is larger than the maximum size, then the group breaks the item into sub-items and repeats the process with the sub-items.  This continues until all items are in the allowed size range.

Principles of Agile Estimation

Agile estimation techniques are collaborative.  All appropriate people are included in the process.  For example the whole Scrum team participates in estimating effort of Product Backlog Items.  Collaborative techniques are also designed so that it is impossible to blame someone for an incorrect estimate: there is no way to trace who estimated what.

Agile estimation techniques are designed to be fast (-er than traditional techniques) and deliberately trade off accuracy.  We are not trying to learn to predict the future… or get better at estimation. Instead, we recognize that estimation is a non-value added activity and minimize it as much as possible.

Most Agile estimation techniques use relative units.  This means that we don’t try to estimate dollars or days directly.  Instead, we use “points” or even qualitative labels and simply compare the items we are estimating to each other.  This takes advantage of the human capacity to compare things to each other and avoids our difficulty in comparing something to an abstract concept (such as dollars or days).

Check out my recent “Agile Planning in a Nutshell” article.

What Other Agile Estimation Methods Are There?  Please let me know in the comments and feel free to include a link!

Please share!

Link: Agile Coaching and Flight Instruction – An Emotional Connection

Learn more about transforming people, process and culture with the Real Agility Program

My friend Mike Caspar has another great blog post: Similarities between Agile Coaching and Flight Instruction.  Check it out!

Please share!

Collaborative Estimation Technique: Challenge, Estimate or Override (CEO)

Learn more about transforming people, process and culture with the Real Agility Program

Great technique described by Shahin Sheidaei on his blog: Challenge, Estimate or Override (CEO) Game for Effective Estimations.  It is much quicker than the Planning Game, and probably a bit slower than the Bucket System.

Please share!

Agile Planning in a Nutshell

Learn more about transforming people, process and culture with the Real Agility Program

Agile methods such as Scrum, Kanban and OpenAgile do not require long-term up-front plans.  However, many teams desire a long-term plan.  This can be thought of as a roadmap or schedule or a release plan.  Agile planning helps us build and maintain long-term plans.

Agile Planning Process

The steps to do this are actually very simple:

  1. Write down all the work to be done.  In Scrum these are called “Product Backlog Items”, in Kanban “Tasks” and in OpenAgile “Value Drivers”.
  2. Do some estimation of the work items.  Many Agile estimation techniques exist including Planning Poker, The Bucket System, Dot Voting, T-Shirt Sizes.  These tools can be applied to many types of estimation.  There are three kinds of estimation that are important for Agile Planning:
    1. Estimating relative business value.  Usually done with the business people including customers and users.
    2. Estimating relative effort.  Usually done by the Agile team that will deliver the work.
    3. Estimating team capacity.  Also done by the Agile team (this is sometimes called “velocity”).
  3. Create the long-term plan.  Use the team capacity estimate and the sum of all the effort estimates to come up with an estimate of the overall time required to do the work.  (In Kanban, which doesn’t have an iterative approach, this is a bit more complicated.)  Use the business value and effort estimates to determine relative return on investment as a way to put work items in a logical sequence.

Agile planning allows a team to update estimates at any time.  Therefore, the techniques used above should not be thought of as a strict sequence.  Instead, as the team and the business people learn, the estimates and long-term plan can be easily updated.  Scrum refers to this ongoing process at “Product Backlog Refinement”.

Principles of Agile Planning

In order to use Agile planning effectively, people must be aware of and support the principles of Agile planning:

  1. Speed over accuracy.  We don’t want to waste time on planning!  Planning in and of itself does not deliver value.  Instead, get planning done fast and use the actual delivery of your Agile team to adjust plans as you go.
  2. Collaborative techniques.  We don’t want to be able to blame individuals if something goes wrong.  Instead, we create safe estimation and planning techniques.  Inevitably, when an estimate turns out to be wrong, it is impossible to blame a single individual for a “mistake”.
  3. Relative units.  We don’t try to estimate and plan based on “real” units such as dollars or hours.  Instead, we use ordering, relative estimation and other relative techniques to compare between options.  Humans are bad at estimating in absolute units.  We are much better at relative estimation and planning.

Please share!

Pitfall of Scrum: Stretch Goals

Learn more about transforming people, process and culture with the Real Agility Program

The team decides on how much work it will do in a Sprint. No one should bring pressure on the team to over-commit. This simply builds resentment, distrust and encourages low-quality work. That said, of course teams can be inspired by challenging overall project or product goals. A stretch goal for a Sprint is just a way to 100% guarantee failure. Even the team should not set its own stretch goals.

There are a few interesting principles that apply here. For example, the Agile Manifesto mentions sustainability:

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

The Agile Manifesto also points out the importance of trust:

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Stretch goals are incompatible with both of these principles from the Agile Manifesto.

There are two types of stretch goals. The first type are those assigned by outsiders to the team. The second type are those which a team sets for itself. Both types are bad.

Stretch Goals Assigned by Outsiders

The worst extreme of this type of stretch goal is also the most common! This is the fixed-scope-fixed-date project deadline. In this type of stretch goal, the project team, doing Scrum or not, is forced to work backwards from the deadline to figure out how to get the work done. If the team can’t figure this out, managers often say things like “re-estimate” or “just get it done.” (Note: another thing that managers do in this situation is even worse: adding people to the project! Check out “The Mythical Man-Month” by F. Brooks for a great analysis of this problem.)

My anecdotal experience with this sort of thing is simple: quality suffers or sustainability suffers. I once worked with three other people on a mission critical project to help two banks with their merger. There was a regulatory deadline for completing the integration of the two existing systems for things like trading, etc. Fixed-scope-fixed-date. Coffee and sleepless nights were our solution since we tried not to sacrifice quality. We actually ended up working in my home for the last few 24-hour stretches so that we would have access to a shower. Suffice it to say, there’s no way we could have sustained that pace. It’s anti-Agile.

A quick search for ideas and opinions about stretch goals makes it very clear that there is no commonly agreed “correct” answer. However, from an Agile perspective stretch goals assigned by outsiders are clearly against the principles of the Agile Manifesto.

Stretch Goals Set by the Scrum Team

The Scrum Guide states:

The number of items selected from the Product Backlog for the Sprint is solely up to the Development Team. Only the Development Team can assess what it can accomplish over the upcoming Sprint.

For emphasis: what it can accomplish – not what it (the Development Team) wants to accomplish, or what it should accomplish, or what it could accomplish if everything goes perfectly. A Development Team should be accomplishing their Sprint plan successfully (all Product Backlog Items done) on a regular basis. Of course, exceptional circumstances may intervene from time to time, but the team should be building trust with stakeholders. Here’s another story:

I had a good friend. We would always go out for coffee together. We just hung out – chatted about life, projects, relationships. Of course, from time-to-time one or the other of us would cancel our plans. That’s just life too. But there came a time when my friend started cancelling more often than not. There was always a good excuse: I’m sick, unexpected visitors, work emergency, whatever. After a little while of this I started to think that cancelling would be the default. I even got to the point where I was making alternative plans even if my friend and I had plans. I got to the point where I no longer trusted my friend. It didn’t matter that the excuses were always good. Trust was broken.

It doesn’t matter why a team fails to meet a goal. It reduces trust. It doesn’t matter why a team succeeds in meeting a goal. It builds trust. Even among team members. A team setting stretch goals is setting itself up for regular failure. Even if the team doesn’t share those stretch goals with outsiders.

Stretch goals destroy trust within the team.

Think about that. When a team fails to meet its own stretch goal, team members will start to look for someone to blame. People look for explanations, for stories. The team will create its own narrative about why a stretch goal was missed. If it happens over and over, that narrative will start to become doubt about the team’s own capacity either by pin-pointing an individual or in a gestalt team sense.

Trust and Agility

The importance of trust cannot be over-stated. In order for individuals to work effectively together, they must trust each other. How much trust? Well, the Agile Manifesto directly addresses trust:

Build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.

Here is my recent YouTube video about stretch goals… check it out and subscribe to our channel!

Please share!

Teams and their proximity to the final user

Learn more about transforming people, process and culture with the Real Agility Program

A great, simple post from Mike Bowler…

Time: Teams that are writing code today that will be used by their customers tomorrow are very focused on what the customers actually need. Teams that are writing code today that won’t be seen by a customer for six months are less engaged.


Mike Caspar
Passionate About Agile

Please share!

Agile Transformation Metrics

Learn more about transforming people, process and culture with the Real Agility Program


When asked to provide metrics to assess “how well” an Agile transformation is going, re-frame the discussion around measuring changes in the impact the IT organization is having (or not) on it’s Business environment, and define a small set of “fitness for purpose” metrics.

The Inevitable Question about Agile Transformation Metrics

Sooner or later, as an IT organization embarks on a transformation towards Agile mindset and practices, someone will be asked to provide “hard evidence” that the effort is paying off, and the conclusion will be that metrics is the vehicle to satisfy that request. What are your Agile transformation metrics?

It’s been my experience that this request usually leads to a discussion about measuring the specific Agile initiatives the IT organization has launched. In organizations where the emphasis has been around engineering disciplines, such metrics might be things like unit test code coverage, or integration build times. If the focus  was around teams and process, then counting number of teams “converted” to Scrum, or people sent to Scrum Master training may appear as the choice.

While those measurement might be useful indicators in some context, they have two problems. First, they are akin to measuring the performance of the car engine without looking outside the window; the engine might be performing well, but if the car doesn’t have the wheels attached, we’re going nowhere. More importantly, though, these figures are usually meaningless for Business stakeholders, who are the ones usually asking for them in the first place.  Agile transformation metrics need to be meaningful to the Business.

Re-framing the Agile Transformation Metrics Question

Agile transformation efforts can be very costly exercises, therefore it is legitimate to ask about the results of such endeavour. The important thing to realize, though, is that this question is really equivalent to another question: “is the IT organization improving its impact on its Business environment.” Another way to put it is, borrowing from the terminology used by the Kanban community: “is the IT organization becoming more and more fit for purpose?” Answering this question, of course, requires a clear understanding of what is that the Business expects from its interactions with IT.

The IT organization can be seen as providing various services to customers. Arguably, if IT has decided to “transform” in some way (perhaps by moving towards an Agile mindset), it’s doing so to improve its impact on those customers, so this is what needs to be measured to know “how the transformation” is going.

Some of those customers are different areas of the organization (like Finance, or HR.) But it doesn’t stop there, because the Business’ engagement with IT doesn’t have value for its own sake. Ultimately, the Business is using IT as a way to optimize its operations so that it can provide external customers with more effective products and services. Moreover, IT is these days the direct channel through which those products and services are delivered to external customers (for example, through web sites and mobile applications.) Therefore, the concept of “fitness for purpose” of the IT organization can be extended to consider the fitness for purpose of the Business respect the external customers it intends to serve.

Defining the “Agile” Transformation Metrics

Measuring “agile transformation success” really means measuring the success of the exchanges between IT and the Business, and between the Business and its external customers.  Measuring the internal processes and practices that IT puts in place as part of that “transformation” is beside the point. This implies starting with a careful definition of the boundaries that delineate the exchanges to be measured. There might be more to external customer fitness for purpose than IT operations, for example, and that needs to be considered when defining Agile transformation metrics, especially if we’re later going to be drawing causation conclusions.

Defining Agile transformation metrics will be, of course,  a highly contextual exercise because every business organization is different.  But we can, however, draw again from the Kanban community for some general guidelines on what to look for. Their thought leaders talk about classifying metrics into 3 categories: fitness for purpose metrics, health indicators and improvement drivers.  Using this framework, when talking about “agile transformation metrics” we are referring mainly to the first category, and perhaps a bit to the second. Based on those, improvement initiatives can be put in place, and perhaps driven with metrics belonging to the third category.

A fitness for purpose metric (also known as KPI) is an indicator of something a customer will care about. This is a key distinction: if the metric is not easily recognizable and meaningful for the customer, then it’s not a KPI. Another key characteristic is that a minimum threshold for its value can be defined: if the metric goes below the threshold, the Business is putting the relation with its customers at risk (perhaps they will walk away, initiate legal actions, etc.). In other words, the Business is no longer “fit for purpose”. We can then measure the effectiveness of the “agile transformation” by analyzing how KPI values over time compare to their respective thresholds. A typical KPI is delivery time, measured from the moment a customer request is accepted and committed to, until the moment it’s delivered to production.  This is usually a good Agile transformation metric.

Health indicators are metrics that are inwards facing. Customers don’t really care about them (or even understand), but they indicate how a given aspect of the system is operating. The key characteristic is that they are not directly actionable; they only provide information that needs to be analyzed and put in context. As the value of a health indicator changes, we can draw some conclusions about how the system works, or explain why something is happening (or not), but it doesn’t necessarily leads to concrete action. Defect count is an example of this. Customers will certainly care about quality of the product, and we can make inferences about that quality by looking at how many defects we have, but the absolute number of defects will not necessarily make the product more or less fit for purpose. It may happen that customers consider the current quality to be “good enough”, irrespective of the number of defects.

Finally, improvement driver metrics are metrics put in place to influence behaviour towards a particular change. Their key characteristic is that they are temporary: we set a target on them and once the target is achieved, the metric is no longer necessary. The reason for this is related to the unintended behaviours that a metric might encourage in people, which may lead to locally optimizing the metric at the expense of other aspects, leading to global sub-optimization of the system. An example is unit testing code coverage: if we have determined that a given service is not fit for purpose and the cause is related to poor unit test coverage, then establishing a target for minimum coverage may influence developers to work on adding tests to reverse the situation.

Please share!