December 16, 2005
Two Motivational Metrics for Agile Teams
In general, an organization should have one metric that is used to measure success. However, along the way, it may be useful to temporarily use other metrics to help motivate, track, or predict work. Here are two metrics that can be used in this temporary manner for Agile Work.
Time to Obstacle Removal
TTOR = sum of calendar time elapsed for all unresolved obstacles divided by number of unresolved obstacles
Why Use TTOR: the team, process owner and product owner need to work hard to remove obstacles quickly. This metric tracks how well this is going.
When Use TTOR: use this only when people and the organization are not driven to remove obstacles. This metric can be tied to bonuses and/or compensation in order to give it teeth. Do not measure individuals with this metric. It is only applicable at the team level or higher (e.g. program or division level).
When to Stop Using TTOR: When TTOR has been reduced to less than the time between status meetings (usually one day), then the organization is doing well and this metric should be discontinued.
Obstacles Removed per Iteration
OR/I = number of obstacles closed as removed (not ignored or deferred) in a single iteration
Why Use OR/I: this is a "game" metric or score that encourages large numbers of obstacles to be identified and removed. The more obstacles removed in an organization or team context, the faster the work can be done.
When Use OR/I: if a team is not identifying obstacles in the status meetings, consider implementing this metric to encourage obstacle identification... and critically... resolution. This metric is for team use only.
When to Stop Using OR/I: this metric stops being useful if team members are spending time concocting obstacles that are trivial or worse, deliberately creating obstacles to be resolved. The Process Facilitator needs to be sensitive to these potential pitfalls of this metric. Once the team is consistently identifying one or two obstacles per status meeting, and they are getting resolved in equal quantity, this metric must be discontinued.
A Word of Caution: there is no substitute for a . These two metrics are contextual and temporary. Do not use them if they are not indicated by the circumstances and be sure to discontinue use when symptoms disappear.
Posted by Mishkin Berteig at 11:57 PM | |
November 16, 2005
A Metric Leading to Success
I just recently read the article by Ron Jeffries: A Metric Leading to Agility. It's a good, well written, well thought out article. If you are in the software business, check it out if you haven't already.
However, much as it is good advice, I believe that I must add a cautionary note.
Why Agility?
The title of the above article, and the metric suggested (Running Tested Features), imply some pretty serious questions: why agility? Aren't we really trying to help organizations succeed? Isn't agility just a means?
I've seen organizations adopt Agile in some form or another for lots of different reasons. The best reason is because it is recognized as a means to succeed. However, as a means, you don't want Agile to drive change in your organization. Something else should be doing that. Some need, some failing, some gap or some pain. And that need should be measured.
Appropriate Metrics
Running Tested Features as a metric is indeed powerful: it is difficult to "game" and it seems to promote the delivery of features, which presumably are valuable. The problem comes with the presumption of value. What if you end up choosing the wrong features? What if you are using agile on a non-new-product, non-software project? What if you don't care what method you use, but you want a metric that will drive success (and might as a by-product encourage Agile approaches)?
One great approach to Metrics comes from the book "Good to Great" by Jim Collins. The research team for the book discovered that every organization (in their sample) that made a sustainable change from mediocre to great results had a central, single metric (among other things) that was used as an economic driver to guide and measure the transformation. I enquired of the author about this: did the research team discover the use of more than one metric? Joanne Ernst on the research team responded:
I discussed your email with Jim, and he says that all of the metrics/measures that were consistently shared by GTG companies were reported in the book – e.g., the economic driver. (Personal Communication, 26 Jun 2005)
When looking at an organizational level, suddenly Running Tested Features seems like it might be a little beside the point. Mary and Tom Poppendieck in "Lean Software Development" discuss using financial models to help drive a team's work to become more Lean, more Agile... and specifically to be aligned with what is going to bring success to the organization.
Using metrics to drive behavior or performance is tricky business. Simpler is better, fewer is better and directly related to overall organizational goals is better. Measuring Running Tested Features can indeed drive a team to become more agile, but that might not be the best thing for the organization, or it may not be applicable to the organization. There is no One True Metric. Each organization and team has to find out what works best based on application of principles and experimentation.
One metric that leads to agile? Maybe. One metric that leads to success? That's what your organization needs to find for itself.
Posted by Mishkin Berteig at 07:15 AM | |
May 12, 2005
Appropriate Metrics - Continued
Lean Metrics... are Lean
In the book Lean Software Development by Mary and Tom Poppendieck, there are a small number of references to metrics: process cycle time, process cycle efficiency, business models. Lean practices focus very much on diagnostic metrics that are used to help people find and eliminate waste and improve value and quality. The other aspect of metrics is to measure up for purposes of motivation and performance measurement.
Conclusions for Scrum and Agile in General
Don't prescribe specific metrics!
Agile work is about an empirical process where a team responds to change, learns, and self-regulates. An agile team has the power to choose their own metrics for their own self-inspection. For upper management, the single economic driver that is part of the "Good to Great" process should be sufficient.
Posted by Mishkin Berteig at 07:14 PM | |
May 11, 2005
Appropriate Metrics
At the Advanced ScrumMaster Training, Ken Schwaber presented a substantial amount of thinking about metrics used with Scrum. The main driver for thinking about metrics has come from implementing Scrum in enterprise situations. Management expects metrics to be used in order to provide visibility into the progress of the Scrum implementation.
While this driver has some legitimacy, there are three main concerns to prescribing metrics for use with Scrum.
1. Planned/Engineering Approach - metrics such as "ideal person days" smell like the bad old way of treating people as resources that can be swapped in and out of a project.
2. Normative vs. Empirical - metrics are often used to set a standard for reasons such as prediction, and expectation. Scrum is about discovery and improvement not prediction and planning.
3. Is Something in Scrum not Working? Is adoption being hindered? Are too many Scrum implementations failing? Are metrics a critical success factor?
Keep these concerns in mind while considerig the uses of metrics.
What are Metrics for?
1. Self-Evaluation over Time - use a metric to track the progress of a group. A measurement is taken at a certain point in time, and then taken repeatedly over intervals. Example: the velocity at which a team completes work can be used to identify problems and opportunities for a team.
2. Control - use a metric to legislate the qualities of some process or attribute of work. A goal or standard is set for a measurement. The size of a queue of projects waiting to be worked upon by a team can be controlled in order to limit the size of work in progress and therefore project inventory.
3. Prediction - use a metric with values collected over time in order to predict future values of that metric. By implication, the metric is used to predict the performance of a system. The number of additional tasks discovered inside an iteration can be tracked and used to determine future expectations on extra tasks discovered.
4. Performance Measurement - use a metric in order to determine rewards and/or punishments and/or adjustments to behavior. An individual on a team may be evaluated based on their productivity as measured by lines of code completed and rewarded or penalized on that basis (not recommended!).
5. Behavior Motivation - use a metric to guide behavior by setting a context for thinking, action and reflection. Focus on an important measure in order to draw attention to improving the attribute with which the metric is associated. Measuring the time elapsed from conception of a project idea to the time it actually delivers valuable results can draw attention to improving speed.
The Lesson from Good to Great
Good to Great: Why Some Companies Make the Leap... and Others Don't by Jim Collins discusses the attributes and behavior that are common and unique to companies that have gone from a long history of mediocre results to a long run of great results. One identified aspect is referred to as the "Hedgehog Principle". In this principle, three questions are answered: "what can we be the best in the world at?", "what can we be passionate about?", "what is my one economic driver?"
The economic driver is a metric. For example, at Walgreens, their driver is profit per customer visit. Other large institutions have other metrics. But all good to great companies have a single economic driver metric that guides all their decision making.
Posted by Mishkin Berteig at 05:21 PM | |
April 20, 2005
Scrum Scorecards - Measurements to See Progress of Agile Adoption
Two interesting visual presentations of the progress of adoption of Scrum practices. These are marginally software-specific but could very easily be adapted to non-software agile work situations.
http://wiki.scrums.org/index.cgi?ScrumScoreboard
Posted by Mishkin Berteig at 04:31 PM | |