Mark Levison has written up a great list of Scrum Smells on his blog Notes from a Tool User. Check them out! There is a lot of great information there. He also asks your help: if you know of common problems with Scrum, see if you can add them to the list!
One of the challenges with agile methods is to get a clear perspective on how to measure process improvements. I recently had a brief discussion with a C-level executive at a small organization about this. His concern was that cycle time was meaningless because it depended so much upon the size of the work package. So how do we use cycle time as a meaningful measurement? What else can we use to measure process improvement?
Let’s look at the difference in measuring cycle time in an agile vs. non-agile environment. Then we’ll get to other measurements.
First, let’s define cycle time. From iSixSigma we have:
Cycle time is the total time from the beginning to the end of your process, as defined by you and your customer. Cycle time includes process time, during which a unit is acted upon to bring it closer to an output, and delay time, during which a unit of work is spent waiting to take the next action.
This definition is important because it gives us a clue about the potential difference between a waterfall vs. agile method of delivering value. Let’s imagine the typical process used in a waterfall environment. The following are the high-level steps:
So from the start of the customer request formally submitted to the time that the fulfillment of that request is made is our true cycle time. There are a few important things to note here. First, there is a queue of work based on requests made but not yet scheduled. There is another queue for work scheduled but not yet started. We know that if we can reduce the size of these queues, we can improve cycle time in a general sense. Second, we know that most organizations of any significant size will have different queues based on the urgency of the request. For example, a high severity bug discovered in the production system of a company’s largest client will be treated differently than a wish list item for a small not-yet-client. These two requests won’t even go in the same queue: the high priority problem will be quickly escalated to a support or development team that can work on it immediately. Third, it is tempting for the development group to measure their local cycle time. This is a Really Bad Idea since it leads to sub-optimizing behaviors. For example, it is easy for the development team to improve their cycle time by sacrificing quality… but this just causes the QA cycle time to increase, and probably the overall cycle time (true cycle time) is affected more than the local improvement in the development group’s cycle time.
Now let’s look at the steps that occur in an ideal agile environment:
So the ideal method of doing agile has a maximum cycle time of two months to deliver from the time a request is made… how many teams are doing this? Not many.
The ideal is extremely difficult to accomplish. Getting to that state requires that the development organization catches up to the business side so that there are zero pending requests at the start of each iteration. It also requires that the business side users and stakeholders are able to articulate their requests so that they are small, and appropriately detailed for the team doing the work.
A realistic agile implementation actually is a lot more messy. Depending on the type of request, the cycle time for a piece of work can vary widely. Some low priority items may take years even in an agile environment. A low priority request is made and approved but then never quite makes it into a project… and then once in a project never quite makes it to the top of the team’s product backlog. This is interesting to look at sometimes, but it points out another important aspect of measuring cycle time: mostly we care about average cycle time (or some other statistically interesting aggregate measure).
The predominant factor in most organizations’ cycle time is the number and size of the queues they use as work is processed. In most organizations there are several queues and most of them contain large numbers of requests or bits of work in process. Queues represent huge amounts of waste. It is easy to see that queue size and cycle time are closely related: the more items in a queue, the longer the cycle time.
This leads to a simple conclusion: regardless of lifecycle approach, reducing the size of an organization’s queues is one of the easiest ways to reduce cycle time. What are some common queues? There are often queues of projects, queues of enhancement requests, queues of defects to be fixed, queues of features, queues of tasks, queues of email (large inboxes), queues of approval requests, queues of production database changes. The number of queues increases the more an organization is oriented around functional groups, and the number of queues decreases the more an organization arranges work to be handled by cross-functional teams.
This is where queueing theory and agile methods intersect really well. Cycle time is related to the load on your system, in particular your units of work processing. In most organizations, teams are created to handle work. The more work given to a team simultaneously, the higher their utilization level. Many organizations like high utilization levels because it gives them a guarantee that people are doing valuable work all the time that they are paid to work. This is a completely false benefit and in fact is extremely destructive to overall productivity. From queueing theory we know that the cycle time for a piece of work increases exponentially to the utilization level. We see this whenever we over-load a server… but for some reason we fail to see this when we overload a person or a team or an organization even though it still happens.
Cycle time is also related to the variability in the size of the work packages. Low variability means that the exponential factor related to load is low, and high variability means that the exponential factor is high. In other words, if you have a highway that only allows motorbikes, you can have a very high load without getting bad traffic jams. On the other hand, if you have a highway that allows anything on it, you get traffic jams even with low levels of load. This is why HOV or commuter lanes and the left lane in multi-lane highways don’t typically allow transport trucks and buses. This result from queueing theory is not intuitively obvious so it is even harder for us to apply to software development.
But apply these two ideas, load and work size variability, we must if we wish to create a high performance development organization. The simplest way to do this is to have a single team work on a single project at a time and use iterations to ensure that the work being done is always exactly the same size - the size of the iteration.
It is possible to have very short iterations and still have a long cycle time. Many organizations make a few common mistakes with agile that cause this. If the work done inside each iteration is restricted to pure development work and everything else is done outside the iterations, then cycle time likely stays long. A common example of this is having the QA folks remain separate from the development team and do their work after a development team releases their work.
There is really only one way to avoid this: have a comprehensive definition of “done” that is met by the team every single iteration. This ensures that all work from idea to release for a given customer request is done inside a single iteration. A side effect of this is that all the pieces of work need to be small. It also gets rid of all the queues except one: the queue of ideas approved for delivery. With a single queue to manage, it becomes easy to measure cycle time, and therefore easy to improve it.
Improving cycle time can now be done in a few ways:
Once you have control of cycle time, it is possible to make reasonable measurements of productivity and two more metrics become extremely important (not that they weren’t important before, but they are easier to work with now). The first is Return on Investment (ROI) and the second is customer satisfaction.
ROI is in its simplest form a measure of how much benefit there is to doing something as compared to the cost of doing it. It takes into account the importance of time and timing, the importance of other options you may have, and of course, hopefully takes into account the business reality of your work. It also takes into account costs.
In software development, the primary cost is the cost of the staff doing the work, and the time factor is your cycle time (Ah! that’s where we use it). If you have a consistent team working on iterations that are always the same size and if you have little or no work being done outside of the iterations, it is very easy to calculate ROI in a useful way. Simply measure how much value a given iteration worth of work will generate and divide by the cost of the team for an iterations (and if the team is not yet doing work as it comes in, take into account the time value of money since the work might not be done for several iterations). Now, productivity is simply a measure of the Return for each Team-Iteration. Dollars/iteration. Simple. If the team’s productivity goes down, you can ask some really simple questions:
Customer satisfaction can be measured in many ways. If you have already started using agile practices, there is a good chance that your customers will already be more satisfied than they were before. This will show up informally through word-of-mouth. However, it is good to have a more systematic way of measuring customer satisfaction. One of the simplest and most commonly used methods of measuring customer satisfaction is the Net Promoter Score. From WikiPedia:
Companies obtain their Net Promoter Score by asking customers a single question (usually, “How likely is it that you would recommend us to a friend or colleague?”). Based on their responses, customers can be categorized into one of three groups: Promoters, Passives, and Detractors. In the net promoter framework, Promoters are viewed as valuable assets that drive profitable growth because of their repeat/increased purchases, longevity and referrals, while Detractors are seen as liabilities that destroy profitable growth because of their complaints, reduced purchases/defection and negative word-of-mouth. Companies calculate their Net Promoter Score by subtracting their % Detractors from their % Promoters.
The Net Promoter Score is closely linked to quality including the hard-to-measure parts of quality like responsiveness, ease of use, and fitness for purpose.
Cycle time also affects customer satisfaction. The faster you can respond to requests by customer, users or other stakeholders, the more likely they are to be satisfied. This happens for two reasons: fast response time means that solutions are more likely to still be useful and correct when actually delivered, and it also gives more opportunities for feedback.
In fact, if we look at these three measures, cycle time, ROI and customer satisfaction, we see that they form a mutually supporting and cross-checking system of ensuring productivity and effectiveness. Measuring anything else muddies the waters and can cause sub-optimal behaviors. The real challenge for most teams is realizing that all their local measures of performance and effectiveness may actually be causing harm (unintentionally) because they draw the team’s attention away from the three organizationally important measures.
Cycle time is the measure that is most closely related to process improvements, but ROI and customer satisfaction should also be used to ensure that process improvements don’t accidentally harm the organization.
I have been reading a book entitled “Agile Project Management with Scrum” by Ken Schwaber. It is an interesting read. The examples and stories that he shares of companies who have struggled with Scrum and those that have succeeded are fantastic. The way Schwaber breaks up the book and explains all the roles then gives example makes it a great learning tool. It is also really funny and clever.
One complaint I have with the book is that it is very technical, it seems that the reader is assumed to have many years of software development experience. It is interesting that the projects that Schwaber discusses that have the most trouble with Scrum are those that are “stuck” in their old ways of working. It’s almost as if the old saying of “A little knowledge is a dangerous thing” is true for Scrum implementations. “Scrum means doing things in small cycles - so I will do everything the same except in shorter cycles.” Anybody ever heard of that type of reasoning?
I definitely recommend this book for those who have considerable experience in the technology field. For those who don’t this book might be challenging at times, espcially with the computer language words that are used.
I want to continually learn for my own personal and professional growth. So IÂ would like to know which books do you suggest? Are there any books that share examples and stories that are not focused on software development? If you disagree which my review of the book please comment.
The Definition of “Done” is an important concept that helps us understand how to produce working, potentially shippable software at the end of every Sprint. Previously, I wrote about how to expand the definition of “done” from the perspective of the team’s skills, capabilities and work processes. This time around, let’s look at it from a more tactical perspective: how do we identify things that should be added to the definition of “done” and when do we do this? (more…)
This might be impossible, but I was thinking that it would be cool to have a single reference of all the possible agile practices. Obviously, since “agile” is not a single defined method, we must take the word “comprehensive” with a bit of humor (or a grain of salt). I’ve attached a spreadsheet that represents my first draft (it’s in OpenOffice.org format so that you don’t have to worry about me spreading viruses - if you want it in MS Office format, email me at mishkin@berteigconsulting.com). I’ve split the practices up into several sections including: “Agile Skeleton”, “Common Practices”, “Basic Scrum Practices”, “Optional Scrum Practices”, “Extreme Programming Practices” and “Lean Practices”. I’ve stopped there because I’m not an expert on other agile methods such as Crystal, Agile Unified Process or Feature Driven Development. I imagine that this list will be useful for teams to do self-assessment and to think about ways they might improve. Perhaps it could be used in a retrospective setting. Berteig Consulting coaches use something similar to this to assess the effectiveness of their engagement with clients. If you think of practices I’ve missed or other potential uses for a list like this, let me know in the comments. My intention is to convert this to a wiki and make it available under a Creative Commons license once it is a little more refined.
Agile Practices List (OpenOffice format - 68KB)
There was an interesting discussion on the LeanAgileScrum Yahoo Group early in December regarding the difference between flow (lean) and iterations (agile) that caught my eye. I only just now have had the time to write about it.
Okay, I’m only here for one day, and the first part of the day was me giving a workshop and then the next part was a presentation on Ruby and Selenium which was interesting (but unfortunately nothing to write home about) and then…
Mary Poppendieck spoke. Here are my notes.
Recently in my ScrumMaster Certification courses, I have been ending the course with a list of the rules of Scrum. This list of rules is based on my understanding and practice and may not be 100% the same as that found in the Scrum books or in other Scrum practitioners’ models. Nevertheless, I just recently checked it against some other online reference material and it seems like a reasonable list of rules. I am interested in any feedback or variations or disagreements…
[Updated: 20070922]
Without further ado, here they are, categorized into Required Rules, Basic Rules and Optional Rules, plus a downloadable PDF version.
In my coaching work, I have often been asked a question about the planning process for iterations, that until just a few days ago, I would actually brush off!!! I didn’t even realize I was doing this, it is only in retrospect that I see this. This question is simple: “how does a team plan for the improvement efforts that come out of the retrospective when they are supposed to be working at maximum velocity when implementing tasks directly related to the items in the work queue?” Or, more simply, “we don’t have time for process improvements.”
Mary and I worked together for a short time almost two years ago now. When we worked together I was impressed by her honesty, insight and wisdom. Here is a brief on-line interview she has done. It is focused on software development, but many of the ideas she discusses can be easily translated into other types of work.
Most of the teams and organizations I coach are working on using agile methods to improve their software development approach. Somewhere along the way, someone has realized that there must be a better way… either better than chaos, or better than bureaucracy. Over the years that I have been practicing agile methods, I have come to believe that quality is not negotiable.
Lean Manufacturing
If I am manufacturing computers, and I receive a large batch of CPU’s… say… Intel Pentium IIIs, I put them in inventory. There is a recurring montetary cost associated with keeping this inventory. There is, however, also a hidden cost. If I use up half my inventory to build computers, and then I inventory those computers, and I ship half of those computers, I now have 3/4 of my initial chips waiting around, earning me no money at all.
Then, Intel ships the new Pentium 4. What happens now? Well, I basically have to throw out my Pentium 3 stock (either by making low-cost, reduced-margin computers just to get rid of them, or by trying to sell the chips wholesale at cut rates). I also have to either slash prices on my remaining stock of computers, or re-fit them with the new processors. All of this is very expensive, and may eliminate most or all of the profits I was expecting to make on the original purchase. It is a scenario that is rife with waste.
Most manufacturing industry players have figured this out, and moved to a Just-In-Time model of production. Toyota pioneered much of this in the 1970s, and now inventory is seen as a liability, rather than an asset. This waste-free approach is a centrepiece of the Lean manufacturing revolution
Waste and Inventory in Software Development
Unfortunately, there is a parallel in computer software creation that has been rather poorly understood by businesses that need custom software development. Waste can be considered as anything that is unfinished, and/or unused. In software development terms, this can be applied to documents that will never be read or that might be unnecessary, and other such things. It is also true of unfinished software.
Traditional Software Development Example
Let us suppose that we ask someone to develop a piece of software with thirty features. Let us further suppose that this software will take about twelve months to produce. Let’s further suppose that ten of these are business-critical features, and that all of the features’ definitions are highly market-driven.
So a traditional software project starts to develop them. First there is a requirements analysis phase, then a design phase. Throughout there are lots of approval stages, sign-offs, etc. Then some team codes up the software starting somewhere around month five. Coding proceeds for about three months, after which the software goes through a testing process in month nine. In theory, at the end of this testing process (month twelve), we are supposed to receive the software for use, and we should be able to receive the value it was supposed to offer. However, defects are found in testing, requiring some re-work. The software is delayed by a further three months, and we finally receive it. By the time we receive it, our competetors have defined new features, and we have to submit new feature requests, whose results will not be seen for several months.
So for the entire development process, we received no business value, and continued to pay. Fifteen months from first investment until we started to achieve results that could mean revenue from the system. At this point, the software is partially obsolete. This is quite similar to the manufacturing scenario above.
Lean and Agile Software Development
Agile and Lean software development practices change this process by delivering business value a little-bit at a time. In an Agile software project, the business specifies what the most important features are (in our case the ten business-critical ones). The team then begins working in small time-boxes - say, two to four weeks. In each time-box, the team works on a small but well-defined list of features taken from the top of the prioritized feature list that we specified. At the end of the time-box, those features that were worked on are delivered to a degree of quality that each of the delivered features would be suitable for use by the customer. The whole might not be ready, but any features worked-on would be complete and production-ready.
If the team can average about three features per month (about what they pulled off in the above example), then by month four, the team could deliver a piece of software that has all ten business-critical features ready for use. The whole might not be ready, but the customers could determine whether those ten were worth taking the software in its current state, and packaging it up and deploying it. Quite simply, the software may not be “done”, but it’s “done enough”. Then the team can continue to roll-out less valueable features from the prioritized list.
Market and customer needs volatility
This becomes even more important when we consider the competition’s changes. In the traditional example above, after fifteen months, additional features were required. These may have been discovered in month six. Traditionally, these could only be considered in month sixteen. In Agile and Lean software development, however, work on these changes could be started in month seven or sooner. So the business received value early, or “just-in-time”, and they could get high-value changes “just-in-time”.
Quality
Because testing is built-in to the process, the customer is able to validate the product at the end of every time-box, so there are no large batches of re-work. The only time large batches of rework are necessary are when large changes to the basic requirements are requested. And then, Agile does not resist the customer’s desire to change, but recognizes it as an essential part of software delivery, and adapts to the changing consditions. As a result, the Agile and Lean methods are optimized to produce quality product in small increments, so quality doesn’t suffer from customers or markets changing their minds.
Summary
Agile and Lean principles are very powerful, and allow for business value to be delivered sooner to end-customers. This allows for better quality and risk management. It also allows strategic and tactical decision-making by executives to be undertaken when such decisions are most needed - not six months too late.
Firms that embrace Lean and Agile development principles are beginning to see the competetive differentiation that Toyota saw vis-a-vis the rest of the automotive industry throughout the 70’s and 80’s. It is only in the last two decades that, after adopting Lean practices, Toyota’s competitors have begun to close the gap. Agile software shops are beginning to achieve these same competetive gains. Those who don’t adapt to just-in-time value delivery - who don’t eliminate waste in their processes - will feel the results as their competitors take products and services to market far faster, and with more responsiveness to their customers.
“If you’re not on the steam-roller, you’re part of the asphalt.” - Lighthouse Design, circa 1996.
Lean Metrics… are Lean
In the book Lean Software Development by Mary and Tom Poppendieck, there are a small number of references to metrics: process cycle time, process cycle efficiency, business models. Lean practices focus very much on diagnostic metrics that are used to help people find and eliminate waste and improve value and quality. The other aspect of metrics is to measure up for purposes of motivation and performance measurement.
Conclusions for Scrum and Agile in General
Don’t prescribe specific metrics!
Agile work is about an empirical process where a team responds to change, learns, and self-regulates. An agile team has the power to choose their own metrics for their own self-inspection. For upper management, the single economic driver that is part of the “Good to Great” process should be sufficient.
See also: Appropriate Metrics and A Metric Leading to Success.
Here are my rough notes from the May 2005 Scrum Gathering in Boston. Regrettably I was not in the room for most of Mike Cohn’s presentation on User Stories… but his book (User Stories Applied : For Agile Software Development) is excellent 
The notes in this entry include predictions from Ken Schwaber, a presentation from Bob Schatz formerly of Primavera on their enterprise-wide implementation of Scrum, a panel discussion with Tim Bacon, Jeff McKenna, and Diana Larsen, moderated by Esther Derby. In the afternoon we heard from Pete Deemer about Yahoo!’s enterprise adoption of Scrum, Mike Cohn about User Stories, and to close the day we had an energetic presentation from Tim Dorsey of WildCard Systems about their enterprise implementation of Scrum.
Waste is the result of activities or environmental conditions that prevent a team from reaching its goal. The opposite of waste is something that adds value (more, faster or higher quality) to the desired result.