Agile Advice - Working With Agile Methods (Scrum, XP, Lean)

AGILE Project Management with Scrum -A book by Ken Schwaber

Prior to the Certified ScrumMaster seminar I attended in August 2008, I read the book by Ken Schwaber called Agile Project Management with Scrum based on a recommendation from Mishkin Berteig.    After attending the seminar and becoming certified as a ScrumMaster I re-read the book.   The second reading was much more valuable than the first for I had a much better understanding of Scrum.   Here are my comments on this book.

What have I learned?

1.    The adoption of Scrum methodology is more about changing roles and behaviours than it is about embracing a new process.

It was obvious to me and to Ken that one of the greatest challenges facing those individuals when moving from a their current environment to a scrum environment was that they would need to change their behaviours.    In the former environment the team member would be directed and inspected based on what their project manager told them to do.  The PM is the boss and the team members are somewhat powerless.  In Scrum the team members take responsibility for their commitments and communicate their accomplishments on a daily basis.  The hardest change occurs when the project manager is asked to become a ScrumMaster.  The project Manager is familiar with assigning tasks and personally inspecting results. In the scrum environment they are the servants of the team, removing obstacles and facilitating the process.   As Ken states in this book some project managers have great difficulty transitioning into the ScrumMaster role.  They are unwilling to give up the power and position as a project master.   It is hard to move from the leader of the pack to become the sheep dog herding the sheep!

2.    Scrum is unforgiving for if you do not apply the fundamental principles it is likely your efforts to adopt Scrum will fail.

As I reviewed the numerous case studies Ken chronicled is was apparent that when organizations, Team members, Product Owners and ScrumMasters followed the terms, conditions and guidelines of the Scrum methodology, they tended to deliver on their commitments.   When they misunderstood, misused or deleted some portion of the methodology they tended not to accomplish their objectives.   The methodology is well thought out and works in many situations when used appropriately.

3.    Scrum enhances individual and team expertise.

I agree and totally support Ken’s opinion about the value of Scrum.   I have no doubt the individual team member is empowered and has a greater sense of achievement.    Obviously based on his case studies, Ken builds a strong case that Scrum allows the team to deliver quicker.  The process is more change adaptive, responsive to customer needs, timely and economical than traditional methods.   Greater energy and capacity is released in the team and individual team members.

Agile Project Management - Certified ScrumMaster Training
Offered by Berteig Consulting Inc. www.berteigconsulting.com

Agile methods promise early, continuous and accelerating return on investment for stakeholders and a high level of job satisfaction for teams. Agile methods are appropriate for a variety of projects including program development, materials development, new product development, IT projects and software development outsourcing. In this three-day course, you will learn the foundational skills of agile process facilitation and project management, focused on the role of the ScrumMaster (also known as: Agile Project Manager, Process Facilitator, Team Coach).
What You Will Learn:

In this 3-Day course participants learn how to guide a team, a project, or an organization into a process of rapid delivery of valuable results. Exercises, case studies, and examples are used to assist participants to develop the knowledge, skills, capabilities and insights needed to make the shift from traditional management practices to Agile practices.

In this workshop you will learn:
• agile project management practices that enable hyper-productive projects;
• agile processes and facilitation tools that improve the productivity of the team;
• agile collaboration methods to remove the barriers between the team and the customer; and,
• to improve the lives of the team members by facilitating creativity and empowerment.
Why is this ScrumMaster course better than others out there?

Our Agile Project Management / ScrumMaster course is unique in many ways. Here is what you get when you attend our course:
• Three intensive days of training with exercises, simulations, discussion and lecture
• Classroom management using agile methods so that you can learn by example
• Course material designed for the ScrumMaster/Project Manager role not diluted to accommodate other roles
• Taught by one of the most experienced Certified Scrum Trainers in the world
• Low class size with a maximum of 15 students compared to some trainers with 20, 30 or even 50 students!
• Electronic copies of all course materials including free updates forever
• 24 PDUs if you are a member of the Project Management Institute
• Certification as a ScrumMaster at the ScrumAlliance that never expires
• One year paid membership in the ScrumAlliance

• Light breakfast and lunch all three days

Note: To learn more about this course and to register, please visit our website:

http://www.berteigconsulting.com/AgileScrumCertificationCSMTrainingOctober2008TorontoOntarioCanada

http://pligg.scrum-on.com/ - Just found this through Niall O’Keeffe who is an attendee at my Certified ScrumMaster course.

It seems that Berteig Consulting is blocked from China, so I’ve made a new site called Scrum Beijing to promote the ScrumMaster training I do there.

Also, the price has been reduced on the Scrum Study Guide and the web site now includes screen shots and a bit more useful information.

Berteig Consulting is starting to formally market OpenAgile as a method for helping small businesses manage their work (in any industry).  Heck, we’re using it ourselves.

And of course, a reminder about the Agile 2008 conference happening in Toronto in August - don’t miss it, it’s going to be fantastic!

Mark Levison has written up a great list of Scrum Smells on his blog Notes from a Tool User.  Check them out!  There is a lot of great information there.  He also asks your help: if you know of common problems with Scrum, see if you can add them to the list!

One of the challenges with agile methods is to get a clear perspective on how to measure process improvements. I recently had a brief discussion with a C-level executive at a small organization about this. His concern was that cycle time was meaningless because it depended so much upon the size of the work package. So how do we use cycle time as a meaningful measurement? What else can we use to measure process improvement?

Let’s look at the difference in measuring cycle time in an agile vs. non-agile environment. Then we’ll get to other measurements.

Cycle Time , Waterfall and Agile

First, let’s define cycle time. From iSixSigma we have:

Cycle time is the total time from the beginning to the end of your process, as defined by you and your customer. Cycle time includes process time, during which a unit is acted upon to bring it closer to an output, and delay time, during which a unit of work is spent waiting to take the next action.

This definition is important because it gives us a clue about the potential difference between a waterfall vs. agile method of delivering value. Let’s imagine the typical process used in a waterfall environment. The following are the high-level steps:

1. Customer / User / Stakeholder sees a need, validates it and submits a request to have that need fulfilled. This is when we start the clock on cycle time.
2. The fulfillment organization (IT, Product Development, R&D) puts the request in a queue, backlog or requirements management system.
3. Along with other requests, the fulfillment organization schedules the work on the request, usually by creating a project to fulfill it and other related requests. The project is estimated at a high level, the current status of in-flight projects is noted, and the new project is prioritized relative to other projects.
4. At some point, based on the schedule and the reality of the work on other projects, the project containing our customer’s request is started. Here, “started” means that detailed requirements are gathered.
5. After sufficient requirements are gathered, a detailed technical analysis is done including architecture, high-level design, risk analysis, etc.
6. Development begins. (Note: many people mistakenly start measuring cycle time here.)
7. Developers and testers work to validate the results of development and fix any problems discovered.
8. Final acceptance testing is done.
9. The results of the project are deployed to users, sold to the client, or in some other way passed back to the original requestor. This is when we stop the clock on cycle time.

So from the start of the customer request formally submitted to the time that the fulfillment of that request is made is our true cycle time. There are a few important things to note here. First, there is a queue of work based on requests made but not yet scheduled. There is another queue for work scheduled but not yet started. We know that if we can reduce the size of these queues, we can improve cycle time in a general sense. Second, we know that most organizations of any significant size will have different queues based on the urgency of the request. For example, a high severity bug discovered in the production system of a company’s largest client will be treated differently than a wish list item for a small not-yet-client. These two requests won’t even go in the same queue: the high priority problem will be quickly escalated to a support or development team that can work on it immediately. Third, it is tempting for the development group to measure their local cycle time. This is a Really Bad Idea since it leads to sub-optimizing behaviors. For example, it is easy for the development team to improve their cycle time by sacrificing quality… but this just causes the QA cycle time to increase, and probably the overall cycle time (true cycle time) is affected more than the local improvement in the development group’s cycle time.

Now let’s look at the steps that occur in an ideal agile environment:

1. As before, the Customer / User / Stakeholder sees a need, validates it and submits a request to have that need fulfilled.
2. That request is immediately placed in a ready state for the next iteration (cycle, sprint) of a delivery team. Elapsed time: maximum one month.
3. Team completes the request including all work to actually deliver/deploy and work is delivered to the stakeholder at the end of the iteration. Elapsed time: maximum two months.

So the ideal method of doing agile has a maximum cycle time of two months to deliver from the time a request is made… how many teams are doing this? Not many.

The ideal is extremely difficult to accomplish. Getting to that state requires that the development organization catches up to the business side so that there are zero pending requests at the start of each iteration. It also requires that the business side users and stakeholders are able to articulate their requests so that they are small, and appropriately detailed for the team doing the work.

A realistic agile implementation actually is a lot more messy. Depending on the type of request, the cycle time for a piece of work can vary widely. Some low priority items may take years even in an agile environment. A low priority request is made and approved but then never quite makes it into a project… and then once in a project never quite makes it to the top of the team’s product backlog. This is interesting to look at sometimes, but it points out another important aspect of measuring cycle time: mostly we care about average cycle time (or some other statistically interesting aggregate measure).

The predominant factor in most organizations’ cycle time is the number and size of the queues they use as work is processed. In most organizations there are several queues and most of them contain large numbers of requests or bits of work in process. Queues represent huge amounts of waste. It is easy to see that queue size and cycle time are closely related: the more items in a queue, the longer the cycle time.

This leads to a simple conclusion: regardless of lifecycle approach, reducing the size of an organization’s queues is one of the easiest ways to reduce cycle time. What are some common queues? There are often queues of projects, queues of enhancement requests, queues of defects to be fixed, queues of features, queues of tasks, queues of email (large inboxes), queues of approval requests, queues of production database changes. The number of queues increases the more an organization is oriented around functional groups, and the number of queues decreases the more an organization arranges work to be handled by cross-functional teams.

Cycle Time and Work Package Size

This is where queueing theory and agile methods intersect really well. Cycle time is related to the load on your system, in particular your units of work processing. In most organizations, teams are created to handle work. The more work given to a team simultaneously, the higher their utilization level. Many organizations like high utilization levels because it gives them a guarantee that people are doing valuable work all the time that they are paid to work. This is a completely false benefit and in fact is extremely destructive to overall productivity. From queueing theory we know that the cycle time for a piece of work increases exponentially to the utilization level. We see this whenever we over-load a server… but for some reason we fail to see this when we overload a person or a team or an organization even though it still happens.

Cycle time is also related to the variability in the size of the work packages. Low variability means that the exponential factor related to load is low, and high variability means that the exponential factor is high. In other words, if you have a highway that only allows motorbikes, you can have a very high load without getting bad traffic jams. On the other hand, if you have a highway that allows anything on it, you get traffic jams even with low levels of load. This is why HOV or commuter lanes and the left lane in multi-lane highways don’t typically allow transport trucks and buses. This result from queueing theory is not intuitively obvious so it is even harder for us to apply to software development.

But apply these two ideas, load and work size variability, we must if we wish to create a high performance development organization. The simplest way to do this is to have a single team work on a single project at a time and use iterations to ensure that the work being done is always exactly the same size - the size of the iteration.

Improving Cycle Time

It is possible to have very short iterations and still have a long cycle time. Many organizations make a few common mistakes with agile that cause this. If the work done inside each iteration is restricted to pure development work and everything else is done outside the iterations, then cycle time likely stays long. A common example of this is having the QA folks remain separate from the development team and do their work after a development team releases their work.

There is really only one way to avoid this: have a comprehensive definition of “done” that is met by the team every single iteration. This ensures that all work from idea to release for a given customer request is done inside a single iteration. A side effect of this is that all the pieces of work need to be small. It also gets rid of all the queues except one: the queue of ideas approved for delivery. With a single queue to manage, it becomes easy to measure cycle time, and therefore easy to improve it.

Improving cycle time can now be done in a few ways:

1. Put a cap on the number of items in the work queue. Since cycle time is directly related to the size of the queues in a system, this is a sure way of putting a maximum on cycle time.
2. Go through all existing requests and throw as many away as possible. This can be tough to do, but if you are able to do a cost benefit analysis, you will typically find that older items in the queue are no longer worth while.
3. Provide more stringent gating functions for allowing requests onto the queue. The few items added, the faster the size of the queue is reduced.
4. And of course, increase the performance of your team(s) so that they go through items on the queue more quickly.

Productivity and Cycle Time

Once you have control of cycle time, it is possible to make reasonable measurements of productivity and two more metrics become extremely important (not that they weren’t important before, but they are easier to work with now). The first is Return on Investment (ROI) and the second is customer satisfaction.

ROI is in its simplest form a measure of how much benefit there is to doing something as compared to the cost of doing it. It takes into account the importance of time and timing, the importance of other options you may have, and of course, hopefully takes into account the business reality of your work. It also takes into account costs.

In software development, the primary cost is the cost of the staff doing the work, and the time factor is your cycle time (Ah! that’s where we use it). If you have a consistent team working on iterations that are always the same size and if you have little or no work being done outside of the iterations, it is very easy to calculate ROI in a useful way. Simply measure how much value a given iteration worth of work will generate and divide by the cost of the team for an iterations (and if the team is not yet doing work as it comes in, take into account the time value of money since the work might not be done for several iterations). Now, productivity is simply a measure of the Return for each Team-Iteration. Dollars/iteration. Simple. If the team’s productivity goes down, you can ask some really simple questions:

• Did the expected return of the work go down? If so, is there more valuable work the team should be doing? This becomes an opportunity for product improvement.
• If not, what caused the team to get less done? Was the work harder than expected? Was there a skill gap? Was there an organizational obstacle that was revealed? Was someone sick? This becomes an opportunity for process and team improvement.

Customer satisfaction can be measured in many ways. If you have already started using agile practices, there is a good chance that your customers will already be more satisfied than they were before. This will show up informally through word-of-mouth. However, it is good to have a more systematic way of measuring customer satisfaction. One of the simplest and most commonly used methods of measuring customer satisfaction is the Net Promoter Score. From WikiPedia:

Companies obtain their Net Promoter Score by asking customers a single question (usually, “How likely is it that you would recommend us to a friend or colleague?”). Based on their responses, customers can be categorized into one of three groups: Promoters, Passives, and Detractors. In the net promoter framework, Promoters are viewed as valuable assets that drive profitable growth because of their repeat/increased purchases, longevity and referrals, while Detractors are seen as liabilities that destroy profitable growth because of their complaints, reduced purchases/defection and negative word-of-mouth. Companies calculate their Net Promoter Score by subtracting their % Detractors from their % Promoters.

The Net Promoter Score is closely linked to quality including the hard-to-measure parts of quality like responsiveness, ease of use, and fitness for purpose.

Cycle time also affects customer satisfaction. The faster you can respond to requests by customer, users or other stakeholders, the more likely they are to be satisfied. This happens for two reasons: fast response time means that solutions are more likely to still be useful and correct when actually delivered, and it also gives more opportunities for feedback.

In fact, if we look at these three measures, cycle time, ROI and customer satisfaction, we see that they form a mutually supporting and cross-checking system of ensuring productivity and effectiveness. Measuring anything else muddies the waters and can cause sub-optimal behaviors. The real challenge for most teams is realizing that all their local measures of performance and effectiveness may actually be causing harm (unintentionally) because they draw the team’s attention away from the three organizationally important measures.

Cycle time is the measure that is most closely related to process improvements, but ROI and customer satisfaction should also be used to ensure that process improvements don’t accidentally harm the organization.

After a rigorous submission process, a total of five presentations by Berteig Consulting staff and affiliates are accepted into the program for the Agile 2008 conference in Toronto. The five presentations are:

Meta-Agile: Using Agile Methods to Deliver Agile Training (3 hours, workshop format) - Mishkin Berteig

Extremely Short Iterations as a Catalyst for Effective Prioritization of Work (30 minutes, experience report) - Mishkin Berteig

The Learning Circle: An Evolution of Agile for Learning Environments (30 minutes, experience report) - Garry Berteig

Using Agile Engineering Tools and Practices to Achieve Organizational Change (90 minutes, talk) - Christian Gruber

Maven and Continuum - Building an Ecosystem for Agile Builds and Testing (3 hours, workshop) - Christian Gruber

Berteig Consulting is also using the vendor talk slot to provide additional value to conference goers with an introduction to using OpenAgile to run a small business.

OpenAgile: A Method for Business Development (30 minutes, vendor talk) - Paul Heidema and Mishkin Berteig

Everywhere I go, there are three practices that make the biggest difference in overall productivity for teams and organizations. All three practices are part of agile methods such as Scrum and Extreme Programming, but you don’t need to be doing these methods to take advantage of these practices. All of them are relatively inexpensive, and the return on investment for these practices is HUGE!!! Without further ado…

1. A Proper Team Room

This is astonishing: you can expect a 60% boost in team productivity from this single practice! The cost of re-stacking your cubes or office spaces is trivial compared to the benefits. If you are going to do this, do it right! Do the research, hire an agile coach or consultant, but make sure it is done well. One organization I worked with was very excited about their new team room setup. They had a nice open-concept layout with lots of windows etc. But they had also made some big mistakes including that all the developers on a single team would have a low wall separating them from each other. Because of poor layout that would block communication paths, their new setup would actually be worse than their old setup! Some research has shown that you can expect as much as a doubling of productivity (reference). This is one practice you don’t want to let your competitors pick up before you do! Here are some tips on agile team room setup.

2. Short Iterations

Once you have set up your team room, it is critical for your team to have something to do! The fastest way to get your team doing something is to start using short cycles of work (iterations, sprints) to deliver valuable results such as working software. Many software development projects use iterations that are two weeks long or even a month long. I strongly recommend iterations that are only one week long. Again, the benefits are incredible: your team will move through the stages of team development (forming, storming, norming and performing - reference) much more quickly than with longer iterations or no iterations… thus leading to high productivity much sooner. The value here is in the time gained. This chart demonstrates how this works:

The short iterations provide a certain type of pressure that forces team and project crisis to happen quickly. However, because iterations deliver working, valuable results, the pressure is not demoralizing, instead it motivates teams to get through the crisis and reach the norming and performing stages of development quickly. Again, to make this work, there are some critical success factors including methods of allowing team commitment, self-organizing and obstacle removal.

3. Test Driven Development

There is a myth that speed and quality are mutually exclusive. This comes from the idea that you need to slow down to make stuff high quality or that you need to sacrifice quality in order to go fast. It is true that initially you might get gains through these approaches. The really amazing thing happens when you try, deliberately and systematically, to do both high speed and high quality work. In software development this is best done through test driven development. In informal polling I’ve done with teams I’ve worked with, test driven development produces a noticeable long-term productivity gain as well as a simultaneous increase in developer and end user satisfaction due to a substantial reduction in defects discovered after the code leaves the developers. I have seen teams doing this that reduce defect rates to 5% (or less!) of what they once were prior to test driven development… while at the same time delivering projects faster than expected. Since substantial expense is squandered on defect management (tools, support teams, user training, lost productivity, etc.) and since staff turnover is also high in IT and high-tech, the results of test driven development on the bottom line are substantial.

Benefit of All Three Practices

If a team and an organization adopt these practices, get through the initial cost of learning them, then I would like to suggest that your teams can easily double their productivity if not more. For a team of 5 people working on a 100 day project this amounts to shortening the project to 50 days (save $200,000) or get twice as much work done. Clearly, an organization that adopted these practices and perfected them would save huge amounts of money and would be able to crush any competitors not doing this.

Previously I wrote a more general treatment of the benefits of agile and an article that lists other resources discussing the benefits of agile.

Any discussion of benefits should at least say a few words about how exactly to measure those benefits! However, I’m out of time. How do you measure the benefits of agile?

Hey folks, if you found this useful, could you please “Digg” and “Reddit” this article?  Thanks!

Not getting the benefits of agile? Consider the Agile Clinic!

Scrum Study Guide, “The Best Tool for New ScrumMasters”, is now available at Scrum Study Guide. This guide is designed to be an editable tool for helping ScrumMasters do their jobs effectively. With the Scrum Study Guide you are able to keep track of the rules of Scrum, to keep structured notes on your own job as the ScrumMaster, to maintain a list of online reference material, to assess the progress of your team, and to organize the obstacles you are working on.  It also contains a wealth of reference information for learning along the way.

This is the project I’ve been working on for the last several months that has reduced my output here on Agile Advice.  It represents a huge investment of my time as well as several other people who have assisted me in this including Paul Heidema and Garry Berteig.  Purchasing the Scrum Study Guide, aside from its usefulness as a tool itself, will also give you substantial discounts on other services offered by Berteig Consulting Inc.  Finally, if you like it, you can help to share it by letting us know who should get a discount on their purchase of the Scrum Study Guide.  Enjoy!

Our earlier post saying that it was available was a mistake.

Sorry for the mistake.

The recent growth in the popularity of agile methods such as Scrum is gratifying. However, I am constantly encountering people looking for the Silver Bullet of software development. In the paper written by Brooks, No Silver Bullet[pdf], he describes “accidental” and “essential” complexity. Agile in no way changes his arguments. What agile methods do is to help remove the accidental complexity associated with people and their interactions. This can lead to substantial increases in productivity, but it does not change the hardness of the underlying problem that is being solved by building a particular software system. In fact, doing a good job with agile methods, in particular Scrum, is extremely hard work due to the deep cultural shifts that must occur in order to get the full benefits.

The Definition of “Done” is an important concept that helps us understand how to produce working, potentially shippable software at the end of every Sprint. Previously, I wrote about how to expand the definition of “done” from the perspective of the team’s skills, capabilities and work processes. This time around, let’s look at it from a more tactical perspective: how do we identify things that should be added to the definition of “done” and when do we do this? (more…)

Berteig Consulting with the help of some partners is now offering a new service called The Agile Clinic. This is not a typical coaching or training session. The entire clinic has a duration of just one day. During this day there are short 30 or 60 minute appointments made by managers, executives, and staff with two experienced agile coaches. These coaches listen to problems presented to them, consult, discover, facilitate, diagnose, and offer solutions. These appointments are designed to be intense and high-impact sessions. Visit www.agileclinic.com to see how this service can add great value and provide fantastic results to your company with a small time cost.

This might be impossible, but I was thinking that it would be cool to have a single reference of all the possible agile practices.  Obviously, since “agile” is not a single defined method, we must take the word “comprehensive” with a bit of humor (or a grain of salt).  I’ve attached a spreadsheet that represents my first draft (it’s in OpenOffice.org format so that you don’t have to worry about me spreading viruses - if you want it in MS Office format, email me at mishkin@berteigconsulting.com).  I’ve split the practices up into several sections including: “Agile Skeleton”, “Common Practices”, “Basic Scrum Practices”, “Optional Scrum Practices”, “Extreme Programming Practices” and “Lean Practices”.  I’ve stopped there because I’m not an expert on other agile methods such as Crystal, Agile Unified Process or Feature Driven Development.  I imagine that this list will be useful for teams to do self-assessment and to think about ways they might improve.  Perhaps it could be used in a retrospective setting.  Berteig Consulting coaches use something similar to this to assess the effectiveness of their engagement with clients.  If you think of practices I’ve missed or other potential uses for a list like this, let me know in the comments.  My intention is to convert this to a wiki and make it available under a Creative Commons license once it is a little more refined.

Agile Practices List (OpenOffice format - 68KB)

I’m working with a number of companies using agile methods that have between 10 and 20 teams all working on the same product/project/program. They didn’t start small. These aren’t cases of organically growing from one good agile team to many good agile teams. Rather, these are organizations that have grown up in a non-agile approach and now want to reap the benefits of agile with their many teams. What is interesting is that these organizations all have some common problems and then all have some unique problems. There isn’t an obvious prescription for how they should be doing their agile implementations. I hope to write a few articles about scaling agile and scrum, and this one is our starting point: what reading should you do if you find yourself in the situation of trying to build a large agile organization.

(more…)