Agile Advice

October 23, 2007

Scrum Rules Cheat Sheet - Updated!

A few weeks ago, after my last update of the Scrum Rules Cheat Sheet, I decided that it might be a good idea to ask Ken Schwaber about what I had developed... He immediately asked if I had checked it against the rules listed in his book Agile Project Management with Scrum. Of course, I hadn't. In fact, although I've read the book, I neglected to read the appendix in which the rules are contained. So, I read it and updated the cheat sheet. I found that there were a few rules that I was "assuming" because I work with Scrum so frequently. As well, there were a few rules that I actually didn't remember. The cheat sheet is now getting to the point where I would like to break it into two pages, but then it wouldn't be a cheat sheet anymore... so with smaller font size... presenting, the new and improved Scrum Rules Cheat Sheet. Here is the original article about the Scrum Rules Cheat Sheet.

Posted by Mishkin Berteig at 08:19 AM | |

October 12, 2007

Agile Classroom Management

I'm fascinated by the idea of applying agile methods outside of software... be it to business management, family and household, or, as I and my father have been exploring over the last two and a half years... agile classroom management. Here's how I do it in my Agile Project Management / ScrumMaster Certification courses:

My father, Garry Berteig, had been using agile methods in his classroom for over two years before I tried it myself. I had gone into one of his courses at Keyano College in Fort McMurray to give a presentation to his second-year media students on agile and Scrum. They used it for a video project (see "A Student Documentary Film Project" and "Scrum Saves the Day for Media Student". Out of that experience, and others, he developed the idea of the Learning Circle as the basic cycle in an agile learning environment.

In the meantime, I became a Certified Scrum Trainer. Back when I did it, this just meant co-teaching a Scrum class with Ken Schwaber. He had a standard slide deck which he presented as he masterfully guided a class through two days of Scrum training. I (as expected) took that deck and made it my own by reformatting, rewording, changing a few of the exercises, and changing the emphasis in a couple of spots. And then I used that deck with minor corrections and updates and some expansion over the course of the next 16 months.

In the meantime, I was also doing a fair amount of training for teams and organizations that didn't care particularly for Scrum, but just wanted to know more about this "agile" thing. Sometimes it would be a one-day course, sometimes as much as five days (that was exhausting!!!). I developed a substantial body of modules including exercises, case studies, and special topics. Eventually, I found that I was having a hard time fitting my fixed agenda materials into the time I had allocated. For example, I was commonly doing a three-day course and would find that by the end of the first day I was a little behind on my agenda and by the middle of the third day I would certainly be running out of time and miss possibly even two or three modules in the agenda.

Finally, last Sprint, I got a big client who asked for training for a large number of their staff. This gave me the opportunity to: a) work with the same materials and agenda at least three times, and b) involve an assistant in all three deliveries. The first time through, I had the typical problem where the third day was a terrible rush. The second time through, it happened again, but I had de-scoped and re-arranged some of the material so it didn't seem as bad. After that second time, my assistant, an excellent coach, David Chilcott, suggested that I try to apply a more flexible... agile... approach to handling the materials. We did a quick revision of the materials to support this idea and then the third class was run as an "agile training environment". What did this mean?

The main insight was that the time allocated to the class was timeboxed. This meant finding a way to prioritize the work of the class so that the majority of the participants would get the best value possible for their time in the class. We did this in two ways:

1) As questions came up, we found that some of them needed to be deferred because they would be answered in a later module or because there was background material that needed to be covered first. So, for every question that needed to be deferred, we wrote it up on a 3x5 note card and put it up on the wall. Now, in order to keep track of this, we also had all the course modules written up on 3x5 note cards and up on the wall too. Most of the question cards would immediately be put up with the associated module card.

2) We drastically cut back the core required modules for the three days and made everything else optional. We then decided on a simple in-class mechanism to choose from those optional modules. At the end of the first day, we had time for one optional module so the class voted and we chose the module based on that vote. Likewise at the end of the second day. By the third day, we had a substantial portion of the day un-allocated. As soon as we finished with the required materials, we got the class to choose the next module to cover. As soon as that module was covered, the class chose again from all those still remaining.

It was a bit of a mess and our mechanisms didn't work perfectly, but it got the job done. At the start of our second day, the cards we were using to track modules and questions looked like this:

After we finished the course, we did an in-depth reflection and decided on some changes to be made. One of the most important was actually very simple: make the module cards much larger so that it was possible to put many question cards right on the module card. Here is what this looked like in a recent class:

The left is the module card at the start of the course with questions that were generated by the course attendees when asked what they wanted to learn from the course. The right is the same module at the end of the course. You can see that one additional question card was added between the start of the course and the time when the module was actually covered. You can also see that I use big check marks to show to the class that the module is complete.

Another change to the method of doing this agile classroom management came in the further refinement of the module content and sizing to be more appropriate for doing the modules in unpredictable sequence. This included things like checking definitions, improving some of the introductory material (to make sure all the concepts were introduced, not just some of them), finding references to other modules and removing them, and in some cases moving material from one module to another. There are still improvements to be made in this area.

Thirdly, we now are better at explaining to the class how this will all work at the start. We are modeling an agile process in the way we run the course. This means that the attendees will have an opportunity to influence what material is covered and in what depth. It also means that if the class takes a lot of time with questions early on with the basic material, it leaves less time at the end for optional modules. The people in the class can now visibly see these tradeoffs.

Here are two photos from a recent class. The first was taken at the end of the first day (note that I forgot to use the check marks on these cards). The second was taken at the end of the third day. You can clearly see how the modules of the third day were chosen entirely by the participants from the available optional modules.

This mechanism of managing a classroom using an agile approach has had some very interesting results:

1) Every class is substantially different. Before, the differences were always in the details: timing, specific questions, etc. Now, the actual material covered is uncertain. I always make adjustments to the course materials after every course. Now, not only are those differences in play, but also the actual modules covered might be different. The fascinating thing is that now, by the time we get to the last module, most of the people in the class are giving off a palpable sense of satisfaction: their urgent questions have been answered. Sometimes, this is so strong that I do the graduation "ceremonies" right before covering the last module so that people not interested in that last module can leave a little early.

2) I get far fewer comments in my feedback forms that indicate a desire that the course was longer. Before I started using this agile learning environment, I would get a substantial number of comments to the effect that it was really too bad there was not another day. Now I rarely get those comments.

3) The attendees in the class see how an agile approach actually works in a real life situation. They see that allowing the requirements to change over time is actually a good thing. Every single time I do this, there is a pattern that shows up on the third day that relates to the way the class prioritizes the next module. I count up the votes and choose the next module to cover. All the other modules have their scores written on them as well. After a module is covered, instead of using those old scores, the class takes another vote. Every single class, the priority changes between votes. This clearly demonstrates the power of allowing stakeholders to change their minds after seeing real results.

4) Participants in the course are engaged at a deeper level. There are lots of "tricks" for keeping people engaged: discussions, exercises, simulations, questions, presentation eye candy, etc. All of those things have their place and are important. However, by having participants steer the course itself, the level of engagement suddenly becomes much more substantial. The participants are responsible for the results of the class in a way that is quite unusual in most educational environments. This responsibility is granted to them and they take it seriously. I rarely see anyone refuse to participate in this process, and I never see anyone attempt to subvert it.

---

If you are interested in learning this mechanism of classroom management, I strongly recommend:

1) Learn up on agile. The openagile.org web site is starting to define an agile method that is abstracted away from the software legacy that agile methods currently are shackled with. This site is still in its infancy so treat it gently :-)

2) Come try one of my courses! (Yes, that's a blatant shameless plug!) You will get to experience and learn the agile method in a focused setting. The courses are IT/software focused in some respects, but since they are non-technical courses, and really about agile project management and human interactions, almost anyone can attend (I've had dancers, administrative assistants, educators, artists, community workers, executives, and actors all attend).

3) Get in touch with me to find out more of the details that didn't make it into this blog entry. The best way to do this is (really) through my business web site: berteigconsulting.com where you can find both email and phone contact information.

---

Well, I finally got around to writing this up. This is part of a series of related articles "tagged" as USEFUL started by Scott who writes at Tyner Blain. His article, which started the game of tag, is called Software Product Success Stories. Scott... I'm sorry this article isn't about software!!!

I would like to now tag a few people who write useful blogs:

Mike Vizdos who writes the Implementing Scrum blog. This blog is best known for the fabulous cartoons of the Chicken and the Pig, but also has a lot of great and practical insight on the challenges of, you guessed it, implementing Scrum!

Tobias Mayer who writes at Agile Thinking. He doesn't write often, but his articles are always fantastic! I hope that Tobias will take the time to write something in this "useful" stream and pass along the tag.

Mark Levison writes Notes From a Tool User and discusses many things, not just agile. His writing is fun and he's in the trenches doing real serious work getting his company to use agile. He's got good insight.

Posted by Mishkin Berteig at 03:46 AM | |

September 24, 2007

Agile Benefits: Five Essential Reasons to Try Agile

Although there many other benefits of agile, and although those provide us with other reasons to try agile, the five essential benefits of agile are:

Rapid Learning - disciplined application of the scientific method to explore the best ways to deliver valuable results.

Early Return on Investment - opportunity to use the results of work starting with the work delivered at the end of the first iteration.

Satisfied Stakeholders - engagement in the process in a way that allows meaningful contributions from all stakeholders.

Increased Control - mechanisms to track/measure and therefore steer the direction that work is going so that it meets goals.

Responsiveness to Change - processes, tools, roles and principles that allow a team and an organization to embrace change rather than reject, control or suffer from change.

These reasons are sufficient and apply to operations work, project work and open-ended research work, whenever humans are involved. The above links take you to more detailed descriptions of each of these benefits.

What are some of the other benefits of agile?

Posted by Mishkin Berteig at 10:47 PM | |

Agile Benefits: Responding to Change

The fact that agile methods increase return on investment, accelerate learning, increase stakeholder satisfaction, and enable better control of work are all an interesting result of this final benefit: responding to change.

Responding to Change

The whole idea of "agile", and one of the primary reasons for the choice of this word is the final item of the Agile Manifesto: "Responding to Change". It is clear that this is one of the distinguishing features of agile as compared to a more traditional waterfall or phase-based approach to working. Not so obvious is that this is also true about agile compared to a more chaotic or ad-hoc way of working.

Agile and Waterfall - Change "Control"

In most waterfall or phase-based approaches to working, change control is a technique used to evaluate change requests and then accept or reject them. The practical consequence of this is that "control" often becomes a euphemism for "prevent at almost any price", and we end up with results that satisfy no-one because life changes.

There are, of course, some changes that are accepted even in a change control environment. These changes tend to be the ones that have both a powerful sponsor and a "wealthy" sponsor. The next problem with change control relates to the fact that change becomes very expensive in a phase-based process. It usually means going back to the "beginning" and re-examining the other requirements, the analysis, the design/architecture, the development/construction, the testing and the impact is necessarily large for all but the most trivial changes.

Agile methods turn this on it's head. By allowing (nay, building in!) change every iteration, every cycle, the team and the organization start to get good at accommodating change. The value of this effect cannot be over-estimated; getting good at doing change is a natural consequence of accepting change all the time in a structured manner, and in fact, leads to all the other four benefits: increased learning, increased ROI, increased stakeholder satisfaction, and increased control. The cost of change stabilizes over time instead of increasing exponentially!

Agile and Chaos - The Illusion of Changability

Unfortunately, I am not aware of any studies that target chaotic (non-process) environments to see what kind of results they have in delivering results. (If anyone is aware of such studies, please let me know in the comments!) That said, I think almost everyone has had some experience in environments where the winds of change were so severe and fickle, and the structures to support working with change were non-existent, that chaos reigned. These environments are almost completely negative: things don't get done in a timely manner nor with an acceptable level of quality.

Often the reason for this lack of structure is about flexibility, needing to work in an environment which is also extremely chaotic. Sometimes its worse though: management or stakeholders are the cause of chaos without understanding the consequences of their actions.

Agile methods put a sufficient structure in place to build some discipline, and still remain adaptable. This sufficient structure is often built by gradual application of various agile practices until there is "just enough". This process does require dedication and will, but because it is gradual, it is often much easier than trying to suddenly put in place a full-scale disciplined process.

As these practices are implemented, the team and organization gradually become better and better at accommodating change.

---

Agile Benefits: Rapid Learning
Agile Benefits: Early Return on Investment
Agile Benefits: Satisfied Stakeholders
Agile Benefits: Increased Control
Agile Benefits: Responsiveness to Change
Agile Benefits: Summary Article

Posted by Mishkin Berteig at 07:35 AM | |

September 06, 2007

Four Methods of Perfecting Agile

Most organizations start doing agile (or Scrum or lean or ...) imperfectly. Someone introduces a few practices or a manager gets a team some training, or a person starts using agile terminology. And things might improve, particularly with the use of iterations. One of the core ideas of agile methods is to have frequent delivery of valuable results. In fact, this core idea can be used to drive the improvement of an agile process. How? Here are four methods of perfecting agile by expanding the definition of done.

Perfecting Agile

Let's suppose you already understand the benefits of agile. With these benefits in mind, you would like to improve the organization's ability to deliver working, valuable results at the end of every iteration so that you can get better at realizing those benefits. The primary way to do this is by expanding the definition of done. You can imagine this like so:

On a regular basis, the team/organization find ways to bring work done in either the preparation stage or the close stage into every iteration of the agile portion of the project. By moving the work from these "bookend" stages into the iterations, you reduce the amount of time spent in those stages and simultaneously create a more complete delivery every iteration. The "definition of done" is now expanded to contain the results or value delivered by the work that was taken out of the startup and shutdown stages of the project. By expanding the definition of done, each iteration delivers a more "complete" increment of value, and there is less work done before or after iterations in order to plan or deliver. This gradual process allows the team to get better at doing agile.

---

There are four methods for transferring work from the start and end of a project into the iterations of a project.

Expand the Agile Team's Skill Set

In some ways, this is the simplest and most common approach to expanding the definition of done in the agile portion of a project. By training, coaching, mentoring, re-assigning or hiring, a team's capacity to do work is expanded and used to expand the definition of done. As a simple example, a software developer might learn to use an automated unit testing framework and therefore expand the definition of done to include some amount of unit test coverage of delivered code. In general, training, coaching and mentoring existing team members should be preferred over adding people to the team since the addition disrupts the team's development and can increase communication overhead among team members.

Expand the Agile Team's Authority

Sometimes, a team is not able to do part of the preparation work or close up work because they are not authorized to do so. This may be a policy, a unspoken assumption or a bureaucratic procedure. By giving the team (or some person on the team) the authority to do this work, the team can find ways to do it every iteration instead of having to work through another non-team individual. Again, a simple example here is a situation where a technical person is given permission to talk directly to an end user in order to reduce the need for up-front requirements gathering and analysis and reduce the need for end-of-project user acceptance testing. The obvious challenge to do this is the question of trust (or lack thereof).

Automate an Existing Manual Process

Automation is often given far less than its due consideration. This is primarily be cause automating a process is an investment of work in and of itself. Fortunately, it is often easy to measure the ROI or savings involved with automation. In many agile environments, heavy automation is critical and a huge enabler for very short iterations. Automated testing, automated translation, automated build processes, are all common areas of improvement. Agile teams should always be looking for opportunities to automate their own work. In this way, the automation work itself is transformed from a separate project to a responsibility of the team.

Remove Wasteful Processes

There are some parts of the project preparation work and the project close up work that are pure waste! There is no independent value to these activities, nor is there indirect value to them. An excellent example of this sort of thing is an approval process that _always_ grants approval ("rubber stamping"). One insurance organization I worked with as they were converting to an agile approach discovered that their "second stage" approvals always allowed proposed projects to proceed. Since they often incurred a 4-6 week delay for this approval process, it became obvious that they should "get rid of it". Now, what they actually did is made it so that it became a parallel review process rather than a gated approval process; this was so that the true purpose of the activity could still be met: to help stakeholders understand the projects that were being worked upon. Here, there is no need to take this approval process and somehow work it into every iteration. An agile approach tends to increase the visibility of the work anyway, so it may be discovered later on that the review process can also be done away with.

---

Agile is often implemented in a limited fashion when it is first adopted by most teams and organizations. The four methods of expanding the definition of done can help a team or organization get better at doing agile and therefore reap more of the benefits of agile. These methods are simple: expand the agile team's capacity, their authority, have them automate manual processes and remove wasteful activities from the process.

Posted by Mishkin Berteig at 03:32 AM | |

August 30, 2007

Agile Retrospectives and the Plan of Action

Bas Vodde has published a good article about making goal oriented action plans for agile projects. It is a nice piece of the puzzle on how to do effective retrospectives. It also nicely ties into the "Learning Circle" Reflection/Learning/Planning/Action steps.

Posted by Mishkin Berteig at 04:59 PM | |

July 25, 2007

Time is Not Negotiable

The Project Management Institute refers to three variables that can be negotiated or constrained for a given project: scope, resources and schedule. Schedule is an interesting "variable" in that we have no choice about how time flows. We can control how much scope to ask for, we can control how much money to put towards the work, but we cannot actually "buy" more time than, say, our competitors. This has important implications which deeply challenge the PMI's PMBoK model of project management.

In a related article "Quality is Not Negotiable" I wrote about the strategic important of maintaining a high level of quality, and that allowing defects into your work is like taking on a debt with an unknown interest rate. With time, the problem is a little different.

The idea of sunk costs, also know as "don't throw good money after bad" is difficult for most people at a psychological level. We often feel like making just a little more investment of time, work or cash will turn an ailing effort around. Unfortunately, a traditional waterfall or phased-based approach to project work increases the psychological pressure to continue working, even when the project is "bad".

We know from the recent Chaos report statistics that two thirds of all IT projects are still either failing or challenged. We know that the bigger the project, the more likely it is to fail. We know that in other industries such as construction and economic development projects also have low success rates. How is it that we put all this money and time into these projects without knowing earlier if there are problems?

The problem is, of course, time. We can't predict the future. The farther out we need to see, the more uncertain the view. Not only that, but we can't "go back" to correct mistakes. We can only correct past mistakes by changing our future behavior... by using up more time.

One organization I worked with wanted to do a small web-based project. They had been "thinking" about this project for three years. At various times they had done requirements gathering and analysis with various stakeholders... and then failed to execute on the project. Finally they got around to starting the project. Using an agile approach, the first release of the project was completed in about eight weeks. How does this relate to time?

Well, first off, the project success was measured only in terms of the eight weeks of labor, not the time value of the sunk costs over the past three years. Now of course, the reasoning behind not including sunk costs is exactly because they are non-recoverable. Unfortunately, that doesn't give a very good picture of the investment made in the project. The fact that the costs are non-recoverable is not because of money, scope or quality (the other project variables) but because time is not negotiable: you can't get it back, you can't get more of it, you can't store it up, and you can't transfer it to other people. Sunk costs are really better thought of as sunk time.

A project is a temporary and one-time endeavor undertaken to create a unique product or service, which brings about beneficial change or added value. [Wikipedia]

This helps to explain why large projects have such low success rates. The larger a project, the larger the sunk time. It's hard to re-start projects, and the larger they are the harder they are to re-start. And yet, the proper approach to sunk costs (time) is to ignore what has gone past and treat your work as if you were starting from scratch. A project approach specifically does not allow you to do this. Hence the fiction of "earned value" (which is a whole other topic).

Only an agile or operational approach allows for low sunk costs. By doing work in extremely small pieces, each of which actually delivers value, an agile approach is best suited for the reality that time is not negotiable. Agile methods acknowledge that the best way to avoid sunk time is to not do much work until you deliver results. As well, since we can't go back to change our mistakes, agile methods allow us to check our work frequently and learn as we go.

In what other ways is time a different sort of variable than scope or cost? In what other ways does an agile approach take this into account?

Posted by Mishkin Berteig at 10:05 AM | |

June 25, 2007

Why Scrum Works

Why does Scrum work? Why do any of the Agile methodologies work? How does Scrum help teams deliver value? How does it help high performance teams form?

This series of posts that will look at why Scrum works on three levels:

• Scrum delivers value to the business
• Scrum helps form high performing teams
• Scrum helps motivate and focus team members

---

Delivering value to the business

Scrum delivers value to the business through:

• Every iteration the product is ready to ship.


• Adapt to changing requirements: Short iterations mean that the Product Owner provides frequent feedback. As result there are frequent small course corrections as opposed to massive changes late in the project.


• Visibility of progress: delivering a working product at the end of every iteration means that the customer, executive sponsors and other interested parties can see the product take shape. They are not surprised six weeks before release.


• Accurate tracking of how much work is left before release: the combination of the product backlog and the team's velocity (aka Yesterday's weather in XP speak) means that you tell how much the team will get done by the release date.


• Lightweight requirements: Since the team is in frequent (preferably daily contact) with the Product Owner, she spends less time writing detailed requirements. Instead she is able focus on making decisions and answering questions from developers.


• Process improvement mechanism to meet business needs: As business needs change – SOX, FDA compliance, through the retrospective the Scrum process can be improved and adapted to meet goals and needs in changing environments.

In short, Scrum provides a way for the business to maximize its ROI by using iterative development to rapidly create working software. It ensures that the team is always working to deliver the highest priority features.

The next post in this series will cover how the elements of Scrum create the right environment for the formation of high performance teams. This post will be hosted in my own blog: Notes From a Tool User

This article written by Mark Levison.

Posted by Guest at 02:54 PM | |

May 24, 2007

Scrum Rules

Recently in my ScrumMaster Certification courses, I have been ending the course with a list of the rules of Scrum. This list of rules is based on my understanding and practice and may not be 100% the same as that found in the Scrum books or in other Scrum practitioners' models. Nevertheless, I just recently checked it against some other online reference material and it seems like a reasonable list of rules. I am interested in any feedback or variations or disagreements...
[Updated: 20070922]

Without further ado, here they are, categorized into Required Rules, Basic Rules and Optional Rules, plus a downloadable PDF version.

Required Rules to Start – the “Agile Skeleton”:
- Full-Time ScrumMaster Identified and Team Members Available to Do Work
- Team Agrees to Demonstrate Working Software in No More Than 30 Days
- Stakeholders Invited to Demonstration

Basic Rules of Scrum to Implement As Soon As Possible:
- Full-Time Product Owner (with Expertise and Authority) Identified
- Product Owner Works With Team and All Other Stakeholders
- Product Backlog Created and Managed by Product Owner
- Daily Scrum Meeting with 3 Questions (Completed? Will Complete? Obstacles?)
- Daily Scrum Meeting Same Place and Time and Less Than 15 Minutes
- Regular Sprint Length (no more than 30 days)
- Sprint Planning Meeting to Create Sprint Backlog of Estimated Tasks
- Sprint Burndown Chart
- Team Room with All Needed Equipment and Supplies
- Retrospective Meeting for Process Improvements
- Definition of “Done”
- Commitment Velocity Calculated (from Sprint Backlog Estimates)
- Team Size 7 +/-2, Maximum of 12
- Cross-Functional Team Including ScrumMaster and Product Owner
- Team Self-Organization - Team Members Volunteer for Tasks
- ScrumMaster Tracking and Removing Obstacles
- Team Safety – No Interruptions to Team's Work During Sprints
- No “Break” Between Sprints
- Sustainable Pace - Timebox Effort, Not Just Schedule
- Quality is Not Negotiable - Defects Go on Top of Product Backlog

Optional Rules of Scrum to Implement Depending on Context:
- Test Driven Work and Continuous Integration
- User Stories as Product Backlog Items (As a I can so that )
- Project Burndown Chart
- Release Burndown Chart
- Planning Velocity Calculated (from Product Backlog Estimates)
- Scrum of Scrums for Multiple Teams
- Canceling the Sprint Early
- Financial Modeling for Product Backlog
- Sprint Backlog Tasks on Big Visible Chart on Wall
- Backup Product Owner Identified
- Team of Volunteers - Self-Selecting
- Rotate the ScrumMaster Duties

---

And here is the downloadable version of the Scrum Rules Cheat Sheet [pdf].

Finally, you can purchase a poster-sized version of this from Cafe Press AgileAdvice Shop for $22.99.

Posted by Mishkin Berteig at 03:01 AM | |

March 05, 2007

A Better Iteration Structure

In my coaching work, I have often been asked a question about the planning process for iterations, that until just a few days ago, I would actually brush off!!! I didn't even realize I was doing this, it is only in retrospect that I see this. This question is simple: "how does a team plan for the improvement efforts that come out of the retrospective when they are supposed to be working at maximum velocity when implementing tasks directly related to the items in the work queue?" Or, more simply, "we don't have time for process improvements."

The source of the problem shows up in the normal* structure of an iteration. This structure is as follows:

Step 1: set a goal by selecting work from the top of the work queue.
Step 2: plan the execution of that work by doing a task breakdown and task-level estimation.
Step 3: do the work according to the tasks (this is the bulk of the time in the iteration).
Step 4: do a demo to review the work - use this demo to adjust the work queue if necessary.
Step 5: do a retrospective to review the "how" and come up with action items to make the work more effective next iteration.

Since iterations follow one after another, this structure when rolled up can be re-written in the following manner:

Action (step 3)
Reflection, Learning (step 4)
Reflection, Learning, Planning (step 5)
Planning (step 1 and 2)

Now if we look at the purpose of the steps in comparison to the type of learning activity taking place, we see that there is a big disconnect between the planning that happens in step 5 (the retrospective) and the planning that happens in steps 1 and 2 at the start of the next iteration. Not only that, but reflection and learning are separated into "what" and "how" so are done twice for each iteration.

Frankly, this causes a lot of confusion: I see it in my coaching when teams try to accommodate the two types of planning, and I see it in my training when I teach the structure of the iteration.

---

So what is to be done? Simplify the structure of the iteration. Instead of thinking of the iteration as a linear block of time that starts with step 1 and ends with step 5, think of it as a cycle that is dominated by Action, and punctuated regularly by Reflection, Learning and Planning. Now, instead of four separate meetings, there is a single meeting every iteration that combines the wrap-up of the previous iteration with the start of the next. Let's call this the Iteration Transition Meeting or the RLP Meeting or the Adapt Meeting... (suggestions welcome!)

The agenda for this meeting is very simple and contains the essential elements that exist in the "old" style. It starts with Reflection where the facts of the Action are examined: what did we build? how did we build it? how did we feel about what we built? etc. This moves fairly quickly to the Learning component where we try to understand what the facts are telling us: what went well? what needs improvement? what insights can we glean? what new questions are open to us? and of course, what did we learn? Now we are prepared to treat Planning holistically, with no "hidden" activities. We examine our Work Queue, re-arrange it, put new items on it, etc. and do our normal task breakdown and estimation. The difference this time is that both direct business tasks and process improvement tasks are included in the same planning activity. This makes task level (commitment) velocity more truthful!

I will admit that this is not something that I have tried yet. I have seen it done accidentally when people start to ignore the action items from the retrospective in the old iteration structure and instead include them in the task planning. When it happened, I thought it was "messy" because it broke the separation between "what" and "how" oriented activities. Now I think that this might be the best way to do things.

Anyone out there willing to give this a try? If so, I would love to spend some time with you working on it - helping, experimenting, and learning from it.

I should mention a couple important points that didn't fit in the above narrative. First, this insight happened as a result of discussions over the past week with my father, Garry Berteig who is using agile methods and the learning circle in his classroom environment. Second, Garry has been doing what I just described in his classroom environment; the main difference is that Garry provides a great deal of the Planning to his students by way of the progression of assignments/activities.

---

One other cool thing: I'm in Frankfurt as I write this on my way to Chennai to deliver a three-day Agile Project Management / ScrumMaster Certification course. I haven't done international travel since I went to Beijing last summer, and this is the first time I am going outside of North America for work! Fun!

---

* Normal here means Scrum and XP for the most part. Lean doesn't have this structure and as for other agile methods, I don't know them well enough to say if this is normal or not.

Posted by Mishkin Berteig at 04:16 AM | |

January 27, 2007

The Freedom of Limited Capacity

Something that I would have thought impossible has happened. By understanding how incredibly limited my capacity to do work is, I am getting a greater and greater sense of freedom and contentment.

My "experiement" to run my business using Agile Work practices is starting to bear some fruit. The trouble is, that so far, that fruit is simple better knowledge of my extremely limited capacity to get things done during a single week-long iteration. I can do 3 or 4 items from my Work Queue which works out to about 15 or 20 tasks. This seems ridiculous to me! Surely one hard-working guy can get more than that done!

But I can't. I'm getting to the end of my fourth iteration and it's the same thing again!

I know some of the reasons for this: interruptions, unexpected work, unremembered work, and my general nature of being easily distracted by shiny things!

In some ways this has been quite frustrating. I have a huge queue of work I want to get done for my business. But the other side of the coin, the one that looks different than I expected, is that knowing just how limited my capacity is... is liberating! I don't feel nearly as stressed as I did four weeks ago. I have the courage to commit at an appropriate level. I can start to believe my plans. I can see - realistically - just how much I need help to achieve my goals. I can see how soon I will be able to do cost/benefit analysis on hiring vs. doing the work myself (can't do that yet). I can even start to believe that I might be able to reach my original goal of reducing my working hours from 80+/week to 50 or less... as long as I make sure that my Work Queue is properly prioritized with that goal in mind.

---

I have worked with a number of teams that have gotten to this point of certainty about their capacity. I've seen this happen in others... and now I recognize it for what it is: relief. One team I worked with after only two iterations had a very clear idea of their capacity, and you could feel the comfort level of everyone with this new agile thing skyrocket. Another team I worked with got a clear sense of its capacity after only three iterations. This team settled in and then started to work to increase their capacity and did a fabulous job by using automation tools, eliminating various organizational bottlenecks, etc. They wow'ed the rest of the organization with their incredible, consistent delivery of value. One of my former students, Dmitri Zimine, gave a presentation at the XPToronto group where he talked about the incredible level of trust that developed between his team and the rest of the organization after consistently, iteration after iteration after iteration meeting their commitments.

All of this is only possible by a rigorous application of timeboxing, careful and consistent task breakdown, and good, honest tracking of results. I wrote a previous article called Seventeen Tips for Iteration Planning. Look it over and see what new ideas you can apply in your situation. If you are frustrated by a team continually over-committing, you will find a lot of valuable advice there.

---

As a parting shot: consider how you feel right now. Are you stressed out? Over committed? Failing to meet your commitments? Feeling guilty? Working insane hours? Not spending enough time with your family? Killing yourself!?

What about the team or organization you work in? Is it constantly being bombarded by problems it can't handle? Surviving on pure heroics?

I can't tell you that following a careful iterative planning approach will fix your problems... but I can tell you that it will reduce your stress levels, help you (or your organization) make appropriate commitments, and let you see the reality of your situation (good or bad).

Posted by Mishkin Berteig at 03:07 AM | |

January 22, 2007

Cancelled Iteration

Last week went totally wonko for Berteig Consulting. My planning was bad, bad, bad!

For planning, I totally forgot about three very important scheduling things:

- a ScrumMaster certification course on Thursday and Friday which required not just time in class, but also three hours of commuting each day.

- my birthday for which I had planned a very informal party on Saturday (normally a very productive work day) which consumed the whole day from morning to late at night.

- the departure of my brother Alexei and his wife Ma Jin so that Sunday would be my last day of visiting, again, taking up my whole day from morning to late at night.

Suffice it to say, a 3-day work week was not what I was expecting when I did my planning. Sometime on Thursday I gave up.

I decided to delay starting a new iteration until I could recover a bit. Over the course of four days I get nearly 200 emails that I actually need to pay attention to at some level or another. So, today has been a recovery day and at the end of the day I will plan my next iteration. Since I did not have a delivery meeting (since I cancelled the iteration), I can only do an informal review of the accomplishments:

- some progress on development work for my client
- some progress on invoicing my client
- some progress on automation of my course listings (Selenium is great!)
- no progress on providing a statement to my other client

The causes of this massive failure are obvious. It should be noted that by doing the ScrumMaster training, I did actually deliver value to Berteig Consulting... it just wasn't the value that I had planned!

Posted by Mishkin Berteig at 05:58 PM | |

January 15, 2007

End of Iteration 2 ("Capacity") and Start of Iteration 3 ("Automation")

This second iteration when much better than the first. I committed to an amount of work that was much closer to my real capacity, and I stayed more focused on that work. Here are the results of my demo, retrospective and planning for Iteration 3 which I am calling "Automation" for reasons which will be described below.

Demo

I committed to the following items from my Work Queue:

I also added these in the middle of the iteration (which is really a "no-no", but I did it anyway to my regret).

What did I actually get done? Here are the stats:

For Iteration 002 - "Capacity":
# Work Queue Items At Start: 5
# Work Queue Items Completed: 3
# Tasks at Start: 32
# Tasks Remaining: 17

What is scary is that my velocity is exactly the same in terms of number of tasks: I really need to start my iterations with only 15 tasks.

Anyway, what I need to do for my next iteration is schedule my "demo" with my other stakeholder (my wife Melanie). Demo'ing to myself in the wee hours of the morning is not really practical in terms of my overall goal for this project (running my business at a sustainable pace).

Retrospective

Here are the things that went well:

- Came much closer to getting everything done that was in my iteration. This was great - I continued to use Tiddly Wiki to track my Work Queue and Tasks.
- Felt less stress. Although I still worked very hard, I was not worried nearly as much about the things still on my Work Queue that weren't in this iteration. I also decided on certain boundaries to work: generally work 4 to 6 hours during the day and another few hours after lights out (usually after 11pm).
- The prioritization worked well - didn't feel impelled to do too much that was _not_ in the iteration. Althought I did take a couple more items in, it was more because of opportunity than anything else. My brother Alexei wanted to help me with my book (for example). Next time around, perhaps I should be more explicit about negotiating with myself about taking something out of the iteration if I bring something in.

Here are the things that need improvment:

- Get even better at judging capacity and committing to an appropriate amount of work. The numbers speak loudly! This next iteration I have to limit myself to 3 items from the Work Queue and about 15 tasks. I have to remind myself that I can always bring more work in if I complete my work early and I still have time.
- Care with long term scheduling - made a mistake in my scheduling with a Client - hopefully it will be okay! This is a bit of a nit, but I need to keep my daybook up-to-date even with long-term stuff.
- Spending time with my family - e.g. playing with kids, helping around house, assisting with homeschooling. This was better than I was doing in Dec., but I need to continue to improve here. I would really like to work with Justice and Haifa on learning Chinese. We've got tons of materials to do it, just need the discipline to get through it together.

Planning

Okay, maybe I'm a glutton for punishment, but I've decided on four items from my Work Queue:

Client2 Invoice
Client2 Work
Automate Listing of Scheduled Courses on 3 Sites
Client3 Statement of Account

The key one is the automation since that will be an investment to save me time later on. In my task breakdown it looks like this (tasks with an "x" are already done from the previous iteration):

So, that's 4 items from the Work Queue and 17 tasks (the ones with "--" in front of them are sub-tasks... not sure how to handle those, but for now I'm not counting them separately.

I'm really excited about this. It sucks to see that I get so little done in a single week, but on the other hand, knowing my capacity allows me to understand just how much work I have on my plate and how badly I need two things: automation and assistance! I'm going to start with automation, but I'm also looking for assistance. The trouble is that I can't affort to hire someone on a salary... (if you know anyone who would be interested in working on commission/revenue-sharing basis, I would love to talk!).

---

One final note: I did manage to include my wife Melanie in my planning for this iteration. She seemed pleased :-)

Posted by Mishkin Berteig at 03:09 PM | |

January 08, 2007

Planning Iteration 0002 "Capacity"

I've completed my iteration planning for my second iteration. As a reminder, I'm doing this because I want to be working only 50 hours per week by July 2007. My sole improvement item from last iteration was to use get better at committing to an amount of work within my capacity. Here's what I have planned:

I have included my complete list of work to be done broken down into tasks. It is anonymized:

Client1 Proposal Writing
- 1 day intermediate team course
- phone consult details
- QA/tools coaching
- email proposal
Client2 Invoice
- get template from them
- collect hours
- collect receipts
- image receipts
- currency convert receipts if needed
- type up hours
- type up receipts
- email/submit
Client2 work
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
India Training
- ping Contact1 "V"
- ping Contact2 "N"
- ping Contact3 "M"
- ping Contact4 "S"
- ping Contact5 "V"
- air reservations
- prepare visa application
- submit visa application
- hotel reservations
Client3 Statement of Account
- equipment
- invoice 1
- invoice 2

As you can see, I have committed to only five items off the Work Queue for this iteration and they add up to thirty-two tasks. This is more than I indicated last night in my retrospective results. I'm nervous about this in two ways. One, I think that I am under-committing and that I'll get into huge trouble if I don't do more than this. Two, I think that I'm over-committing and that with my brother visiting, even doing this much smaller amount of work will be challenging. Still, I have tried to make both the items from the Work Queue and the tasks smaller than in my previous iteration. I guess I'll see in a week!

Posted by Mishkin Berteig at 11:01 AM | |

First Interation Ending

My first iteration using Agile Work for my business development has come to a close. Here is what I did for a "demo" and retrospective.

Demo

This was simple. I looked at my selected Work Queue items and my tasks and check to see what I had accomplished (finished or partially finished). Here are the statistics (grim and discouraging as they are):

For Iteration 001 - "Beginnings":
# Work Queue Items At Start: 11
# Work Queue Items Completed: 2
# Tasks at Start: 43
# Tasks Remaining: 28

From these numbers, for my next iteration I should probably be hesitant about committing to any more than 2 items from the Work Queue and any more than 15 tasks.

Considering how many work hours I have in a week, 43 tasks is absolutely ridiculous... unless they are really small, which many of them weren't. For example, I had one task as "Integrate part one of Alexei's feedback into my eBook" which when I did it actually took about three hours. Its just one sample, but 3 x 43 is 129 hours and is just not possible for a single human being in 7 days.

Aside from the value-oriented items from the Work Queue, I also did the work to finish a more complete Work Queue. It now has 37 items including those which remain un-finished from this first iteration. It is poorly prioritized, but that will improve over time. As well, I suspect there are quite a few things missing that I forgetting. I should probably get my wife, Melanie, to look over it and make some suggestions. I also have some old "to do" lists for my business which might remind me of some things to add onto it near the bottom.

Retrospective

I did a little mini-retrospective to find the three things that went well this iteration, and the three things that need improvement. I also decided on a clear action item to take to try to improve my next iteration. (Another clear reason to use Card Meeting - worked great for my retrospective :-) .)

Three Things that Went Well:
- Referred to my iteration tasks frequently. I kept my TiddlyWiki up in my browser and checked it several times a day.
- Spent time with my family and friends despite huge workload. We had guests and family visiting and I got some good time in with them. I also spent a good amount of time with Melanie and my kids. I had to make a conscious effort to do this, but since this is the goal of my using Agile Work (to have a good life/work balance), I figured I might as well start making some progress on it right away!
- Had courage to do some development work which required changes to other developers' work. I ran into a problem with some of the work I am doing as a contract. I dug around and realized that any fix I made would be felt by other parts of the codebase as API's changed. I made the fix and then tracked down the dependencies and fixed everything up. Fun!

Three Things that Need Improvement:
- Pay more attention to my schedule / calendar so as to reduce problems with work/family boundaries. Still a loooooong way to go on this one!
- Spend more work time "on-task", rather than on things not on my list. I ended up doing a bunch of work stuff that wasn't in my iteration plan. Partly I did it because I got bored of what I had planned, and partly becuase I forgot to focus on my plans. It would probably help if I had a Process Facilitator breathing down my neck :-)
- Make realistic plans given my capacity for work. This is the biggie! It is going to be very hard for me to keep to a small commitment. But considering the my brother is visiting from Beijing with his wife this week and next, I really have to try to get this right.

It is that last point that I will use as the basis for my next improvement. For my next iteration planning meeting (to be held in the morning after a far-to-brief sleep), I will commit to a maximum of 3 items from the Work Queue. This may mean breaking the items down into smaller bits. I'll talk with the Queue Master (myself) tomorrow morning about it :-)

Posted by Mishkin Berteig at 04:37 AM | |

January 02, 2007

Top Ten Most Popular Entries from 2006

If you are new to Agile Advice, these are not just some of the most popular articles, they are also some of the best! Take a look around; you will find ideas to inspire you, challenge your thinking and maybe that one little thing that will make the difference in applying agile methods in your situation.

1. How Two Hours Can Waste Two Weeks - 25,617 unique views
This little hypothetical story by Dmitri Zimine was very popular on Reddit, and Joel on Software ranted a bit about it.

2. The Case for Context Switching - 2,936 unique views
My rebuttal to Joel's rant. Goes well with Dmitri's article. Emphasizes the idea of building trust in an organization.

3. The Qualities of an Ideal Test - 1,579 unique views
Six qualities that will help make your tests as valuable as can be. Similar to the ACID properties of databases or the INVEST properties of user stories.

4. The Pros and Cons of Short Iterations - 1,555 unique views
A few words that will help you decide how long to make your iteration length. This is an important decision, and most teams and organizations don't know the factors involved.

5. Five Signs of Trouble in an Iteration - 1,489 unique views
A good howto on using burndown charts to discover problems in an iteration.

6. Seventeen Tips for Iteration Planning - 1,427 unique views
A good list of hints and tips that can make the difference between struggling with iteration planning and having it go smoothly and effectively. This is a key part of the Agile Work process, so getting good at it is a high priority for any team new to Agile Work.

7. Change is Natural - "Embrace Change" - 1,397 unique views
A short riff on the universality of change. Also introduces the idea of the "horizon of predictability".

8. The Art of Obstacle Removal - 1,287 unique views
This is a good reference article on types of obstacles and methods for removing them... a critical practice for Process Facilitators.

9. The Seven Core Practices of Agile Work - 1,285 unique views
Agile Work is really quite simple. This is a concise list of the practices that make up this effective methodology.

10. Interview with Alistair Cockburn - Agile and House Renovations - 902 unique views
Applying agile methods to home and garden renovations! Learn a bit about how this luminary of the agile world has taken agile practices out of the software world and into the hardware world.

Posted by Mishkin Berteig at 06:32 PM | |

My First Iteration Planning

I've done my iteration planning for my first iteration called "Beginnings". The length of my iterations is one week (including weekends). Here is the list of backlog items that I have committed to (some detail removed):

Catch up on Scrum course follow-up work
Advertize future courses
Finish eBook 2nd Draft
- email request for feedback return
- go through feedback
- finish references
- finish diagrams
Invoice
Finalize India Trip
Interview (Tuesday)
Meet up with and co. (Sunday)
Continue development work for

I have broken all of these into tasks, but only put here the ones for finishing my eBook since the rest get into lots of private detail.

I'm currently using TiddlyWiki to track my Work Queue, my tasks and my recurring/scheduled duties. I create a new wikiword for each iteration and copy items from the Work Queue into that section. I also have a menu that has links to my daily, weekly and monthly tasks.

I'm not doing any estimation on either Work Queue items or tasks. This may become a problem, but for now I'm going to try doing the work without the estimation overhead.

I also spoke with my wife, Melanie who is my business partner about all this. She agrees that using Agile Work is a good step to take and looks forward to all the benefits of my being more organized :-)

Posted by Mishkin Berteig at 12:05 PM | |

December 22, 2006

Technical Debt

Last night I was thinking more about the analogy of technical debt. In this analogy, design and quality flaws in a team's work become a "debt" that must eventually be paid back. This analogy is fantastic because it can be taken just a little bit further to understand just how bad defects are...

Debt is sometimes useful. Financial debt can be used for risk reduction, investment, and emergencies. But it can also cause problems. Too much debt becomes hard to manage. If debt maintenance costs more than revenue (minus other expenses), then you're going down!

Technical debt is a little different than financial debt.

Suppose I went into a boardroom full of high-power executives for some large company. (Never mind how I got there.) And I pitched this fabulous idea: I'll give them a bunch of money for them to use for operations, expansion, whatever. All they have to agree to is that I choose the interest rate when I ask for repayment... trust me!

I would be thrown out of that room so fast!

That is what we do when we encourage teams to take on technical debt.

There is no way to know the interest rate on a defect. Part of the cost of a defect is obvious: how much time and materials will it take to repair the immediate problem. (Although even that is often hard to measure.) But there are also lots of non-obvious and probably non-measurable costs. How much effort will it take to get to the root cause of the defect so that it doesn't occur a second time? How much will it affect our "goodwill" and thus reduce further and repeat sales? How much will the existence of one defect hide the existence of other defects (with their own costs)? How much will the defect demoralize the team and increase staff turnover or reduce productivity? How much of an opportunity will the defect create for competitors? How much will the defect increase maintenance and support costs?

In other words, every time someone asks a team to let quality slide, they are asking the team (and the organization) to take on debt with an unknown interest rate. Which is lunacy.

Read more about this:

Quality is Not Negotiable

Voluntary Technical Debt - James Shore talks about a situation where a conscious decision to take on technical debt led to positive results.

Technical Debt: The Threshold of Acceptable Pain - Steve Bate talks about skill and sensitivity to technical debt. Here's a great quote from this article:

What if someone has a very high threshold of pain? I’d expect to see lots of crud (technical debt) in their code.... The high threshold developer doesn’t mind spending weeks on new features that would otherwise require days or hours with clean code. And they don’t mind staying at work all night fixing bugs before a major release or spending countless hours babysitting the production system after it’s been released. It doesn’t seem to bother them to spend significant time away from family and friends or to have limited time or other interests and hobbies. ...they sometimes experience pleasure at being the “hero” who saved the company from the bugs they created.

An Incremental Technique to Pay Off Testing Technical Debt - Johanna Rothman talks about technical debt and describes a simple risk-oriented approach to reducing it.

Posted by Mishkin Berteig at 08:23 AM | |

November 24, 2006

How to Avoid Context Switching

Given the huge interest in the article by Dmitri Zimine about context switching, and despite a couple of good articles about how to determine iteration length, there has been no empirical method described, only reasoning processes. This article describes a simple method to quickly determine iteration length by experimental means.

Step One

Start with an iteration length that feels right. Use the two articles below to try decide a reasonable length. In software development, this should be no longer that four weeks. In larger volunteer communities it might be as long as three months. In a work environment where you have to deliver daily, your iteration length might be two hours.

Step Two

Build your Work Queue, and plan your first iteration. Mark the tasks that come out of your iteration planning meeting so that you know that they are tasks that were planned. As you go through this first iteration, track all the interruptions you get by writing up a task for each interruption. Each interruption task should be marked so that you know it was an interruption. If you are using note cards on a visible task board, I like to have "normal" tasks on white cards and interruption tasks on fluorescent orange cards to help them stand out!

Step Three

At the end of your iteration, determine which interruption tasks could have been deferred. In other words, determine which were truly urgent and needed to be handled in the already short time of your iteration, and which could have been put on the Work Queue, prioritized, and therefore deferred to a later iteration. This will require collaborating with your Queue Master and possibly other stakeholders.

Step Four

Count how many non-deferrable interruption tasks your team had in the iteration. For this experimental method, you can assume that this is going to be your normal number of interruptions. Divide the length of your iteration by the number of non-deferrable interruptions. For example, if you had a ten day iteration, and two non-deferrable interruptions, you would have a result of five days. Also consider what was the maximum reasonable response time for these non-deferrable interruptions. The lower of these two values becomes your experimentally determined iteration length. But you are not quite done!

Step Five

Do it all again to verify your iteration length. Note that because of your shortened iteration length, you hopefully will have far fewer non-deferrable interruptions. After your second (shortened) iteration, you can adjust the iteration length shorter if necessary... but don't adjust longer (for now).

---

I've worked with enough teams to know that for a substantial portion of them, this method would result in very very short iterations. In the software world, a team is often asked to work on a project and support their previous project. This support work tends to mean dealing with various bugs in deployed software. This is one reason why I have become such a strong advocate that quality is not negotiable.

I worked with one team that did something similar to this method (although not as rigorously) and we decided that we needed to try an iteration length of two days. This was painful for the team, but they badly wanted to build trust with their stakeholders. Their interruptions were causing them to constantly fail on their commitments. By switching to these two day iterations, they were able to defer the bulk of their interruptions and meet the commitment they made as a team in the iteration planning meeting.

---

Articles about iteration length:

Mike Cohn's article: "Selecting the Right Iteration Length for Your Software Development Process"

Mishkin Berteig's article: "The Pros and Cons of Short Iterations"

---

Now that you have gotten to the end of the article, I can admit something to you: this article is badly named. It should be named "How to determine how often to context switch so that you can meet your commitments and build trust with your stakeholders."

What this article doesn't help with is the pain of context switching. The teams that I have see that use short iteration lengths all find ways of making context switching less painful. Automation, good tool choices, workspace arrangements, etc. all play a part in this. But there is no secret ingredient to make context switching painless. Sorry!

Posted by Mishkin Berteig at 07:55 AM | |

November 15, 2006

The Case for Context Switching

Recently, Dimitri Zimine wrote an excellent little story about context switching. Joel Spolsky writes in "From the 'You Call this Agile' Department":

Dmitri is only looking at one side of the cost/benefit equation. He's laid out a very convincing argument why Sarah should not interrupt her carefully planned two week iteration, but he hasn't even mentioned arguments for the other side: the important sale that will be lost.

Okay... I'll bite.

Let's look at that argument from the perspective of the sales person first since that is where Joel's concern starts:

The Sales Guy Perspective

"I need the 'ezhibal' feature now! Big bucks coming soon if we can get it now."

Let's suppose that this urgent email has come in somewhere near the start of our two week iteration. The normal response to this in Scrum is to push back a little. The ScrumMaster says something to the effect of: "Look if you wait 7 days we can put this on the top of the list for our next iteration."

First reaction, and it's a normal one, is for Sales Guy to freak out. I've actually heard people saying things like "You're going to lose your job over this! I'm getting the VP involved and he's not going to like it" and then they stalk off to find the big dog to come and bark at us. Anyway, let's pretend that the Sales Guy is willing to be reasonable and not instantly escalate the "problem". So what he actually says is: "Look, this is super important, it'll probably only take a few minutes for us to talk about it and then we can figure out how long it will take to fix. Let's just do a quick phone call and yadda, yadda, yadda, blah dee blah..."

---

Enough of the Sales Guy perspective.

Nowadays, if I'm in this situation, I do a value assessement. I tell the team to keep working on their plan, nothing's changed yet, and I sit down with Sales Guy and the person who's sponsoring the current work and we start a discussion about the options of which there are really only two that work in Scrum:

1. Stay the Course
2. Cancel the Iteration

First, let's talk about how we decide which option to take. Then we'll talk about why.

Deciding on the option is easy. You look at the value of the work currently being done and compare it to the value of the work that Sales Guy needs. You take into account probabilities. If Sales Guy doesn't have a signed contract, then it's hard to day if there's going to be any real revenue from the 'ezhibal' feature. Still, you might be able to do an assessment of the probabilities based on your level of trust and history with the client, etc. You also need to take into account the time value of money. Does delaying the current work have consequences for another client or stakeholder? What is the cost of those consequences.

This is a relatively simple cost/benefit analysis and comparison. If you're not comfortable with money and numbers and spreadsheets, you better get comfortable!

---

Okay, so we have a way of comparing the two bits of work. Now let's look at the two "allowed" responses and a third "bad" response.

Stay the Course

Turns out that the potential benefit of the stuff Sales Guy wants is not quite as high as the potential benefit of the stuff that Suzie Stakeholder prioritized for the current iteration. Well, that's easy. Put the request from Sales Guy in our prioritized list of work and get to it when there is an appropriate level of benefit relative to the other work.

Cancel the Iteration

The stuff Sales Guy wants is super-valuable. So let's get the whole team to stop what they are doing and everyone supports this very valuble work. Stopping the whole team is appropriate because obvioulsy, the stuff they're working on isn't as valuable. Oh, and because we treat a team as a unit in Scrum. And because the team needs to commit to work, not individuals. This isn't so obvious... more later.

Peel Sarah off to do the 'ezhibal' Feature

This is what normally happens, and in a "normal" non-agile environment, it's probably okay. In a non-agile environment, Sarah hasn't made any commitments (she's been forced to agree to dates and scope, etc., but she hasn't made a commitment that she is empowered to live up to). So if she goes off and does this one little thing, it probably will be just business as usual. In an agile environment, the team has made a commitment and doing this work this way invalidates the team's commitment.

---

Why do we do it this way? The main reason is around trust and commitment. Yup, it's that soft icky stuff that we're so incredibly bad at that customers think that bugs are normal, that management shoves the kitchen sink into projects in the frustrated hope that they'll get something out of the IT team at the end of the project. Sound familiar to anyone?

Anyway. An iteration is a commitment. It is a firm and solid commitment. The team as a group of smart and honorable people is making a definite commitment to the rest of the organization to get a certain amount of work done in a fixed amount of time. In return, management is agreeing to give the team every support in reaching this commitment. When a team is new at this, they might get it wrong. But having done this with dozens of teams now, I know that after a few tries, the team gets the hang of it and commits to appropriate amounts of work, and management provides appropriate levels of support.

This commitment is essential for developing trust. And anything that comes in the way of the team meeting its commitment is considered "BAD". An obstacle to do away with.

This is interesting, because Joel's second example is about defects. And I strongly agree that defects are "BAD" and need to be dealt with at a very high level of priority. The reason is simple: they prevent a team from meeting its commitment.

---

One team I was coaching was constantly bombarded by these types of it'll-just-take-a-few-minutes-need-it-asap requests. They had many stakeholders and very very limited resources to service these requests. They had several small projects that were taking literally years to do because they couldn't get enough concentrated time on any one thing. This was considered normal and good in their environment.

The trouble is, no one had really looked at the overall consequences. Everyone was doing local optimization. For us geeks, we all know that local optimization is something to be avoided if possible. We climb a peak only to discover that we have to climb back down a ways to get up to the higher peak we now see is next to this one. We climb up that one only to discover yet another higher peak even further along thus requiring us to climb down and up again... When really what we should have done is stepped back a ways, looked at the lay of the land and said, "hey, that peak over there is the highest of them all, let's go climb that one."

Scrum helps us avoid local optimization by forcing all feature requests for a team to be prioritized in a list of work, and by allowing the team to complete small pieces of work so it actually gets _something_ done that you can learn from.

---

Joel said:

Agile is not supposed to be about swapping out one set of bureaucratic, rigid procedures for another equally rigid set of procedures that still doesn't take customer's needs into account.

True enough! But it also demonstrates a serious lack of understanding about what is being done in Dmitri's example! First of all, without being agile at all, it is possible to switch from 18 month projects to two week projects. Both can be bureaucratic as you please (well, actually, there's only so much bureaucracy you can cram into two weeks and still get something done). The shorter projects will allow you to be much more responsive to customer needs... by definition!

So what happens when you add in all the other things that agile really is about? Transparency. Truthfulness. Creativity. Learning. Meta-Learning. Prioritization. Self-Organizating Teams. Eliminating Waste.

Well, first of what you get is something that's damn hard to do right. It goes against almost everything we've been taught to do: the extreme of heroics of the extreme of careful planning and process.

Secondly, what you get is something that needs safety zones and meta-rules. Like mutual, freely-given, team-to-stakeholders commitment. Like assuming positive intent.

And thirdly, what you get is an environment where not only is the business getting what it needs more than it used to, but also, the team likes working with the business, and the business likes working with the team.

---

I admit that the point Joel is making isn't too different. Yes: look at the costs and the benefits. But agile isn't quite about instantaneous responsiveness. That's a red herring and I'm suprised that Joel threw it's stinky carcass into the discussion. Agile is also about balancing that responsiveness with the overall view of value for the team and the organization. The tool for doing that is the prioritized list of work, not the email message from Sales Guy to Sarah.

Posted by Mishkin Berteig at 04:13 AM | |

November 14, 2006

Process Facilitator "Smells"

I have now trained over one hundred people in my Agile Project Managmenet / ScrumMaster Certification course. I'm starting to see and hear some of the results of this training. There are a couple specific "smells" that I have become aware of.

Fortunately, I've been able to provide coaching to some of the organizations that have sent people to my course. There are quite a number of good things that happen, but there are a couple of things that seem to be "natural" misunderstandings.

1. Spectating
   I put a lot of emphasis on the idea of a self-organizing team in my course. There are a number of exercises, an hour-long section, and many other points during the course when it comes up. With all of this emphasis, it seems that a few people have come away from the course with an extremely hands-off approach to the Process Facilitator role (ScrumMaster/Agile Project Manager). I think this is a natural and probably good reaction to the heavy-handed command & control approach that these people come from. However, there are a few things that should be considered minimum levels of engagement (listed below).
2. Problem Solving
   There is also a great deal of emphasis put in the course on removing obstacles. I have seen several cases where it becomes the habit of the Process Facilitator to start solving every problem. This can happen in day-to-day work, and also in the retrospectives. Again, this seems to be a natural consequence of the desire to get in there and be of value. However, if the Process Facilitator writes down all the "things that need impovement" from the retrospective and then says "Okay! I'll take care of these things." then you know that the Process Facilitator has gone too far.

Appropriate Process Facilitator Engagement

Here are a few ideas on an appropriate level of engagement. Finding the right balance of enagement is not easy and there is no exact formula to follow. Partly it depends on your personality and skills as a Process Facilitator, partly it depends on the capabilities of the team, and partly it depends on the constraints of your work environment. Nevertheless, there are some types of engagement that you can persue with confidence. Here are a few concrete guidelines:

• Team Membership
  If you aren't saying "we" when referring to the team, you aren't engaged enough. The Process Facilitator is a member of the team.
• Colocation
  Most of your time should be spent sitting with the team. If the team isn't colocated, then you should be spending most of your time physically visiting each team member.
• Obligations
  The Process Facilitator is a catalyst. You must track all the obstacles and "needs improvement" things that the team raises, and then make sure that they get fixed... without going in and fixing everything yourself. Ultimately, your job is to work yourself out of a job.
• Assertiveness
  The organization that the team is working in will be creating some of the obstacles that the team is facing. Being assertive is critical to helping the organization face and then overcome some of these obstacles.

Posted by Mishkin Berteig at 01:49 PM | |

November 12, 2006

More on Scaling Agile

One problem with having multiple teams working on the same project will be the tendency to compare the teams' performance. Why is this a problem?

Why Not Compare Team Performance?

One of the main reasons is that the teams need to be collaborating not competing. There can be a small amount of friendly competition that might come naturally, but as soon as management starts paying attention to differences in team performance, the competition starts to get serious.

In the case of multiple teams working on the same project, the goal should be to maximize overall performance, not individual team performance. Too much competition can lead to unintentional or deliberate sabotage: failed communication, incomplete communication and downright dishonesty.

Motivating Teams without Comparing Them!

As Mary Poppendieck has written, measure up [pdf]. In a single-team situation this means that individuals are measured and rewarded based on team performance (their sphere of influence). In a multi-team environment, that means that the group of teams should be measured as a group and compensated as a group. This will encourage all teams to work towards the success of the overall project. I personally believe there is some room for individual-based compensation, but the way it is handled needs to be done so that it does not encourage sub-optimal behavior.

As well, each team can keep track of their velocity. Some coaches recommend using "ideal hours" or some other units to determine velocity (velocity = estimated work remaining completed / iteration). The trouble with doing this with multiple teams is that there will be a very real tendency to want to compare each team's velocity. Since velocity is a useful measure for team capacity, it is important to still have a way to measure it. One simple way to do this to prevent comparison is to use different units for each team. Team One might be measuring velocity in Estimated Ping Pong Balls Completed / Iteration... Team Two in Estimated Bananas Completed / Iteration... Team Three in Estimated Bogo-MIPS Completed / Iteration... etc. etc.

Motivating Collaboration

First off, management must make visible commitments to eliminating barriers to collaboration. For example, it is a great sign of commitment to re-organize office spaces so that all the teams are sitting near to each other. Every time the Process Facilitator identifies an obstacle that relates to collaboration (tools, process, physical environment, etc.) management should get right on it and fix it.

An ongoing investment in team-building training, workshops and exercises is also helpful. This type of investment helps people gain the skills necessary to work effectively with other people. Again, individuals need to see and believe that management cares about and values teams.

Finally, one of my pet peeves: when a project is over, keep the team together! Do not break them up and redistribute your "resources" to other efforts. The value of those people working together is substantial. The value of those people working together as a high-performance team is incredible! In a multi-team situation, it is reasonable to take teams from the overall group and re-distribute their efforts... but just don't break up the individual teams.

---

Miscellaneous Notes on Scaling Agile:

Twelve is still the maximum recommended size for a single agile team. Seven is really the sweet spot. A team larger than twelve people just takes too long to get into the Performing stage of team development. If you feel like your project needs more than twelve people actively involved, then you probably want to split into two or more teams... and then you have "scaled" agile.

If you have three teams of five people (or some similar configuration of people just over the 12 person limit), then they will work as a very close-knit group and a lot of the time will act as if they were a single team. They will probably plan iterations and do demos and retrospectives together.

Twelve teams working on the same project at once is about the maximum number before communication overhead is slowing everyone down too much. This is largely a factor of trust: with 150 or fewer people involved in an effort, it is possible for everyone to know everyone. More than that many people and it is no longer possible. Trust is just not an option anymore and bureaucratic controls take over.

If for some reason you need to do something in a small amount of time and you think it's going to take more than twelve teams of twelve people...? Break the effort into smaller chunks. Divide and conquer. Division can be across functional areas, structural areas or time.

Although I have heard of agile methods being used with groups larger than this, I haven't heard any success stories :-)

Check out my earlier introductory article on Scaling Agile in a large project situation.

---

Dean Leffingwell has an article about practices needed for scaled-up efforts at the Agile Journal. I glanced through it, but I admit that after I disagreed with his very first point (Intentional Architecture), I started to pay less attention.

He claims that refactoring of large systems is not possible (or at least infeasible). The odd thing is, most large projects that I have been involved with are being done exactly because an old system is not refactorable. A large telecom system, a large insurance system, a large data warehouse and a large GIS system are all being done with scaled up agile methods exactly because the old systems that are currently in place have become ossified to the point where they must be replaced.

These old systems were originally built with phase-based development approaches. At some point, people stopped refactoring because they were not given the space to do so. This drop in code quality turned into technical debt. The technical debt accumulated to the point where it was unbearable (maintenance costs, cost of change, etc.).

The problem with intentional architecture is that it goes back to the old assumption that you can do a good design for a system without the constant feedback from review, deployment and use done on a very short cycle time. Over and over, we are faced with the painful consequences of this attitude, and that is one of the key reasons we started to work with agile methods in the first place.

---

Martin Fowler makes a good case that scaling agile is the last thing you should do. I don't disagree! Scale your agile teams at your own risk!

It's nice for me to be able to say that I've worked on some agile projects over $10,000,000 in size, but the fact is that the cost could have been reduced substantially if the team size was lowered and the deadline extended. It is (relatively) simple to do a cost/benefit analysis of cross-team-coordination-overhead vs. the time value of early delivery of more functionality... although I've never seen anyone do it! If you know of an example of an organization doing this in a realistic way, I'd love to hear about it!

---

Are there other ways of supporting cross-team collaboration that you have seen?

Posted by Mishkin Berteig at 09:13 AM | |

November 07, 2006

Scaling Agile Projects

More and more, organizations are applying agile methods to large projects or efforts that require more than a single team. There are three dimensions or concerns of coordination. It is critical that all three be addressed, but there are many ways of addressing them. Here I will simply list these three types of coordination and make some simple suggestions of how to implement them.

I have now had the opportunity to work with several clients where they are applying agile methods to projects with budgets that are greater than $10,000,000. All of them are using multiple teams to work on the same overall project/program. Out of this experience (along with some good reading along the way), I have come to understand that the following three types of coordination are the essential ones:

Value

In order to maintain the "early and frequent" delivery of value that agile methods propose, it is important that the work of the effort be coordinated to maximize early delivery of value. From this perspective, there are often many cooks in the kitchen. I have seen a "Product Owner Team", a "Customer Team", and a number of variations of this type. In order to do the coordination work effectively, it is still necessary to make sure of two things:

1. Maintain a single Work Queue that prioritizes the work and from which all the teams select items.
2. Have a single person in the "buck stops here" role who can make final binding decisions about work priority and content.

These two items have some implications for the organization.

First, the teams must be organized to be generalist: each team should be able to handle any item on the Work Queue. Not every team is going to be equal in abilities and this can be accomodated in a number of ways. My favorite so far comes from an excellent agile coach Dave West who suggested that teams bid on the items in the Work Queue at the start of each iteration. This should be done in a collaborative fashion so that it isn't just a simple low-bid-gets-the-work, but rather the teams learn from each other and have an opportunity to adjust their bids.

The second implication is that the customer or product team (Queue Master) must have the availability to support multiple teams in a timely fashion. Ideally, there are individuals on each team who can make judgement calls about features, functionality, constraints etc. on the work and provide quick answers to questions. This is not always easy since the people doing this often have a special area of expertise and it is difficult for them to work outside this area. Just as team members are asked to become generalizing specialists, so must the people who are responsible for determining value in a project.

Process

An agile process endeavors to provide a minimally structured way to do three things: deliver value early, then learn about what is high in value and deliver that more, and finally, learn how to deliver value more effectively.

That third activity, learning how to deliver value more effectively, is facilitated by the Process Facilitator. The Process Facilitator keeps a visible list of obstacles and works collaboratively with the Team and the Stakeholders to resolve obstacles on the list.

In a multi-team environment, there may be a single Process Facilitator working with each team. Like with the Work Queue, it is often necessary to have a single Record of Obstacles for the entire project.

Technique

People develop skill and knowledge in the use of their tools. Most types of work have a special vocabularly that only makes sense to other people also doing that work. For example, the field of computer programming has programming languages, integrated development environments, build tools, testing tools, algorithms, and a host of other techniques. The field of film-making has cameras, film, directorial techniques, lighting, story structure, it's own esoteric vocabulary, and other techniques. Likewise for construction, law, medicine, drama, education, etc. etc. etc.

In a large Agile Work project, teams need a way to coordinate their technique to produce integrated, consistent and compatible results. As well, individuals on the teams may discover or create new ways of doing things that would be valuable for the other teams to know about and use.

The most effective way of coordinating technique across teams is for strong members of each team to gather regularly to review the way work is being done. This "technical support group" can look at tools, reuse, automation, patterns, vocabularly and any other "how to" aspects of the work. It is critical that these people are actively involved in work being done on a delivery team so that efforts of the technical support group do not become academic or "ivory tower".

In certain environments, it may be useful to have this techincal support group become a team with a clear allocation of time apart from the regular delivery teams. In this case, this technical support team would have its own Work Queue that consisted of requests, ideas, concerns, and opportunities presented by the regular delivery teams.

---

I have seen all three aspects of coordination implemented in large multi-team projects. Some of the common challenges include:

1. Generalist Teams.
   It is difficult enough to create cross-functional teams where people are willing to become generalizing specialists. While it is important to create generalist teams, most organizations should expect to set up non-ideal specialist teams (sometimes by line-of-business) and support their development into generalist teams.
2. Technical Coordination.
   Often organizations have a design or technical review group composed of the "experts". These people are often isolated from the actual work being performed by the teams. It is difficult, yet critical, that these people actually be involved in day-to-day work on the teams.

Posted by Mishkin Berteig at 09:15 AM | |

September 12, 2006

Yet Another Big Picture of an Agile Process

I created this image for a presentation targetted to management. It's a good high-level overview of what goes on in an Agile Work effort. Feel free to use this or the high res version... just keep the copyright notice on and put a link somewhere in your email/document/presentation/webpage/etc. back to Agile Advice. Suggestions on ways to improve the diagram would be greatly appreciated. I will try to update it... and maybe even get my incredibly talented visual artist brother Alexei to re-do it.

Here's the high resolution version, good for nice printed material: Agile Work - The Whole Process.

Posted by Mishkin Berteig at 08:52 AM | |

August 29, 2006

Seventeen Tips for Iteration Planning

Agile Work requires a team to take work items from a prioritized list, break those items into small tasks and then execute those tasks inside of the timebox of an iteration. When first trying agile, many teams have trouble with the task breakdown done in the iteration planning meeting. Here are some hints and tips for making this critical part of the agile process more effective.

1. Work items are, among their other qualities, valuable to the customer or stakeholders. Think of these work items as being built out of many smaller pieces. You can imagine a toy model made out of Lego bricks. Each brick, by itself, is of no value. It is only when they are put together that they become useful. The tasks you create for each work item are often small enough that they do not have any value on their own.

2. The word "task" implies activity... but the tasks you create for your work items do not need to be activity based. Tasks can be effective if they are written as components or pieces of the work item you are building. Here is an article about creating good tasks.

3. Sometimes if the work item is not well understood, you might find that a "research" or "experiment" task is a good starting place. Try to be as specific as possible. When writing the task description, spell out what exactly is the goal of your research, and possibly list what options you are going to research.

4. When first starting out with Agile Work, many teams find it difficult to do a good job of iteration planning in the fixed amount of time allocated to it. Consider shortening your iteration length so you can practice this skill more frequently. (Remember to make the planning meeting shorter too!)

5. Make sure that everyone has index cards and a pen to write with! Team members shouldn't have to wait for a "scribe" to write down a task. In many ways this is a brainstorming session. The Process Facilitator can collect all the cards after the meeting is finished if they need to be recorded more formally.

6. Do a first pass by creating "big" tasks... then break them up into smaller tasks if you have time. Since this meeting is timeboxed, it is better to get all the work broken down into big chunks than to break down only a small part of the work into very fine chunks.

7. If the same task keeps showing up for all your work items, it probably shouldn't be a task... instead it should probably be a process step or constraint or condition of satisfaction for the work you are doing. For example, if you always have to write a document that follows a template to record what you have done for each work item, then writing that document can be shown as part of your task board.

8. It's okay for tasks to be _very_ small.

9. Share your tasks! If you write a task down without telling the rest of your team, they can't use your idea to generate more tasks, nor can they improve on your idea.

10. Generating tasks in the iteration planning meeting is a problem solving and creative process. This is where you do a lot of your analysis and design work. This is where you struggle through options and choose _how_ to build/do your work.

11. Consider creativity technique such as light-weight brainstorming for generating lots of ideas quickly. Any technique you use should be streamlined for quick results.

12. Don't worry about administrative stuff while you are generating tasks. For example, if you normally put task cards up on a wall, wait until after the meeting is over to do this. Likewise, if you normally enter them into an electronic tools, wait until the meeting is over to do this.

13. Make sure you have scrap paper or a good whiteboard convenient for notes and drawings so that you can quickly model your solution for the work item. (Check out Agile Modelling for a discussion of this in the software realm.)

14. Remember that it's okay if you end the meeting with an imperfect list of tasks. You will make corrections throughout the iteration. It is more important to maintain the discipline of the timeboxed meeting length, than to get the tasks right up front.

15. The whole team must participate: everyone's experience, skill, expertise and insight are needed to do the best job of generating tasks. Just because it won't be a perfect list doesn't mean you can do a shoddy job of it!

16. You need quick access to information about the work items you are planning. You also need quick access to other relevent information. A computer with a web browser open to Google is a great tool to have at hand.

17. If you don't have anyone on your team who has lots of diverse experience and expertise, then consider inviting someone like this as a guest to help you out. It is much more difficult to do the necessary problem solving if you new to the medium in which you are doing the solving. Such a guest would need some time before the meeting to be prepared.

Posted by Mishkin Berteig at 10:44 PM | |

June 14, 2006

Agile Work Artifacts - Tasks

I recently described the Work Item List. The second type of artifact used in an Agile Work environment is the Task. At its most basic, a Task is a simple description of how to do some bit of work towards completing an item in the Work Item List. However, there are some important things to remember when using Tasks.

What is a Task

Normally, an item from the Work Item List is broken down into multiple Tasks. These Tasks are all the things that need to be done or built to get the item into a deliverable state.

In general, the process of creating a bundle of Tasks for a given Work Item is a design or analysis process. It is a problem-solving process. The Tasks themselves represent the solution: the building blocks of the structure of the Work Item. Tasks do not normally represent the problem solving or analysis process.

Here are two contrasting (somewhat contrived) examples of Tasks generated for a "Home Office Addition" to make this clear:

Example 1 Bad:

• determine requirements for home office
• design home office layout
• check home office layout with client
• choose furniture and decor
• build home office
• install furniture and decor
• unpack books and files into home office

Example 2 Good:

• 300 sq. ft. maple hardwood
• 4' x 7' custom desk
• Aeron chair
• red leather sofa
• 16' custom bookshelves
• 22" double French doors
• 8' 4-panel vinyl triple glaze bay window
• 36 2x4x10' metal studs
• ... drywall
• ... other building materials and furniture and decor

Generating Tasks

The whole team works together to come up with the Tasks for every Work Item being worked upon during the iteration. The majority of the Tasks are determined during the timeboxed iteration planning meeting. Sometimes, the iteration planning meeting is not long enough to get all the Tasks defined. Sometimes new Tasks are discovered during the course of the iteration. Either way, it is fine to make changes to the bundle of Tasks as the work proceeds.

Special Tasks

Due to the timeboxed nature of the iteration planning meeting, a Team will sometimes find itself knowing that there is more analysis or problem-solving needed for a given Work Item. In this case, a special Task can be created to indicate that this analysis work needs to be completed. For example, the task might be labled "Finish analysis for Work Item 62", or it might say "Team meeting to finish Task generation for Work Item 62". Other ways of writing these Tasks include "Research...", "Solve..." or "Discuss...". Everyone needs to recognize that these tasks represent risk and uncertainty about how to get the work of the iteration complete.

Sometimes work needs to be done that is outside of the control of the Team. The Team should always try to find creative solutions to avoid this situation, but it is almost inevitable that Tasks will come up for which the Team does not have the expertise or the authority. These Tasks need to be treated very carefully since they can seriously hinder a Team's forward progress. The Process Facilitator should usually consider these Tasks as obstacles to be removed.

Working on Tasks

Typically, a single person takes responsibility for a single Task at a time. A person may end up working on other tasks for two reasons. First, if the Task he/she is responsible for has an unavoidable wait, then that person may go find someone else to assist until the Task is ready to be worked on again. Secondly, another person may have an urgent need for assistance and call other people away from their current Tasks.

Tracking the Work

In their most basic form, Tasks are assumed to all be roughly the same size or effort and small enough that they are only done or not-done. Therefore, tracking the work on the Tasks throughout the course of the iteration is kept very simple: how many tasks are remaining to complete in the remaining time of the iteration? There is no ambiguity: Team members report on completion of tasks to the Process Facilitator who updates a burndown chart to show the quantity of remaining work.

Gotcha's

Tasks must be written to avoid including wait time. For example, a Task that says "Get the parts delivered" actually includes three steps: make the request for the parts, wait for the parts to be delivered, receive the parts. Since it is often hard to predict how long a "wait" will be, it is important to break this down into two separate tasks: "order the parts" and "receive the parts" and then treat the wait time as an obstacle to be removed/minimized.

Posted by Mishkin Berteig at 09:34 AM | |

June 06, 2006

Agile Work Artifacts - the Work Queue

Agile Work requires only a very small number of simple "artifacts". The most basic is the Work Queue. This is very similar to the Scrum Product Backlog but there are some differences too. The Work Queue is like a carefully managed To-Do list. This article details the use of the Work Queue.

Basic Description

The Work Queue is a list of work to be done by a person, team, organization or community. The list consists of brief descriptions of what to deliver, not how to do it. The list is prioritized so that the item on the top of the list is the single most important thing to get done, with items below that being successively lower priority. All items on the Work Queue contribute directly to an overall goal. The person holding the Product Owner role manages the list.

Place in the Process

The place of the Work Queue in the process is very simple. The Work Queue is initially created after the overall goal is determined. Ideally, it is created before work towards the goal is started, but often Agile Work will be adopted for an ongoing effort, in which case work has already started. The Work Queue is constantly maintained by the Queue Master so that it maintains the above mentioned qualities. One or more of the items from the top of the Work Queue are selected to be worked on. As an item on the list is completed, it is removed from the Work Queue. The number of items on the list that are being worked on simultaneously will depend on the capacity of the people doing the work. If iterations are being used to plan work, then the Work Queue is updated every iteration.

How is it Created/Managed

The Product Owner is responsible for creating and managing the Work Queue.

The initial creation of this Work Queue is a simple process: based on the goals to be attained, and based on an other factors that are relevent, the Product Owner drafts an initial version of the Work Queue. The time spent on creating this initial version should be minimized using three techniques:

1. Keep the Work Queue small by recording only high priority items that need to be done immediately. Don't add items that are uncertain or will be done far in the future. (See "The Horizon of Predictability" for more information about this.)

2. Keep the Work Queue simple by focusing on point-form high-level descriptions of the items in the list. Don't worry about any details about how the item will be accomplished, who will do it, when it will be done, dependencies with other items, or technical or conditional details.

3. In advance, allocate a fixed amount of time to draft the Work Queue... and don't spend any more time on it than that. Get the assistance of the Process Facilitator to keep on track. The list doesn't have to be perfect or complete... you only need enough on the list to allow the work to start.

Ongoing management of the Work Queue consists of removing completed items or items that are no longer needed, adding new items as they are discovered, merging or splitting items as more is learned about them, and reprioritizing items. All of these tasks can be performed at any time. As well, the Work Queue should always be in a state of readiness such that the top items are ready to be selected for work.

The Product Owner is responsible for answering any questions that may arise about the work as it is progressing. If someone doesn't know what is meant by an item, then work on that item should halt until the Product Owner can clarify.

At no time is the Work Queue "frozen" or finalized. It is changing throughout the entire time that people are working towards the goal. Items that are completed are removed from the list, and do not need to be archived or recorded anywhere else, nor is it necessary to save versions of the list as it is changed.

Gating/Queuing

The Work Queue is a queue of work in process (WIP) that is built up in front of the people who are doing the work. As such, it is advisable to keep the list short. The Product Owner acts as a gate to the list: only those things that reasonably contribute to the goal and which are relatively immanent in implementation should be added to the Work Queue. If someone suggests something be added to the Work Queue, the Product Owner must, of course, take that under advisement and collaborate with the suggestor, but is not obligated to add the suggestion to the list.

The Product Owner can manage the size of the Work Queue by deciding how many iterations of future work are allowed on it. In other words, the number of items on the Work Queue is limited to how many can be accomplished in a fixed number of iterations. As the Team takes a number of items off the Work Queue, space is freed to add an equivalent amount of work back onto the Work Queue. The specific number of iterations chosen for this will vary depending on your circumstances.

Differences from the Scrum Product Backlog

The Work Queue does not require two things that are part of the Scrum Product Backlog: item estimates and item values. Adding this information to the Work Queue can be useful in many situations, but is not an essential part of the list.

As well, the management of the Work Queue is different from the Scrum Product Backlog in that the Product Owner is not obligated to put every suggestion that comes their way onto the list.

Advanced Use of the Work Queue

Consider adding the following information to each item on the list:

1. An estimate of effort. Make sure that these estimates are created by the whole group of people who will be doing the work.

2. An estimate of value. The Product Owner needs to be responsible for determining value, but the whole group of people doing the work and all the other stakeholders need to understand how that value was determined.

3. Cycle time tracking. When an item is added to the list, record the date/time it is added, then record the the date/time it is completed and removed from the list. The use of cycle time can assist in making a work process more efficient.

Posted by Mishkin Berteig at 04:24 PM | |

September 26, 2005

Wideband Delphi Estimation Technique

Here's an excellent introductory article on the wideband delphi estimation technique. Typically wideband delphi is used to estimate software development efforts, but can be used in almost any domain of work. This method might be applied to estimating effort for items in a work list at either a project level or tasks in an iteration work list. The way it is described, it sounds fairly heavy-weight, potentially taking several hours for a relatively small list of work items. However, it is worthwhile for process facilitators and product owners to be aware of these sorts of methods if problems with estimating occur in a project team. The wideband delphi model is related to the Delphi method.

[Update 2007/05/24]

Agile Work and Scrum use a very different method of estimating work that has one very important and incredible property: people don't have to get better at estimating for them to get better at committing to work!

The basic idea can be thought of as "commitment velocity". The method here is to use an arbitrary unit of effort that the team applies to estimating all tasks relative to each other. At the start of an iteration, the team can use any collaborative method (including wideband delphi) to come up with estimates for tasks. The team sums up the estimates for all the work of the iteration. Then, at the end of the iteration the team looks at the estimates for all the remaining undone work (if any) and subtracts that from the start of iteration sum. This is the Commitment Velocity. Now on their next iteration, they do their estimation work again, relative to the tasks in the previous iteration. If the sum of the estimates is larger than the Commitment Velocity, then the team needs to de-scope or find more efficient solutions to their work. This process usually converges after only three iterations (assuming: constant team composition, constant iteration length).

This method is taught in detail in my Agile Project Management courses, and there is a little bit more information here: Planning vs. Commitment. The agile estimation method of Commitment Velocity is an application of the Central Limit Theorem from statistics.

[Update: 2007/09/11]

Wikipedia now has an article on Wideband Delphi. The article points out some interesting potential problems with the wideband delphi method including corruption of the group doing the estimation and suppression of minority opinions. I know that in my experience as a coach, I have seen numerous occasions of both types of problem with estimation processes. The fact that wideband delphi is susceptible to these problems is more a factor of the human side of things than the method itself.

Interestingly, the wideband delphi method of estimation is very similar to the "Planning Poker" method of estimation used in many agile and scrum project environments. This method is also covered in the Agile Project Management courses.

Other links to wideband delphi:

Wideband Delphi Estimation Process

The ProjectInitiation.com website has a wideband delphi worksheet (MS Word) available.

Project Estimation: Wideband Delphi

Posted by Mishkin Berteig at 12:40 AM | |

September 06, 2005

Process Facilitator Role

I've been thinking a lot about the roles on Agile Work projects. Here is a possible "mission statement" or definition for the Process Facilitator:

The Process Facilitator is a person who is both experienced with Agile Work and trained as a facilitator. The Process Facilitator acts as a coach to the team to monitor the process, foster the understanding of the Agile Work Axioms, the development of the Agile Work Disciplines and adherence to the Agile Work practices. The goal of the Process Facilitator is to assist a team to become "performing" so that they are able to actively and independently persue continuous learning and improvement.

Also Known As: Scrum Master, Coach, and previously referred to as the "Process Owner"

Never Spread Your Facilitator Butter too Thin

Posted by Mishkin Berteig at 12:27 AM | |

May 17, 2005

The Qualities of an Ideal Test

These six qualities of tests describe how to make a test as effective and as useful as possible. The qualities are similar in style to the INVEST qualities of user stories - but they don't form a nice acronym.

Decisive: The test contains all the information required to automatically determine success or failure. Manual inspection of test results is not necessary to determine success or failure. The test is expressed in a way that produces a pass/fail answer rather than a numerical or qualitative result. Decisive tests are often expressed as assertions.

Valid: The test produces a result that matches the intention for the work artifact under test. Test failure indicates failure in the artifact under test and test success indicates correct operation or configuration of the artifact under test. Simply put: the result of a test should be true.

Complete: The test contains all the information it needs to run correctly with a given test harness and work artifact under test. The test performs all activities and provides all the data necessary. The test does not require any input outside of itself in order to run.

Repeatable: The test always gives the same results if the test harness and the artifact under test are the same. The test is created in such a way that the result is deterministic. Even if the system under test is not deterministic, the test should be created to account for that (possibly with statistical analysis) and produce a deterministic result.

Isolated: The test result is not affected by other tests run before it nor does a test affect the results of tests run after it. The test and the test harness work together to clean up after every test is run. A collection of tests can be run in any order and always produce the same results. Any test that depends upon the results or side-effects of a previous test is not isolated.

Automated: The test requires only a start signal in order to run to completion in a finite amount of time. No manual intervention is required after the test is started. Tests that are automated can be put together into a test suite and run one after another without human intervention. Automation of tests requires the creation of a test harness system.

---

Update 20060106

I've added a little bit more detail to the above qualities. I also wanted to say a few words about my experience with testing:

I first started doing test-driven development almost seven years ago after reading the book Refactoring by Martin Fowler. I picked it up in Windsor Ontario while I was driving to San Francisco for my first contract position as an independent. I had to stay the night there due to the immigration office being closed so I read a lot of the book in my motel room that night. By time I got to California four days later, I was totally convinced that I should be using JUnit and refactoring for everything I did. Fortunately my project was ammenable to this approach. Over the next four months I built a Java JMS layer on top of Tibco Rendezvous. In the process I discovered methods for doing unit tests for asynchronous multi-threaded distributed functionality. And my tests satisfied all the above qualities. I delivered code with 100% test coverage and zero defects discovered until more than two years later when a very rare and obscure deadlock issue was found. Suffice it to say, I was convinced from a quality perspective.

But there is more. To build tests that follow all the above qualities, you need code that is testable. I have come to believe that testability is the most important architectural attribute of code for most software. The implications of having testable code include code that is easily changed, that is verifiable, and that is easy to understand. Refactoring plays a big role here too.

For getting a basic but very practical sense of test driven development, I strongly recomment Test Driven Development: By Example by Kent Beck.

---

I have also written a couple of articles recently about quality:

Quality is Not Negotiable in which I talk about how to handle defects - by always treating them as top priority work items.

and

Technical Debt in which I expand on this common analogy between lack of testing and debt to show that technical debt is even worse than financial debt.

Posted by Mishkin Berteig at 06:48 PM | |

May 16, 2005

Iterative Delivery

Work can often be divided up so that the smaller pieces are valuable on their own. By dividing work this way, a team can deliver value incrementally. The team can choose a short period of time called an iteration and select a small amount of work to complete in that time. This work should be valuable on its own. For example, if a team is building something, then at the end of each iteration whatever is built is usable as it is. This means that each iteration includes all the planning and design as well as construction or creation necessary to deliver a final product or result.

For example, a volunteer group may desire to attract new members. A non-agile approach would have the group plan their membership campaign completely before actually executing on it. An agile approach using iterative delivery would have the group plan a small piece of work that will attract some small number of new members, execute it, and then start a new iteration. One iteration may cover the creation of and delivery of a door-to-door flyer in a neighborhood. Another iteration may cover the design, creation and publishing of a small advertisement in a local newspaper. Each iteration includes all the steps necessary to produce a furthering of the group's goal of attracting new members.

In a business environment, iterative delivery allows for a much faster return on investment. The following diagram compares delivering value iteratively with a non-agile project delivery where results are delivered only at the end of the project:

One can see clearly from the diagrams that the non-agile delivery of value at the end of a project is also extremely risk prone and suseptible to change. If the project is cancelled just before it delivers, then a fairly substantial amount of effort is wasted. In the agile iterative delivery situation, an endeavor can be cancelled at almost any time and it is likely that substantial value has already been delivered.

Even if the work cannot actually be delivered incrementally, it almost always can be divided in a way so that it can be inspected in stages. Either method of dividing work allows us to do the work in iterations.

Iterations are fixed and consistent units of time during which work is performed and between which planning, inspection and adjustment is done. The empowered team will decide on the length of iterations for their work. As a rule of thumb iterations should be shorter than the horizon of predictability. Generally, iterations should never be longer than one month, no matter what the endeavor.

At the end of each iteration, a demonstration of the work completed is given to the stakeholders in order to amplify learning and feedback. Between iterations, the stakeholders collaborate with the team to prioritize the remaining work and choose what will be worked on during the next iteration. During the iteration, the stakeholders need to be accessible for questions and clarifications.

Iterative and incremental delivery is used to allow for the early discovery and correction of mistakes and the incorporation of learning and feedback while at the same time delivering value early.

Posted by Mishkin Berteig at 10:27 PM | |

May 15, 2005

What To Do With the Horizon of Predictability

In a previous entry about constant change, the idea of a horizon of predictability was introduced. This concept, along with the agile discipline of amplifying learning, suggest a strategy for addressing problems in a project.

Shorten the length of the iterations you are using. Contract your "planning horizon". The length of your iterations should be motivated by the horizon of predictability for your environment. If your project encounters trouble, particularly of the sort where it looks like you might not accomplish your commitments for an iteration, then shortening the length of iterations will enable you to resolve your problems.

First off, by shortening your iteration length, your opportunities for learning become more frequent.

Secondly, a contracted planning horizon will put you more firmly inside the horizon of predictability... and therefore there will be fewer unexpected changes (on the whole, not in any specific iteration).

A related article is The Pros and Cons of Short Iterations.

Posted by Mishkin Berteig at 06:34 PM | |

May 13, 2005

Self Organizing Systems FAQ

Just a link to the Self Organizing System FAQ - glanced through it, it looks amazing.

Posted by Mishkin Berteig at 10:29 PM | |

Amplify Learning

Learning is the result of both encountering new experiences and deliberate experimentation. Learning creates new knowledge, increased volition and improved action.

Some of people's best learning comes from “failure”. An essential component of learning is feedback.

Learning and feedback can be amplified in several ways. Provide opportunities for learning through books, training courses, coaching, deliberate exposure to diverse work, and deliberate experimentation. Frequently ask the questions “why?” and “how?” and answer the honestly and deeply. Increase the level and quality of communication among the stakeholders and team members. Inspect work in progress frequently or even continuously. Learning accelerates greatly if a culture of learning is created where individuals feel free to experiment and take initiative even on critical aspects of the work.

Learning and feedback support all three agile principles. People become more effective creators as they learn. People are better able to adapt to and embrace change as they learn. And a person's span of perception increases as they learn. Any increase in learning or feedback leads to an increase in the manifestation of the principles.

Learning and feedback can be amplified rapidly, but an empowered team is necessary to effectively take advantage of this amplification. If a team is not empowered, then rapid learning can lead to frustration. Amplified learning and feedback result in excitement, enthusiasm and playfulness, rapid problem solving, high quality work results and satisfied stakeholders.

An excellent analysis of learning at a social or group level is presented in Progress and Its Problems by Larry Laudan. In this very well written book, Laudan takes a look at the history and philosophy of science and develops a model for learning that is applicable to the development of agile methods.

Posted by Mishkin Berteig at 01:12 PM | |

April 28, 2005

Plan Methods Oppose Agile Work Axioms

Plan driven methodologies which attempt to mechanize the process of doing work are in opposition to the three Agile Work Principles.

We are Creators
A plan methodology attempts to define intermediate and end work products independently of the input and effort of those who perform the work of creating the work products. This disenfranchises people from their work and leads to low morale. It also establishes a heirarchy of value for the people working on an effort where those who create the plans are perceived as more important or valuable than those who execute on the plans.

Change is Natural
This principle is usually acknowledged, but is usually described as a "problem" to be dealt with rather than as a basic principle to be fully embraced. A plan methodology has "change control" or "change management" and "risk management" and puts the whole notion of change in a negative light. This approach also disenfranchises people because they are constantly placed in opposition to reality.

Reality is Perceived
Plans attempt to legislate reality. "Thus and so must the project go" results in a constant struggle between the plan and peoples' perception of reality. Plans marginalize the importance of perception on the belief that reality can be objectively understood. If reality can be objectively understood, then it can be mechanistically manipulated. Thus results can be pre-determined without regard for the perception of those results.

Posted by Mishkin Berteig at 10:54 PM | |

April 27, 2005

Eliminate Waste

Waste is the result of activities or environmental conditions that prevent a team from reaching its goal. The opposite of waste is something that adds value (more, faster or higher quality) to the desired result.

The whole notion of eliminating waste comes from lean manufacturing. More recently, Mary and Tom Poppendieck applied this idea to software in their book "Lean Software Development: An Agile Toolkit for Software Development Managers". In this (excellent) book, the authors list the wastes of manufacturing and the wastes of software. Here I have summarized and generalized these types of wastes so that they apply in any situation:

The Seven Wastes

1. waiting caused by delays, unreadiness, or simple procrastination

2. partially done work or inventory caused by sub-optimal workflow

3. extra processing or processes caused by poor organization or bureaucracy

4. defects and rework caused by insufficient skill, tools, inspection or filtering

5. movement of people or work caused by physical separation

6. overproduction or extra features caused by working towards speculative goals

7. task switching caused by multiple commitments

As wastes are eliminated or reduced, a team will function faster and with higher quality. However, not all waste can be eliminated. Sometimes waste is legislated, sometimes waste is an unavoidable byproduct of work, sometimes mistakes are made, and sometimes it takes a great deal of effort to eliminate a waste.

In order to eliminate waste, first waste has to be detected and identified, then the underlying causes of the waste have to be identified, and finally changes to the work environment need to be made to both eliminate the cause of the waste and the waste itself. Many agile work practices help with this process.

Value stream mapping is one particular tool that can be used by a team or organization to identify wasteful activities. The team describes the amount of time that work takes to go through each activity in their overall work process. Next, the team determines if each activity adds value or does not add value to the end goal. All activities are subject to speed improvements, and activities that do not add value are subject to elimination.

In order to determine the causes of waste, special attention should be paid to incentives and motivations. Wasteful behavior often exists because there is some incentive for people to do it. Sometimes these incentives are explicit, but sometimes they are the side-effects of other things going on in the team's environment. Changing the incentives can be an effective way of reducing waste.

By eliminating waste, the team will find it has reduced frustrations, and enabled greater productivity and creativity. The team will also increase its speed and delivery of value, and at the same time reduce defects.

Posted by Mishkin Berteig at 07:40 PM | |

April 18, 2005

Book Review: "Good to Great" by Jim Collins

Summary: In this well-written, engaging book, author Jim Collins describes six critical factors that he and his research team found common in companies that transformed themselves from a long period of mediocre or bad results to a long period of great financial results. Although focused on publicly-traded companies in the United States, the results of this research can easily be extended to apply to other types of organizations.

Good to Great describes the following six concepts:

1. Level 5 Leadership: personal humilty and professional discipline in an organization's leader are the starting point for the transformation.
2. First Who... Then What: don't worry about what to do until the right people are in the right positions in an organization.
3. Confront the Brutal Facts (Yet Never Lose Faith): careful and honest examination of an organization's current situation is the foundation for finding a path forward.
4. The Hedgehog Concept (Simplicity within the Three Circles): discover a simple business concept that people can be passionate about, that works with a single key economic driver and that the organization can be the best in the world.
5. A Culture of Discipline: disciplined people removes the need for heirarchy, disciplined thought removes the need for bureaucracy and disciplined action removes the need for excessive controls.
6. Technology Accelerators: pioneer the application of carefully selected technologies without relying on technology for transformation.

Some of these concepts are very close to the underlying prinicples of agile work. For example, in the Scrum methodology, the first five principles are all represented. The Scrum Master embodies some of the attributes of level 5 leadership. A self-organizing team gets the right people in the right position. The daily scrum and the sprint planning meetings are designed to help the team confront the brutal facts. The sprint goal embodies at a local level the simplicity of the hedgehog concept. And the practices in general are a manifestation of a culture of discipline.

Who Should Read this Book? Anyone interested in creating a lasting positive transformation in an organization should read this book. In particular, individuals who are coaching teams or organizations should read this book since the concepts also appear to apply at a sub-organizational level.

Posted by Mishkin Berteig at 07:29 PM | |

April 14, 2005

Stealth Methodology Adoption

This link was seen on a scrum-toronto list, referring to an e-book called Stealth Methodology Adoption. The title is brilliant, and reflects, in my view, a significant means of adoption of Agile technologies at this point in the maturity of this market.

More projects than not, a small straw poll of friends and family in the business indicated, where Agile was used, used it under the radar screen. Often artifacts were translated into the artifacts of more traditional PMOs, and not a word was said. However, this is a dangerous approach, however necessary at times. The partial implementation of an Agile method, without buy-in in certain quarters can lead to a failure to realize the benefits of the method. If the practices that were used can be blamed, they will. People love to find scapegoats, and there are costs to agile that can be pinned. For example, if the project fails, but TDD was used in isolation, or used badly, it can be held to account as having taken more time than just writing code. If Pair Programming is used in an otherwise traditional project that fails, it can be identified as a halving of productivity. If daily scrums and backlog are used (again, by themselves, or without experience), they can be seen as a failure to gather sufficient requirements and manage to those requirements.

It take subtlety and wisdom to know how much agile to put into an organization, how much visibility, and which particular practices will garner the most benefit in a particular area. Most agile methods have a combination of practices that encourage emergent behaviour. But in many cases, the practices themselves are emergent. The interaction of daily scrums with a scrum master that's working the group to a sprint backlog, and working with a product owner to refine the product backlog for the next sprint - without interrupting the sprint - these have powerful effects on productivity when put together. But, for example, all you have to do is interrupt the sprint and start messing with sprint backlog, or have a product owner / scrum master that doesn't do the work of refining a good product backlog and you end up with developers who don't know which way to turn at a given moment.

Similarly, TDD - if you are writing tests first, but you're not actually defining "done" properly, (ie. sufficiently covering the desired behaviour and edge and negative cases) then you can end up having someone write a lot of tests that are practically meaningless to the code in question. This means that you don't get the benefits of the testing, plus you incur the costs not only of the tests, but also of the wasted time sifting through test code that was not written against the requirements of the interface.

In a lot of these situations, it is judgement that comes from experience that is the real solution. There are few formulae that can practically resolve these questions. The benefits of NOT implementing Agile in stealth-mode, therefore, include the ability to bring in experienced mentors who can provide this kind of judgement to an organization new to agile approaches. Most "stealth agile" projects don't have budget for such. But books like the afore-linked can help communicate some of this wisdom and judgement.

Posted by Christian Gruber at 07:08 AM | |

April 13, 2005

Backlogs in an Organization - Levels of Queues

Queues of work form at three types of levels in an agile organization.

At the largest level is the project portfolio. The queue for this level contains all the projects that are not yet being actively worked upon by project teams.

At the intermediate level is the backlog of project functionality. The queue for this level contains packages of business function or infrastructure components necessary to implement business function. These packages are selected by a project team to fit into a single iteration.

The packages in turn are also elements in a queue. This smallest level of queue contains individual tasks required to implement all the business function and infrastructure that make up a selected package of work. The team members select tasks off this queue based on priority and dependencies.

In many agile methods, the queue management approach is fairly explicit at the intermediate and small levels. However, very little is said about the largest level. Some organizations have solved this by limiting the size of projects:

To be successful, high-tech CIOs recommend biting off projects in small chunks.... Gregoire notes that Dell is growing so fast that at the end of an 18-month project, the company would be significantly different from when it began. "A project has to take less than six months [to complete]. That's the only way we can make sure [it stays] with the business," he says. (http://www.cio.com/archive/120198/tech.html)

Posted by Mishkin Berteig at 06:19 PM | |