November 02, 2007
Dynamic Strategic Planning
I ran across this idea of Dynamic Strategic Planning [pdf] a couple nights ago at a meeting where a friend and I were talking about agile applied to non-technology work. He told me about Richard de Neufville who had come up with this concept of making strategic plans that were adaptable. It involves the use of Real Options, negotiation and other aspects of planning. I don't know anything more than what is written in the paper I have linked to... do any of you have experience with Dynamic Strategic Planning that you could tell us about? Are there any good books about this that you would recommend?
Posted by Mishkin Berteig at 06:26 AM | |
October 23, 2007
Truthfulness in Agile
I believe in Agile. To the best of my knowledge and based on my experience, Agile methods are the best way for teams, organizations and communities to get stuff done. Doesn't mean it's easy though! The biggest challenge of Agile is in the need for truthfulness.
Most of us live and work in environments where the truth is not totally welcome. Maybe it's "little white lies". Maybe it's hiding behind the raw facts. Maybe it's withholding information. Maybe it's exaggeration. Maybe it's "rose colored glasses". Maybe it's self-preservation. Maybe it's getting attention. Maybe it's lack of confidence or over-confidence. Truthfulness is compromised on a regular basis in most work environments and in most interpersonal relationships.
Agile methods have several powerful benefits. To get those benefits, agile needs to be done right. To do agile right, requires truthfulness.
Why does agile require truthfulness? Because agile methods have fewer checks and balances, fewer gating points, far fewer bureaucratic procedures, than other methods. This lack of safety mechanisms means that it is possible to subvert a team using an agile method more easily than it is to subvert a high-ceremony, high-rigor, high-paperwork type method.
That said, there are
- some
Here is a worst case scenario.
Someone on the team is frustrated or angry about something. This person, let's call him Joe, decides he's going to lash out. He starts by spreading some not-very-nice-nor-accurate gossip that gets the other team members to not trust each other as much. He then makes up some sort of technical risk factor and starts to inflate his estimates and convinces a couple other people to do so as well... just in case. The team's velocity appears stable, but their actual output has decreased. The team is not yet mature enough in their interactions so these things are not mentioned in the retrospective. Joe gradually finds ways to demoralize everyone on the team. The team starts having more and more problems with code quality which slows it down. The process facilitator, Mary, notices these things. However, Joe was planning ahead. Joe has been talking with Mary to help her see how everyone else on the team is subverting the process, thus deflecting attention away from himself. Mary, also relatively new to her role, isn't careful about checking facts and steps on some toes. Now she is almost powerless to influence the team to get it back on track... and the final blow comes as Joe starts raising obstacles in the daily status meeting that are "made up" and designed to be difficult if not impossible for Mary to help with thus solidifying her impotence in the minds of the rest of the team. Ug.
Is there any way out of this? Not really. Intervention is possible by an experienced coach, but not guaranteed to succeed. The best thing to do is probably to disband the team get an expert retrospective facilitator in to try and extract some of lessons, and then try again with a new team.
What else could be done? Is there a specific intervention that could be performed given that we don't know that Joe is behind it all? Could this really happen in an agile environment?
Posted by Mishkin Berteig at 09:05 AM | |
October 12, 2007
Agile Classroom Management
I'm fascinated by the idea of applying agile methods outside of software... be it to business management, family and household, or, as I and my father have been exploring over the last two and a half years... agile classroom management. Here's how I do it in my Agile Project Management / ScrumMaster Certification courses:
My father, Garry Berteig, had been using agile methods in his classroom for over two years before I tried it myself. I had gone into one of his courses at Keyano College in Fort McMurray to give a presentation to his second-year media students on agile and Scrum. They used it for a video project (see "A Student Documentary Film Project" and "Scrum Saves the Day for Media Student". Out of that experience, and others, he developed the idea of the Learning Circle as the basic cycle in an agile learning environment.
In the meantime, I became a Certified Scrum Trainer. Back when I did it, this just meant co-teaching a Scrum class with Ken Schwaber. He had a standard slide deck which he presented as he masterfully guided a class through two days of Scrum training. I (as expected) took that deck and made it my own by reformatting, rewording, changing a few of the exercises, and changing the emphasis in a couple of spots. And then I used that deck with minor corrections and updates and some expansion over the course of the next 16 months.
In the meantime, I was also doing a fair amount of training for teams and organizations that didn't care particularly for Scrum, but just wanted to know more about this "agile" thing. Sometimes it would be a one-day course, sometimes as much as five days (that was exhausting!!!). I developed a substantial body of modules including exercises, case studies, and special topics. Eventually, I found that I was having a hard time fitting my fixed agenda materials into the time I had allocated. For example, I was commonly doing a three-day course and would find that by the end of the first day I was a little behind on my agenda and by the middle of the third day I would certainly be running out of time and miss possibly even two or three modules in the agenda.
Finally, last Sprint, I got a big client who asked for training for a large number of their staff. This gave me the opportunity to: a) work with the same materials and agenda at least three times, and b) involve an assistant in all three deliveries. The first time through, I had the typical problem where the third day was a terrible rush. The second time through, it happened again, but I had de-scoped and re-arranged some of the material so it didn't seem as bad. After that second time, my assistant, an excellent coach, David Chilcott, suggested that I try to apply a more flexible... agile... approach to handling the materials. We did a quick revision of the materials to support this idea and then the third class was run as an "agile training environment". What did this mean?
The main insight was that the time allocated to the class was timeboxed. This meant finding a way to prioritize the work of the class so that the majority of the participants would get the best value possible for their time in the class. We did this in two ways:
1) As questions came up, we found that some of them needed to be deferred because they would be answered in a later module or because there was background material that needed to be covered first. So, for every question that needed to be deferred, we wrote it up on a 3x5 note card and put it up on the wall. Now, in order to keep track of this, we also had all the course modules written up on 3x5 note cards and up on the wall too. Most of the question cards would immediately be put up with the associated module card.
2) We drastically cut back the core required modules for the three days and made everything else optional. We then decided on a simple in-class mechanism to choose from those optional modules. At the end of the first day, we had time for one optional module so the class voted and we chose the module based on that vote. Likewise at the end of the second day. By the third day, we had a substantial portion of the day un-allocated. As soon as we finished with the required materials, we got the class to choose the next module to cover. As soon as that module was covered, the class chose again from all those still remaining.
It was a bit of a mess and our mechanisms didn't work perfectly, but it got the job done. At the start of our second day, the cards we were using to track modules and questions looked like this:
After we finished the course, we did an in-depth reflection and decided on some changes to be made. One of the most important was actually very simple: make the module cards much larger so that it was possible to put many question cards right on the module card. Here is what this looked like in a recent class:
The left is the module card at the start of the course with questions that were generated by the course attendees when asked what they wanted to learn from the course. The right is the same module at the end of the course. You can see that one additional question card was added between the start of the course and the time when the module was actually covered. You can also see that I use big check marks to show to the class that the module is complete.
Another change to the method of doing this agile classroom management came in the further refinement of the module content and sizing to be more appropriate for doing the modules in unpredictable sequence. This included things like checking definitions, improving some of the introductory material (to make sure all the concepts were introduced, not just some of them), finding references to other modules and removing them, and in some cases moving material from one module to another. There are still improvements to be made in this area.
Thirdly, we now are better at explaining to the class how this will all work at the start. We are modeling an agile process in the way we run the course. This means that the attendees will have an opportunity to influence what material is covered and in what depth. It also means that if the class takes a lot of time with questions early on with the basic material, it leaves less time at the end for optional modules. The people in the class can now visibly see these tradeoffs.
Here are two photos from a recent class. The first was taken at the end of the first day (note that I forgot to use the check marks on these cards). The second was taken at the end of the third day. You can clearly see how the modules of the third day were chosen entirely by the participants from the available optional modules.
This mechanism of managing a classroom using an agile approach has had some very interesting results:
1) Every class is substantially different. Before, the differences were always in the details: timing, specific questions, etc. Now, the actual material covered is uncertain. I always make adjustments to the course materials after every course. Now, not only are those differences in play, but also the actual modules covered might be different. The fascinating thing is that now, by the time we get to the last module, most of the people in the class are giving off a palpable sense of satisfaction: their urgent questions have been answered. Sometimes, this is so strong that I do the graduation "ceremonies" right before covering the last module so that people not interested in that last module can leave a little early.
2) I get far fewer comments in my feedback forms that indicate a desire that the course was longer. Before I started using this agile learning environment, I would get a substantial number of comments to the effect that it was really too bad there was not another day. Now I rarely get those comments.
3) The attendees in the class see how an agile approach actually works in a real life situation. They see that allowing the requirements to change over time is actually a good thing. Every single time I do this, there is a pattern that shows up on the third day that relates to the way the class prioritizes the next module. I count up the votes and choose the next module to cover. All the other modules have their scores written on them as well. After a module is covered, instead of using those old scores, the class takes another vote. Every single class, the priority changes between votes. This clearly demonstrates the power of allowing stakeholders to change their minds after seeing real results.
4) Participants in the course are engaged at a deeper level. There are lots of "tricks" for keeping people engaged: discussions, exercises, simulations, questions, presentation eye candy, etc. All of those things have their place and are important. However, by having participants steer the course itself, the level of engagement suddenly becomes much more substantial. The participants are responsible for the results of the class in a way that is quite unusual in most educational environments. This responsibility is granted to them and they take it seriously. I rarely see anyone refuse to participate in this process, and I never see anyone attempt to subvert it.
If you are interested in learning this mechanism of classroom management, I strongly recommend:
1) Learn up on agile. The openagile.org web site is starting to define an agile method that is abstracted away from the software legacy that agile methods currently are shackled with. This site is still in its infancy so treat it gently :-)
2) Come try one of my courses! (Yes, that's a blatant shameless plug!) You will get to experience and learn the agile method in a focused setting. The courses are IT/software focused in some respects, but since they are non-technical courses, and really about agile project management and human interactions, almost anyone can attend (I've had dancers, administrative assistants, educators, artists, community workers, executives, and actors all attend).
3) Get in touch with me to find out more of the details that didn't make it into this blog entry. The best way to do this is (really) through my business web site: berteigconsulting.com where you can find both email and phone contact information.
Well, I finally got around to writing this up. This is part of a series of related articles "tagged" as USEFUL started by Scott who writes at Tyner Blain. His article, which started the game of tag, is called Software Product Success Stories. Scott... I'm sorry this article isn't about software!!!
I would like to now tag a few people who write useful blogs:
Mike Vizdos who writes the Implementing Scrum blog. This blog is best known for the fabulous cartoons of the Chicken and the Pig, but also has a lot of great and practical insight on the challenges of, you guessed it, implementing Scrum!
Tobias Mayer who writes at Agile Thinking. He doesn't write often, but his articles are always fantastic! I hope that Tobias will take the time to write something in this "useful" stream and pass along the tag.
Mark Levison writes Notes From a Tool User and discusses many things, not just agile. His writing is fun and he's in the trenches doing real serious work getting his company to use agile. He's got good insight.
Posted by Mishkin Berteig at 03:46 AM | |
September 24, 2007
Agile Benefits: Five Essential Reasons to Try Agile
Although there many other benefits of agile, and although those provide us with other reasons to try agile, the five essential benefits of agile are:
Rapid Learning - disciplined application of the scientific method to explore the best ways to deliver valuable results.
Early Return on Investment - opportunity to use the results of work starting with the work delivered at the end of the first iteration.
Satisfied Stakeholders - engagement in the process in a way that allows meaningful contributions from all stakeholders.
Increased Control - mechanisms to track/measure and therefore steer the direction that work is going so that it meets goals.
Responsiveness to Change - processes, tools, roles and principles that allow a team and an organization to embrace change rather than reject, control or suffer from change.
These reasons are sufficient and apply to operations work, project work and open-ended research work, whenever humans are involved. The above links take you to more detailed descriptions of each of these benefits.
What are some of the other benefits of agile?
Posted by Mishkin Berteig at 10:47 PM | |
August 22, 2007
Stuck? Try Extreme Obstacle Removal!
What happens when you are iterating away, your team is totally groking agile, delivering great results every couple of weeks, and then unexpectedly, suddenly and firmly everyone is stuck!? An obstacle has come along that forces a full stop. A barrier has been placed in the path. What do you do?
I know of a team that was in exactly this situation. They were building software on BEA's Weblogic platform. They discovered that a defect in the platform made any forward progress impossible (until the defect was remedied). So they canceled their iteration (good), tried to work in an open ended mode without iterations (really really really bad!) and then got absolutely nothing done for four months...
Encountering large obstacles like this puts a team in a nasty danger zone!!!!
Me, I like to consider the extremes:
1) Dump the vendor. This means building the component from scratch or replacing it with a component from another vendor or changing the whole platform. Although this sounds extreme, a self-organizing team should be aware that this is an option that might make the most sense given the circumstances. Keep iterating.
2) Make the vendor part of the team. Insist that they provide a dedicated, on-site person who is expert on the component you are using, insist that they provide you with source code. Both of these are so that you can cross-train in-house team members to solve this problem. Keep iterating.
3) Cancel the entire effort until the vendor (or some other entity) resolves the problem. Get the team to stay together but work on something else in the corporate portfolio that is valuable enough to justify the cost of the team. Let them work in short iterations so that there is little delay from the time the component is fixed until the team finishes an iteration on the other project. Keep iterating.
In my opinion, if you can't do one of the above three options, it is because your project is not valuable enough and no one is willing to admit it. (Actually, there might be other reasons too, but this is the one I would really be worried about.) Don't forget the Art of Obstacle Removal!
That said, there are of course half measures... just make sure you KEEP ITERATING!!! Whatever your current iteration length, find a way to deliver potentially shippable software at the end of every iteration.
Posted by Mishkin Berteig at 12:40 PM | |
July 25, 2007
Time is Not Negotiable
The Project Management Institute refers to three variables that can be negotiated or constrained for a given project: scope, resources and schedule. Schedule is an interesting "variable" in that we have no choice about how time flows. We can control how much scope to ask for, we can control how much money to put towards the work, but we cannot actually "buy" more time than, say, our competitors. This has important implications which deeply challenge the PMI's PMBoK model of project management.
In a related article "Quality is Not Negotiable" I wrote about the strategic important of maintaining a high level of quality, and that allowing defects into your work is like taking on a debt with an unknown interest rate. With time, the problem is a little different.
The idea of sunk costs, also know as "don't throw good money after bad" is difficult for most people at a psychological level. We often feel like making just a little more investment of time, work or cash will turn an ailing effort around. Unfortunately, a traditional waterfall or phased-based approach to project work increases the psychological pressure to continue working, even when the project is "bad".
We know from the recent Chaos report statistics that two thirds of all IT projects are still either failing or challenged. We know that the bigger the project, the more likely it is to fail. We know that in other industries such as construction and economic development projects also have low success rates. How is it that we put all this money and time into these projects without knowing earlier if there are problems?
The problem is, of course, time. We can't predict the future. The farther out we need to see, the more uncertain the view. Not only that, but we can't "go back" to correct mistakes. We can only correct past mistakes by changing our future behavior... by using up more time.
One organization I worked with wanted to do a small web-based project. They had been "thinking" about this project for three years. At various times they had done requirements gathering and analysis with various stakeholders... and then failed to execute on the project. Finally they got around to starting the project. Using an agile approach, the first release of the project was completed in about eight weeks. How does this relate to time?
Well, first off, the project success was measured only in terms of the eight weeks of labor, not the time value of the sunk costs over the past three years. Now of course, the reasoning behind not including sunk costs is exactly because they are non-recoverable. Unfortunately, that doesn't give a very good picture of the investment made in the project. The fact that the costs are non-recoverable is not because of money, scope or quality (the other project variables) but because time is not negotiable: you can't get it back, you can't get more of it, you can't store it up, and you can't transfer it to other people. Sunk costs are really better thought of as sunk time.
A project is a temporary and one-time endeavor undertaken to create a unique product or service, which brings about beneficial change or added value. [Wikipedia]
This helps to explain why large projects have such low success rates. The larger a project, the larger the sunk time. It's hard to re-start projects, and the larger they are the harder they are to re-start. And yet, the proper approach to sunk costs (time) is to ignore what has gone past and treat your work as if you were starting from scratch. A project approach specifically does not allow you to do this. Hence the fiction of "earned value" (which is a whole other topic).
Only an agile or operational approach allows for low sunk costs. By doing work in extremely small pieces, each of which actually delivers value, an agile approach is best suited for the reality that time is not negotiable. Agile methods acknowledge that the best way to avoid sunk time is to not do much work until you deliver results. As well, since we can't go back to change our mistakes, agile methods allow us to check our work frequently and learn as we go.
In what other ways is time a different sort of variable than scope or cost? In what other ways does an agile approach take this into account?
Posted by Mishkin Berteig at 10:05 AM | |
July 19, 2007
Agile is Not Communism
Last week I taught an introductory course on Agile Work. Normally this is pretty easy stuff. However, I was teaching this course in Bucharest, Romania (cool), and I have come across a substantial, strong and vigorous objection to agile (also cool, but challenging too). Several people in my class are asserting that agile is just like communism and since communism failed, agile is not likely to succeed either. I'm looking for help on this!
I have Googled "agile communism" and come up with some interesting reading:
Does Scrum/XP Violate the Agile Manifesto?
The Agile Method and Other Fairy Tales: QED
I have also done some quick research on communism to check that I understand the comparison and objection. Here is a wiktionary definition of communism:
1. A term used to refer to a number of political philosophies or ideologies which share the belief in the virtue of holding the production resources collectively. 2. A society organized in line with the communist theories, the ultimate aim of which is the abolition of the state and the creation of a classless, stateless society whose members produce according to their abilities and take what they need.
So let's start with that definition.
Holding Production Resources Collectively
Also called: common ownership of the means of production
I suppose there are a few ways to look at this. In an agile environment which encourages (for example) collective code ownership, it might seem like holding the production resources collectively. However, the code is actually the result of production, rather than the Means of Production. This distinction is not trivial. The means of production for a team in an agile environment still include both the tools and the raw materials upon which those tools exercise. In software development (and in most creative work nowadays), the tools are computers and software and other electronic gadgets such as video cameras, telecomm systems, etc. The raw materials are typically intellectual constructs such as images, sounds, ideas, processes (and of course their legal counterparts such as trademarks, copyrights, and patents). Agile methods do not require the team of workers to own these means, nor do agile methods forbid workers from owning these means. In fact, There is one important way in which agile methods are decidedly not communist: every individual owns their own creativity, experience, and knowledge and is only asked to share willingly (and usually in exchange for pay such as salary, stock options or outright corporate ownership). I believe this passage clarifies things nicely:
Marxists define economic systems in terms of how the means of production are used, and which social class controls them. Thus, in capitalism, the means of production are controlled by the bourgeoisie, (the "capitalists" - the owners of capital), while in socialism they are controlled by the people's elected representatives and in communism they are controlled collectively by the people themselves. [Means of Production]
Agile methods, if anything, tend towards capitalism in this regard. (Although a whole other question could be asked about just how much control the owners of a corporation really have given delegated authority through the board of directors to the chief executive staff _and_ the abrogated authority through mutual funds, pension funds, holding companies, and trusts _and_ the limitations on that control through the blunt instrument of voting shares _and_ the influences on that control through the control of information by financial analysts and the media...)
Ultimate Aim of The Creation of a Classless, Stateless Society
Well, this certainly isn't the aim of agile methods that I am aware of. The aims of agile methods as I understand them includes:
- Building stuff that stakeholders like
- Creating an environment for team members to exercise their creativity
- Doing all this in a way that responds well to pervasive frequent change
Again, I can understand why there might be some confusion here. Agile methods promote these three aims by doing something that looks just a little like a classless organizational structure. Typically, agile (and lean) environments start to have a higher manager to staff ratio (fewer managers), encourage self-organizing, cross-functional teams, and emphasize team goal setting, commitment and accountability. This might seem classless (and stateless/managementless) until one examines what is not said: Agile does not claim that every team member is exactly equal, it does not require that every team member do exactly the same thing, it does not require that every team member give up _all_ their individual preferences (although certainly it would be hard for someone who didn't like talking to other people to be part of an agile team... so I guess some individual preferences won't work in an agile team), it does not encourage every team member to do exactly the same amount of work regardless of if you are measuring effort or output.
Now admittedly, when I am presenting examples of self-organizing teams, I do sometimes use examples that come close to classless stateless teams... but that's just to show that this is a possibility and allowable in an agile environment, not that it is the only way.
Those practices in agile methods that do seem like classlessness and statelessness are not aims... they are means. A self-organizing, self-managing team, is means to an end. It is not an aim in itself. Why does this matter? For the simple reason that it is always bad to confuse means and ends. The end cannot justify the means (classlessness and statelessness imposed by revolution)... and nor can the means justify the ends (classlessness and statelessness that results in apathy, boredom and low productivity). So the fact that self-organization is a means is a way for us to decide if it is worthwhile. The evidence for self-organizing teams in a business context is strong (lots of links and articles and books on this blog as well as others). Most team members enjoy being self-directed in a way that is collaborative with other professionals. So both the ends are good - productivity - and the means are good - happy people.
Members Produce According to Their Abilities and Take What They Need
To me, "produce according to their abilities" sounds a lot like a tautology. Certainly, team members can produce no more than their abilities! From an agile perspective, this isn't even what we are interested in... we are interested in the multiplicative effect of teams of people working together effectively so that the result of the work is greater than the sum of the individual abilities. There is now substantial evidence that this is not just possible, but a likely outcome of group work (see Research on Group Effectiveness vs Individuals among many others). My favorite phrase for this is "Unity in Diversity" which describes a group of people who have united around a common goal, but with each individual having talents, experiences, knowledge, patterns of behavior, and insights that are all different from each other.
The second part of the phrase "take what they need" is another piece of this pie that has absolutely no relationship to agile methods. Again, there is evidence that this is specifically not supported. Lean methods encourage compensation that is based on a number of things including: contribution to results in one's sphere of influence (rather than the more limited sphere of responsibility) and the number of skills you have mastered and use to contribute to the work.
Disconnecting reward from results is an obvious problem. While people do have altruistic feelings, we also are clearly motivated by praise
Some Similarities
One of the attendees of the course, a woman by the name of Christina, provided me with some notes based on her understanding of Agile and Communism and she listed these similarities:
- They both rely on participation, rather than executing orders.
- One aim is less frustration.
- They use some similar phrases such as multidisciplinary / cross-functional.
- They both ask people to evaluate each other.
And What about Reality?
Well, I haven't lived in a communist environment so I admit that my ability to talk intelligently about this is not what I would like it to be. I am interested in people out there who might be able to help me with this.
Here are some questions for people who have actually experienced communism:
- What are the actual, in-practice similarities between agile and communism?
- What are the in-practice differences between agile and communism?
- Why did communism fail and how might this affect the success of agile methods?
- How can we safeguard agile from falling into the same problems?
Can anyone out there help please?
Posted by Mishkin Berteig at 06:13 PM | |
April 20, 2007
Leadership
"A leader is a person who assists a group of people become a self-organizing, self-managing and self-sustaining team so that he/she no longer has to be a leader for that group." - Just an idea I had. What do you think?
Posted by Mishkin Berteig at 03:35 AM | |
March 05, 2007
A Better Iteration Structure
In my coaching work, I have often been asked a question about the planning process for iterations, that until just a few days ago, I would actually brush off!!! I didn't even realize I was doing this, it is only in retrospect that I see this. This question is simple: "how does a team plan for the improvement efforts that come out of the retrospective when they are supposed to be working at maximum velocity when implementing tasks directly related to the items in the work queue?" Or, more simply, "we don't have time for process improvements."
The source of the problem shows up in the normal* structure of an iteration. This structure is as follows:
Step 1: set a goal by selecting work from the top of the work queue.
Step 2: plan the execution of that work by doing a task breakdown and task-level estimation.
Step 3: do the work according to the tasks (this is the bulk of the time in the iteration).
Step 4: do a demo to review the work - use this demo to adjust the work queue if necessary.
Step 5: do a retrospective to review the "how" and come up with action items to make the work more effective next iteration.
Since iterations follow one after another, this structure when rolled up can be re-written in the following manner:
Action (step 3)
Reflection, Learning (step 4)
Reflection, Learning, Planning (step 5)
Planning (step 1 and 2)
Now if we look at the purpose of the steps in comparison to the type of learning activity taking place, we see that there is a big disconnect between the planning that happens in step 5 (the retrospective) and the planning that happens in steps 1 and 2 at the start of the next iteration. Not only that, but reflection and learning are separated into "what" and "how" so are done twice for each iteration.
Frankly, this causes a lot of confusion: I see it in my coaching when teams try to accommodate the two types of planning, and I see it in my training when I teach the structure of the iteration.
So what is to be done? Simplify the structure of the iteration. Instead of thinking of the iteration as a linear block of time that starts with step 1 and ends with step 5, think of it as a cycle that is dominated by Action, and punctuated regularly by Reflection, Learning and Planning. Now, instead of four separate meetings, there is a single meeting every iteration that combines the wrap-up of the previous iteration with the start of the next. Let's call this the Iteration Transition Meeting or the RLP Meeting or the Adapt Meeting... (suggestions welcome!)
The agenda for this meeting is very simple and contains the essential elements that exist in the "old" style. It starts with Reflection where the facts of the Action are examined: what did we build? how did we build it? how did we feel about what we built? etc. This moves fairly quickly to the Learning component where we try to understand what the facts are telling us: what went well? what needs improvement? what insights can we glean? what new questions are open to us? and of course, what did we learn? Now we are prepared to treat Planning holistically, with no "hidden" activities. We examine our Work Queue, re-arrange it, put new items on it, etc. and do our normal task breakdown and estimation. The difference this time is that both direct business tasks and process improvement tasks are included in the same planning activity. This makes task level (commitment) velocity more truthful!
I will admit that this is not something that I have tried yet. I have seen it done accidentally when people start to ignore the action items from the retrospective in the old iteration structure and instead include them in the task planning. When it happened, I thought it was "messy" because it broke the separation between "what" and "how" oriented activities. Now I think that this might be the best way to do things.
Anyone out there willing to give this a try? If so, I would love to spend some time with you working on it - helping, experimenting, and learning from it.
I should mention a couple important points that didn't fit in the above narrative. First, this insight happened as a result of discussions over the past week with my father, Garry Berteig who is using agile methods and the learning circle in his classroom environment. Second, Garry has been doing what I just described in his classroom environment; the main difference is that Garry provides a great deal of the Planning to his students by way of the progression of assignments/activities.
One other cool thing: I'm in Frankfurt as I write this on my way to Chennai to deliver a three-day Agile Project Management / ScrumMaster Certification course. I haven't done international travel since I went to Beijing last summer, and this is the first time I am going outside of North America for work! Fun!
* Normal here means Scrum and XP for the most part. Lean doesn't have this structure and as for other agile methods, I don't know them well enough to say if this is normal or not.
Posted by Mishkin Berteig at 04:16 AM | |
January 23, 2007
The Agile Zealot's Handbook
This is great! I often call myself an Agile Zealot to my clients. (Usually, they smile... and if they don't I tend to have a short relationship with them!) So here it is, the Agile Zealot's Handbook.
And, since I've got a dead horse lying around waiting to be beaten up some more, I've re-written it (the Agile Zealot's Handbook, not the dead horse) to be non-software oriented. Presenting... the new and improved... non-software oriented... readable by anyone... Agile Zealot's Creed:
VALUE
IF you don't repeatedly deliver results
that produce actual value
for a real customer
into your real world environment
frequently enough to respond to change...
TECHNIQUE
IF you're not paying constant attention to technical excellence
with simple, effective tools, processes and reuse
driven by uncompromising dedication to quality
and the discipline to test everything...
LEARN
IF you're not deliberately going
through stages of planning, acting
reflecting and learning
as frequently as you are delivering results
over and over and over...
TEAM
IF your team is not empowered to self-organise,
does not sit together and engage in face-to-face communication,
does not include your customer
and all the necessary skills to make its own decisions and take immediate action...
THEN YOU HAVE COMPROMISED YOUR AGILITY
(and good luck to you... you'll need it!)
Posted by Mishkin Berteig at 08:41 PM | |
January 22, 2007
Never Spread Your Facilitator Butter too Thin
I like butter, and when I eat a piece of freshly baked bread, a good hunk of butter thickly spread is an essential ingredient. I don't skimp on the butter. Using the butter for one piece of bread and spreading it across two or three slices saves cost, but at what price!? Why, the price is my enjoyment.
So how does this relate to Process Facilitators?
Sometimes, organizations try to "optimize" their costs by spreading Process Facilitators thinly. This is a false economy. Having a Process Facilitator for multiple teams is probably better than having no Process Facilitator, but if the organization really believes in the value of the work the team is doing, and the value of the removal of obstacles, then there should be no problem having one Process Facilitator per team.
I strongly advise that there be only one Process Facilitator per team for the simple reason that the Process Facilitator is a member of the team. The Process Facilitator does not _individually_ commit to the team's goal for an iteration, but the Process Facilitator must feel a great deal of responsibility for that goal given that this person is responsible for tracking, facilitating and in some cases actually removing obstacles to the team's ability to meet its goal. As well, the Process Facilitator protects the team from new impediments.
I think that for rare individuals with relatively established teams, it would be possible to be the Process Facilitator for more than one team effectively. Remember that the ideal team size is 7+/-2. For much smaller teams, (4 or fewer people), you might find that having a single person dedicated to the role of Process Facilitator is too much overhead. But rather than sharing one Process Facilitator across multiple small teams, consider having one individual on each team who has some substantial portion of their time dedicated to those duties.
A Process Facilitator who is not part of the team cannot be properly informed of the needs, difficulties and successes of the team. This only leads to frustrated teams, frustrated Process Facilitators and organizations that improve much more slowly than they could.
Never spread your Process Facilitator too thin.
Posted by Mishkin Berteig at 12:24 PM | |
January 17, 2007
The Inner Ring
Here's a slightly off-topic, but nevertheless excellent read: "The Inner Ring" by C S Lewis. This is a talk given by C S Lewis to what seems to be a group of university students. In it, he describes the notion of the inner ring and the desire to be "in". It is amazing how much our culture in North America and our corporate culture is driven by this desire. I'll leave it to you to decide if this is good or bad.
Posted by Mishkin Berteig at 04:23 PM | |
January 02, 2007
Top Ten Most Popular Entries from 2006
If you are new to Agile Advice, these are not just some of the most popular articles, they are also some of the best! Take a look around; you will find ideas to inspire you, challenge your thinking and maybe that one little thing that will make the difference in applying agile methods in your situation.
1. How Two Hours Can Waste Two Weeks - 25,617 unique views
This little hypothetical story by Dmitri Zimine was very popular on Reddit, and Joel on Software ranted a bit about it.
2. The Case for Context Switching - 2,936 unique views
My rebuttal to Joel's rant. Goes well with Dmitri's article. Emphasizes the idea of building trust in an organization.
3. The Qualities of an Ideal Test - 1,579 unique views
Six qualities that will help make your tests as valuable as can be. Similar to the ACID properties of databases or the INVEST properties of user stories.
4. The Pros and Cons of Short Iterations - 1,555 unique views
A few words that will help you decide how long to make your iteration length. This is an important decision, and most teams and organizations don't know the factors involved.
5. Five Signs of Trouble in an Iteration - 1,489 unique views
A good howto on using burndown charts to discover problems in an iteration.
6. Seventeen Tips for Iteration Planning - 1,427 unique views
A good list of hints and tips that can make the difference between struggling with iteration planning and having it go smoothly and effectively. This is a key part of the Agile Work process, so getting good at it is a high priority for any team new to Agile Work.
7. Change is Natural - "Embrace Change" - 1,397 unique views
A short riff on the universality of change. Also introduces the idea of the "horizon of predictability".
8. The Art of Obstacle Removal - 1,287 unique views
This is a good reference article on types of obstacles and methods for removing them... a critical practice for Process Facilitators.
9. The Seven Core Practices of Agile Work - 1,285 unique views
Agile Work is really quite simple. This is a concise list of the practices that make up this effective methodology.
10. Interview with Alistair Cockburn - Agile and House Renovations - 902 unique views
Applying agile methods to home and garden renovations! Learn a bit about how this luminary of the agile world has taken agile practices out of the software world and into the hardware world.
Posted by Mishkin Berteig at 06:32 PM | |
December 22, 2006
Technical Debt
Last night I was thinking more about the analogy of technical debt. In this analogy, design and quality flaws in a team's work become a "debt" that must eventually be paid back. This analogy is fantastic because it can be taken just a little bit further to understand just how bad defects are...
Debt is sometimes useful. Financial debt can be used for risk reduction, investment, and emergencies. But it can also cause problems. Too much debt becomes hard to manage. If debt maintenance costs more than revenue (minus other expenses), then you're going down!
Technical debt is a little different than financial debt.
Suppose I went into a boardroom full of high-power executives for some large company. (Never mind how I got there.) And I pitched this fabulous idea: I'll give them a bunch of money for them to use for operations, expansion, whatever. All they have to agree to is that I choose the interest rate when I ask for repayment... trust me!
I would be thrown out of that room so fast!
That is what we do when we encourage teams to take on technical debt.
There is no way to know the interest rate on a defect. Part of the cost of a defect is obvious: how much time and materials will it take to repair the immediate problem. (Although even that is often hard to measure.) But there are also lots of non-obvious and probably non-measurable costs. How much effort will it take to get to the root cause of the defect so that it doesn't occur a second time? How much will it affect our "goodwill" and thus reduce further and repeat sales? How much will the existence of one defect hide the existence of other defects (with their own costs)? How much will the defect demoralize the team and increase staff turnover or reduce productivity? How much of an opportunity will the defect create for competitors? How much will the defect increase maintenance and support costs?
In other words, every time someone asks a team to let quality slide, they are asking the team (and the organization) to take on debt with an unknown interest rate. Which is lunacy.
Read more about this:
Voluntary Technical Debt - James Shore talks about a situation where a conscious decision to take on technical debt led to positive results.
Technical Debt: The Threshold of Acceptable Pain - Steve Bate talks about skill and sensitivity to technical debt. Here's a great quote from this article:
What if someone has a very high threshold of pain? I’d expect to see lots of crud (technical debt) in their code.... The high threshold developer doesn’t mind spending weeks on new features that would otherwise require days or hours with clean code. And they don’t mind staying at work all night fixing bugs before a major release or spending countless hours babysitting the production system after it’s been released. It doesn’t seem to bother them to spend significant time away from family and friends or to have limited time or other interests and hobbies. ...they sometimes experience pleasure at being the “hero” who saved the company from the bugs they created.
An Incremental Technique to Pay Off Testing Technical Debt - Johanna Rothman talks about technical debt and describes a simple risk-oriented approach to reducing it.
Posted by Mishkin Berteig at 08:23 AM | |
December 18, 2006
Lean, Agile and Capitalism - Just a Thought
It occurred to me to ask: If the "invisible hand" in the free markets of capitalism is making for efficient markets, efficient work... then why is there some much room for improvement when we start using non-competitive, collaborative techniques such as lean and agile?
And if these collaborative techniques work on a small scale to improve efficiency, does this mean that we could do this across organizations as a "replacement" for capitalism somehow?
In agile methods, we "assume positive intent" on the part of individuals. What if we could do this across organizations? I'm not living in a dream world yet, but I think I have an inkling of what it might look like: Toyota and its collaborative, leaned-out supply chain.
Posted by Mishkin Berteig at 06:39 PM | |
December 07, 2006
Peformance Goals - The Wisdom of Teams
As I continue my enthralled read through "The Wisdom of Teams: Creating the High-Performance Organization
The Performance Goal
This concept and practice is an essential condition for a team to become a high performance team. The Performance Goal is a specific, measurable, challenging goal that is given to and/or adopted by the team. It is a statement or description of a goal that answers "why?" and "what?" questions, but specifically avoids answering "how?". It is not a description of activities, it is a statement of desired results. The team is left with the full authority to answer "how?" and implement it.
This concept is essential for setting the initial boundaries of self-organization. By defining "what" and "why", the team is left free to be creative about the solution. The Performance Goal is also essential to building team accountability (as opposed to individual or externalized accountability). Every action, plan, mistake and success are oriented around the Performance Goal.
From the book:
The hunger for performance is far more important to team success than team-building exercises, special incentives, or team leaders with ideal profiles. In fact, teams often form around such challenges without any help or support from management. Conversely, potential teams without such challenges usually fail to become teams.
I would also like to point out a great blog entry I found that shows some of the other side of dealing with teams and present some cautionary words about the potential pitfalls of working in teams.
In an Agile Work environment, the starting point for a performance goal is simply the delivery of valuable work at the end of their very first iteration. This is often a substantial challenge to a team and an organization. For some teams that have worked for a long time in a "waterfall" or phase-based project environment, it can be almost unthinkable that valuable results could be delivered in one tenth or one twentieth of the "normal" amount of time.
However, simply delivering value at the end of each iteration is probably not going to sustain the development of a high performance team for very long. Rather, the overall objective or goal of the project has to be important and compelling. Much work these days is _not_ important and compelling. In fact, many people become cynical about work because they are stuck doing a high proportion of work that is bureaucratic or due to chaotic circumstances.
As a reminder, the books "Good to Great
I frequently work with organizations where a team has been formed up, told to use agile methods, and then also told how to do their work. Really great examples of this are things like: "we want you to self-organize, but you have to build this huge system using J2EE." The the problem with this is simply that it may in fact be ten times less expensive to build the system with Ruby. However, someone has decided (possibly for defensible reasons) that J2EE is the technology platform that must be used. In this circumstance, someone external to the team has stepped over the boundary of "why" and "what" and also included some "how" in the team's goals. The team is not even allowed to consider the possibility that something might work just as well and be much less expensive. Not only that, but the stakeholders haven't even really stated "why" the system is being built and so the team can't evaluate technology choices. There is no standard around which to self-organize. I admit that I am using a simplistic example here, but the pattern is something that I have seen over and over again.
Posted by Mishkin Berteig at 11:44 PM | |
November 24, 2006
How to Avoid Context Switching
Given the huge interest in the article by Dmitri Zimine about context switching, and despite a couple of good articles about how to determine iteration length, there has been no empirical method described, only reasoning processes. This article describes a simple method to quickly determine iteration length by experimental means.
Step One
Start with an iteration length that feels right. Use the two articles below to try decide a reasonable length. In software development, this should be no longer that four weeks. In larger volunteer communities it might be as long as three months. In a work environment where you have to deliver daily, your iteration length might be two hours.
Step Two
Build your Work Queue, and plan your first iteration. Mark the tasks that come out of your iteration planning meeting so that you know that they are tasks that were planned. As you go through this first iteration, track all the interruptions you get by writing up a task for each interruption. Each interruption task should be marked so that you know it was an interruption. If you are using note cards on a visible task board, I like to have "normal" tasks on white cards and interruption tasks on fluorescent orange cards to help them stand out!
Step Three
At the end of your iteration, determine which interruption tasks could have been deferred. In other words, determine which were truly urgent and needed to be handled in the already short time of your iteration, and which could have been put on the Work Queue, prioritized, and therefore deferred to a later iteration. This will require collaborating with your Queue Master and possibly other stakeholders.
Step Four
Count how many non-deferrable interruption tasks your team had in the iteration. For this experimental method, you can assume that this is going to be your normal number of interruptions. Divide the length of your iteration by the number of non-deferrable interruptions. For example, if you had a ten day iteration, and two non-deferrable interruptions, you would have a result of five days. Also consider what was the maximum reasonable response time for these non-deferrable interruptions. The lower of these two values becomes your experimentally determined iteration length. But you are not quite done!
Step Five
Do it all again to verify your iteration length. Note that because of your shortened iteration length, you hopefully will have far fewer non-deferrable interruptions. After your second (shortened) iteration, you can adjust the iteration length shorter if necessary... but don't adjust longer (for now).
I've worked with enough teams to know that for a substantial portion of them, this method would result in very very short iterations. In the software world, a team is often asked to work on a project and support their previous project. This support work tends to mean dealing with various bugs in deployed software. This is one reason why I have become such a strong advocate that quality is not negotiable.
I worked with one team that did something similar to this method (although not as rigorously) and we decided that we needed to try an iteration length of two days. This was painful for the team, but they badly wanted to build trust with their stakeholders. Their interruptions were causing them to constantly fail on their commitments. By switching to these two day iterations, they were able to defer the bulk of their interruptions and meet the commitment they made as a team in the iteration planning meeting.
Articles about iteration length:
Mike Cohn's article: "Selecting the Right Iteration Length for Your Software Development Process"
Mishkin Berteig's article: "The Pros and Cons of Short Iterations"
Now that you have gotten to the end of the article, I can admit something to you: this article is badly named. It should be named "How to determine how often to context switch so that you can meet your commitments and build trust with your stakeholders."
What this article doesn't help with is the pain of context switching. The teams that I have see that use short iteration lengths all find ways of making context switching less painful. Automation, good tool choices, workspace arrangements, etc. all play a part in this. But there is no secret ingredient to make context switching painless. Sorry!
Posted by Mishkin Berteig at 07:55 AM | |
November 21, 2006
Obstacle Types
The Art of Obstacle Removal is an important thing for the Process Facilitator to get good at. I was working with a client today and realized that there are really three broad categories of obstacles and only one category is easy to see. The other two are much harder for people to identify and deal with.
Category One: Tangible Barriers
This is the easy type to identify. Physical obstacles such as distance or walls are the most obvious. And usually people have no trouble identifying other tangible barriers such as bureaucracy or bad habits.
Category Two: Obstacles of Absence
These types of obstacles are harder to identify because they often take creative thinking to see a possibility of improvement by adding something to the environment. Some general sub-categories include a lack of trust, a lack of automation and a lack of knowledge.
Category Three: Chaos
These is often the hardest type of obstacle to identify. Conflict between team members is hard to speak honestly about. Too much context switching is also hard to acknowledge because it often seems like it's a good thing to switch to something else - there's always some sort of reason.
Are there any broad categories that I've missed? Any good examples of these different types of obstacles?
Posted by Mishkin Berteig at 05:29 PM | |
November 17, 2006
Thinking/Doing Loops
Ryan Cooper, who recently attended one of my agile training courses said:
It doesn't make sense to separate the thinking from the doing.
So the real question is: why do we do this so often?
Levels of Separation
It is probably good to note that there are levels of separation. If you're in a big bureaucratic organization you will find hugely separated thinking and doing.
For example, at one large organization that I know well, there is a part of the organization dedicated to determining the best possible process to follow for all the different types of projects that are run: software projects, business projects, marketing projects, etc. The people in this process group sit together and think about all sorts of interesting things like how to meet regulatory compliance constraints, how to measure things really well, how to avoid most mistakes, how to make sure that there is a careful balance of responsibilities, how to make sure that everyone who cares about a project gets a say, and how to make sure that everyone who cares about a project can throw in their version of the kitchen sink. These people get around to interviewing the managers of the people who actually work in these projects... maybe once every six months.
The people "working" in the process group are almost completely separated from the reality of the people doing the work. The feedback cycle is at least twelve months. Let's say that a project starts out using Version N of the process defined by the process group. The project runs for six months. The process people interview the manager of the project and incorporate the feedback in Version N+1 of the process which is released into the wild six months later.
The trouble is that the twelve months for the feedback cycle is actually the least bad it could possibly be. More likely it looks like this: six month project? Too small, the feedback isn't terribly important. Two year, forty person project? That's more like it! Let's get lots of feedback in the post-mortem for this project! Project starts with Version N of the process. Thirty-six months later the project finishes (if lucky!). Post-mortem done. a few months later someone from the process group receives the post-mortem report (200 pages of boring, poorly written text). The process group decides that the $100 million worth of feedback deserves its own project. The process group launches a major process revision project based on the post-mortem report. One year later, the process group releases Version N+1 of the process. Total elapsed time for feedback: fifty+ months! Total waste of time: yes.
Okay, so that's a worst-case scenario.
There are lots of smaller ways that we separate thinking from doing.
Individual
The most direct pairing of thinking and doing happens at the level of the individual person. A person's own thoughts and creative efforts are directing their actions, and they are observing and learning from those actions to change future thoughts. This feedback loop is the easiest and most natural (although sometimes we forget to learn from the results of our actions). Sometimes, this feedback loop is so tight that we get into a state of flow, the "zone" and it seems as if our actions and our thoughts are perfectly at one.
This level is limited in that many thing cannot be done by the efforts of a single human being. So we move up a level.
Team
The team works closely together so that thinking and doing is separated at most by trusted high-bandwidth low-latency communications such as a few people working together at a whiteboard, or two people working at a computer together. All the members of the team have equal participation in the thinking/doing loop.
Again, it is possible for a team to get into the "zone" and become a high-performing or "real" team
This level of separation of thinking/doing does present some challenges: it is more difficult to get into a state of flow as a team, it is easier for the work to be disrupted by the environment, and sometimes the communication breaks down.
Group or Organization
There are some rare organizations which maintain an incredibly productive state above the team level. Toyota is oft-cited, but there are others too such as the "Good to Great
More often than not, however, groups and organizations fall into various degenerative types of thinking/doing separation. In software development, this is typified by the handoffs between people doing requirements to different people doing architecture to different people doing analysis and detailed design to different people doing development to different people doing testing. And that's not even taking into account project managers, managers, marketing, sales and customers! Yikes! No wonder phase-based software development is so painful!
Obligatory Conclusion
Agile methods help tighten up the thinking/doing loop by requiring cross-functional teams with product (what) and process (how) and technique (work) skills and authority actually on the team.
P.S. This is an expansion on the Cycles of the Mind stuff.
Posted by Mishkin Berteig at 02:54 AM | |
November 15, 2006
The Case for Context Switching
Recently, Dimitri Zimine wrote an excellent little story about context switching. Joel Spolsky writes in "From the 'You Call this Agile' Department":
Dmitri is only looking at one side of the cost/benefit equation. He's laid out a very convincing argument why Sarah should not interrupt her carefully planned two week iteration, but he hasn't even mentioned arguments for the other side: the important sale that will be lost.
Okay... I'll bite.
Let's look at that argument from the perspective of the sales person first since that is where Joel's concern starts:
The Sales Guy Perspective
"I need the 'ezhibal' feature now! Big bucks coming soon if we can get it now."
Let's suppose that this urgent email has come in somewhere near the start of our two week iteration. The normal response to this in Scrum is to push back a little. The ScrumMaster says something to the effect of: "Look if you wait 7 days we can put this on the top of the list for our next iteration."
First reaction, and it's a normal one, is for Sales Guy to freak out. I've actually heard people saying things like "You're going to lose your job over this! I'm getting the VP involved and he's not going to like it" and then they stalk off to find the big dog to come and bark at us. Anyway, let's pretend that the Sales Guy is willing to be reasonable and not instantly escalate the "problem". So what he actually says is: "Look, this is super important, it'll probably only take a few minutes for us to talk about it and then we can figure out how long it will take to fix. Let's just do a quick phone call and yadda, yadda, yadda, blah dee blah..."
Enough of the Sales Guy perspective.
Nowadays, if I'm in this situation, I do a value assessement. I tell the team to keep working on their plan, nothing's changed yet, and I sit down with Sales Guy and the person who's sponsoring the current work and we start a discussion about the options of which there are really only two that work in Scrum:
- Stay the Course
- Cancel the Iteration
First, let's talk about how we decide which option to take. Then we'll talk about why.
Deciding on the option is easy. You look at the value of the work currently being done and compare it to the value of the work that Sales Guy needs. You take into account probabilities. If Sales Guy doesn't have a signed contract, then it's hard to day if there's going to be any real revenue from the 'ezhibal' feature. Still, you might be able to do an assessment of the probabilities based on your level of trust and history with the client, etc. You also need to take into account the time value of money. Does delaying the current work have consequences for another client or stakeholder? What is the cost of those consequences.
This is a relatively simple cost/benefit analysis and comparison. If you're not comfortable with money and numbers and spreadsheets, you better get comfortable!
Okay, so we have a way of comparing the two bits of work. Now let's look at the two "allowed" responses and a third "bad" response.
Stay the Course
Turns out that the potential benefit of the stuff Sales Guy wants is not quite as high as the potential benefit of the stuff that Suzie Stakeholder prioritized for the current iteration. Well, that's easy. Put the request from Sales Guy in our prioritized list of work and get to it when there is an appropriate level of benefit relative to the other work.
Cancel the Iteration
The stuff Sales Guy wants is super-valuable. So let's get the whole team to stop what they are doing and everyone supports this very valuble work. Stopping the whole team is appropriate because obvioulsy, the stuff they're working on isn't as valuable. Oh, and because we treat a team as a unit in Scrum. And because the team needs to commit to work, not individuals. This isn't so obvious... more later.
Peel Sarah off to do the 'ezhibal' Feature
This is what normally happens, and in a "normal" non-agile environment, it's probably okay. In a non-agile environment, Sarah hasn't made any commitments (she's been forced to agree to dates and scope, etc., but she hasn't made a commitment that she is empowered to live up to). So if she goes off and does this one little thing, it probably will be just business as usual. In an agile environment, the team has made a commitment and doing this work this way invalidates the team's commitment.
Why do we do it this way? The main reason is around trust and commitment. Yup, it's that soft icky stuff that we're so incredibly bad at that customers think that bugs are normal, that management shoves the kitchen sink into projects in the frustrated hope that they'll get something out of the IT team at the end of the project. Sound familiar to anyone?
Anyway. An iteration is a commitment. It is a firm and solid commitment. The team as a group of smart and honorable people is making a definite commitment to the rest of the organization to get a certain amount of work done in a fixed amount of time. In return, management is agreeing to give the team every support in reaching this commitment. When a team is new at this, they might get it wrong. But having done this with dozens of teams now, I know that after a few tries, the team gets the hang of it and commits to appropriate amounts of work, and management provides appropriate levels of support.
This commitment is essential for developing trust. And anything that comes in the way of the team meeting its commitment is considered "BAD". An obstacle to do away with.
This is interesting, because Joel's second example is about defects. And I strongly agree that defects are "BAD" and need to be dealt with at a very high level of priority. The reason is simple: they prevent a team from meeting its commitment.
One team I was coaching was constantly bombarded by these types of it'll-just-take-a-few-minutes-need-it-asap requests. They had many stakeholders and very very limited resources to service these requests. They had several small projects that were taking literally years to do because they couldn't get enough concentrated time on any one thing. This was considered normal and good in their environment.
The trouble is, no one had really looked at the overall consequences. Everyone was doing local optimization. For us geeks, we all know that local optimization is something to be avoided if possible. We climb a peak only to discover that we have to climb back down a ways to get up to the higher peak we now see is next to this one. We climb up that one only to discover yet another higher peak even further along thus requiring us to climb down and up again... When really what we should have done is stepped back a ways, looked at the lay of the land and said, "hey, that peak over there is the highest of them all, let's go climb that one."
Scrum helps us avoid local optimization by forcing all feature requests for a team to be prioritized in a list of work, and by allowing the team to complete small pieces of work so it actually gets _something_ done that you can learn from.
Joel said:
Agile is not supposed to be about swapping out one set of bureaucratic, rigid procedures for another equally rigid set of procedures that still doesn't take customer's needs into account.
True enough! But it also demonstrates a serious lack of understanding about what is being done in Dmitri's example! First of all, without being agile at all, it is possible to switch from 18 month projects to two week projects. Both can be bureaucratic as you please (well, actually, there's only so much bureaucracy you can cram into two weeks and still get something done). The shorter projects will allow you to be much more responsive to customer needs... by definition!
So what happens when you add in all the other things that agile really is about? Transparency. Truthfulness. Creativity. Learning. Meta-Learning. Prioritization. Self-Organizating Teams. Eliminating Waste.
Well, first of what you get is something that's damn hard to do right. It goes against almost everything we've been taught to do: the extreme of heroics of the extreme of careful planning and process.
Secondly, what you get is something that needs safety zones and meta-rules. Like mutual, freely-given, team-to-stakeholders commitment. Like assuming positive intent.
And thirdly, what you get is an environment where not only is the business getting what it needs more than it used to, but also, the team likes working with the business, and the business likes working with the team.
I admit that the point Joel is making isn't too different. Yes: look at the costs and the benefits. But agile isn't quite about instantaneous responsiveness. That's a red herring and I'm suprised that Joel threw it's stinky carcass into the discussion. Agile is also about balancing that responsiveness with the overall view of value for the team and the organization. The tool for doing that is the prioritized list of work, not the email message from Sales Guy to Sarah.
Posted by Mishkin Berteig at 04:13 AM | |
November 14, 2006
Process Facilitator "Smells"
I have now trained over one hundred people in my Agile Project Managmenet / ScrumMaster Certification course. I'm starting to see and hear some of the results of this training. There are a couple specific "smells" that I have become aware of.
Fortunately, I've been able to provide coaching to some of the organizations that have sent people to my course. There are quite a number of good things that happen, but there are a couple of things that seem to be "natural" misunderstandings.
- Spectating
I put a lot of emphasis on the idea of a self-organizing team in my course. There are a number of exercises, an hour-long section, and many other points during the course when it comes up. With all of this emphasis, it seems that a few people have come away from the course with an extremely hands-off approach to the Process Facilitator role (ScrumMaster/Agile Project Manager). I think this is a natural and probably good reaction to the heavy-handed command & control approach that these people come from. However, there are a few things that should be considered minimum levels of engagement (listed below). - Problem Solving
There is also a great deal of emphasis put in the course on removing obstacles. I have seen several cases where it becomes the habit of the Process Facilitator to start solving every problem. This can happen in day-to-day work, and also in the retrospectives. Again, this seems to be a natural consequence of the desire to get in there and be of value. However, if the Process Facilitator writes down all the "things that need impovement" from the retrospective and then says "Okay! I'll take care of these things." then you know that the Process Facilitator has gone too far.
Appropriate Process Facilitator Engagement
Here are a few ideas on an appropriate level of engagement. Finding the right balance of enagement is not easy and there is no exact formula to follow. Partly it depends on your personality and skills as a Process Facilitator, partly it depends on the capabilities of the team, and partly it depends on the constraints of your work environment. Nevertheless, there are some types of engagement that you can persue with confidence. Here are a few concrete guidelines:
- Team Membership
If you aren't saying "we" when referring to the team, you aren't engaged enough. The Process Facilitator is a member of the team. - Colocation
Most of your time should be spent sitting with the team. If the team isn't colocated, then you should be spending most of your time physically visiting each team member. - Obligations
The Process Facilitator is a catalyst. You must track all the obstacles and "needs improvement" things that the team raises, and then make sure that they get fixed... without going in and fixing everything yourself. Ultimately, your job is to work yourself out of a job. - Assertiveness
The organization that the team is working in will be creating some of the obstacles that the team is facing. Being assertive is critical to helping the organization face and then overcome some of these obstacles.
Posted by Mishkin Berteig at 01:49 PM | |
November 12, 2006
More on Scaling Agile
One problem with having multiple teams working on the same project will be the tendency to compare the teams' performance. Why is this a problem?
Why Not Compare Team Performance?
One of the main reasons is that the teams need to be collaborating not competing. There can be a small amount of friendly competition that might come naturally, but as soon as management starts paying attention to differences in team performance, the competition starts to get serious.
In the case of multiple teams working on the same project, the goal should be to maximize overall performance, not individual team performance. Too much competition can lead to unintentional or deliberate sabotage: failed communication, incomplete communication and downright dishonesty.
Motivating Teams without Comparing Them!
As Mary Poppendieck has written, measure up [pdf]. In a single-team situation this means that individuals are measured and rewarded based on team performance (their sphere of influence). In a multi-team environment, that means that the group of teams should be measured as a group and compensated as a group. This will encourage all teams to work towards the success of the overall project. I personally believe there is some room for individual-based compensation, but the way it is handled needs to be done so that it does not encourage sub-optimal behavior.
As well, each team can keep track of their velocity. Some coaches recommend using "ideal hours" or some other units to determine velocity (velocity = estimated work remaining completed / iteration). The trouble with doing this with multiple teams is that there will be a very real tendency to want to compare each team's velocity. Since velocity is a useful measure for team capacity, it is important to still have a way to measure it. One simple way to do this to prevent comparison is to use different units for each team. Team One might be measuring velocity in Estimated Ping Pong Balls Completed / Iteration... Team Two in Estimated Bananas Completed / Iteration... Team Three in Estimated Bogo-MIPS Completed / Iteration... etc. etc.
Motivating Collaboration
First off, management must make visible commitments to eliminating barriers to collaboration. For example, it is a great sign of commitment to re-organize office spaces so that all the teams are sitting near to each other. Every time the Process Facilitator identifies an obstacle that relates to collaboration (tools, process, physical environment, etc.) management should get right on it and fix it.
An ongoing investment in team-building training, workshops and exercises is also helpful. This type of investment helps people gain the skills necessary to work effectively with other people. Again, individuals need to see and believe that management cares about and values teams.
Finally, one of my pet peeves: when a project is over, keep the team together! Do not break them up and redistribute your "resources" to other efforts. The value of those people working together is substantial. The value of those people working together as a high-performance team is incredible! In a multi-team situation, it is reasonable to take teams from the overall group and re-distribute their efforts... but just don't break up the individual teams.
Miscellaneous Notes on Scaling Agile:
Twelve is still the maximum recommended size for a single agile team. Seven is really the sweet spot. A team larger than twelve people just takes too long to get into the Performing stage of team development. If you feel like your project needs more than twelve people actively involved, then you probably want to split into two or more teams... and then you have "scaled" agile.
If you have three teams of five people (or some similar configuration of people just over the 12 person limit), then they will work as a very close-knit group and a lot of the time will act as if they were a single team. They will probably plan iterations and do demos and retrospectives together.
Twelve teams working on the same project at once is about the maximum number before communication overhead is slowing everyone down too much. This is largely a factor of trust: with 150 or fewer people involved in an effort, it is possible for everyone to know everyone. More than that many people and it is no longer possible. Trust is just not an option anymore and bureaucratic controls take over.
If for some reason you need to do something in a small amount of time and you think it's going to take more than twelve teams of twelve people...? Break the effort into smaller chunks. Divide and conquer. Division can be across functional areas, structural areas or time.
Although I have heard of agile methods being used with groups larger than this, I haven't heard any success stories :-)
Check out my earlier introductory article on Scaling Agile in a large project situation.
Dean Leffingwell has an article about practices needed for scaled-up efforts at the Agile Journal. I glanced through it, but I admit that after I disagreed with his very first point (Intentional Architecture), I started to pay less attention.
He claims that refactoring of large systems is not possible (or at least infeasible). The odd thing is, most large projects that I have been involved with are being done exactly because an old system is not refactorable. A large telecom system, a large insurance system, a large data warehouse and a large GIS system are all being done with scaled up agile methods exactly because the old systems that are currently in place have become ossified to the point where they must be replaced.
These old systems were originally built with phase-based development approaches. At some point, people stopped refactoring because they were not given the space to do so. This drop in code quality turned into technical debt. The technical debt accumulated to the point where it was unbearable (maintenance costs, cost of change, etc.).
The problem with intentional architecture is that it goes back to the old assumption that you can do a good design for a system without the constant feedback from review, deployment and use done on a very short cycle time. Over and over, we are faced with the painful consequences of this attitude, and that is one of the key reasons we started to work with agile methods in the first place.
Martin Fowler makes a good case that scaling agile is the last thing you should do. I don't disagree! Scale your agile teams at your own risk!
It's nice for me to be able to say that I've worked on some agile projects over $10,000,000 in size, but the fact is that the cost could have been reduced substantially if the team size was lowered and the deadline extended. It is (relatively) simple to do a cost/benefit analysis of cross-team-coordination-overhead vs. the time value of early delivery of more functionality... although I've never seen anyone do it! If you know of an example of an organization doing this in a realistic way, I'd love to hear about it!
Are there other ways of supporting cross-team collaboration that you have seen?
Posted by Mishkin Berteig at 09:13 AM | |
November 07, 2006
Scaling Agile Projects
More and more, organizations are applying agile methods to large projects or efforts that require more than a single team. There are three dimensions or concerns of coordination. It is critical that all three be addressed, but there are many ways of addressing them. Here I will simply list these three types of coordination and make some simple suggestions of how to implement them.
I have now had the opportunity to work with several clients where they are applying agile methods to projects with budgets that are greater than $10,000,000. All of them are using multiple teams to work on the same overall project/program. Out of this experience (along with some good reading along the way), I have come to understand that the following three types of coordination are the essential ones:
Value
In order to maintain the "early and frequent" delivery of value that agile methods propose, it is important that the work of the effort be coordinated to maximize early delivery of value. From this perspective, there are often many cooks in the kitchen. I have seen a "Product Owner Team", a "Customer Team", and a number of variations of this type. In order to do the coordination work effectively, it is still necessary to make sure of two things:
- Maintain a single Work Queue that prioritizes the work and from which all the teams select items.
- Have a single person in the "buck stops here" role who can make final binding decisions about work priority and content.
These two items have some implications for the organization.
First, the teams must be organized to be generalist: each team should be able to handle any item on the Work Queue. Not every team is going to be equal in abilities and this can be accomodated in a number of ways. My favorite so far comes from an excellent agile coach Dave West
The second implication is that the customer or product team (Queue Master) must have the availability to support multiple teams in a timely fashion. Ideally, there are individuals on each team who can make judgement calls about features, functionality, constraints etc. on the work and provide quick answers to questions. This is not always easy since the people doing this often have a special area of expertise and it is difficult for them to work outside this area. Just as team members are asked to become generalizing specialists, so must the people who are responsible for determining value in a project.
Process
An agile process endeavors to provide a minimally structured way to do three things: deliver value early, then learn about what is high in value and deliver that more, and finally, learn how to deliver value more effectively.
That third activity, learning how to deliver value more effectively, is facilitated by the Process Facilitator. The Process Facilitator keeps a visible list of obstacles and works collaboratively with the Team and the Stakeholders to resolve obstacles on the list.
In a multi-team environment, there may be a single Process Facilitator working with each team. Like with the Work Queue, it is often necessary to have a single Record of Obstacles for the entire project.
Technique
People develop skill and knowledge in the use of their tools. Most types of work have a special vocabularly that only makes sense to other people also doing that work. For example, the field of computer programming has programming languages, integrated development environments, build tools, testing tools, algorithms, and a host of other techniques. The field of film-making has cameras, film, directorial techniques, lighting, story structure, it's own esoteric vocabulary, and other techniques. Likewise for construction, law, medicine, drama, education, etc. etc. etc.
In a large Agile Work project, teams need a way to coordinate their technique to produce integrated, consistent and compatible results. As well, individuals on the teams may discover or create new ways of doing things that would be valuable for the other teams to know about and use.
The most effective way of coordinating technique across teams is for strong members of each team to gather regularly to review the way work is being done. This "technical support group" can look at tools, reuse, automation, patterns, vocabularly and any other "how to" aspects of the work. It is critical that these people are actively involved in work being done on a delivery team so that efforts of the technical support group do not become academic or "ivory tower".
In certain environments, it may be useful to have this techincal support group become a team with a clear allocation of time apart from the regular delivery teams. In this case, this technical support team would have its own Work Queue that consisted of requests, ideas, concerns, and opportunities presented by the regular delivery teams.
I have seen all three aspects of coordination implemented in large multi-team projects. Some of the common challenges include:
- Generalist Teams.
It is difficult enough to create cross-functional teams where people are willing to become generalizing specialists. While it is important to create generalist teams, most organizations should expect to set up non-ideal specialist teams (sometimes by line-of-business) and support their development into generalist teams. - Technical Coordination.
Often organizations have a design or technical review group composed of the "experts". These people are often isolated from the actual work being performed by the teams. It is difficult, yet critical, that these people actually be involved in day-to-day work on the teams.
Posted by Mishkin Berteig at 09:15 AM | |
September 05, 2006
The Seven Core Practices of Agile Work
Agile Work consists of seven core practices. These practices form a solid starting point for any person, team or community that wishes to follow the Middle Way to Excellence.
Self-Organizing Team
Any group of people that wish to be an Agile Team need to take the initiative to determine for themselves how they are going to work (process) and how they are going to do the work (product). The term "team" really applies quite broadly to any size group of people that are working together towards a common goal.
Teams go through stages of development as they perform their work. The most important result of team development is the team itself, and not the specific skills and abilities that the individuals learn.
If the team is part of a broader organization, that organization must give the team the authority, space and safety to learn to be self-organizing. The organization's leadership is responsible for determining the "why?", some constraints on "how?", and then letting the team respond to the need as best as it can.
Also Known As: Whole Team (Extreme Programming), Cross-Functional Team (business management).
Deliver Frequently
Agile Work uses short fixed periods of time to frame the process of delivering something of value. Each of these iterations or timeboxes is structured so that the team or group actually finishes a piece of work and delivers it to stakeholders. Then, the team builds on what has previously been delivered to do it again in the same short amount of time.
The sooner that valuable results can be delivered, the more value can be obtained from those results. This extra value is derived from opportunities such as earlier sales, competitive advantage, early feedback, and risk reduction.
There is an explicit tradeoff: the shorter the time to delivery, the smaller the piece of value will be. But, like investing in one's retirement account, the earlier you start, even with small amounts of money, the better off you are in the long run.
Also Known As: Sprint (Scrum), Iteration (Extreme Programming), Timeboxing (generic), Time Value of Money (accounting).
Plan to Learn
Every type of work is governed by a Horizon of Predictability. Any plan that extends beyond this horizon of predictability is bound to fail. Agile work uses an explicit learning cycle tied in with the planning of work to accomodate this inevitable change.
First, a goal is required. This goal can be long-term. Teams using Agile Work then create a queue of work items to be done in order to reach this goal. Each iteration, some of these items are selected, finished and then the queue is adjusted. The changes in the work queue are based on external factors, and learning that the team does as it goes.
One of the most effective methods for the team to learn about how it is doing its work is the retrospective. After each delivery of results, the team holds a retrospective to examine how it can improve.
Also Known As: Inspect and Adapt (Scrum), Kaizen (Lean), Adaptive Planning (generic).
Communicate Powerfully
A team needs to have effective means of communicating, both amongst team members and also to stakeholders. To Communicate Powerfully, a team needs to prefer in-person communication over distributed communication. Synchronous over asynchronous communication. High-bandwidth over low-bandwidth communication. Multi-mode communication over single-mode communication.
The results of failing to communicate powerfully include wasted time for waiting, misunderstandings leading to defects or re-work, slower development of trust, slower team-building, and ultimately a failure to align perceptions of reality.
The single most effective means to communicate powerfully, is to put all the team in a room together where they can do their work, every day for the majority of the work time.
Some types of work do not lend themselves to this approach (e.g. creating a documentary video), but every effort should be made to improve communication.
Also Known As: Visibility (Scrum), Whole Team and Team Room (Extreme Programming), War Room (business management).
Test Everything
Defects are one of the most critical types of waste to eliminate from a work process. By testing everything, by driving all the work of a team by creating test cases to check the work, a team can reach extremely high quality levels. This ability to prevent defects is so important that only an executive level decision should be considered sufficient to allow defects into a work process. Quality is not negotiable.
In Agile Work, removing a defect is the only type of work that takes priority over any new features/functionality/production. If the end result desired is to maximize value, then removing defects is an important means to that end.
A team has an ethical duty to discover new ways to effectively test their work. This can be through the use of tools, various feedback mechanisms, automation, and good old problem-solving abilities.
Also Known As: Canary in the Coal Mine (Scrum), Test-Driven Development (Extreme Programming), Defects per Opportunities (Six-Sigma).
Measure Value
Since Reality is Perceived, it is important for an agile team and organization to have a clear method of describing and perceiving what is important for the organization. Measuring value is a critical method for describing and perceiving what is important.
A single metric can be used to drive all the measurement and goal-setting and rewards in an organization. All other measurements are secondary and must be treated as such: limited in use and temporary.
There are many things which are easier to measure than value. It is often easy to measure cost, or hours worked, or defects found, or estimate vs. actual... etc. However, all of these other measurements either implicitly or explicitly drive sub-optimal behavior.
Also Known As: Measuring Results (Scrum), ROI (business management), Economic Driver (Good to Great), Running Tested Features (Extreme Programming).
Clear the Path
Everyone in an organization using Agile Work takes responsibility for clearing the path, removing the obstacles that prevent work from being done effectively. Clearing the Path doesn't just mean expedient, quick fixes to problems, but rather taking the time to look at an obstacle and do the best possible to remove it permanently so that it never blocks the path again.
In the Agile Work method, the Process Facilitator is the person who is responsible for tracking obstacles and ensuring that the path is cleared. To do this, the Process Facilitator maintains a Record of Obstacles.
Clearing the Path is sometimes painful work that exposes things we would rather not deal with. As a result, it is critical that people build their capacity for truthfulness and work to develop trust amongst each other. Building a capacity for truthfulness is not something that can be done by using an explicit process.
Also Known As: Removing Obstacles (Scrum), Stopping the Line (Lean).
Remember also, that these practices must always be viewed and implemented in the context of the Agile Axioms. These axioms provide a check to ensure that the practices are not being applied blindly, but rather applied appropriately to the given situation.
Posted by Mishkin Berteig at 09:09 AM | |
August 23, 2006
United Action
A few nights ago, I was having a conversation with my father, Garry Berteig, who is also an agile coach. He had an interesting insight based on a recent experience in Beijing: unity is a precondition to assistance.
In Beijing, he was helping my brother, Alexei, get ready for his upcoming wedding to a wonderful Chinese woman, Ma Jin. The three of them were out shopping. Alexei needed new furniture. As they shopped, Alexei and Ma Jin discussed the various items, their prices, qualities, etc. It took quite a while, but eventually they agreed upon a set of furniture. They became united in their understanding and perception of reality, and agreed without hesitation.
At this point, my father was able to offer his assistance to purchase the furniture as a gift for their wedding. Once Alexei and Ma Jin were united, it was possible to not just offer the assistance, but actually execute on it. If they had not come to a unity of vision, it would have been a mistake to execute on the assistance.
I believe this applies in teams and organizations. In agile environments, managers become "servant leaders". This service is only effective once the self-organizing team is united in its understanding of its needs. If the team is not united, then the service of the manager can lead to problems: some members of the team may feel discounted, some may feel (falsely) that management supports their side, and finally, management may not be able to offer whole-hearted support in the futu