November 02, 2007
Dynamic Strategic Planning
I ran across this idea of Dynamic Strategic Planning [pdf] a couple nights ago at a meeting where a friend and I were talking about agile applied to non-technology work. He told me about Richard de Neufville who had come up with this concept of making strategic plans that were adaptable. It involves the use of Real Options, negotiation and other aspects of planning. I don't know anything more than what is written in the paper I have linked to... do any of you have experience with Dynamic Strategic Planning that you could tell us about? Are there any good books about this that you would recommend?
Posted by Mishkin Berteig at 06:26 AM | |
October 23, 2007
Truthfulness in Agile
I believe in Agile. To the best of my knowledge and based on my experience, Agile methods are the best way for teams, organizations and communities to get stuff done. Doesn't mean it's easy though! The biggest challenge of Agile is in the need for truthfulness.
Most of us live and work in environments where the truth is not totally welcome. Maybe it's "little white lies". Maybe it's hiding behind the raw facts. Maybe it's withholding information. Maybe it's exaggeration. Maybe it's "rose colored glasses". Maybe it's self-preservation. Maybe it's getting attention. Maybe it's lack of confidence or over-confidence. Truthfulness is compromised on a regular basis in most work environments and in most interpersonal relationships.
Agile methods have several powerful benefits. To get those benefits, agile needs to be done right. To do agile right, requires truthfulness.
Why does agile require truthfulness? Because agile methods have fewer checks and balances, fewer gating points, far fewer bureaucratic procedures, than other methods. This lack of safety mechanisms means that it is possible to subvert a team using an agile method more easily than it is to subvert a high-ceremony, high-rigor, high-paperwork type method.
That said, there are
- some
Here is a worst case scenario.
Someone on the team is frustrated or angry about something. This person, let's call him Joe, decides he's going to lash out. He starts by spreading some not-very-nice-nor-accurate gossip that gets the other team members to not trust each other as much. He then makes up some sort of technical risk factor and starts to inflate his estimates and convinces a couple other people to do so as well... just in case. The team's velocity appears stable, but their actual output has decreased. The team is not yet mature enough in their interactions so these things are not mentioned in the retrospective. Joe gradually finds ways to demoralize everyone on the team. The team starts having more and more problems with code quality which slows it down. The process facilitator, Mary, notices these things. However, Joe was planning ahead. Joe has been talking with Mary to help her see how everyone else on the team is subverting the process, thus deflecting attention away from himself. Mary, also relatively new to her role, isn't careful about checking facts and steps on some toes. Now she is almost powerless to influence the team to get it back on track... and the final blow comes as Joe starts raising obstacles in the daily status meeting that are "made up" and designed to be difficult if not impossible for Mary to help with thus solidifying her impotence in the minds of the rest of the team. Ug.
Is there any way out of this? Not really. Intervention is possible by an experienced coach, but not guaranteed to succeed. The best thing to do is probably to disband the team get an expert retrospective facilitator in to try and extract some of lessons, and then try again with a new team.
What else could be done? Is there a specific intervention that could be performed given that we don't know that Joe is behind it all? Could this really happen in an agile environment?
Posted by Mishkin Berteig at 09:05 AM | |
October 12, 2007
Agile Classroom Management
I'm fascinated by the idea of applying agile methods outside of software... be it to business management, family and household, or, as I and my father have been exploring over the last two and a half years... agile classroom management. Here's how I do it in my Agile Project Management / ScrumMaster Certification courses:
My father, Garry Berteig, had been using agile methods in his classroom for over two years before I tried it myself. I had gone into one of his courses at Keyano College in Fort McMurray to give a presentation to his second-year media students on agile and Scrum. They used it for a video project (see "A Student Documentary Film Project" and "Scrum Saves the Day for Media Student". Out of that experience, and others, he developed the idea of the Learning Circle as the basic cycle in an agile learning environment.
In the meantime, I became a Certified Scrum Trainer. Back when I did it, this just meant co-teaching a Scrum class with Ken Schwaber. He had a standard slide deck which he presented as he masterfully guided a class through two days of Scrum training. I (as expected) took that deck and made it my own by reformatting, rewording, changing a few of the exercises, and changing the emphasis in a couple of spots. And then I used that deck with minor corrections and updates and some expansion over the course of the next 16 months.
In the meantime, I was also doing a fair amount of training for teams and organizations that didn't care particularly for Scrum, but just wanted to know more about this "agile" thing. Sometimes it would be a one-day course, sometimes as much as five days (that was exhausting!!!). I developed a substantial body of modules including exercises, case studies, and special topics. Eventually, I found that I was having a hard time fitting my fixed agenda materials into the time I had allocated. For example, I was commonly doing a three-day course and would find that by the end of the first day I was a little behind on my agenda and by the middle of the third day I would certainly be running out of time and miss possibly even two or three modules in the agenda.
Finally, last Sprint, I got a big client who asked for training for a large number of their staff. This gave me the opportunity to: a) work with the same materials and agenda at least three times, and b) involve an assistant in all three deliveries. The first time through, I had the typical problem where the third day was a terrible rush. The second time through, it happened again, but I had de-scoped and re-arranged some of the material so it didn't seem as bad. After that second time, my assistant, an excellent coach, David Chilcott, suggested that I try to apply a more flexible... agile... approach to handling the materials. We did a quick revision of the materials to support this idea and then the third class was run as an "agile training environment". What did this mean?
The main insight was that the time allocated to the class was timeboxed. This meant finding a way to prioritize the work of the class so that the majority of the participants would get the best value possible for their time in the class. We did this in two ways:
1) As questions came up, we found that some of them needed to be deferred because they would be answered in a later module or because there was background material that needed to be covered first. So, for every question that needed to be deferred, we wrote it up on a 3x5 note card and put it up on the wall. Now, in order to keep track of this, we also had all the course modules written up on 3x5 note cards and up on the wall too. Most of the question cards would immediately be put up with the associated module card.
2) We drastically cut back the core required modules for the three days and made everything else optional. We then decided on a simple in-class mechanism to choose from those optional modules. At the end of the first day, we had time for one optional module so the class voted and we chose the module based on that vote. Likewise at the end of the second day. By the third day, we had a substantial portion of the day un-allocated. As soon as we finished with the required materials, we got the class to choose the next module to cover. As soon as that module was covered, the class chose again from all those still remaining.
It was a bit of a mess and our mechanisms didn't work perfectly, but it got the job done. At the start of our second day, the cards we were using to track modules and questions looked like this:
After we finished the course, we did an in-depth reflection and decided on some changes to be made. One of the most important was actually very simple: make the module cards much larger so that it was possible to put many question cards right on the module card. Here is what this looked like in a recent class:
The left is the module card at the start of the course with questions that were generated by the course attendees when asked what they wanted to learn from the course. The right is the same module at the end of the course. You can see that one additional question card was added between the start of the course and the time when the module was actually covered. You can also see that I use big check marks to show to the class that the module is complete.
Another change to the method of doing this agile classroom management came in the further refinement of the module content and sizing to be more appropriate for doing the modules in unpredictable sequence. This included things like checking definitions, improving some of the introductory material (to make sure all the concepts were introduced, not just some of them), finding references to other modules and removing them, and in some cases moving material from one module to another. There are still improvements to be made in this area.
Thirdly, we now are better at explaining to the class how this will all work at the start. We are modeling an agile process in the way we run the course. This means that the attendees will have an opportunity to influence what material is covered and in what depth. It also means that if the class takes a lot of time with questions early on with the basic material, it leaves less time at the end for optional modules. The people in the class can now visibly see these tradeoffs.
Here are two photos from a recent class. The first was taken at the end of the first day (note that I forgot to use the check marks on these cards). The second was taken at the end of the third day. You can clearly see how the modules of the third day were chosen entirely by the participants from the available optional modules.
This mechanism of managing a classroom using an agile approach has had some very interesting results:
1) Every class is substantially different. Before, the differences were always in the details: timing, specific questions, etc. Now, the actual material covered is uncertain. I always make adjustments to the course materials after every course. Now, not only are those differences in play, but also the actual modules covered might be different. The fascinating thing is that now, by the time we get to the last module, most of the people in the class are giving off a palpable sense of satisfaction: their urgent questions have been answered. Sometimes, this is so strong that I do the graduation "ceremonies" right before covering the last module so that people not interested in that last module can leave a little early.
2) I get far fewer comments in my feedback forms that indicate a desire that the course was longer. Before I started using this agile learning environment, I would get a substantial number of comments to the effect that it was really too bad there was not another day. Now I rarely get those comments.
3) The attendees in the class see how an agile approach actually works in a real life situation. They see that allowing the requirements to change over time is actually a good thing. Every single time I do this, there is a pattern that shows up on the third day that relates to the way the class prioritizes the next module. I count up the votes and choose the next module to cover. All the other modules have their scores written on them as well. After a module is covered, instead of using those old scores, the class takes another vote. Every single class, the priority changes between votes. This clearly demonstrates the power of allowing stakeholders to change their minds after seeing real results.
4) Participants in the course are engaged at a deeper level. There are lots of "tricks" for keeping people engaged: discussions, exercises, simulations, questions, presentation eye candy, etc. All of those things have their place and are important. However, by having participants steer the course itself, the level of engagement suddenly becomes much more substantial. The participants are responsible for the results of the class in a way that is quite unusual in most educational environments. This responsibility is granted to them and they take it seriously. I rarely see anyone refuse to participate in this process, and I never see anyone attempt to subvert it.
If you are interested in learning this mechanism of classroom management, I strongly recommend:
1) Learn up on agile. The openagile.org web site is starting to define an agile method that is abstracted away from the software legacy that agile methods currently are shackled with. This site is still in its infancy so treat it gently :-)
2) Come try one of my courses! (Yes, that's a blatant shameless plug!) You will get to experience and learn the agile method in a focused setting. The courses are IT/software focused in some respects, but since they are non-technical courses, and really about agile project management and human interactions, almost anyone can attend (I've had dancers, administrative assistants, educators, artists, community workers, executives, and actors all attend).
3) Get in touch with me to find out more of the details that didn't make it into this blog entry. The best way to do this is (really) through my business web site: berteigconsulting.com where you can find both email and phone contact information.
Well, I finally got around to writing this up. This is part of a series of related articles "tagged" as USEFUL started by Scott who writes at Tyner Blain. His article, which started the game of tag, is called Software Product Success Stories. Scott... I'm sorry this article isn't about software!!!
I would like to now tag a few people who write useful blogs:
Mike Vizdos who writes the Implementing Scrum blog. This blog is best known for the fabulous cartoons of the Chicken and the Pig, but also has a lot of great and practical insight on the challenges of, you guessed it, implementing Scrum!
Tobias Mayer who writes at Agile Thinking. He doesn't write often, but his articles are always fantastic! I hope that Tobias will take the time to write something in this "useful" stream and pass along the tag.
Mark Levison writes Notes From a Tool User and discusses many things, not just agile. His writing is fun and he's in the trenches doing real serious work getting his company to use agile. He's got good insight.
Posted by Mishkin Berteig at 03:46 AM | |
September 24, 2007
Agile Benefits: Five Essential Reasons to Try Agile
Although there many other benefits of agile, and although those provide us with other reasons to try agile, the five essential benefits of agile are:
Rapid Learning - disciplined application of the scientific method to explore the best ways to deliver valuable results.
Early Return on Investment - opportunity to use the results of work starting with the work delivered at the end of the first iteration.
Satisfied Stakeholders - engagement in the process in a way that allows meaningful contributions from all stakeholders.
Increased Control - mechanisms to track/measure and therefore steer the direction that work is going so that it meets goals.
Responsiveness to Change - processes, tools, roles and principles that allow a team and an organization to embrace change rather than reject, control or suffer from change.
These reasons are sufficient and apply to operations work, project work and open-ended research work, whenever humans are involved. The above links take you to more detailed descriptions of each of these benefits.
What are some of the other benefits of agile?
Posted by Mishkin Berteig at 10:47 PM | |
August 22, 2007
Stuck? Try Extreme Obstacle Removal!
What happens when you are iterating away, your team is totally groking agile, delivering great results every couple of weeks, and then unexpectedly, suddenly and firmly everyone is stuck!? An obstacle has come along that forces a full stop. A barrier has been placed in the path. What do you do?
I know of a team that was in exactly this situation. They were building software on BEA's Weblogic platform. They discovered that a defect in the platform made any forward progress impossible (until the defect was remedied). So they canceled their iteration (good), tried to work in an open ended mode without iterations (really really really bad!) and then got absolutely nothing done for four months...
Encountering large obstacles like this puts a team in a nasty danger zone!!!!
Me, I like to consider the extremes:
1) Dump the vendor. This means building the component from scratch or replacing it with a component from another vendor or changing the whole platform. Although this sounds extreme, a self-organizing team should be aware that this is an option that might make the most sense given the circumstances. Keep iterating.
2) Make the vendor part of the team. Insist that they provide a dedicated, on-site person who is expert on the component you are using, insist that they provide you with source code. Both of these are so that you can cross-train in-house team members to solve this problem. Keep iterating.
3) Cancel the entire effort until the vendor (or some other entity) resolves the problem. Get the team to stay together but work on something else in the corporate portfolio that is valuable enough to justify the cost of the team. Let them work in short iterations so that there is little delay from the time the component is fixed until the team finishes an iteration on the other project. Keep iterating.
In my opinion, if you can't do one of the above three options, it is because your project is not valuable enough and no one is willing to admit it. (Actually, there might be other reasons too, but this is the one I would really be worried about.) Don't forget the Art of Obstacle Removal!
That said, there are of course half measures... just make sure you KEEP ITERATING!!! Whatever your current iteration length, find a way to deliver potentially shippable software at the end of every iteration.
Posted by Mishkin Berteig at 12:40 PM | |
July 25, 2007
Time is Not Negotiable
The Project Management Institute refers to three variables that can be negotiated or constrained for a given project: scope, resources and schedule. Schedule is an interesting "variable" in that we have no choice about how time flows. We can control how much scope to ask for, we can control how much money to put towards the work, but we cannot actually "buy" more time than, say, our competitors. This has important implications which deeply challenge the PMI's PMBoK model of project management.
In a related article "Quality is Not Negotiable" I wrote about the strategic important of maintaining a high level of quality, and that allowing defects into your work is like taking on a debt with an unknown interest rate. With time, the problem is a little different.
The idea of sunk costs, also know as "don't throw good money after bad" is difficult for most people at a psychological level. We often feel like making just a little more investment of time, work or cash will turn an ailing effort around. Unfortunately, a traditional waterfall or phased-based approach to project work increases the psychological pressure to continue working, even when the project is "bad".
We know from the recent Chaos report statistics that two thirds of all IT projects are still either failing or challenged. We know that the bigger the project, the more likely it is to fail. We know that in other industries such as construction and economic development projects also have low success rates. How is it that we put all this money and time into these projects without knowing earlier if there are problems?
The problem is, of course, time. We can't predict the future. The farther out we need to see, the more uncertain the view. Not only that, but we can't "go back" to correct mistakes. We can only correct past mistakes by changing our future behavior... by using up more time.
One organization I worked with wanted to do a small web-based project. They had been "thinking" about this project for three years. At various times they had done requirements gathering and analysis with various stakeholders... and then failed to execute on the project. Finally they got around to starting the project. Using an agile approach, the first release of the project was completed in about eight weeks. How does this relate to time?
Well, first off, the project success was measured only in terms of the eight weeks of labor, not the time value of the sunk costs over the past three years. Now of course, the reasoning behind not including sunk costs is exactly because they are non-recoverable. Unfortunately, that doesn't give a very good picture of the investment made in the project. The fact that the costs are non-recoverable is not because of money, scope or quality (the other project variables) but because time is not negotiable: you can't get it back, you can't get more of it, you can't store it up, and you can't transfer it to other people. Sunk costs are really better thought of as sunk time.
A project is a temporary and one-time endeavor undertaken to create a unique product or service, which brings about beneficial change or added value. [Wikipedia]
This helps to explain why large projects have such low success rates. The larger a project, the larger the sunk time. It's hard to re-start projects, and the larger they are the harder they are to re-start. And yet, the proper approach to sunk costs (time) is to ignore what has gone past and treat your work as if you were starting from scratch. A project approach specifically does not allow you to do this. Hence the fiction of "earned value" (which is a whole other topic).
Only an agile or operational approach allows for low sunk costs. By doing work in extremely small pieces, each of which actually delivers value, an agile approach is best suited for the reality that time is not negotiable. Agile methods acknowledge that the best way to avoid sunk time is to not do much work until you deliver results. As well, since we can't go back to change our mistakes, agile methods allow us to check our work frequently and learn as we go.
In what other ways is time a different sort of variable than scope or cost? In what other ways does an agile approach take this into account?
Posted by Mishkin Berteig at 10:05 AM | |
July 19, 2007
Agile is Not Communism
Last week I taught an introductory course on Agile Work. Normally this is pretty easy stuff. However, I was teaching this course in Bucharest, Romania (cool), and I have come across a substantial, strong and vigorous objection to agile (also cool, but challenging too). Several people in my class are asserting that agile is just like communism and since communism failed, agile is not likely to succeed either. I'm looking for help on this!
I have Googled "agile communism" and come up with some interesting reading:
Does Scrum/XP Violate the Agile Manifesto?
The Agile Method and Other Fairy Tales: QED
I have also done some quick research on communism to check that I understand the comparison and objection. Here is a wiktionary definition of communism:
1. A term used to refer to a number of political philosophies or ideologies which share the belief in the virtue of holding the production resources collectively. 2. A society organized in line with the communist theories, the ultimate aim of which is the abolition of the state and the creation of a classless, stateless society whose members produce according to their abilities and take what they need.
So let's start with that definition.
Holding Production Resources Collectively
Also called: common ownership of the means of production
I suppose there are a few ways to look at this. In an agile environment which encourages (for example) collective code ownership, it might seem like holding the production resources collectively. However, the code is actually the result of production, rather than the Means of Production. This distinction is not trivial. The means of production for a team in an agile environment still include both the tools and the raw materials upon which those tools exercise. In software development (and in most creative work nowadays), the tools are computers and software and other electronic gadgets such as video cameras, telecomm systems, etc. The raw materials are typically intellectual constructs such as images, sounds, ideas, processes (and of course their legal counterparts such as trademarks, copyrights, and patents). Agile methods do not require the team of workers to own these means, nor do agile methods forbid workers from owning these means. In fact, There is one important way in which agile methods are decidedly not communist: every individual owns their own creativity, experience, and knowledge and is only asked to share willingly (and usually in exchange for pay such as salary, stock options or outright corporate ownership). I believe this passage clarifies things nicely:
Marxists define economic systems in terms of how the means of production are used, and which social class controls them. Thus, in capitalism, the means of production are controlled by the bourgeoisie, (the "capitalists" - the owners of capital), while in socialism they are controlled by the people's elected representatives and in communism they are controlled collectively by the people themselves. [Means of Production]
Agile methods, if anything, tend towards capitalism in this regard. (Although a whole other question could be asked about just how much control the owners of a corporation really have given delegated authority through the board of directors to the chief executive staff _and_ the abrogated authority through mutual funds, pension funds, holding companies, and trusts _and_ the limitations on that control through the blunt instrument of voting shares _and_ the influences on that control through the control of information by financial analysts and the media...)
Ultimate Aim of The Creation of a Classless, Stateless Society
Well, this certainly isn't the aim of agile methods that I am aware of. The aims of agile methods as I understand them includes:
- Building stuff that stakeholders like
- Creating an environment for team members to exercise their creativity
- Doing all this in a way that responds well to pervasive frequent change
Again, I can understand why there might be some confusion here. Agile methods promote these three aims by doing something that looks just a little like a classless organizational structure. Typically, agile (and lean) environments start to have a higher manager to staff ratio (fewer managers), encourage self-organizing, cross-functional teams, and emphasize team goal setting, commitment and accountability. This might seem classless (and stateless/managementless) until one examines what is not said: Agile does not claim that every team member is exactly equal, it does not require that every team member do exactly the same thing, it does not require that every team member give up _all_ their individual preferences (although certainly it would be hard for someone who didn't like talking to other people to be part of an agile team... so I guess some individual preferences won't work in an agile team), it does not encourage every team member to do exactly the same amount of work regardless of if you are measuring effort or output.
Now admittedly, when I am presenting examples of self-organizing teams, I do sometimes use examples that come close to classless stateless teams... but that's just to show that this is a possibility and allowable in an agile environment, not that it is the only way.
Those practices in agile methods that do seem like classlessness and statelessness are not aims... they are means. A self-organizing, self-managing team, is means to an end. It is not an aim in itself. Why does this matter? For the simple reason that it is always bad to confuse means and ends. The end cannot justify the means (classlessness and statelessness imposed by revolution)... and nor can the means justify the ends (classlessness and statelessness that results in apathy, boredom and low productivity). So the fact that self-organization is a means is a way for us to decide if it is worthwhile. The evidence for self-organizing teams in a business context is strong (lots of links and articles and books on this blog as well as others). Most team members enjoy being self-directed in a way that is collaborative with other professionals. So both the ends are good - productivity - and the means are good - happy people.
Members Produce According to Their Abilities and Take What They Need
To me, "produce according to their abilities" sounds a lot like a tautology. Certainly, team members can produce no more than their abilities! From an agile perspective, this isn't even what we are interested in... we are interested in the multiplicative effect of teams of people working together effectively so that the result of the work is greater than the sum of the individual abilities. There is now substantial evidence that this is not just possible, but a likely outcome of group work (see Research on Group Effectiveness vs Individuals among many others). My favorite phrase for this is "Unity in Diversity" which describes a group of people who have united around a common goal, but with each individual having talents, experiences, knowledge, patterns of behavior, and insights that are all different from each other.
The second part of the phrase "take what they need" is another piece of this pie that has absolutely no relationship to agile methods. Again, there is evidence that this is specifically not supported. Lean methods encourage compensation that is based on a number of things including: contribution to results in one's sphere of influence (rather than the more limited sphere of responsibility) and the number of skills you have mastered and use to contribute to the work.
Disconnecting reward from results is an obvious problem. While people do have altruistic feelings, we also are clearly motivated by praise
Some Similarities
One of the attendees of the course, a woman by the name of Christina, provided me with some notes based on her understanding of Agile and Communism and she listed these similarities:
- They both rely on participation, rather than executing orders.
- One aim is less frustration.
- They use some similar phrases such as multidisciplinary / cross-functional.
- They both ask people to evaluate each other.
And What about Reality?
Well, I haven't lived in a communist environment so I admit that my ability to talk intelligently about this is not what I would like it to be. I am interested in people out there who might be able to help me with this.
Here are some questions for people who have actually experienced communism:
- What are the actual, in-practice similarities between agile and communism?
- What are the in-practice differences between agile and communism?
- Why did communism fail and how might this affect the success of agile methods?
- How can we safeguard agile from falling into the same problems?
Can anyone out there help please?
Posted by Mishkin Berteig at 06:13 PM | |
April 20, 2007
Leadership
"A leader is a person who assists a group of people become a self-organizing, self-managing and self-sustaining team so that he/she no longer has to be a leader for that group." - Just an idea I had. What do you think?
Posted by Mishkin Berteig at 03:35 AM | |
March 05, 2007
A Better Iteration Structure
In my coaching work, I have often been asked a question about the planning process for iterations, that until just a few days ago, I would actually brush off!!! I didn't even realize I was doing this, it is only in retrospect that I see this. This question is simple: "how does a team plan for the improvement efforts that come out of the retrospective when they are supposed to be working at maximum velocity when implementing tasks directly related to the items in the work queue?" Or, more simply, "we don't have time for process improvements."
The source of the problem shows up in the normal* structure of an iteration. This structure is as follows:
Step 1: set a goal by selecting work from the top of the work queue.
Step 2: plan the execution of that work by doing a task breakdown and task-level estimation.
Step 3: do the work according to the tasks (this is the bulk of the time in the iteration).
Step 4: do a demo to review the work - use this demo to adjust the work queue if necessary.
Step 5: do a retrospective to review the "how" and come up with action items to make the work more effective next iteration.
Since iterations follow one after another, this structure when rolled up can be re-written in the following manner:
Action (step 3)
Reflection, Learning (step 4)
Reflection, Learning, Planning (step 5)
Planning (step 1 and 2)
Now if we look at the purpose of the steps in comparison to the type of learning activity taking place, we see that there is a big disconnect between the planning that happens in step 5 (the retrospective) and the planning that happens in steps 1 and 2 at the start of the next iteration. Not only that, but reflection and learning are separated into "what" and "how" so are done twice for each iteration.
Frankly, this causes a lot of confusion: I see it in my coaching when teams try to accommodate the two types of planning, and I see it in my training when I teach the structure of the iteration.
So what is to be done? Simplify the structure of the iteration. Instead of thinking of the iteration as a linear block of time that starts with step 1 and ends with step 5, think of it as a cycle that is dominated by Action, and punctuated regularly by Reflection, Learning and Planning. Now, instead of four separate meetings, there is a single meeting every iteration that combines the wrap-up of the previous iteration with the start of the next. Let's call this the Iteration Transition Meeting or the RLP Meeting or the Adapt Meeting... (suggestions welcome!)
The agenda for this meeting is very simple and contains the essential elements that exist in the "old" style. It starts with Reflection where the facts of the Action are examined: what did we build? how did we build it? how did we feel about what we built? etc. This moves fairly quickly to the Learning component where we try to understand what the facts are telling us: what went well? what needs improvement? what insights can we glean? what new questions are open to us? and of course, what did we learn? Now we are prepared to treat Planning holistically, with no "hidden" activities. We examine our Work Queue, re-arrange it, put new items on it, etc. and do our normal task breakdown and estimation. The difference this time is that both direct business tasks and process improvement tasks are included in the same planning activity. This makes task level (commitment) velocity more truthful!
I will admit that this is not something that I have tried yet. I have seen it done accidentally when people start to ignore the action items from the retrospective in the old iteration structure and instead include them in the task planning. When it happened, I thought it was "messy" because it broke the separation between "what" and "how" oriented activities. Now I think that this might be the best way to do things.
Anyone out there willing to give this a try? If so, I would love to spend some time with you working on it - helping, experimenting, and learning from it.
I should mention a couple important points that didn't fit in the above narrative. First, this insight happened as a result of discussions over the past week with my father, Garry Berteig who is using agile methods and the learning circle in his classroom environment. Second, Garry has been doing what I just described in his classroom environment; the main difference is that Garry provides a great deal of the Planning to his students by way of the progression of assignments/activities.
One other cool thing: I'm in Frankfurt as I write this on my way to Chennai to deliver a three-day Agile Project Management / ScrumMaster Certification course. I haven't done international travel since I went to Beijing last summer, and this is the first time I am going outside of North America for work! Fun!
* Normal here means Scrum and XP for the most part. Lean doesn't have this structure and as for other agile methods, I don't know them well enough to say if this is normal or not.
Posted by Mishkin Berteig at 04:16 AM | |
January 23, 2007
The Agile Zealot's Handbook
This is great! I often call myself an Agile Zealot to my clients. (Usually, they smile... and if they don't I tend to have a short relationship with them!) So here it is, the Agile Zealot's Handbook.
And, since I've got a dead horse lying around waiting to be beaten up some more, I've re-written it (the Agile Zealot's Handbook, not the dead horse) to be non-software oriented. Presenting... the new and improved... non-software oriented... readable by anyone... Agile Zealot's Creed:
VALUE
IF you don't repeatedly deliver results
that produce actual value
for a real customer
into your real world environment
frequently enough to respond to change...
TECHNIQUE
IF you're not paying constant attention to technical excellence
with simple, effective tools, processes and reuse
driven by uncompromising dedication to quality
and the discipline to test everything...
LEARN
IF you're not deliberately going
through stages of planning, acting
reflecting and learning
as frequently as you are delivering results
over and over and over...
TEAM
IF your team is not empowered to self-organise,
does not sit together and engage in face-to-face communication,
does not include your customer
and all the necessary skills to make its own decisions and take immediate action...
THEN YOU HAVE COMPROMISED YOUR AGILITY
(and good luck to you... you'll need it!)
Posted by Mishkin Berteig at 08:41 PM | |
January 22, 2007
Never Spread Your Facilitator Butter too Thin
I like butter, and when I eat a piece of freshly baked bread, a good hunk of butter thickly spread is an essential ingredient. I don't skimp on the butter. Using the butter for one piece of bread and spreading it across two or three slices saves cost, but at what price!? Why, the price is my enjoyment.
So how does this relate to Process Facilitators?
Sometimes, organizations try to "optimize" their costs by spreading Process Facilitators thinly. This is a false economy. Having a Process Facilitator for multiple teams is probably better than having no Process Facilitator, but if the organization really believes in the value of the work the team is doing, and the value of the removal of obstacles, then there should be no problem having one Process Facilitator per team.
I strongly advise that there be only one Process Facilitator per team for the simple reason that the Process Facilitator is a member of the team. The Process Facilitator does not _individually_ commit to the team's goal for an iteration, but the Process Facilitator must feel a great deal of responsibility for that goal given that this person is responsible for tracking, facilitating and in some cases actually removing obstacles to the team's ability to meet its goal. As well, the Process Facilitator protects the team from new impediments.
I think that for rare individuals with relatively established teams, it would be possible to be the Process Facilitator for more than one team effectively. Remember that the ideal team size is 7+/-2. For much smaller teams, (4 or fewer people), you might find that having a single person dedicated to the role of Process Facilitator is too much overhead. But rather than sharing one Process Facilitator across multiple small teams, consider having one individual on each team who has some substantial portion of their time dedicated to those duties.
A Process Facilitator who is not part of the team cannot be properly informed of the needs, difficulties and successes of the team. This only leads to frustrated teams, frustrated Process Facilitators and organizations that improve much more slowly than they could.
Never spread your Process Facilitator too thin.
Posted by Mishkin Berteig at 12:24 PM | |
January 17, 2007
The Inner Ring
Here's a slightly off-topic, but nevertheless excellent read: "The Inner Ring" by C S Lewis. This is a talk given by C S Lewis to what seems to be a group of university students. In it, he describes the notion of the inner ring and the desire to be "in". It is amazing how much our culture in North America and our corporate culture is driven by this desire. I'll leave it to you to decide if this is good or bad.
Posted by Mishkin Berteig at 04:23 PM | |
January 02, 2007
Top Ten Most Popular Entries from 2006
If you are new to Agile Advice, these are not just some of the most popular articles, they are also some of the best! Take a look around; you will find ideas to inspire you, challenge your thinking and maybe that one little thing that will make the difference in applying agile methods in your situation.
1. How Two Hours Can Waste Two Weeks - 25,617 unique views
This little hypothetical story by Dmitri Zimine was very popular on Reddit, and Joel on Software ranted a bit about it.
2. The Case for Context Switching - 2,936 unique views
My rebuttal to Joel's rant. Goes well with Dmitri's article. Emphasizes the idea of building trust in an organization.
3. The Qualities of an Ideal Test - 1,579 unique views
Six qualities that will help make your tests as valuable as can be. Similar to the ACID properties of databases or the INVEST properties of user stories.
4. The Pros and Cons of Short Iterations - 1,555 unique views
A few words that will help you decide how long to make your iteration length. This is an important decision, and most teams and organizations don't know the factors involved.
5. Five Signs of Trouble in an Iteration - 1,489 unique views
A good howto on using burndown charts to discover problems in an iteration.
6. Seventeen Tips for Iteration Planning - 1,427 unique views
A good list of hints and tips that can make the difference between struggling with iteration planning and having it go smoothly and effectively. This is a key part of the Agile Work process, so getting good at it is a high priority for any team new to Agile Work.
7. Change is Natural - "Embrace Change" - 1,397 unique views
A short riff on the universality of change. Also introduces the idea of the "horizon of predictability".
8. The Art of Obstacle Removal - 1,287 unique views
This is a good reference article on types of obstacles and methods for removing them... a critical practice for Process Facilitators.
9. The Seven Core Practices of Agile Work - 1,285 unique views
Agile Work is really quite simple. This is a concise list of the practices that make up this effective methodology.
10. Interview with Alistair Cockburn - Agile and House Renovations - 902 unique views
Applying agile methods to home and garden renovations! Learn a bit about how this luminary of the agile world has taken agile practices out of the software world and into the hardware world.
Posted by Mishkin Berteig at 06:32 PM | |
December 22, 2006
Technical Debt
Last night I was thinking more about the analogy of technical debt. In this analogy, design and quality flaws in a team's work become a "debt" that must eventually be paid back. This analogy is fantastic because it can be taken just a little bit further to understand just how bad defects are...
Debt is sometimes useful. Financial debt can be used for risk reduction, investment, and emergencies. But it can also cause problems. Too much debt becomes hard to manage. If debt maintenance costs more than revenue (minus other expenses), then you're going down!
Technical debt is a little different than financial debt.
Suppose I went into a boardroom full of high-power executives for some large company. (Never mind how I got there.) And I pitched this fabulous idea: I'll give them a bunch of money for them to use for operations, expansion, whatever. All they have to agree to is that I choose the interest rate when I ask for repayment... trust me!
I would be thrown out of that room so fast!
That is what we do when we encourage teams to take on technical debt.
There is no way to know the interest rate on a defect. Part of the cost of a defect is obvious: how much time and materials will it take to repair the immediate problem. (Although even that is often hard to measure.) But there are also lots of non-obvious and probably non-measurable costs. How much effort will it take to get to the root cause of the defect so that it doesn't occur a second time? How much will it affect our "goodwill" and thus reduce further and repeat sales? How much will the existence of one defect hide the existence of other defects (with their own costs)? How much will the defect demoralize the team and increase staff turnover or reduce productivity? How much of an opportunity will the defect create for competitors? How much will the defect increase maintenance and support costs?
In other words, every time someone asks a team to let quality slide, they are asking the team (and the organization) to take on debt with an unknown interest rate. Which is lunacy.
Read more about this:
Voluntary Technical Debt - James Shore talks about a situation where a conscious decision to take on technical debt led to positive results.
Technical Debt: The Threshold of Acceptable Pain - Steve Bate talks about skill and sensitivity to technical debt. Here's a great quote from this article:
What if someone has a very high threshold of pain? I’d expect to see lots of crud (technical debt) in their code.... The high threshold developer doesn’t mind spending weeks on new features that would otherwise require days or hours with clean code. And they don’t mind staying at work all night fixing bugs before a major release or spending countless hours babysitting the production system after it’s been released. It doesn’t seem to bother them to spend significant time away from family and friends or to have limited time or other interests and hobbies. ...they sometimes experience pleasure at being the “hero” who saved the company from the bugs they created.
An Incremental Technique to Pay Off Testing Technical Debt - Johanna Rothman talks about technical debt and describes a simple risk-oriented approach to reducing it.
Posted by Mishkin Berteig at 08:23 AM | |
December 18, 2006
Lean, Agile and Capitalism - Just a Thought
It occurred to me to ask: If the "invisible hand" in the free markets of capitalism is making for efficient markets, efficient work... then why is there some much room for improvement when we start using non-competitive, collaborative techniques such as lean and agile?
And if these collaborative techniques work on a small scale to improve efficiency, does this mean that we could do this across organizations as a "replacement" for capitalism somehow?
In agile methods, we "assume positive intent" on the part of individuals. What if we could do this across organizations? I'm not living in a dream world yet, but I think I have an inkling of what it might look like: Toyota and its collaborative, leaned-out supply chain.
Posted by Mishkin Berteig at 06:39 PM | |
December 07, 2006
Peformance Goals - The Wisdom of Teams
As I continue my enthralled read through "The Wisdom of Teams: Creating the High-Performance Organization
The Performance Goal
This concept and practice is an essential condition for a team to become a high performance team. The Performance Goal is a specific, measurable, challenging goal that is given to and/or adopted by the team. It is a statement or description of a goal that answers "why?" and "what?" questions, but specifically avoids answering "how?". It is not a description of activities, it is a statement of desired results. The team is left with the full authority to answer "how?" and implement it.
This concept is essential for setting the initial boundaries of self-organization. By defining "what" and "why", the team is left free to be creative about the solution. The Performance Goal is also essential to building team accountability (as opposed to individual or externalized accountability). Every action, plan, mistake and success are oriented around the Performance Goal.
From the book:
The hunger for performance is far more important to team success than team-building exercises, special incentives, or team leaders with ideal profiles. In fact, teams often form around such challenges without any help or support from management. Conversely, potential teams without such challenges usually fail to become teams.
I would also like to point out a great blog entry I found that shows some of the other side of dealing with teams and present some cautionary words about the potential pitfalls of working in teams.
In an Agile Work environment, the starting point for a performance goal is simply the delivery of valuable work at the end of their very first iteration. This is often a substantial challenge to a team and an organization. For some teams that have worked for a long time in a "waterfall" or phase-based project environment, it can be almost unthinkable that valuable results could be delivered in one tenth or one twentieth of the "normal" amount of time.
However, simply delivering value at the end of each iteration is probably not going to sustain the development of a high performance team for very long. Rather, the overall objective or goal of the project has to be important and compelling. Much work these days is _not_ important and compelling. In fact, many people become cynical about work because they are stuck doing a high proportion of work that is bureaucratic or due to chaotic circumstances.
As a reminder, the books "Good to Great
I frequently work with organizations where a team has been formed up, told to use agile methods, and then also told how to do their work. Really great examples of this are things like: "we want you to self-organize, but you have to build this huge system using J2EE." The the problem with this is simply that it may in fact be ten times less expensive to build the system with Ruby. However, someone has decided (possibly for defensible reasons) that J2EE is the technology platform that must be used. In this circumstance, someone external to the team has stepped over the boundary of "why" and "what" and also included some "how" in the team's goals. The team is not even allowed to consider the possibility that something might work just as well and be much less expensive. Not only that, but the stakeholders haven't even really stated "why" the system is being built and so the team can't evaluate technology choices. There is no standard around which to self-organize. I admit that I am using a simplistic example here, but the pattern is something that I have seen over and over again.
Posted by Mishkin Berteig at 11:44 PM | |
November 24, 2006
How to Avoid Context Switching
Given the huge interest in the article by Dmitri Zimine about context switching, and despite a couple of good articles about how to determine iteration length, there has been no empirical method described, only reasoning processes. This article describes a simple method to quickly determine iteration length by experimental means.
Step One
Start with an iteration length that feels right. Use the two articles below to try decide a reasonable length. In software development, this should be no longer that four weeks. In larger volunteer communities it might be as long as three months. In a work environment where you have to deliver daily, your iteration length might be two hours.
Step Two
Build your Work Queue, and plan your first iteration. Mark the tasks that come out of your iteration planning meeting so that you know that they are tasks that were planned. As you go through this first iteration, track all the interruptions you get by writing up a task for each interruption. Each interruption task should be marked so that you know it was an interruption. If you are using note cards on a visible task board, I like to have "normal" tasks on white cards and interruption tasks on fluorescent orange cards to help them stand out!
Step Three
At the end of your iteration, determine which interruption tasks could have been deferred. In other words, determine which were truly urgent and needed to be handled in the already short time of your iteration, and which could have been put on the Work Queue, prioritized, and therefore deferred to a later iteration. This will require collaborating with your Queue Master and possibly other stakeholders.
Step Four
Count how many non-deferrable interruption tasks your team had in the iteration. For this experimental method, you can assume that this is going to be your normal number of interruptions. Divide the length of your iteration by the number of non-deferrable interruptions. For example, if you had a ten day iteration, and two non-deferrable interruptions, you would have a result of five days. Also consider what was the maximum reasonable response time for these non-deferrable interruptions. The lower of these two values becomes your experimentally determined iteration length. But you are not quite done!
Step Five
Do it all again to verify your iteration length. Note that because of your shortened iteration length, you hopefully will have far fewer non-deferrable interruptions. After your second (shortened) iteration, you can adjust the iteration length shorter if necessary... but don't adjust longer (for now).
I've worked with enough teams to know that for a substantial portion of them, this method would result in very very short iterations. In the software world, a team is often asked to work on a project and support their previous project. This support work tends to mean dealing with various bugs in deployed software. This is one reason why I have become such a strong advocate that quality is not negotiable.
I worked with one team that did something similar to this method (although not as rigorously) and we decided that we needed to try an iteration length of two days. This was painful for the team, but they badly wanted to build trust with their stakeholders. Their interruptions were causing them to constantly fail on their commitments. By switching to these two day iterations, they were able to defer the bulk of their interruptions and meet the commitment they made as a team in the iteration planning meeting.
Articles about iteration length:
Mike Cohn's article: "Selecting the Right Iteration Length for Your Software Development Process"
Mishkin Berteig's article: "The Pros and Cons of Short Iterations"
Now that you have gotten to the end of the article, I can admit something to you: this article is badly named. It should be named "How to determine how often to context switch so that you can meet your commitments and build trust with your stakeholders."
What this article doesn't help with is the pain of context switching. The teams that I have see that use short iteration lengths all find ways of making context switching less painful. Automation, good tool choices, workspace arrangements, etc. all play a part in this. But there is no secret ingredient to make context switching painless. Sorry!
Posted by Mishkin Berteig at 07:55 AM | |
November 21, 2006
Obstacle Types
The Art of Obstacle Removal is an important thing for the Process Facilitator to get good at. I was working with a client today and realized that there are really three broad categories of obstacles and only one category is easy to see. The other two are much harder for people to identify and deal with.
Category One: Tangible Barriers
This is the easy type to identify. Physical obstacles such as distance or walls are the most obvious. And usually people have no trouble identifying other tangible barriers such as bureaucracy or bad habits.
Category Two: Obstacles of Absence
These types of obstacles are harder to identify because they often take creative thinking to see a possibility of improvement by adding something to the environment. Some general sub-categories include a lack of trust, a lack of automation and a lack of knowledge.
Category Three: Chaos
These is often the hardest type of obstacle to identify. Conflict between team members is hard to speak honestly about. Too much context switching is also hard to acknowledge because it often seems like it's a good thing to switch to something else - there's always some sort of reason.
Are there any broad categories that I've missed? Any good examples of these different types of obstacles?
Posted by Mishkin Berteig at 05:29 PM | |
November 17, 2006
Thinking/Doing Loops
Ryan Cooper, who recently attended one of my agile training courses said:
It doesn't make sense to separate the thinking from the doing.
So the real question is: why do we do this so often?
Levels of Separation
It is probably good to note that there are levels of separation. If you're in a big bureaucratic organization you will find hugely separated thinking and doing.
For example, at one large organization that I know well, there is a part of the organization dedicated to determining the best possible process to follow for all the different types of projects that are run: software projects, business projects, marketing projects, etc. The people in this process group sit together and think about all sorts of interesting things like how to meet regulatory compliance constraints, how to measure things really well, how to avoid most mistakes, how to make sure that there is a careful balance of responsibilities, how to make sure that everyone who cares about a project gets a say, and how to make sure that everyone who cares about a project can throw in their version of the kitchen sink. These people get around to interviewing the managers of the people who actually work in these projects... maybe once every six months.
The people "working" in the process group are almost completely separated from the reality of the people doing the work. The feedback cycle is at least twelve months. Let's say that a project starts out using Version N of the process defined by the process group. The project runs for six months. The process people interview the manager of the project and incorporate the feedback in Version N+1 of the process which is released into the wild six months later.
The trouble is that the twelve months for the feedback cycle is actually the least bad it could possibly be. More likely it looks like this: six month project? Too small, the feedback isn't terribly important. Two year, forty person project? That's more like it! Let's get lots of feedback in the post-mortem for this project! Project starts with Version N of the process. Thirty-six months later the project finishes (if lucky!). Post-mortem done. a few months later someone from the process group receives the post-mortem report (200 pages of boring, poorly written text). The process group decides that the $100 million worth of feedback deserves its own project. The process group launches a major process revision project based on the post-mortem report. One year later, the process group releases Version N+1 of the process. Total elapsed time for feedback: fifty+ months! Total waste of time: yes.
Okay, so that's a worst-case scenario.
There are lots of smaller ways that we separate thinking from doing.
Individual
The most direct pairing of thinking and doing happens at the level of the individual person. A person's own thoughts and creative efforts are directing their actions, and they are observing and learning from those actions to change future thoughts. This feedback loop is the easiest and most natural (although sometimes we forget to learn from the results of our actions). Sometimes, this feedback loop is so tight that we get into a state of flow, the "zone" and it seems as if our actions and our thoughts are perfectly at one.
This level is limited in that many thing cannot be done by the efforts of a single human being. So we move up a level.
Team
The team works closely together so that thinking and doing is separated at most by trusted high-bandwidth low-latency communications such as a few people working together at a whiteboard, or two people working at a computer together. All the members of the team have equal participation in the thinking/doing loop.
Again, it is possible for a team to get into the "zone" and become a high-performing or "real" team
This level of separation of thinking/doing does present some challenges: it is more difficult to get into a state of flow as a team, it is easier for the work to be disrupted by the environment, and sometimes the communication breaks down.
Group or Organization
There are some rare organizations which maintain an incredibly productive state above the team level. Toyota is oft-cited, but there are others too such as the "Good to Great
More often than not, however, groups and organizations fall into various degenerative types of thinking/doing separation. In software development, this is typified by the handoffs between people doing requirements to different people doing architecture to different people doing analysis and detailed design to different people doing development to different people doing testing. And that's not even taking into account project managers, managers, marketing, sales and customers! Yikes! No wonder phase-based software development is so painful!
Obligatory Conclusion
Agile methods help tighten up the thinking/doing loop by requiring cross-functional teams with product (what) and process (how) and technique (work) skills and authority actually on the team.
P.S. This is an expansion on the Cycles of the Mind stuff.
Posted by Mishkin Berteig at 02:54 AM | |
November 15, 2006
The Case for Context Switching
Recently, Dimitri Zimine wrote an excellent little story about context switching. Joel Spolsky writes in "From the 'You Call this Agile' Department":
Dmitri is only looking at one side of the cost/benefit equation. He's laid out a very convincing argument why Sarah should not interrupt her carefully planned two week iteration, but he hasn't even mentioned arguments for the other side: the important sale that will be lost.
Okay... I'll bite.
Let's look at that argument from the perspective of the sales person first since that is where Joel's concern starts:
The Sales Guy Perspective
"I need the 'ezhibal' feature now! Big bucks coming soon if we can get it now."
Let's suppose that this urgent email has come in somewhere near the start of our two week iteration. The normal response to this in Scrum is to push back a little. The ScrumMaster says something to the effect of: "Look if you wait 7 days we can put this on the top of the list for our next iteration."
First reaction, and it's a normal one, is for Sales Guy to freak out. I've actually heard people saying things like "You're going to lose your job over this! I'm getting the VP involved and he's not going to like it" and then they stalk off to find the big dog to come and bark at us. Anyway, let's pretend that the Sales Guy is willing to be reasonable and not instantly escalate the "problem". So what he actually says is: "Look, this is super important, it'll probably only take a few minutes for us to talk about it and then we can figure out how long it will take to fix. Let's just do a quick phone call and yadda, yadda, yadda, blah dee blah..."
Enough of the Sales Guy perspective.
Nowadays, if I'm in this situation, I do a value assessement. I tell the team to keep working on their plan, nothing's changed yet, and I sit down with Sales Guy and the person who's sponsoring the current work and we start a discussion about the options of which there are really only two that work in Scrum:
- Stay the Course
- Cancel the Iteration
First, let's talk about how we decide which option to take. Then we'll talk about why.
Deciding on the option is easy. You look at the value of the work currently being done and compare it to the value of the work that Sales Guy needs. You take into account probabilities. If Sales Guy doesn't have a signed contract, then it's hard to day if there's going to be any real revenue from the 'ezhibal' feature. Still, you might be able to do an assessment of the probabilities based on your level of trust and history with the client, etc. You also need to take into account the time value of money. Does delaying the current work have consequences for another client or stakeholder? What is the cost of those consequences.
This is a relatively simple cost/benefit analysis and comparison. If you're not comfortable with money and numbers and spreadsheets, you better get comfortable!
Okay, so we have a way of comparing the two bits of work. Now let's look at the two "allowed" responses and a third "bad" response.
Stay the Course
Turns out that the potential benefit of the stuff Sales Guy wants is not quite as high as the potential benefit of the stuff that Suzie Stakeholder prioritized for the current iteration. Well, that's easy. Put the request from Sales Guy in our prioritized list of work and get to it when there is an appropriate level of benefit relative to the other work.
Cancel the Iteration
The stuff Sales Guy wants is super-valuable. So let's get the whole team to stop what they are doing and everyone supports this very valuble work. Stopping the whole team is appropriate because obvioulsy, the stuff they're working on isn't as valuable. Oh, and because we treat a team as a unit in Scrum. And because the team needs to commit to work, not individuals. This isn't so obvious... more later.
Peel Sarah off to do the 'ezhibal' Feature
This is what normally happens, and in a "normal" non-agile environment, it's probably okay. In a non-agile environment, Sarah hasn't made any commitments (she's been forced to agree to dates and scope, etc., but she hasn't made a commitment that she is empowered to live up to). So if she goes off and does this one little thing, it probably will be just business as usual. In an agile environment, the team has made a commitment and doing this work this way invalidates the team's commitment.
Why do we do it this way? The main reason is around trust and commitment. Yup, it's that soft icky stuff that we're so incredibly bad at that customers think that bugs are normal, that management shoves the kitchen sink into projects in the frustrated hope that they'll get something out of the IT team at the end of the project. Sound familiar to anyone?
Anyway. An iteration is a commitment. It is a firm and solid commitment. The team as a group of smart and honorable people is making a definite commitment to the rest of the organization to get a certain amount of work done in a fixed amount of time. In return, management is agreeing to give the team every support in reaching this commitment. When a team is new at this, they might get it wrong. But having done this with dozens of teams now, I know that after a few tries, the team gets the hang of it and commits to appropriate amounts of work, and management provides appropriate levels of support.
This commitment is essential for developing trust. And anything that comes in the way of the team meeting its commitment is considered "BAD". An obstacle to do away with.
This is interesting, because Joel's second example is about defects. And I strongly agree that defects are "BAD" and need to be dealt with at a very high level of priority. The reason is simple: they prevent a team from meeting its commitment.
One team I was coaching was constantly bombarded by these types of it'll-just-take-a-few-minutes-need-it-asap requests. They had many stakeholders and very very limited resources to service these requests. They had several small projects that were taking literally years to do because they couldn't get enough concentrated time on any one thing. This was considered normal and good in their environment.
The trouble is, no one had really looked at the overall consequences. Everyone was doing local optimization. For us geeks, we all know that local optimization is something to be avoided if possible. We climb a peak only to discover that we have to climb back down a ways to get up to the higher peak we now see is next to this one. We climb up that one only to discover yet another higher peak even further along thus requiring us to climb down and up again... When really what we should have done is stepped back a ways, looked at the lay of the land and said, "hey, that peak over there is the highest of them all, let's go climb that one."
Scrum helps us avoid local optimization by forcing all feature requests for a team to be prioritized in a list of work, and by allowing the team to complete small pieces of work so it actually gets _something_ done that you can learn from.
Joel said:
Agile is not supposed to be about swapping out one set of bureaucratic, rigid procedures for another equally rigid set of procedures that still doesn't take customer's needs into account.
True enough! But it also demonstrates a serious lack of understanding about what is being done in Dmitri's example! First of all, without being agile at all, it is possible to switch from 18 month projects to two week projects. Both can be bureaucratic as you please (well, actually, there's only so much bureaucracy you can cram into two weeks and still get something done). The shorter projects will allow you to be much more responsive to customer needs... by definition!
So what happens when you add in all the other things that agile really is about? Transparency. Truthfulness. Creativity. Learning. Meta-Learning. Prioritization. Self-Organizating Teams. Eliminating Waste.
Well, first of what you get is something that's damn hard to do right. It goes against almost everything we've been taught to do: the extreme of heroics of the extreme of careful planning and process.
Secondly, what you get is something that needs safety zones and meta-rules. Like mutual, freely-given, team-to-stakeholders commitment. Like assuming positive intent.
And thirdly, what you get is an environment where not only is the business getting what it needs more than it used to, but also, the team likes working with the business, and the business likes working with the team.
I admit that the point Joel is making isn't too different. Yes: look at the costs and the benefits. But agile isn't quite about instantaneous responsiveness. That's a red herring and I'm suprised that Joel threw it's stinky carcass into the discussion. Agile is also about balancing that responsiveness with the overall view of value for the team and the organization. The tool for doing that is the prioritized list of work, not the email message from Sales Guy to Sarah.
Posted by Mishkin Berteig at 04:13 AM | |
November 14, 2006
Process Facilitator "Smells"
I have now trained over one hundred people in my Agile Project Managmenet / ScrumMaster Certification course. I'm starting to see and hear some of the results of this training. There are a couple specific "smells" that I have become aware of.
Fortunately, I've been able to provide coaching to some of the organizations that have sent people to my course. There are quite a number of good things that happen, but there are a couple of things that seem to be "natural" misunderstandings.
- Spectating
I put a lot of emphasis on the idea of a self-organizing team in my course. There are a number of exercises, an hour-long section, and many other points during the course when it comes up. With all of this emphasis, it seems that a few people have come away from the course with an extremely hands-off approach to the Process Facilitator role (ScrumMaster/Agile Project Manager). I think this is a natural and probably good reaction to the heavy-handed command & control approach that these people come from. However, there are a few things that should be considered minimum levels of engagement (listed below). - Problem Solving
There is also a great deal of emphasis put in the course on removing obstacles. I have seen several cases where it becomes the habit of the Process Facilitator to start solving every problem. This can happen in day-to-day work, and also in the retrospectives. Again, this seems to be a natural consequence of the desire to get in there and be of value. However, if the Process Facilitator writes down all the "things that need impovement" from the retrospective and then says "Okay! I'll take care of these things." then you know that the Process Facilitator has gone too far.
Appropriate Process Facilitator Engagement
Here are a few ideas on an appropriate level of engagement. Finding the right balance of enagement is not easy and there is no exact formula to follow. Partly it depends on your personality and skills as a Process Facilitator, partly it depends on the capabilities of the team, and partly it depends on the constraints of your work environment. Nevertheless, there are some types of engagement that you can persue with confidence. Here are a few concrete guidelines:
- Team Membership
If you aren't saying "we" when referring to the team, you aren't engaged enough. The Process Facilitator is a member of the team. - Colocation
Most of your time should be spent sitting with the team. If the team isn't colocated, then you should be spending most of your time physically visiting each team member. - Obligations
The Process Facilitator is a catalyst. You must track all the obstacles and "needs improvement" things that the team raises, and then make sure that they get fixed... without going in and fixing everything yourself. Ultimately, your job is to work yourself out of a job. - Assertiveness
The organization that the team is working in will be creating some of the obstacles that the team is facing. Being assertive is critical to helping the organization face and then overcome some of these obstacles.
Posted by Mishkin Berteig at 01:49 PM | |
November 12, 2006
More on Scaling Agile
One problem with having multiple teams working on the same project will be the tendency to compare the teams' performance. Why is this a problem?
Why Not Compare Team Performance?
One of the main reasons is that the teams need to be collaborating not competing. There can be a small amount of friendly competition that might come naturally, but as soon as management starts paying attention to differences in team performance, the competition starts to get serious.
In the case of multiple teams working on the same project, the goal should be to maximize overall performance, not individual team performance. Too much competition can lead to unintentional or deliberate sabotage: failed communication, incomplete communication and downright dishonesty.
Motivating Teams without Comparing Them!
As Mary Poppendieck has written, measure up [pdf]. In a single-team situation this means that individuals are measured and rewarded based on team performance (their sphere of influence). In a multi-team environment, that means that the group of teams should be measured as a group and compensated as a group. This will encourage all teams to work towards the success of the overall project. I personally believe there is some room for individual-based compensation, but the way it is handled needs to be done so that it does not encourage sub-optimal behavior.
As well, each team can keep track of their velocity. Some coaches recommend using "ideal hours" or some other units to determine velocity (velocity = estimated work remaining completed / iteration). The trouble with doing this with multiple teams is that there will be a very real tendency to want to compare each team's velocity. Since velocity is a useful measure for team capacity, it is important to still have a way to measure it. One simple way to do this to prevent comparison is to use different units for each team. Team One might be measuring velocity in Estimated Ping Pong Balls Completed / Iteration... Team Two in Estimated Bananas Completed / Iteration... Team Three in Estimated Bogo-MIPS Completed / Iteration... etc. etc.
Motivating Collaboration
First off, management must make visible commitments to eliminating barriers to collaboration. For example, it is a great sign of commitment to re-organize office spaces so that all the teams are sitting near to each other. Every time the Process Facilitator identifies an obstacle that relates to collaboration (tools, process, physical environment, etc.) management should get right on it and fix it.
An ongoing investment in team-building training, workshops and exercises is also helpful. This type of investment helps people gain the skills necessary to work effectively with other people. Again, individuals need to see and believe that management cares about and values teams.
Finally, one of my pet peeves: when a project is over, keep the team together! Do not break them up and redistribute your "resources" to other efforts. The value of those people working together is substantial. The value of those people working together as a high-performance team is incredible! In a multi-team situation, it is reasonable to take teams from the overall group and re-distribute their efforts... but just don't break up the individual teams.
Miscellaneous Notes on Scaling Agile:
Twelve is still the maximum recommended size for a single agile team. Seven is really the sweet spot. A team larger than twelve people just takes too long to get into the Performing stage of team development. If you feel like your project needs more than twelve people actively involved, then you probably want to split into two or more teams... and then you have "scaled" agile.
If you have three teams of five people (or some similar configuration of people just over the 12 person limit), then they will work as a very close-knit group and a lot of the time will act as if they were a single team. They will probably plan iterations and do demos and retrospectives together.
Twelve teams working on the same project at once is about the maximum number before communication overhead is slowing everyone down too much. This is largely a factor of trust: with 150 or fewer people involved in an effort, it is possible for everyone to know everyone. More than that many people and it is no longer possible. Trust is just not an option anymore and bureaucratic controls take over.
If for some reason you need to do something in a small amount of time and you think it's going to take more than twelve teams of twelve people...? Break the effort into smaller chunks. Divide and conquer. Division can be across functional areas, structural areas or time.
Although I have heard of agile methods being used with groups larger than this, I haven't heard any success stories :-)
Check out my earlier introductory article on Scaling Agile in a large project situation.
Dean Leffingwell has an article about practices needed for scaled-up efforts at the Agile Journal. I glanced through it, but I admit that after I disagreed with his very first point (Intentional Architecture), I started to pay less attention.
He claims that refactoring of large systems is not possible (or at least infeasible). The odd thing is, most large projects that I have been involved with are being done exactly because an old system is not refactorable. A large telecom system, a large insurance system, a large data warehouse and a large GIS system are all being done with scaled up agile methods exactly because the old systems that are currently in place have become ossified to the point where they must be replaced.
These old systems were originally built with phase-based development approaches. At some point, people stopped refactoring because they were not given the space to do so. This drop in code quality turned into technical debt. The technical debt accumulated to the point where it was unbearable (maintenance costs, cost of change, etc.).
The problem with intentional architecture is that it goes back to the old assumption that you can do a good design for a system without the constant feedback from review, deployment and use done on a very short cycle time. Over and over, we are faced with the painful consequences of this attitude, and that is one of the key reasons we started to work with agile methods in the first place.
Martin Fowler makes a good case that scaling agile is the last thing you should do. I don't disagree! Scale your agile teams at your own risk!
It's nice for me to be able to say that I've worked on some agile projects over $10,000,000 in size, but the fact is that the cost could have been reduced substantially if the team size was lowered and the deadline extended. It is (relatively) simple to do a cost/benefit analysis of cross-team-coordination-overhead vs. the time value of early delivery of more functionality... although I've never seen anyone do it! If you know of an example of an organization doing this in a realistic way, I'd love to hear about it!
Are there other ways of supporting cross-team collaboration that you have seen?
Posted by Mishkin Berteig at 09:13 AM | |
November 07, 2006
Scaling Agile Projects
More and more, organizations are applying agile methods to large projects or efforts that require more than a single team. There are three dimensions or concerns of coordination. It is critical that all three be addressed, but there are many ways of addressing them. Here I will simply list these three types of coordination and make some simple suggestions of how to implement them.
I have now had the opportunity to work with several clients where they are applying agile methods to projects with budgets that are greater than $10,000,000. All of them are using multiple teams to work on the same overall project/program. Out of this experience (along with some good reading along the way), I have come to understand that the following three types of coordination are the essential ones:
Value
In order to maintain the "early and frequent" delivery of value that agile methods propose, it is important that the work of the effort be coordinated to maximize early delivery of value. From this perspective, there are often many cooks in the kitchen. I have seen a "Product Owner Team", a "Customer Team", and a number of variations of this type. In order to do the coordination work effectively, it is still necessary to make sure of two things:
- Maintain a single Work Queue that prioritizes the work and from which all the teams select items.
- Have a single person in the "buck stops here" role who can make final binding decisions about work priority and content.
These two items have some implications for the organization.
First, the teams must be organized to be generalist: each team should be able to handle any item on the Work Queue. Not every team is going to be equal in abilities and this can be accomodated in a number of ways. My favorite so far comes from an excellent agile coach Dave West
The second implication is that the customer or product team (Queue Master) must have the availability to support multiple teams in a timely fashion. Ideally, there are individuals on each team who can make judgement calls about features, functionality, constraints etc. on the work and provide quick answers to questions. This is not always easy since the people doing this often have a special area of expertise and it is difficult for them to work outside this area. Just as team members are asked to become generalizing specialists, so must the people who are responsible for determining value in a project.
Process
An agile process endeavors to provide a minimally structured way to do three things: deliver value early, then learn about what is high in value and deliver that more, and finally, learn how to deliver value more effectively.
That third activity, learning how to deliver value more effectively, is facilitated by the Process Facilitator. The Process Facilitator keeps a visible list of obstacles and works collaboratively with the Team and the Stakeholders to resolve obstacles on the list.
In a multi-team environment, there may be a single Process Facilitator working with each team. Like with the Work Queue, it is often necessary to have a single Record of Obstacles for the entire project.
Technique
People develop skill and knowledge in the use of their tools. Most types of work have a special vocabularly that only makes sense to other people also doing that work. For example, the field of computer programming has programming languages, integrated development environments, build tools, testing tools, algorithms, and a host of other techniques. The field of film-making has cameras, film, directorial techniques, lighting, story structure, it's own esoteric vocabulary, and other techniques. Likewise for construction, law, medicine, drama, education, etc. etc. etc.
In a large Agile Work project, teams need a way to coordinate their technique to produce integrated, consistent and compatible results. As well, individuals on the teams may discover or create new ways of doing things that would be valuable for the other teams to know about and use.
The most effective way of coordinating technique across teams is for strong members of each team to gather regularly to review the way work is being done. This "technical support group" can look at tools, reuse, automation, patterns, vocabularly and any other "how to" aspects of the work. It is critical that these people are actively involved in work being done on a delivery team so that efforts of the technical support group do not become academic or "ivory tower".
In certain environments, it may be useful to have this techincal support group become a team with a clear allocation of time apart from the regular delivery teams. In this case, this technical support team would have its own Work Queue that consisted of requests, ideas, concerns, and opportunities presented by the regular delivery teams.
I have seen all three aspects of coordination implemented in large multi-team projects. Some of the common challenges include:
- Generalist Teams.
It is difficult enough to create cross-functional teams where people are willing to become generalizing specialists. While it is important to create generalist teams, most organizations should expect to set up non-ideal specialist teams (sometimes by line-of-business) and support their development into generalist teams. - Technical Coordination.
Often organizations have a design or technical review group composed of the "experts". These people are often isolated from the actual work being performed by the teams. It is difficult, yet critical, that these people actually be involved in day-to-day work on the teams.
Posted by Mishkin Berteig at 09:15 AM | |
September 05, 2006
The Seven Core Practices of Agile Work
Agile Work consists of seven core practices. These practices form a solid starting point for any person, team or community that wishes to follow the Middle Way to Excellence.
Self-Organizing Team
Any group of people that wish to be an Agile Team need to take the initiative to determine for themselves how they are going to work (process) and how they are going to do the work (product). The term "team" really applies quite broadly to any size group of people that are working together towards a common goal.
Teams go through stages of development as they perform their work. The most important result of team development is the team itself, and not the specific skills and abilities that the individuals learn.
If the team is part of a broader organization, that organization must give the team the authority, space and safety to learn to be self-organizing. The organization's leadership is responsible for determining the "why?", some constraints on "how?", and then letting the team respond to the need as best as it can.
Also Known As: Whole Team (Extreme Programming), Cross-Functional Team (business management).
Deliver Frequently
Agile Work uses short fixed periods of time to frame the process of delivering something of value. Each of these iterations or timeboxes is structured so that the team or group actually finishes a piece of work and delivers it to stakeholders. Then, the team builds on what has previously been delivered to do it again in the same short amount of time.
The sooner that valuable results can be delivered, the more value can be obtained from those results. This extra value is derived from opportunities such as earlier sales, competitive advantage, early feedback, and risk reduction.
There is an explicit tradeoff: the shorter the time to delivery, the smaller the piece of value will be. But, like investing in one's retirement account, the earlier you start, even with small amounts of money, the better off you are in the long run.
Also Known As: Sprint (Scrum), Iteration (Extreme Programming), Timeboxing (generic), Time Value of Money (accounting).
Plan to Learn
Every type of work is governed by a Horizon of Predictability. Any plan that extends beyond this horizon of predictability is bound to fail. Agile work uses an explicit learning cycle tied in with the planning of work to accomodate this inevitable change.
First, a goal is required. This goal can be long-term. Teams using Agile Work then create a queue of work items to be done in order to reach this goal. Each iteration, some of these items are selected, finished and then the queue is adjusted. The changes in the work queue are based on external factors, and learning that the team does as it goes.
One of the most effective methods for the team to learn about how it is doing its work is the retrospective. After each delivery of results, the team holds a retrospective to examine how it can improve.
Also Known As: Inspect and Adapt (Scrum), Kaizen (Lean), Adaptive Planning (generic).
Communicate Powerfully
A team needs to have effective means of communicating, both amongst team members and also to stakeholders. To Communicate Powerfully, a team needs to prefer in-person communication over distributed communication. Synchronous over asynchronous communication. High-bandwidth over low-bandwidth communication. Multi-mode communication over single-mode communication.
The results of failing to communicate powerfully include wasted time for waiting, misunderstandings leading to defects or re-work, slower development of trust, slower team-building, and ultimately a failure to align perceptions of reality.
The single most effective means to communicate powerfully, is to put all the team in a room together where they can do their work, every day for the majority of the work time.
Some types of work do not lend themselves to this approach (e.g. creating a documentary video), but every effort should be made to improve communication.
Also Known As: Visibility (Scrum), Whole Team and Team Room (Extreme Programming), War Room (business management).
Test Everything
Defects are one of the most critical types of waste to eliminate from a work process. By testing everything, by driving all the work of a team by creating test cases to check the work, a team can reach extremely high quality levels. This ability to prevent defects is so important that only an executive level decision should be considered sufficient to allow defects into a work process. Quality is not negotiable.
In Agile Work, removing a defect is the only type of work that takes priority over any new features/functionality/production. If the end result desired is to maximize value, then removing defects is an important means to that end.
A team has an ethical duty to discover new ways to effectively test their work. This can be through the use of tools, various feedback mechanisms, automation, and good old problem-solving abilities.
Also Known As: Canary in the Coal Mine (Scrum), Test-Driven Development (Extreme Programming), Defects per Opportunities (Six-Sigma).
Measure Value
Since Reality is Perceived, it is important for an agile team and organization to have a clear method of describing and perceiving what is important for the organization. Measuring value is a critical method for describing and perceiving what is important.
A single metric can be used to drive all the measurement and goal-setting and rewards in an organization. All other measurements are secondary and must be treated as such: limited in use and temporary.
There are many things which are easier to measure than value. It is often easy to measure cost, or hours worked, or defects found, or estimate vs. actual... etc. However, all of these other measurements either implicitly or explicitly drive sub-optimal behavior.
Also Known As: Measuring Results (Scrum), ROI (business management), Economic Driver (Good to Great), Running Tested Features (Extreme Programming).
Clear the Path
Everyone in an organization using Agile Work takes responsibility for clearing the path, removing the obstacles that prevent work from being done effectively. Clearing the Path doesn't just mean expedient, quick fixes to problems, but rather taking the time to look at an obstacle and do the best possible to remove it permanently so that it never blocks the path again.
In the Agile Work method, the Process Facilitator is the person who is responsible for tracking obstacles and ensuring that the path is cleared. To do this, the Process Facilitator maintains a Record of Obstacles.
Clearing the Path is sometimes painful work that exposes things we would rather not deal with. As a result, it is critical that people build their capacity for truthfulness and work to develop trust amongst each other. Building a capacity for truthfulness is not something that can be done by using an explicit process.
Also Known As: Removing Obstacles (Scrum), Stopping the Line (Lean).
Remember also, that these practices must always be viewed and implemented in the context of the Agile Axioms. These axioms provide a check to ensure that the practices are not being applied blindly, but rather applied appropriately to the given situation.
Posted by Mishkin Berteig at 09:09 AM | |
August 23, 2006
United Action
A few nights ago, I was having a conversation with my father, Garry Berteig, who is also an agile coach. He had an interesting insight based on a recent experience in Beijing: unity is a precondition to assistance.
In Beijing, he was helping my brother, Alexei, get ready for his upcoming wedding to a wonderful Chinese woman, Ma Jin. The three of them were out shopping. Alexei needed new furniture. As they shopped, Alexei and Ma Jin discussed the various items, their prices, qualities, etc. It took quite a while, but eventually they agreed upon a set of furniture. They became united in their understanding and perception of reality, and agreed without hesitation.
At this point, my father was able to offer his assistance to purchase the furniture as a gift for their wedding. Once Alexei and Ma Jin were united, it was possible to not just offer the assistance, but actually execute on it. If they had not come to a unity of vision, it would have been a mistake to execute on the assistance.
I believe this applies in teams and organizations. In agile environments, managers become "servant leaders". This service is only effective once the self-organizing team is united in its understanding of its needs. If the team is not united, then the service of the manager can lead to problems: some members of the team may feel discounted, some may feel (falsely) that management supports their side, and finally, management may not be able to offer whole-hearted support in the future as complaints come in.
Posted by Mishkin Berteig at 01:20 PM | |
August 21, 2006
Quality is Not Negotiable
Most of the teams and organizations I coach are working on using agile methods to improve their software development approach. Somewhere along the way, someone has realized that there must be a better way... either better than chaos, or better than bureaucracy. Over the years that I have been practicing agile methods, I have come to believe that quality is not negotiable.
The problem is this: when you have a defect in your work, you have no way of predicting how long it will take to remove the defect and remove it in such a way that it never occurs again. This is because the process of understanding a defect is unbounded: it is investigatory and exploratory and experimental. Once the defect is fully understood, the solution becomes clear and therefore estimable. Until that time, a defect represents an unknown amount of risk.
Before I started using agile methods, and in particular Test-Driven Work, I built software poorly. I delivered software which had unknown numbers of defects. I would often find myself trying to implement a feature (something of value) only to be thwarted by a defect in the software built so far. Until I could fix the defect, I was blocked from adding in the new feature.
The moment I started using Test-Driven Work to build my software, was the moment that I could start accurately predicting how long it would take me to build new features. I started delivering defect-free software. (Yes, it's possible!) I delivered a complex multi-threaded, guaranteed-delivery, message-oriented-middleware framework that went through QA and Pre-Production testing with zero defects found. It was in production for two years before an obscure multi-threaded non-critical error was found... and none after that for a few more years. This sort of level of quality should be normal, not something to brag about.
Test-Driven Work depends on a clear understanding of how to make good tests. It also depends on an extremely high level of professional discipline. And most importantly, it depends on a supportive organization and management.
Often times, management, marketing or sales will put pressure on a development organization to "just get it done", "don't worry about doing it right", etc. Unfortunately, most developers and development managers have not been instilled with a strong sense of professional ethics. Much more often than not, the response of the developers will be "yes, sir!" and then to go off and build crappy software that on the surface satisfies the pressure, but in the long term ends of sinking the organization.
How does this happen? It's simple. Every time the developers sacrifice quality for schedule/cost/features, they incur a debt. Sometimes this is called "technical debt". I like to call it lying.
Despite my preference for the more direct terminology, the term "technical debt" is a useful euphemism. As you accumulate debt, you have to make interest payments to maintain the debt. This shows up in the workarounds, the design cruft, the inelegant solutions, and ultimately in the slow down of the team in implementing new features. It is the cause of the industry's propensity to rebuild products and systems from scratch every 3 to 5 years, and occasionally it is the cause of the failure of a business.
The only way to deal with technical debt is to confront it directly. First, the development team needs to start to refuse to compromise. No new work will be done without adhering to the highest standards of quality. Secondly, the team must start to clean up the old stuff. Doing this takes, courage, time and therefore money.
While I don't recommend people lose their jobs for this, I do recommend that people look for other work if their organization is not willing to make this transition.
On a softer note, I also acknowledge that sometimes people make honest mistakes that end up introducing defects. This is fine... just make sure that as individuals and as an organization you are investing the time and effort to make sure this happens less and less frequently. Training, reading, practice, collaboration, all can help with this process. One can't go from poor quality to perfect quality instantly... it can take time to make the changes necessary.
Here's a software-specific article that articulates these ideas exceptionally clearly: Why Expensive Bugs are Cheap to Fix.
Here's a follow-up article about technical debt.
Posted by Mishkin Berteig at 03:35 PM | |
August 17, 2006
The Queue Master
Given my interest in applying agile methods outside of software new product development, I've been often uncomfortable with the term "Product Owner". My problem is two-fold: it's too specific a term (referring to product development), and it fails to denote the responsibilities except in the most generic fashion. In other words, I think that it is both too specific, and at the same time, not specific enough. Updated the Agile Work cheat sheets to reflect this new terminology.
Several weeks ago an insightful software engineer I was talking with by the name of Sin Stokic came up with a great alternative name for the role: Queue Master.
This name appeals to me by solving my two problems with the Product Owner term:
1. It is a term that is not specific to a problem domain. Agile methods can be applied to all sorts of non-product type of work including operations, services, remediation and exploration. The term "Queue Master" is generic enough to cover all these types of work.
2. It is a term that denotes the actual function that the person filling this role performs, namely, controlling the queue of work that is being fed to the performing team. Since so much of agile methods rests theoretically on queuing theory, this is an appropriate, and more specific term than "Product Owner".
Additionally, the term "master" is beneficial because it implies a great deal of control and/or expertise that is an essential aspect of this role. The Queue Master must be 1) the master who manages the behavior and properties of the queue of work (the Work Item List), and 2) have mastery of the subject matter contained in the queue of work items.
I would now re-write my previous definition of this role as follows:
The Queue Master holds the final authority for determining the value, priority and details of all work done by an Agile team by controlling the content and ordering of the Work Item List. The Queue Master wields this authority by virtue of a deep knowledge of the goal and end results desired as well as a respected position among all the stakeholders of the endeavor.
I have updated the Agile Work cheat sheets to reflect this new terminology.
All that said, I'm open to suggestions for even better names for this role. I also considered a number of variants such as "Queue Manager", "Queue Facilitator" and "Queue Keeper" (which has a nice alliteration, but is also a little corny). Thoughts? Reactions?
Posted by Mishkin Berteig at 12:57 PM | |
August 13, 2006
Missing Terminology
I discovered that the agile methods that I know well (XP, Scrum, Lean) have no word or term for an important part of their process!
I'm working with an organization that is trying out 2-day long iterations (why is a whole other story). In order to track their work effectively, we are going to be doing brief status checks to update a burndown chart every hour. What is the period of time called between those status checks?
In most agile methods, that period of time is always 24 hours in duration - daily. Therefore it is easy to talk about without a special name for it...
But if the period of time between status checks is longer or shorter than a day, then it becomes more awkward to talk about it. So, I've decided to give it a name:
Work Period
The team I am working with is using a 1 hour work periods inside a 2 day iteration. They have 10 work periods each iteration.
Scrum recommends a 24 hour work period inside a 30 day Sprint.
In my training classes, I run an agile simulation with a 10 minute work period inside a 50 minute iteration.
In the last work period, I completed my two form processing tasks. Next work period I will get the database code for the report completed. I have no obstacles to report from this last work period.
I like the term "work period" because it is simple and generic. It also evokes the term "periodic" and the idea of a concentrated effort on the task at hand.
Thoughts? Any agile methods that _do_ have a term to refer to this period of time between status meetings?
Posted by Mishkin Berteig at 09:38 PM | |
June 19, 2006
A Ninth Barrier to Effective Listening
I provided a link in a previous entry to an article about eight barriers to effective listening. Well, I was recently talking with someone about this and he pointed out a very important ninth barrier to effective listening.
The Ninth Barrier to Effective Listening
Exhausted, sleepy, tired, spent, wasted, enervated, asleep.
This is not rocket science folks. If you are too tired to listen to what others are saying, then you can't listen effectively regardless of the existence or lack of the other eight barriers.
In the courses I run, I always remind people to get a good sleep so they can be alert for the course. It is wasteful of their time to be unable to concentrate or even actually falling asleep. I do my best to make my course interesting and active, but I know myself that if I'm tired, no matter how interesting the material, I can't learn it.
For myself, I don't drink caffienated beverages regularly. Why? So that I can use them as "emergency fuel" when I really need them. It sometimes happens that I can't get to sleep at a decent hour before an important meeting or a course I'm delivering or some important reading I need to do. Caffiene helps, but only if I'm not using it regularly.
Posted by Mishkin Berteig at 11:45 AM | |
May 31, 2006
The Human Touch
If you are given a problem to solve, how much does the presentation of that problem matter to your ability to solve it? Imagine that it's a simple problem... imagine that it is presented in two different ways, both of them simple. It turns out that presentation differences can still make a huge difference. In fact, there is a way to present problems that makes them substantially easiers to solve: make them people problems.
In The Tipping Point: How Little Things Can Make a Big Difference
Consider the following brain teaser. Suppose I give you four cards labeled with the letters A and D and the numberals 3 and 6. The rule of the game is that a card with a vowel on it always has an even number on the other side. Which of the cards would you have to turn over to prove this rule to be true?
Go ahead and take a few moments to figure that out before continuing on to the answer, and keep track of how long you work on it...
The answer is two: the A card and the three card. The overwhelming majority of people given this test, though, don't get it right. They tend to answer just the A card, or the A and the six. It's a hard question. But now let me pose another question. Suppose four people are drinking in a bar. One is drinking Coke. One is sixteen. One is drinking beer and one is twenty-five. Given the rule that no one under twenty -one is allowed to drink beer, which of those people's IDs do we have to check to make sure the law is being observed?
How long does it take you to figure that out?
Now the answer is easy. In fact, I'm sure that almost everyone will get it right: the beer drinker and the sixteen-year-old. But, as the psychologist Leda Cosmides (who dreamt up this example) points out, it is exactly the same puzzle as the A, D, 3, and 6 puzzle. The difference is that it is framed in a way that makes it about people, instead of about numbers, and as human beings we are a lot more sophisticated about each other than we are about the abstract world.
Now unless you had heard about this before, I suspect you were pretty suprised. I know I was! I always considered myself to be a very good abstract thinker/problem solver. In fact, I considered myself to be well above average in that regard for a number of reasons: I was always very good at math without every memorizing a single formula (I always made them up as I went along as long as I remembered the _idea_ of the formula), I was an excellent programmer in a number of different computer languages including assembler, Miranda, Java, Prolog, Pascal, and Objective-C, and finally, I'm always solving problems by moving the problem to a new level of abstraction - solving the meta-problem first.
So what does all this have to do with agile work methods? Quite a few things actually:
1. Obviously, if you can frame a problem as a people problem, it will be easier to solve... and most problems start out this way!
We tend to try to abstract problems, make them more generic or general purpose in the hopes that they can be communicated more precisely and can be solved in a way that will accomodate contingencies we haven't thought of yet. But all the effort we put into abstracting the statement of the problem ends up costing us doubly: in the initial abstraction and in the difficulty of solution that results. So if you have a team that is solving a people problem, make sure to keep it a people problem when you give it to the team!
2. If you have a problem that is given to you in an abstract form, try to convert it to a people problem before trying to solve it.
In all likelihood, the moment you do the conversion, you will quickly see the solution. It may even feel like the process of de-abstraction is a problem-solving process. You may have to make really odd connections to make the problem a people problem but it will likely be worthwhile.
3. Dealing with people rather than abstractions on a day-to-day basis will always result in a more effective interaction.
Sending printed documents, writing emails, manipulating symbols are all interesting ways to communicate, but fundamentally, you are communicating with other people. If you can make that communication as direct as possible - phone, video conference, in-person - then there will be far less effort involved in understanding the communication, and far more effort can be allocated to high-bandwidth communication. This obviously has special relevence for teams: get people in the same room as much of the time as possible.
In the software world, there is one technique that I give teams and that is the use of Personas to assist in solving a software problem. The place I first encountered Personas is in the excellent book The Inmates Are Running the Asylum
The bare essense of the Persona is to create a fictional person who represents a user or actor or stakeholder or customer of whatever it is you are building. This fictional person should have a name and all conversation about the thing being built should be couched in the personal language of these Persona's names. A Persona should also have a short history, a photo and some description of their needs, goals or desires. All of this helps to frame everything about a software project as a people problem... and thus makes it much easier to discuss and solve.
Posted by Mishkin Berteig at 11:12 PM | |
April 21, 2006
Agile Adoption Stages for Teams
We know that teams go through identifiable stages of development: forming, storming, norming and performing (1). What exactly does this look like for an Agile team?
Forming
Here the team is typically innundated with three sources of new information: the Agile process and practices, the nature of the project and the other people in the team. This can be overwhelming and people will react in diverse ways: calm wait-and-see, rebelliousness, passive-aggressive, excitement, etc. If the team has an effective coach or mentor shepherding them through this, then feelings will tend towards excitement. The reality of learning so much at the same time will make the first few weeks of the team's time together quite exhausting. People will be actively fighting old habits, and people around the team will be asking lots of questions. Retrospectives will usually show that the team is impressed with their own teamwork and communication and will also show some disappointment with specific agile work practices.
Storming
After only one or two iterations, the team will transition into the Storming stage of development. Because Agile methods "front-load" the learning and the crisis, this forming stage comes fast, but it is also relatively mild. (Front-loading the learning means that all the problems that an organization has that hold it back from delivering quality work quickly are made visible in the first couple of iterations.) People are not used to a project being in crisis right at the start. It is critical for a coach or mentor or manager to be aware of this effect and expecting it. Again, for emphasis: an Agile project is in crisis immediately!... and this is perfectly normal and healthy. If the organization and the team are able to find means of dealing with this early crisis, then the project will continue and build larger and larger successes. On the other hand, if the organization or the team try to ignore or hide the problems, then very quickly work will revert to the old way: bureaucracy or chaos.
Norming
After about four to eight iterations, the team will reach a fairly comfortable place: the basic agile processes and practices are understood, the organization and the team have removed some basic obstacles to getting work done (and consciously left some obstacles in place in all likelihood), and everyone on the team has a basic level of comfort with their role. The challenge at this stage is to avoid falling into the trap of complacency. Although comfortable, this level of performance is probably not all that much better than the old way. There will be real advantages: regular delivery of work, good communication between stakeholders and the team. But there will be many obstacles still to be removed, and the team has a long way to go in its development. If the team becomes complacent, then it is critical that a catalyst be introduced to incite the team to further development. Often, this can be as simple as a systematic and intensive program of capability building. As team members learn and practice new skills: process skills, technical skills, people skills, strategic skills, business skills... and as they become more and more aware of each other's capabilities, they will also become more and more aware of areas for improvement. Incentives need to be provided to help team members focus dilligently on self-improvement and team improvement. The iteration retrospectives become critical to help with this process... the tricky bit is that this is the stage when people start to think the retrospectives are no longer necessary!
Performing
The transition into the Performing stage for an agile team is gradual and happens over a fairly extended period of time. The definition of "getting to done" will gradually expand to allow the team to go from zero to full delivery of the end results every single iteration. There will be a temptation to split up the team and use these experienced team members to seed new agile teams - resist this temptation! Breaking up the team at this point destroys the value of time and effort invested in the team. It is much more effective to start a new team from scratch. The essence of a performing agile team is not the transferrable knowledge about agile processes and practices. Rather, the most important result of the team-building process combined with the agile process is the team itself.
Posted by Mishkin Berteig at 11:57 PM | |
April 20, 2006
An Introduction to General Systems Thinking
I recently completed reading An Introduction to General Systems Thinking
The Subject
This book is about science, philosophy, behavior, organizations, organisms, problems, solutions, faith, reason, and everything in between. Specifically, it is about a general approach to dealing with systems given the limitations of our human abilities.
The Ideas
One of the strongest ideas in the whole book is that there is a class of systems for which we have only very poor tools to understand them. These systems, which he calls "organized complexity" are contrasted to "organized simplicity (machines)" and "unorganized complexity (aggregates)". Machines can be dealt with purely analytically and in a deterministic manner. Aggregates can be dealt with statistically. Systems that are in the organized complexity category are too complex for a purely analytical approach but too simple for a reasonable statistical analysis. The book is focused on methods for dealing with this class of systems.
The Writing
Mr. Weinberg's writing is, first and foremost, engaging. He writes in an informal voice that is a wonderful complement to the subject matter. Even with the informal tone, he is nevertheless able to communicate some tricky ideas with a great deal of precision and clarity. His use of examples, diagrams, stories and quotes throughout the book is excellent. Although I did not do the exercises he includes, upon reading them, I was satisfied to see that they are all interesting. Since I intend to re-read the book in six months or so, I will also publicly commit to doing a good chunck of the exercises too. Maybe you'll see some of my efforts here since many of them are apropos to Agile Work.
How Does This Apply to Agile Work?
Much of the emphasis in agile methods is on the intractability of building a perfect plan for a set of work (particularly in software projects). The group of human beings that are building something is in itself a system of "organized complexity". As a result, it is impossible to treat that group of people as a system that can be made to work in a deterministic manner. We simply can't account for all the variables. At the same time, we would like to have a lot more certainty about the behavior of the group than we could just using statistical methods. (Check out Systematic HR for some related articles.) Agile methods help us find this middle way that gives us a very good shot at reaching our goals, but acknowledges our inability to determine precisely how we'll get there. A good understanding of Systems Thinking helps us comprehend the necessity and applicability of agile methods.
The Table of Contents
The chapter and section titles of this book tell a great deal about the scope of the work. I have reproduced the table of contents here for your reference.
Chapter 1 The Problem
- The complexity of the World
- Mechanism and Mechanics
- The Square Law of Computation
- The Simplificiation of Science and the Science of Simplification
- Statistical Mechanics and the Law of Large Numbers
- The Law of Medium Numbers
Chapter 2 The Approach
- Organism, Analogy, and Vitalism
- The Scientist and His Categories
- The Main Article of General Systems Faith
- The Nature of General Systems Laws
- Varieties of Systems Thinking
Chapter 3 System and Illusion
- A System Is a Way of Looking at the World
- Absolute and Relative Thinking
- A System is a Set
- Observers and Observations
- The Principle of Indifference
Chapter 4 Interpreting Observations
- States
- The Eye-Brain Law
- The Generalized Thermodynamic Law
- Functional Notation and Reductionist Thought
- Incompleteness and Overcompleteness
- The Generalized Law of Complementarity
Chapter 5 Breaking Down Observations
- The Metaphors of Science
- Boundaries and Things
- Qualities and the Principle of Invariance
- Partitions
- The Strong Connection Law
Chapter 6 Describing Behavior
- Simulation - The White Box
- State Spaces
- Time as a Standard of Behavior
- Behavior in Open Systems
- The Principle of Indeterminability
Chapter 7 Some Systems Questions
- The Systems Triumvirate
- Stability
- Survival
- Identity
- Regulation and Adaptation
- The Used Car Law
Also, check it out, Mr. Weinberg has started a blog on writing fiction. He also helps run the AYE Conference (amplifying your effectiveness).
Posted by Mishkin Berteig at 05:18 PM | |
April 12, 2006
Follow the Principles and Adjust the Practices
In "Built to Last : Successful Habits of Visionary Companies
Follow the Principles
What exactly are the principles? The foundation starts with Trust and Truthfulness. "Truthfulness is the foundation of all human virtues." Everything we do with agile should be about truthfulness (visibility, transparency) and building trust.
With this as a strong foundation, we can look at the Agile Axioms:
We are Creators
Reality is Perceived
Change is Natural
All of the other principles and practices associated with Agile Work flow from these basic assumptions about the world. We can't prove that the above three axioms are "true". But they either resonate with us or they don't. If they do, then it will be easy to use these axioms as a checkpoint for all the activities we engage in using Agile Work, wherever we apply it.
We are creators... therefore we derive our sense of value from our ability to create. If our creations are accepted by others, our team, our stakeholders or our community, all the better. But fundamentally, this is inherent to us as human beings. However, sometimes this natural drive is suppressed or repressed. In order to activate it, we need to work in empowered teams.
Have you ever experienced inspiration or "flow" or joy when working with someone else? Perhaps you were solving a problem. Perhaps you were playing a musical instrument - jamming - and got into a fertile groove. Perhaps you were teaching your children and created the light of understanding in them. Perhaps you built a beautiful set of bookshelves for your home. Or maybe you told a joke that created a brief moment of genuine levity in a group of friends. We are all constantly creating!
This basic principle then means that Agile Work methods and practices should not be imposed. Taught to us, perhaps... given to us as a template, perhaps... but once we understand the practices and are familiar with them, we should immediately be given the freedom to use the learning cycle to be creative with the process and practices of Agile Work itself. If we do not participate in creation, we become dis-empowered and that eventually leads to resentment or apathy.
Reality is perceived... therefore we need to work hard to build a common perception of reality if we are to work together effectively. We need to amplify our learning. We can't assume that our own understanding of a situation is going to be shared by others. At the very least we need to check: "do you see this?"
Let's recognize that in some way or another we are all blind:
Again, the learning cycle comes into play. The guidance, detachment, love, courage and search we go through all help us to build a common understanding of reality. This allows us to see new ways to apply the Agile Work principles and practices that make sense not only to our context, but also to everyone else participating in the work.
Change is natural... therefore instead of fighting change, we need to anticipate it, adjust to it, embrace it, and be gracious or even enthusiastic. Not only does change happen to us, but we also instigate change. If things get to boring, whatever the circumstance, we find ways to change things. We rebel at stasis and ennui.
Each practice and procedure done in the context of Agile Work must be explicity and implicitly accomodating of change. If a procedure can't tolerate change it will either lead to a dissonance or conflict... or if we are embracing change, then we will modify or discard the procedure. Our creative nature loves to create, but if we become too attached, too "in love" with our creations, we will support them past their point of relevance.
Our latest greatest idea will be good for a while. But eventually change will make it irrelevent.
So we see that all three Agile Axioms are also interrelated.
Our creations will be washed away through change and if we are lucky or wise we will perceive the change in reality - be truthful to ourselves and others - and allow a new creation to take the place of the old one.
When we perceive a certain truth, and try to share that with others, we will be asking those others to change their own perceptions. This change can be difficult and may even require the destruction of a mental model created with love and care over a lifetime. Sensitivity to this loss and encouragement to build a new creation will help build a shared perception... as long as we too are open to new perceptions!
Adjust the Practices
And of course, all this foundation of creation, perception and change must be connected to the practical day-to-day reality of our lives. Our family lives, our work lives, our social lives, our volunteer lives, our intellectual lives, our emotional lives, our spiritual lives... our whole lives.
The Agile Work practices are simple to state:
Manage Ourselves
Deliver Frequently
Adapt our Plans
Communicate Powerfully
Test Everything
Measure Value
Remove Obstacles
These practices provide a starting point. A basic set of activities that will assist you, your team or your organization to advance quickly towards whatever goal you have set for yourselves. The way these practices succeed is by making sure that the Agile Axioms are always remembered and their implications accepted. These practices will set up a virtuous circle by building trust and allowing truthfulness. More trust and truthfulness will allow a fuller and more nuanced expression of the practices...
But if these practices become canonized, if they become a rote process imposed and followed blindly, then it means that we have lost sight of the Axioms. We have forgotten to check our practices against the context of creation, perception and change.
The reason we follow these practices is because we believe that we are all creators, that we can learn from our diverse perception of reality and that change is a force of growth. We don't believe these Axioms because we blindly perform these practices.
This is all available as a nicely formatted pdf: Agile Axioms - a Brief Exposition.
Posted by Mishkin Berteig at 01:45 AM | |
April 05, 2006
A Simple Analogy for a Simple Process
Mike Dwyer has written up a very nice analogy between the simplicity of the Scrum method and the simplicity of the waltz.
I used the waltz analogy a while ago in Scrum development, partially because I was looking at Tim Lister on the cover of Waltzing with Bears and partially because it is such a simple set of rule to dance by and mostly because of the last chick flick I was noble enough not to grab the switcher on was all about ball room dancing.Now wouldn't you know that the two remaining grey cells in my head bumped into a synapse - BAM Headache!
First for all of you who escaped taking dancing lessons the waltz is an extraordinarily simple dance. 1,2,3 1,2,3 right foot, bring left foot to right foot, one step to right. Bring left foot to right. Repeat.
This means Tim and the Bear can do it, rhythmically challenged people men like me can too and so can the folks in Ballroom Dancing. All are called waltzing – except; One is what you do to read about risk, one is beaten into you at dance class, and one is a form of competition that combines Olympic caliber training with the clothes racks from Vegas.
What makes them all the waltz? 1,2,3 1,2,3 right foot, bring left foot to right foot, one step to right. Bring left foot to right.
What makes Jeff's, or Glen's, or my use of the same principles of classic Scrum not Scrum? product backlog sprint backlog, product owner defined DONE. Timeboxed iterations, Daily Update, Impediment resolution, we all have a product backlog, we all have a product owner driven way to select work for a timebox, and we all communicate daily the critical information to each other and those that are concerned. Most of all we all measure by a simple DONE or NOT based on Product Owner acceptance.
Break anyone of these simple rules and you are not just failing at Scrum you are failing period. Goes for waltzing as well as Scrum.
Oh yes there is one more thing that waltzing and Scrum have and that is a rhythm capable of maintaining the dance at different tempos for different levels of skill. This way everyone enjoys what they are doing, does the job to the expectations of the Product Owner, and has the opportunity to improve.
Posted by Mishkin Berteig at 11:34 AM | |
April 04, 2006
Connecting Vocabularies - Cycles of the Mind
At the Coach's meeting ten days ago, several of the attendees used a series of phrases to refer to the learning process or cycle that Agile Work promotes. These vocabularies all have a different slant or implication but they all map to each other fairly well.
I have created a basic graphical representation of the relationships between the vocabularies.
The first is from Garry Berteig's Learning Circle. The second is Boyd's OODA Cycle. The third is attributed to be the scientific method. The fourth is from a Persian philosopher and teacher 'Abdu'l-Baha. The fifth is from Ken Schwaber's Scrum methodology.
If you have heard of other vocabularies for describing this cycle of learning I would love to hear about them. Vocabularies from the sciences, social sciences, humanities, arts, etc. would all be welcome.
I find that these various vocabularies are useful for different circumstances, but also for people who maybe don't "get it" with one vocabulary will "get it" with another. Our affinity to various words and the connections they make for us can be very powerful.
Posted by Mishkin Berteig at 11:54 AM | |
March 30, 2006
Bombs and Agile
The coach's gathering last weekend also got me thinking about the ethics of Agile Work and coaching. Is it okay to use agile methods for destructive purposes?
Let's first look at the Agile Software Manifesto for guidance. We see four statements of values and a number of principles. None of them provide an ethical framework that helps us determine where to use agile methods. In fact, there are many types of work that we could ask an equivalent set of questions about:
Is it okay to use agile methods to assist research in bio-weapons?
Is it okay to use agile methods to build software systems for nuclear missiles?
Is it okay to use agile methods to run a hate campaign?
Is it okay to use agile methods to ... ?
The agile community lacks a statement of ethics equivalent to the Hippocratic Oath. Do we even need one? As coaches, should our Middle Way to Excellence be grounded in a strong moral sense or is the middle way adrift?
I feel like we need a moral grounding. I think that the basis of it should be the recognition of the Unity of Humanity. I believe that both justice and mercy are important. Trust and Truthfulness are part of the foundation as well.
Is there any way to state a professional creed for Agile coaches that we can all agree upon? Has anyone tried?
For what it's worth, the ICF has a code of ethics that might be a starting point.
Posted by Mishkin Berteig at 01:04 PM | |
March 20, 2006
Methods of Prioritization
In Jean Tabaka's new book, "Collaboration Explained : Facilitation Skills for Software Project Leaders
1) Round the Group prioritization:
group size 3-8
item list size < 15
Items are written on cards and placed in random order linearly either vertically or horizontally.
The members of the group each take turns placing the items in the order they think is the proper priority order. While doing so, each person moving the cards is welcome to explain their reasoning. However, the other group members refrain from commenting on the new prioritization.
This continues around the group as many times as it takes to find a stable order.
2) Ping Pong Balls:
group size 1-12
item list size > 15
(Thanks to Ken Schwaber for this method)
A fixed number of ping pong ball units are given to the group. The ping pong balls represent units of one dimension for prioritization such as value, risk or cost.
The group discusses how to allocate ping pong balls to each item in a dynamic fashion until everyone agrees that the allocation makes sense.
For very large lists, this is easiest to do in a spreadsheet with fewer people involved.
3) Variation: 2-stage multi-voting with voter freedom
group size 5-20
item list size < 50
This is identical to the public multi-voting system Jean describes with the following changes:
First, there is no restriction on how votes are allocated to items. A person can put multiple votes on a single item and can withhold some or all of her votes.
Second, after everyone has "finished" voting, the facilitator calls for everyone to step back and think about the results. Some discussion is allowed about the consequences of the results. Finally, everyone is given an opportunity to move their votes.
For large groups with large lists this can be somewhat awkward as people might forget where they voted. In this case, and if anonymity is not required, each person can use small post-it tabs with an identifying mark on them so that they can easily be moved around in the second stage.
Posted by Mishkin Berteig at 01:25 PM | |
March 14, 2006
Agile or Not Agile?
Every once in a while the del.icio.us tag for Agile turns up something really interesting. This evening, I found this article about the ongoing use of the term "Agile". The article is brief and a little weak, but it brings up a concern that is always niggling in the back of my mind. Interestingly enough, a good friend of mine, Christian Gruber, emailed me another web page of similar import...
In this registration page for Rules of Enterprise Agility, we read about something that really has nothing to do with the Agile Manifesto, nor the Agile Axioms.
Both of these examples are signs of two things:
1. The growing popularity of the term "Agile".
2. The growing dilution of the meaning of the term.
How can we fight this? Should we fight this?
I think it is very important to constantly call attention to the fact that Agile is about the minimum process and tools that can possibly work, and only in the context of valuing individuals, interactions and teams more than those tools and processes.
Trust is the Foundation of Agile Work
Technology, tools, process, even good ideas and good organizations do not create trust. People create trust by being trustworthy: honoring their commitments, striving for excellence, truthfulness, courage. One of the fundamental problems afflicting organizations is the lack of trust: between management and employees, between business and IT, between experts of various sorts, between coworkers.
This lack of trust is institutionalized in many ways including bureaucracy and legal frameworks.
The only way to change this state of affairs is to build trust. And the only way to build trust is to embody trustworthiness in yourself so that by example and by your words you can help others to become more trustworthy.
Agile methods put in place mechanisms that assist in building trust. But those mechanisms are merely a means to an end. Let us never forget that.
Posted by Mishkin Berteig at 12:27 AM | |
March 12, 2006
Work Item Backlogs as Queues - Agile vs. Lean
A recent discussion on the Scrum Development list (Start of Discussion) provides a good follow up to my parting words in The Art of Obstacle Removal about agile practices themselves becoming obstacles. I have excerpted a small amount of the discussion and added my own comments here.
In the agile community, most of us have bought into and adopted the Agile mental model. But that mental model has assumptions embedded into that may not be correct under all circumstances. Part of that mental model includes the efficacy of the work item backlog as a tool for tracking, prioritizing and communicating work to be done in the future.
I will be the first to say that Agile is far better than waterfall in almost every situation (the exception being the mythical project with stable requirements, business environment, technology and team).
That said, let's look at backlogs from another perspective: if a backlog was the only thing you delivered to the customer, would they pay for it? If you spent even just a few hours coaching a business representative to build a backlog, but there was no team to implment it, would that person really find value in the backlog itself? Would they be able to deliver ROI just from a backlog? I think the answer is a clear "no".
That set of questions may seem silly, but from a lean perspective, they are among the most important questions. Is an artifact/process value-added or non-value-added?
Since the backlog is clearly not a value added artifact/process (pause for effect)... it is waste!
When one is doing agile effectively, the backlog may in fact be one of the larger sources of waste! If the team has a stable velocity, there will come a point when the backlog becomes the constraint on the overall cycle time going from idea to ROI. If the backlog is large/long, and as Mary Poppendieck said if there is more than ...maybe two or three
sprints of work.... in there, then it may be time to find ways of constraining the size of the backlog. I have two suggestions:
Capacity of Team(s):
Find a way to increase the capacity of the team so that the backlog size reduces and then goes into a steady state. This may mean augmenting the staff.
Gate Backlog Items by ROI
(This is just theory to me at this point.) Make it a condition that all backlog items must have a positive ROI. In other words, the cost of building the backlog item needs to be less than the value delivered, taking into account time value of money. Don't let items onto the backlog unless this positive ROI can be demonstrated.
I believe the lack of this second discipline (assigning ROI to work items) is one of the main reasons that most agile methods such as Scrum allow an unlimited backlog size: there is no realistic way to gate the work that gets onto the backlog!
Until teams get to Agile nirvana (stable velocity, continuous delivery of business value, high-performance cohesive teams), having an unlimited backlog seems like a very reasonable thing: it's not the constraint or the primary source of waste. However, eventually the backlog will become a primary source of waste and it needs to be treated in a disciplined manner.
With a stable and well-functioning team, what other ways might there be to reduce the size of the backlog so that cycle time is substantially reduced?
Posted by Mishkin Berteig at 03:31 PM | |
March 10, 2006
The Art of Obstacle Removal
One of the best ways to go faster is to remove the things that slow you down. This "obstacle removal" is an integral part of many agile methods including Scrum and Lean. Sometimes it is obvious where an obstacle is. There are a few small things that can be done easily to go faster. But to get going really fast, we need to have a deeper understanding of obstacles... and the Art of Obstacle Removal.
What are Obstacles?
An obstacle is any behavior, physical arrangement, procedure or checkpoint that makes getting work done slower without adding any actual contribution to the work. Activities that do add value to our work may be slowed down by obstacles, but are not obstacles in and of themselves.
Obstacles and Waste
Obstacles are the causes of waste in a process. There are many types of waste, and for every type of waste there are many possible sources (obstacles).
Types of Obstacles
Personal
Personal obstacles are related to us as individuals. There are several levels at which these obstacles can show up.
Outside factors in our lives such as illness or family obligations can become obstacles to our work at hand. These obstacles are hard to remove or avoid. Even if we would want to avoid an obstacle such as illness, it is hard to do anything about it in an immediate sense. However, as part of our committment to the group we are working with, we should consider doing things to generally improve our health. Good sleep, healthy and moderate eating, exercise and avoidance of illness-causing things and circumstances are all possible commitments we can make to the group. Likewise, we can make sure our personal affairs are in order so that unexpected events have the least impact possible. This topic is vast and there are many good sources of information.
Physical Environment
Obstacles in the physical environment can consist of barriers to movement or communication, or a lack of adequate physical resources. Sometimes these obstacles are easy to see because their effects are immediate. For example, if a team room lacks a whiteboard for diagrams, keeping notes, etc., then the team may not be able to communicate as effectively.
Other physical obstacles are not so obvious. The effects of physical environment can be subtle and not well-understood. Poor ergonomics take weeks, months or years for their effects to be felt... but it is inevitable. A too-small team room can lead to a feeling of being cooped up and desperation to get out... and eventually to resentment. Again this can take weeks or months.
Here are some guidelines on a good team room.
Knowledge
A lack of knowledge or the inability to access information are obstacles. A team composed of junior people who don't have diverse experience and who don't have a good knowledge of the work they are doing will have trouble working effectively. There may be barriers preventing the team from learning. Common barriers include over-work leading to a lack of time or mental energy for learning. With junior people in particular, there is a lot of pressure to be productive and that can often be at the expense of a solid foundation of learning.
Other times, knowledge-related barriers can be more immediate. If a critical piece of information is delayed or lost this can have a large impact on an Agile team that is working in short cycles. The team may be temporarily halted while they wait for information. Building effective information flow is critical to a team's performance.
Organizational
One of the best sources of information about this is the two books by Jim Collins: "Good to Great" (Review) and "Built to Last" (Review).
Cultural
In organizational culture, one thing I constantly see is a public espoused value of teamwork, but a conflicting behavior of individual performance reviews and ranking. This is cultural. It is also a barrier to the effective functioning of an Agile team. For corporate environments I highly recommend the Corporate Culture Survival Guide by Edgar Schein.
Dis-Unity
Dis-unity is one of the most subtle and common forms of obstacle. Competition, legal and cultural assumption of the goodness of "opposition" and habits of interaction including gossip and backbiting all combine to make united action and thought very difficult.
This is an extremely deep topic. There are many tools and techniques available to assist with team building. If you are interested in this topic, I highly recommend reading "The Prosperity of Humankind".
Removing Obstacles
The ability to identify obstacles and understand why they are causing problems is only the first step in removing obstacles. In Agile Work, the person primarily responsible for identifying and removing obstacles is the Process Facilitator. The Process Facilitator has several approaches available for the removal of obstacles. A process facilitator has similar responsibilities to a change agent.
Direct
Deal with the obstacle directly without involving other people. This can be as simple as getting up and moving an obstacle impairing vision, or as nuanced as running interviews and workshops throughout an organization to gradually change a cultural obstacle.
Command and Control
Identify the obstacle and give precise instructions for its removal to a person who will directly perform the removal. This can sometimes work if removing an obstacle takes a great deal of time, effort or specialized skills that you yourself do not posess. However, the overall approach of "command and control" is not recommended for Agile environments since it is disempowering.
Influence
Identify the obstacle and suggest means to deal with it to a person who has the authority or influence to get others to deal with it. This indirect method of obstacle removal can be slow and frustrating. However it usually has better long-term effects than command and control.
Support
Offer to assist and encourage the removal of obstacles that have been identified by other people. In many respects this is a very effective method. It can assist with team-building and learning by example. People are usually grateful for assistance.
Coaching
Train others on the art of obstacle removal including obstacle identification, types of obstacles and strategies for dealing with obstacles. Observe people's attempts to remove obstacles and give them feedback on their actions.
Creating a Culture of Obstacle Removal
Encourage and measure obstacle removal at all organizational levels until it becomes habitual. In many ways this is the essense of the lean organization.
Strategies for Dealing with Obstacles
Diagrams are a great way of communicating the essense of a concept. Feel free to share the following diagrams with anyone (but of course keep the copyright notice on them).
Remove
Remove the obstacle altogether. This method of dealing with an obstacle is usually the most immediately effective, but is also one of the most difficult methods.
The best way to actually remove an obstacle is to get at the root cause of the obstacle and change that. This type of change results in the longest-lasting and most stable elimination of an obstacle.
Move Aside
Take the obstacle and put it in a place or situation where it is no longer in the path of the team.
In a team's physical environment, this may be as simple as changing the tools that the team is using. For example, if the team is all in a room together, move computer monitors that are blocking team member's views of each other. If there is a useless checkpoint that work results have to go through, get management to eliminate it.
Shield
Build a shield or barrier to hide the obstacle so that it's effects no longer touch your team.
If a team is distracted by noisy neighbors, put up a sound barrier. If a team is unable to see their computers due to late afternoon sunlight, put up window shades. If a manager is bothering the team with meetings or tasks unrelated to the work of the team, then put yourself between the team and the manager (or get someone in upper management to do that).
Shielding is excellent for immediate relief, but remember that the obstacle is still there and may become a problem again if the shield cannot be maintained.
Transform
Change the structure or form of the obstacle so that it no longer affects effectiveness.
In general, this method requires a great deal of creativity and open-mindedness. This is one that works particularly well on people who are obstacles: convert them into friends of the team!
For example if the team needs approval of an expert who is not part of the team, this can cause extra work preparing documentation for this person and long delays while the expert revies the documents. If the expert becomes part of the team, then they are well-informed of the work being done and can give approval with very little overhead.
If done well, this can be a very long-lasting method of dealing with an obstacle. Make sure that the transformation is true and that it takes hold... and beware that the obstacle doesn't revert back to its old nature.
Counteract
Find an activity that negates the effects of the obstacle by boosting effectiveness in another area.
As a coach or Process Facilitator, this is what we spend our time in early in a team's adoption of Agile Work: we get them to work in the same room, use iterations and adaptive planning, we focus them on delivering work valued by the stakeholders as defined by the Product Owner. All these things are enhancing the team's ability to get work done without actually directly dealing with any obstacles.
Watch out for barriers avoided this way to come back and bite you later on.
Removing Obstacles and Learning
Organizational learning, as well as adult learning have a strong relationship to obstacle removal. Organizational learning can be either single-loop or double-loop learning. Adult learning can be either normal or transformative. We can approach obstacle removal from a surface level where we only deal with the immediate symptom, or we can work at a deeper level where we deal with the symptom and its chain of preceding causes. One effective method for examining the deeper causes is the 5-why's exercise.
Obstacles Inherent in Agile
Agile methods do not perfectly eliminate all obstacles. Some obstacles that are inherent in agile methods include overhead due to planning meetings at the start of iterations, the use of a dedicated process facilitator. As well, the use of iterations can become a barrier to certain types of work items: repeating items, investment in infrastructure, one-off tasks that are not directly related to the work at hand.
At some point, our teams will have matured to the point where agile methods are no longer necessary and we can pick and choose what parts of agile we use.
Go Forth and Demolish Obstacles!
As a Process Facilitator, coach, ScrumMaster, manager, change agent or stealth agile advocate, you have the ability and the knowledge to make a big difference in people's lives and in the success of the organizations they work within. Removing obstacles is one of the most important duties you have.
Do you have stories about obstacles you have removed or seen removed that have made a big difference? We would love the hear the anecdotal side of this as well!
Posted by Mishkin Berteig at 01:10 PM | |
March 02, 2006
Five Signs of Trouble in an Iteration
During the course of an iteration, an agile team is able to track it's own progress through the use of burndown charts. The team and the process facilitator can use the burndown chart to watch for signs of trouble. As a coach, I find the following five burndown shapes are common indicators of trouble.
But let's first show the idealized "normal" burndown. This burndown shape shows that the team is on-track to get to done exactly at the end of the iteration. The work remaining has had a constant slope down through the course of the iteration.
Now the warning signs:
1. Too Much Work
In this burndown, the team sees that it has likely selected too much work for the iteration. The process facilitator and the product owner need to work with the team to decide how to change the scope so that the team can get done. One might be tempted to add people to the team to get the work done faster, but this is rarely successful.
2. Not Getting Done
Frequently, when a team is just starting out with agile work, it commits to slightly too much work. It is hard to detect this at the beginning of an iteration. Instead, it is only near the end of the iteration that it becomes apparent that the team isn't quite going to make it to done. The process facilitator must work with the team to determine the root causes of this and change things so it doesn't happen again.
3. Early Learning
When a team is starting the iteration, it can discover new things about the work it has committed to accomplishing. This can include dependencies, new tasks, extra components that need to be built, etc. This discovery process is normal, but needs to be monitored closely. If the burndown doesn't start going down again after about 15% of the iteration is over, then there is likely trouble brewing.
4. Unresolved Obstacles
Sometimes the team will run into an obstacle that will completely stop their progress. Although the process facilitator is responsible for dealing with obstacles, it may be impossible for an obstacle to be fixed quickly. If the team finds that all the work it committed to for the iteration is dependent on someone who is sick or on vacation, that can lead to an unresolvable dead halt. This may be an opportunity to cancel the iteration.
One interesting example of this was observed by a coach I work with frequently, Deborah Hartmann. She was away from her team for a few days and when she came back, she observed this "flatline" burndown. After poking around a little bit to try and find the obstacle, someone finally described what had happened. At the time of the flatline, a Vice President in the organization had come into the team's area, looked around, and loudly declared something to the effect of, "I don't know why you guys are doing this project. It's totally worthless!" Suffice it to say that the team was seriously demoralized. It wasn't a technical obstacle, it wasn't a procedural or bureaucratic obstacle, it wasn't even a cultural obstacle. It was just a serious blow to morale. Of course, to fix this, the actual sponsor of the project had to be brought in to assure the team that it was actually an important project and that there were people in the organization who needed the work that the team was doing.
5. Scope Creep
This last case is rarer for an agile team if the team and the product owner have been educated well about the rules. Nevertheless, it can happen. The product owner, or some other stakeholder, pushes the team to add scope to the iteration. The process facilitator is normally responsible for preventing this from happening. If despite this it does happen, the burndown chart makes the consequences of this scope creep.
Special Quiz Section!!!
What are the possible explanations for the following burndown shape? I have heard/experienced at least six different possible reasons. Leave your answers in the comments.
Update: at Agile Chronicles, there is a short article about this topic which mentions one more burndown chart pattern that is a bad sign: the "Late-Breaking Decline".
Posted by Mishkin Berteig at 01:34 PM | |
February 22, 2006
Agile Work Uses Lean Thinking - Whitepaper Published
Berteig Consulting Inc. has published a whitepaper about some of the relationships between Lean and Agile. The paper is based on three prior entries here on Agile Advice relating to Queuing Theory, Empirical Process Control and Self-Organizing Teams.
For more in-depth reading, I highly recommend the following three books:
This gives a great introduction to Lean thinking applied to software and specifically agile software development.
2. Agile Project Management with Scrum by Ken Schwaber
This books provides some excellent background on empirical process control and how it relates to agile project management.
3. The Goal by Eliyahu Goldratt
A business novel chronicling the experiences of a manufacturing plant manager learning about the Theory of Constraints (closely related to Lean).
Posted by Mishkin Berteig at 01:13 PM | |
December 10, 2005
The Toyota Production System
Here's a good article about the Toyota Production System (TPS). Agile work takes many of the ideas here and adapts them to a non-manufacturing situation. There's an interesting comment about the "5 Whys"....
From the linked article:
When an error occurs, the first thing that needs to be done is fix the error. Minoura recalls that Ohno used to order them to ask the question "Why?" five times over because "that way you'll find the root cause, and if you get rid of that it'll never happen again." However, Minoura emphasizes that on-the-spot observation rather than deduction is the only correct way to answer a "Why?" question. "I'm always struck that the five-why method doesn't seem to be working as well as it should be because there's been a lack of practical training. The reason is that they end up falling back on deduction. Yes, deduction. So when I ask them 'Why?' they reel off five causes as quick as a flash by deduction. Then I ask them five whys again for each of the causes they came up with. The result is that they start falling back on deduction again, and so many causes come back that you end up totally confused as to which of them is important."
Posted by Mishkin Berteig at 12:00 AM | |
November 22, 2005
Cynicism, Apathy and Agile
I am currently one of those consultants working in a large organization that is trying to implement Agile.
Without a doubt there is huge, mostly unconscious, resistance to the cultural change that Agile requires of a highly-regulated command and control organization. Yet even with that, introducing a few simple agile practices in a reasonable fashion such as iterative delivery, daily status for self-organization, colocated teams and a few information radiators has made a huge difference in teams' ability to deliver and in their satisfaction with their work. The jury is still out about this change at this organization: will it be permanent? or will it be another fad? or will it remain a weak version of agile?
Like almost any idealistic movement, Agile cannot succeed in the short term. People's hearts need to be transformed, their habits changed and their thoughts changed. This can happen quickly for some, but for most it is a long process often preceded by the pain of doing things badly. For the most part, Agile is a movement that is being driven by the grassroots: folks in IT who see it as the better way. However, even if every programmer and project manager in an organization decideds that Agile is the way to go, it could still fail before it wins. The corporate culture change required to fully adopt agile can take years and years and can be scuttled at any time by the whim of those in power. If that happens, some of those folks who wanted Agile will become cynical and immunized to the idealistic pull of Agile.
The faster we go, the less patience we have, the more people will become immunized.
The attraction of the financial side of things for consulting, training, publishing, all threaten to poison our community. In the Baha'i Community, to which I belong, our religious laws prohibit "professional" Baha'is. The mix of money with an ideal would destroy the goals of our community: peace, unity. Of course, that means it takes hundreds of years if not thousands to build our capacity, to demonstrate to others the bounties we have to offer.
In the Agile community, like in many other places, there is no sense that this is a movement to span hundreds of years, nor even decades. It is a young set of ideas and has only been visible for six or seven years. We don't have the patience to wait decades, partly because we are excited, but also partly because we all need to earn a living and I don't think any of us really want to be economic martyrs to the cause of Agile. Nevertheless, sometimes we do have to have courage and walk away from a bad deal.
Thanks for Tobias Mayer for a message on the ScrumDevelopment Yahoo! group. That message inspired this one.
Posted by Mishkin Berteig at 11:43 PM | |
November 14, 2005
Agile Work Axioms - Discussion
It's taken a while to get the phrasing for the Agile Work Axioms to the point that it is currently at. I've had personal conversations, one-on-one with a few people that I trust in order to see if what I have written makes sense. But the time has come to ask you, dear reader, for help.
I am trying to formulate a really simple way of describing the underlying assumptions and beliefs about life, the universe and everything that are driving the success of Agile. Right now I have this:
Agile Work Axioms:
We are Creators
Reality is Perceived
Change is Natural
I've written just a bit more than that and it can be found at the Agile Work Axioms web site.
What do you think? Do these relate well to what we find in the Agile Software Development Manifesto (notice the "software development" stuck in there... I'm hoping the Agile Work Axioms apply to other types of work besides software development)? Is there anything missing? Do these make sense? Why?
Posted by Mishkin Berteig at 02:11 AM | |
November 09, 2005
Agile Work Uses Lean Thinking - Empirical Process Control
This is Part 2 of a 3 part series.
Part 1
Part 3 - not posted yet :-)
Some work processes cannot be perfectly controlled nor perfectly defined. There may be non-linear interactions between steps in a process or there may be creative input from a human required. Processes with these qualities require empirical process control.
The basic attribute of empirical process control constitutes a continuous cycle of inspecting the process for correct operation and results and adapting the process as needed. A simple example of this is detecting impending failure of equipment by constantly monitoring the operation of that equipment. For Agile Work, the book Agile Software Development with Scrum provides an excellent chapter about this topic of Empirical Process Control.
In human processes like those to which Agile Work applies, the frequency of inspecting and adapting must match the needs of the process. Many projects occur in the context of constant change. This constant change makes long-term planning a wasteful effort. Rather, short-term planning with constant feedback provides a simple inspect and adapt cycle. This cycle can play out at different levels: daily for a team, monthly for a client of the team. The team inspects and adapts daily at the level of the tasks that it is performing. The client inspects and adapts monthly at the level of the team's actual delivered results.
Both lean and agile methods claim to increase both speed and quality. Many people believe that there are four constraints in a system that can be controlled: speed (or schedule, or time to market, or process cycle time), quality (number of defects), scope (how much functionality), and cost savings (how much to spend on the work). Frequently, management believes that one has to trade off between these four constraints; spend more money, get more scope; lower quality, go faster. But in fact, lean and agile strongly support the idea that as you increase quality, you also increase speed... you just have to do it right.
In Agile Work, increasing speed and quality is done in three ways. First, increase the frequency and quality of communication among team members so that errors are detected early or avoided altogether. Second, drive the work with the creation and execution of automated testing. No work is done without a test in place to check if it is done correctly. This constant testing means that work is always defect-free and therefore very little time/money is spent on fixing defects. Third, eliminate wasteful work steps or obstacles to performance of work. This last one is difficult to do an bears closer examination.
Wasteful work is done in every process, no matter how efficient. Lean tells us that there are several types of waste in a manufacturing process. Those types of waste have analogies in Agile Work. For example, documenting something you plan to do instead of just doing it is wasteful. Another example is waiting while someone completes work that you depend upon. Any step or task that does not add value to the final product of an effort is waste. This standard is very high and most organizations have about 80% of their efforts going into wasteful tasks. An organization that has done an initial cut of wasteful work might stand at about 50% waste. The leanest organizations, such as Toyota, stand at about 20% waste.
Agile work eliminates waste in the form of barriers or obstacles that come up when a team is trying to go fast. Sometimes this is in the form of waiting for another group to do something for the agile team... an outsourced request for service. Sometimes waste is in the form of corporate standards or policies around documentation of work. The Process Facilitator role in an agile team has responsibility for working with the team and others to help overcome these obstacles.
Posted by Mishkin Berteig at 02:08 PM | |
October 11, 2005
Is There a Single "Most Important" Agile Work Practice?
There are a few times that I have been involved with implementing agile pratices without management knowledge or direct support. In these cases it has usually been necessary to gradually introduce the practices. An unsupportive or apathetic environment cannot be changed instantly and big-bang introduction of agile tends to bring too much negative attention too quickly.
In reflecting on those experiences, as well as "normal" agile implementations, I have felt that there are some specific practices that can stand alone.
Self-Organizing Teams
The practice of a self-organizing team consists of frequent regular status meetings, face-to-face, reporting to the other team members accomplishments, work commitments and obstacles. Scrum has a very strict method of doing this on a daily basis but I have found it valuable to do more or less frequently depending on the team and its environment (generally any less than every second day is not enough). The team, or some assistant of some sort, tracks the barriers and works to resolve them quickly. Management, if it exists, must be contacted through trusted channels to assist with the removal of barriers. And stakeholders must be able to attend the status meetings or receive reports immediately after the meetings.
This single practice tends to have the ability to bootstrap the others. The identification and clearing of barriers provides a way for the team to practice all three Agile Work Disciplines (Empower the Team, Amplify Learning, Eliminate Waste). Reporting accomplishments to the other team members Amplifies Learning. Committing to work is empowering.
Some teams have done only this single agile practice and seen great improvements in productivity, morale, and stakeholder satisfaction. However, there are some pitfalls that must be acknowledged and dealt with.
Pitfall: Speculative Work
The team can tend towards speculative work if there is no strong representative of the stakeholders. This does not always happen since most people are sincere in their desire to "make a difference". However, if as a team you adopt only this practice and find yourselves doing lots of "what-if?" or "wouldn't it be neet if..." or "what exactly is our purpose?" discussions, then you need to find some external stakeholder support for your effort.
Pitfall: Failing to Deliver
In many organizations, failure to deliver is an endemic problem and a self-organizing team will break through and start delivering. However, failure to deliver can also become a cultural mindset for an organization or group. A self-organizing team must maintain a goal (not a plan) for itself, and that goal must include delivering something valuable. Again, finding an external stakeholder to support the team's efforts can help to avoid this pitfall.
Pitfall: No Barriers
Sometimes a team will get into a habit where no new barriers are being exposed. This can often happen when the progress in the work becomes steady and is recognizably better than it was before. The team falls into a "local optimum". In this case, the team needs a fresh way to view their work. This can happen in a number of ways: a crisis, an external observer, or a change in environment among others.
...
Do you have experience with successful but incomplete agile implementations? I would love to hear of other experiences and opinions about this.
Posted by Mishkin Berteig at 02:42 PM | |
October 10, 2005
Agile, the Adult Educator and Ethical Considerations
A review of Tara J. Fenwick's “Limits of the Learning Organization: A Critical Look” (article found in Learning for life: Canadian readings in adult education
This article is a critique of learning organization literature (as presented in the works of Peters, Senge, Watkins, Marsick, Argyris, Schon and others). I chose to do a review of it because learning organization literature can and does inform the work of Agile practitioners. The writer, Tara Fenwick, offers a critique of this literature as an academic and practitioner in the field of adult education. Even though the language and tone of the article is judgmental and at times affronting to the corporate trainer audience, it is never-the-less challenging and valuable because she raises interesting ethical questions that can serve as cautions against potential trends that can distort agile practice. I will summarize her argument in the some of the areas most relevant to Agile practice.
Fenwick's summary of the model of learning organization found in the literature, is an organization that: “creates continuous learning opportunities, promotes inquiry and dialog, encourages collaboration and team learning, establishes systems to capture and share learning, empowers people toward collective vision and connects the organization to its environment.”
The following is a summary list of some of Fenwick's critiques:
Who's Interests are Served
Although the learning organization literature holds great promise for a more humanitarian and egalitarian workplace, it has the potential to distort learning “into a tool for competitive advantage” and in turn, exploit people as resources in the pursuit of profit. To explore this idea she asks a valuable question: “Who's interests are being served by the concept of learning organization, and what relations of power does it help to secure?” She argues that learning organization literature tends to serve the interests of educators working as trainers in organizations and managers interested in their own self preservation.
How Learning is Defined
Learning, in learning organization literature seems to be defined as that which benefits the organization, all other learning falls into the dysfunctional category. This perspective negates other ways that people create meaning and learn and potentially causes a person to become “alienated from their own meaning and block flourishing of this learning into something to benefit the community.”
Assumptions about Learners
The learning organization literature subordinates employees by seeing them as “undifferentiated learners-in-deficit”. Educators and managers are the architects of the learning organization while employees are busy “learning more, learning better and faster” trying to correct their knowledge deficit. In the learning organization workers become responsible for the health of the organization without the authority to determine alternative ways to reach that health. The fear of being left behind in a quickly changing market environment is used to create anxiety and fear as motivations for learning. All of these factors serve to put serious limits on the potential of people to learn in the work environment.
Diversity and Privilege Overlooked
Perspectives of race, class and gender -which research has shown affects the way people learn and collaborate- are lacking in the literature. Fenwick challenges the notion of achieving a democratically ideal situation for open dialog -that the learning organization literature calls for- when all people in the work place do not “have equal opportunity to participate, reflect, and refute one another” (for example because of the status of ones job, character, gender, class, age etc.)
Fenwick sheds a clear light on where the good philosophies of the learning organization are found wanting. The Agile community can benefit from asking some of the same ethical questions she asks in relation to our work. Her critique is a good challenge for Agile practitioners. It challenges us to:
- Continue to strive for higher levels of ethical excellence in our work
- To consider issues of power in our work
- To become conscious of how we use our own power
- To give thought to what voices are included / excluded in the creation of the learning organization
- Pay attention to how we motivate learners
- How to foster collaborative environments that are conscious of the privileging of some over others
- Think about who decides what is valuable knowledge and learning and how that affects the knowledge creation process
Reflecting on these issues will go a long way to contributing to the development of agile practice.
The full text of an old version of Fenwick's article can be found here.
Posted by Shabnam Tashakour at 09:35 PM | |
October 07, 2005
Agile Coach/Mentor Job Description (Process Facilitator)
Given the Agile Axioms and Disciplines then an agile coach or mentor should have some really specific experience and capabilities. This list constitutes a first attempt at a job description.
Experience:
- working in strictly timeboxed iterations with adaptive planning using some sort of prioritized work list
- working in a "test-driven" manner (e.g. writing a document for a client where the client specifies acceptance criteria)
- participating in frequent status meetings where the team members report to each other, commit to work and identify barriers
- building and maintaining big visible charts to show progress and status (e.g. the standard thermometer chart to show progress towards a numerical goal)
- fashioning appropriate tracking and performance metrics that encourage team work rather than individual competition
- helping other people to adopt and adapt all these practices
Capabilities:
- promoting collaboration in challenging circumstances
- searching for truth/a solution relentlessly
- honesty and trustworthiness
- a learning attitude (proactive and learning from mistakes)
- humility and assertiveness
- guiding people without controlling them
- detachment (ability to step out of a situation)
- an attitude of service without expecting recompense
- understanding of transformative learning
- conflict resolution as learning (not negotiation)
- encouraging creativity
Not Required:- technical experience related to the work of the team - the Agile Coach (process facilitator) should not be a performer on the team
- domain experience related to the goal of the work - the Agile Coach should not be a direct stakeholder in the results of the work
However, technical experience and domain experience can sometimes help...
Suggestions and ideas are greatly appreciated!
Posted by Mishkin Berteig at 12:33 PM | |
September 29, 2005
Transformative Learning and Agile
It seems to me that most people who have had any kind of success on serious projects, or in life, can probably point to a profound collaborative experience at the core of that experience. In my last posting, "tools vs. capabilities" I said that because Agile is fundamentally a process of collaboration and our culture is fundamentally is a culture of contest, we need to recognize that learning Agile requires a transformation at the level of character more than methodology. Despite the fact that we may have had profound experiences with collaboration, because we are also deeply influenced by our environment, there are limits to what we can understand about it. We need not look further than the agile disciplines to see how most of our working and social practices are not supportive of Agile perspectives. For example empowering the team and the concept of self-organizing team is a direct challenge to most of our social, economic, cultural, community and familial structures which are essentially hierarchical. The discipline of amplifying learning is a direct challenge to the practice of excessive specialization which manifests itself in the form of expert elitism. How can any one of us ever hope to have a culture of learning and innovation if we come from a culture of expertise and hierarchy based on that expertise?
This is where transformative learning comes in. Agile requires of us not just an ordinary, but transformative learning experience. When we learn, we take something new and fit it into an old category or assign an old meaning to it. Categories are ways in which we organize our learning, they can also be called frames of reference. If we encounter an experience for which we have no category it is hard to understand it. For example have you ever been in a conversation or taken part in a course where what you were learning was so foreign to you that you didn't even know what kinds of questions to ask to help you understand it?
Our frames of reference are shaped through the influence of our culture, language, and experiences, which all interact to set boundaries to future learning. This is because outside of these categories it is impossible for us even to register something new, let alone seek out its reality in an unprejudiced manner.
How often do you find yourself in a new learning situation where you feel overwhelmed, frustrated or even angry? It is possible that at those times you may be at the threshold of a transformative learning experience. You can have two reactions: one would be to dig deep and try to figure out why you are disturbed and see what insights you are led to and the other would be to just give up on the idea and find arguments against it.
Another way to recognize a potential opportunity for tranformative learning is to reflect on the following question: have you ever had an experience where you were faced with some new learning and because you have had a similar experience or because for some reason you see yourself as an expert in that field you have not been able to derive the proper learning from that experience? You may have realized this at a later time after numerous interactions with a similar experience where you slowly started to recognize gaps in your own understanding.
In order to derive the full benefit of a new experience that doesn't fit into the realm of our experience we must have a transformative learning experience. A transformative learning experience is an experience that requires of us to examine the values and limitations of our old categories and assign new meanings to them. This does not mean that all of our previous learning is invalid. A transformative learning experience allows us to expand our frames of reference to allow for more complexity and at times possibly to integrate two previously perceived dichotomous approaches.
For a detailed introduction to transformative learning theories, its thinkers and history check out this link:
http://en.wikipedia.org/wiki/Transformative_learning on Wikipedia.
Posted by Shabnam Tashakour at 10:50 PM | |
September 28, 2005
Agile Infrastructure Projects - Lessons Learned
I've worked as an agile coach on three infrastructure/maintenance projects in a row. One was a software/hardware upgrade, one was implementing agile for a defects/enhancements team, and my most recent was a data warehouse decommissioning project. In all cases, the interesting part for me was taking the basic principles of agile and applying them in a way that works when not doing new product development. Here are some lessons I've learned:
1. Figure out what is going to deliver value (usually cost savings). In the case of infrastructure projects, one is usually focused on cost savings. Find a way to tie your work items directly to cost savings. You need a good financial model to do this. Mary and Tom Poppendieck talk about this a little in their Lean Software Development book. In the decommissioning program, there was a very explicit dollar cost associated with disk space and cpu utilization. Every user/MB decommissioned saved a measurable amount of money. As well, it allowed us to easily prioritize our backlog.
2. Focus project/program organization more on Lean principles than agile. A good understanding of queuing theory will go a long way to helping with throughput. In a team doing defects/enhancements work, the small pieces lend themselves well to certain types of streaming through the team. Iterations are not necessary to chunck work. Instead, iterations become checkpoints solely for process reflection.
3. Technical infrastructure projects can benefit greatly from automation. Test automation including test generation can sometimes be possible. Automating parts of a regularly repeated process that is used for every work item can be extremely beneficial for increasing speed. In the case of the decommissioning effort where every database table needs to be considered separately and where they all go through the same process for decommisioning, there are many opportunities for automation. The project/program/team can invest in doing this automation to great benefit to NPV.
4. The basic axioms (We are Creators, Reality is Perceived, Change is Natural) and disciplines (Empower the Team, Amplify Learning, Eliminate Waste) still apply. Even though it is not "new" product development, the creativity of people is essential for problem solving, and finding ways to do the work faster. Stakeholders still need to have their perception of reality acknowledged, and the teams still has to do constant checking to make sure they are on track with that perception. And of course, things are always changing including priorities, our understanding of the work, resource availability etc. Having an empowered team makes short work of many obstacles, but that wouldn't happen without an explicit acknowledgement that we have to constantly be learning and eliminating waste. Teams get better and better at these disciplines over time.
I would be very interested to hear other peoples experiences with infractructural/operational projects.
Posted by Mishkin Berteig at 12:00 PM | |
September 27, 2005
Tools Versus Capabilities Approach To Agile Training
Which approach is most valuable in training that fosters collaborative work for the purpose of optimizing the performance of an organization: a tools / methodologies approach or an inner capabilities approach? The typical orientation that most organizations take is often external and rule-based. This consists of creating methodologies, rules, boundaries, systems and processes to enhance collaboration.
These external approaches ultimately fail to have a lasting effect on people and the culture of the organization because they don't address change at the level of habits of mind. People then work in the new structure with the same patterns of behaviour. Behind this kind of surface approach to change are assumptions about human nature. At worst this consists of a belief that people are base (greedy, selfish etc.) by nature. At best that people are fundamentally good but cannot improve except through external measures. It is true that we need external systems and structures to give expression to our inner capabilities, to test, foster and develop them in action. However all the investment that companies make in tools, systems, methodologies are obviously not enough. We need both external and internal approaches to training people in collaborative processes. Systems and tools provide only a framework that then need to be filled in with character. At the core of Agile there are disciplines (such as Empower the Team, Amplifly Learning) without which the methodologies would have no life. The practice of the disciplines fostered by the development of inner capabilities infuses life into the Agile methods and at the same time the methods act on and reinforce the inner practice of the disciplines.
As Agile champions (coaches, facilitators, practitioners) we must invest energy on fostering -through modelling and coaching- the development of inner capabilities. The Agile community will benefit from an identification of core capabilities required and a deep exploration of how to foster their development in individuals, teams and organizations.
Although it is our nature to organize in groups and we may have much experience with collaboration, we nevertheless live in a culture of contest and individualism. Out of this culture comes a set of belief systems which are so deeply rooted in our lives that we are not fully conscious of them and their affect on us. These belief systems cannot change easily through the introduction of external structures alone.
Posted by Shabnam Tashakour at 12:44 PM | |
September 15, 2005
Personal Philosophy of Adult Education
The following is my approach as an educator to my work in community and organizational development. I have come to this understanding mainly through experience, a great deal of mentoring and study.
Please note that when I use the term “teacher” in this document I also mean consultant, mentor, coach etc. The term “student” is also interchangeable with organization or community. The term education is interchangeable with organizational or community development consulting.
Validation: a starting point
Education should start from, affirm and validate the experience, insights and knowledge of the individual. This is a foundation for education that honours and respects the student. Recognizing the nobility of the student allows her an active role in her own learning. The role of the teacher is to facilitate learning by drawing on the experience of the student, to build on that experience through the acquisition of new insights, knowledge and skills.
Learning must be self-directed. The teacher may have a number of wonderful things to teach, but if the student does not believe that they are relevant to her, she will not be engaged. This is especially true for teachers who are working in communities that they are not a part of. The teacher must engage in careful investigation in order to understand the situation of the student, which includes attentive listening, as well as a genuine interest in the needs of the student, before proceeding along any line of instruction. Taking her cue from the students, the teacher must work with the individual / group to create a learning environment in which everyone takes responsibility for their own learning. In this kind of environment the teacher is not an expert and does not do the students’ learning for her. The teacher can use questions to assist the student to understand, instead of delivering answers. The teacher should also encourage an environment of learning that recognizes mistakes as part of the learning process. The learning environment should create in the student a hunger for the acquisition of knowledge, insights and skills beyond the direct experience with the teacher.
Encouragement: the key to self-directed learning
Once the experience of the student has been validated and her needs established, education should be challenging but not obtrusive and challenges must be presented with respect and encouragement. Encouragement versus excessive criticism leads to individual initiative instead of paralysis. The natural result of an encouraging and challenging learning environment is self-discipline and self-correction instead of external discipline (control) and constant external correction.
A transformative, holistic approach centred in humility and service
The learning environment should foster humility in both the student and teacher. Most contemporary approaches to education are materialistic; the student pays, studies, receives a degree, becomes an “expert”, etc. The whole educational experience, from the teachers to administrators, cultivates in the student a sense of self is that is based solely on the expertise and knowledge gained. The “Expert” attitude in the community development environment is often not useful because the work in the field is so complex. Many stakeholders have keys to the process, as a result, the “expert” attitude devalues the knowledge of others and tends to taint the path to solutions with conflict and ego. Another consequence of the expert mentality in the community is dependency; people are divorced from the solution to problems that they all contribute to and to which they all hold the keys. Instead of drawing on the knowledge of the stakeholders, the expert renders her own knowledge most valuable which in turn causes them to discard volition and succumb to a state of perpetual dependency on one expert after the other. Community members or institutions are robbed of the ability to play a central role in their own lives as a direct result of being robbed of opportunities to play central roles in the decision-making process of their community.
With humility at the centre of all learning, the purpose of education becomes transformation. We learn so that we, our communities and our institutions can improve and change for the better. Also as learning is applied to community efforts, individual capacity unfolds and is developed. Learning for its own sake is valuable, but learning for positive social change, makes the acquisition of knowledge, skills and insights relevant and engaging in the face of community development challenges. Learning then becomes intimately connected with action and is corrected and refined through action. This infuses a powerful sense of purpose and meaning in the learning process, especially as successes are realized.
Principle-based approach facilitates ownership
Education should cultivate a sense of personal ownership in the learning process and community life. Fostering a sense of personal ownership comes with educating students to have a mature perspective about their own learning as well as the changes they desire to implement in the community. It involves helping students learn the capability of ‘becoming’ the change that they want to see, as well as finding positive starting points in desperate situations and building on them. A mature outlook demands that students have a principle-based approach to problem solving versus a rule-based approach. Education then becomes not only a process of acquiring knowledge but centred on capacity building for individuals, institutions and groups. Fostering the development of capacities needed to overcome obstacles also requires a principle-based approach, embodying principles such as perseverance, human rights and dignity, building unity in diversity etc.
Integration and balance of methods essential
Education should be methodical and balanced. It should aim to acknowledge, validate and employ different learning paradigms: those of science, spirituality, culture and the arts. Systems of education that value science above the arts or spirituality are destructive to the individual and community as they create an imbalanced view of the world and rob people of a diversity of perspectives and tools that they need to face complex challenges. An educational program should strive to address the mental, emotional, spiritual and physical needs of students and not focus too much on merely one dimension of life. This is especially important in communities that have experienced extreme marginalization (colonization, oppression) where healing and wellness must play a significant role in the learning process.
Modelling Change
A key ingredient to success in transformational education is the example of the educator. As people, naturally we do what we know and what we have experienced. In order to change our patterns of behavior we need to begin having fundamentally different experiences than what we have known. The educator must be able to assist in the creation of such experiences. To do this she must be capable of modelling what is being taught and through constant critical self-reflection strive to exemplify in every action empowering ideals.
Summary
Learning and education are indispensable to all community efforts for positive change. The job of an adult educator is to assist individuals, the community and its institutions to adopt a posture of learning. This begins with working with the experience of the student, fostering self-directed learning and follows as the teacher interacts with the student to challenge and assist her to new levels of learning. With humility at the centre of all learning efforts, dependency on “experts” can be replaced with volition and independent decision-making. The potential of the individual further unfolds as she applies her learning to service to the community. Attention to capacity building and cultivating a sense of personal ownership -in the process of learning and community building- deepens the experience and truly engages the student in taking an active role in the development of her life. Utilizing all systems of learning in the education process ensures balance of methods and helps cultivate the infinite and diverse capabilities of human potential. Ultimately the success of an educator rests on the degree to which she is able to model the change she is fostering.
Posted by Shabnam Tashakour at 05:04 PM | |
September 06, 2005
Process Facilitator Role
I've been thinking a lot about the roles on Agile Work projects. Here is a possible "mission statement" or definition for the Process Facilitator:
The Process Facilitator is a person who is both experienced with Agile Work and trained as a facilitator. The Process Facilitator acts as a coach to the team to monitor the process, foster the understanding of the Agile Work Axioms, the development of the Agile Work Disciplines and adherence to the Agile Work practices. The goal of the Process Facilitator is to assist a team to become "performing" so that they are able to actively and independently persue continuous learning and improvement.
Also Known As: Scrum Master, Coach, and previously referred to as the "Process Owner"
Never Spread Your Facilitator Butter too Thin
Posted by Mishkin Berteig at 12:27 AM | |
August 25, 2005
The Role of the Process Owner
The following is an edited version of a post to the Scrum Development Yahoo! group made by Dave Barrett (CSM) (used by permission):
The ScrumMaster (Process Owner) role itself doesn't automatically imply any degree of authority, although at times it does help when you need a little clout to clear some impediments or to negotiate with other departments. More than that, the ScrumMaster role does carry a responsibility to provide some leadership to the team. Even a self-organizing team needs leadership!So I would say that it helps if the ScrumMaster has a little bit of
seniority over the rest of the team, it also helps if the ScrumMaster
approaches the role as a coach and leader, rather than as a supervisor.On paper it looks like the ScrumMaster only has a few tasks:
1. Chair the Scrums, and make sure they happen each day. (The self-organizing team's regular status meetings.)
2. Chair the Sprint (Iteration) Review and Planning meetings.
3. Produce the burndown chart. (An information radiator used to indicate the amount of work left in the product work item list and in the iteration work item list.)
4. Do the team's "paperwork" - publish the Sprint Backlog (product work item list) once it has been set and so on.
5. Clear impediments brought up during the Scrums.In reality, the ScumMaster needs to do a lot of other things. There's all of the leadership stuff, keep the team happy, productive and motivated. There's the political aspect, keeping other groups and
departments happy and out of the team's hair. As the in-house "expert" on Scrum, you need to referee on points of procedure and theory. Often you need to champion Scrum within the organization.Really, I don't see a huge difference between the roles of Project Manager and Scrum Master. Semantics mostly. Project Manager almost seems to imply a "command and control" approach, Scrum thrives best
without that. I wouldn't make a junior programmer a project manager, nor would I make him a Scrum Master.
Posted by Mishkin Berteig at 09:32 AM | |
August 18, 2005
Harvard Business Review Article
I highly recommend this article on Collaboration Rules. Great stuff in there about developing teams, developing organizations and how important communication and trust are to doing so. The article draws examples from and compares the open-source development and maintenance of the Linux kernal and the operation of the Toyota Production System.
Posted by Mishkin Berteig at 10:48 PM | |
August 17, 2005
The Viable Systems Model
Agile Work is a system that can be created inside many types of organizations and work environments. I recently came across an interesting article about the viability of systems. The article describes an interesting recursive breakdown of systems into sub-systems of specific types. Over the course of the next few weeks, I will use this model to try to analyze Agile Work to see if it is viable.
Posted by Mishkin Berteig at 08:36 AM | |
August 10, 2005
Optimizing a Team Room
Some less-obvious hints for creating a team room that promotes collaboration and effective communication.
- don't underestimate the importance of plants and natural light for overall morale and health
- encourage the team both individually and as a group to personalize the team space: kid's art, photos, trinkets, food, special chairs/ergonomic stuff
- encouraging a playful atmosphere if your group is quiet and shy is easier in a bullpen and this in turn leads to better morale
- encourage the team to notice traffic patterns (both physical and communication) and optimize the space to account for these patterns
- play games in the team room during lunch breaks or after-hours and have the results of the games as part of the visual environment
Posted by Mishkin Berteig at 03:13 PM | |
August 09, 2005
The Transparent Society
The Transparent Society, an essay by David Brin is an excellent statement about the possibilities and challenges that technology presents to us as a society. What is interesting about this paper is that it presents a possible society that is very similar to some of the goals in establishing an agile environment: open communication, accountability, free access to information and status, and close collaboration.
Posted by Mishkin Berteig at 11:58 PM | |
August 06, 2005
Generalizing Specialists
The term "generalizing specialists" has come to mean an individual who has a particular area of deep expertise but also has experience in a large number of other areas that may not be directly related to their core area. This type of person typically has strong talent in their specialty but also has a generally strong talent for learning new skills and ideas quickly. The origin of the term seems to be in the software industry referring to programmers who can do other software-development related tasks.
In self-organizing teams, a generalizing specialist is a more valuable team member than a pure specialist. The pure specialist often has an attitude that they should not need to do work outside their specialty. This can be destructive to the team's morale. On the other hand, the generalizing specialist is willing and able to learn new skills - to stretch as the needs of the team change. And since change is natural, this is an essential attitude for team members.
However, we are usually trained, and strongly encouraged to have a deep specialty. This approach to education and training is a natural consequence to the typical organizational model for work and society. Therefore, if a team is converting to agile work methods, people need to be coached to stretch themselves and learn new things. For some people, particularly those who already have multiple hobbies outside work, this is an easy transition to make. For others, it is a very difficult transition. In some extreme cases, this may call for the removal of someone from the team. (Note: I have never seen this myself and I only mention it with great reservation. I strongly feel that only those who could be called "ill" will be so incapable of changing their way of working. For other recalcitrants, it is usually a matter of motivation.)
Other terms that are similar to "generalizing specialist" include "craftsperson", "renaissance man", and "polymath".
Posted by Mishkin Berteig at 08:05 PM | |
August 04, 2005
Just In Case You Haven't Seen It Yet
There is a fantastic article about software productivity: http://www.joelonsoftware.com/articles/HighNotes.html. I love Joel's writing style, and this article in particular has important lessons for us all, regardless of our profession: find what you can be the best at, and do that. Interestingly enough this is part of the message of the book Good to Great but applied to a whole corporation. It also applies in the context of self-organizing teams: each individual should be able to find/learn in what way they can best contribute and do that more than they do other stuff.
Posted by Mishkin Berteig at 11:41 AM | |
August 02, 2005
Memory and Mystery
My father, Garry Berteig, recently took a trip to China to visit my brother in Beijing. Garry is an artist and an educator of great skill and insight. While there, he had the following insight:
Memory (traditional forms) is undone by Mystery (abstract forms). The next step is to combine the two into new forms in a postmodern sense... unity through diversity.
I believe this can be related to the idea of levels of mastery which I first encountered in Alistair Cockburns excellent book "Agile Software Development". Since I don't have the book in front of me, I have to go from memory. First there is rote learning, memorization of predefined forms. Then comes understanding of why the forms are the way they are. Finally comes the wisdom and experience to innovate on the forms.
If anyone else has any ideas about this, I would love to discuss them!
Posted by Mishkin Berteig at 12:00 PM | |
May 17, 2005
The Qualities of an Ideal Test
These six qualities of tests describe how to make a test as effective and as useful as possible. The qualities are similar in style to the INVEST qualities of user stories - but they don't form a nice acronym.
Decisive: The test contains all the information required to automatically determine success or failure. Manual inspection of test results is not necessary to determine success or failure. The test is expressed in a way that produces a pass/fail answer rather than a numerical or qualitative result. Decisive tests are often expressed as assertions.
Valid: The test produces a result that matches the intention for the work artifact under test. Test failure indicates failure in the artifact under test and test success indicates correct operation or configuration of the artifact under test. Simply put: the result of a test should be true.
Complete: The test contains all the information it needs to run correctly with a given test harness and work artifact under test. The test performs all activities and provides all the data necessary. The test does not require any input outside of itself in order to run.
Repeatable: The test always gives the same results if the test harness and the artifact under test are the same. The test is created in such a way that the result is deterministic. Even if the system under test is not deterministic, the test should be created to account for that (possibly with statistical analysis) and produce a deterministic result.
Isolated: The test result is not affected by other tests run before it nor does a test affect the results of tests run after it. The test and the test harness work together to clean up after every test is run. A collection of tests can be run in any order and always produce the same results. Any test that depends upon the results or side-effects of a previous test is not isolated.
Automated: The test requires only a start signal in order to run to completion in a finite amount of time. No manual intervention is required after the test is started. Tests that are automated can be put together into a test suite and run one after another without human intervention. Automation of tests requires the creation of a test harness system.
Update 20060106
I've added a little bit more detail to the above qualities. I also wanted to say a few words about my experience with testing:
I first started doing test-driven development almost seven years ago after reading the book Refactoring by Martin Fowler. I picked it up in Windsor Ontario while I was driving to San Francisco for my first contract position as an independent. I had to stay the night there due to the immigration office being closed so I read a lot of the book in my motel room that night. By time I got to California four days later, I was totally convinced that I should be using JUnit and refactoring for everything I did. Fortunately my project was ammenable to this approach. Over the next four months I built a Java JMS layer on top of Tibco Rendezvous. In the process I discovered methods for doing unit tests for asynchronous multi-threaded distributed functionality. And my tests satisfied all the above qualities. I delivered code with 100% test coverage and zero defects discovered until more than two years later when a very rare and obscure deadlock issue was found. Suffice it to say, I was convinced from a quality perspective.
But there is more. To build tests that follow all the above qualities, you need code that is testable. I have come to believe that testability is the most important architectural attribute of code for most software. The implications of having testable code include code that is easily changed, that is verifiable, and that is easy to understand. Refactoring plays a big role here too.
For getting a basic but very practical sense of test driven development, I strongly recomment Test Driven Development: By Example 
I have also written a couple of articles recently about quality:
Quality is Not Negotiable in which I talk about how to handle defects - by always treating them as top priority work items.
and
Technical Debt in which I expand on this common analogy between lack of testing and debt to show that technical debt is even worse than financial debt.
Posted by Mishkin Berteig at 06:48 PM | |
May 15, 2005
Truck Factor
Truck Factor (definition): "The number of people on your team who have to be hit with a truck before the project is in serious trouble"
Clearly "hit by a truck" is an extreme thought however you could easily substitute "take vacation at the same time" to get the same idea. If any part of your project has a truck factor of one then you are in a particularly fragile situation. If that one person leaves or is unable to work on the project, you will suffer the consequences.
Over time, anyone can be replaced. Truck factor is an indication of how expensive it will be to replace specific people.
In an ideal situation, everyone on the team will know all parts of the system so that the loss of any one person would have minimal impact. In reality, many projects rely on one or more "heros" who are the only one who understand certain critical parts of the system. When these heros leave (and you should assume they will), you must be prepared to recover.
If you have a hero on your team, the best thing you can do is reassign that person to a different part of the system. This will allow the replacement to ramp up while the hero is still available for support. If you wait until the hero has left then the ramp-up will be significantly more expensive.
An added benefit to reassigning the hero is that this person will now have the opportunity to work on something different. Since the hero's tend to be the most technically competent members of the team, this will usually mean that the new area will improve once the hero has worked on it for a while.
Truck factor is a quick metric that will highlight potential problems in your project. Having hero's on your team can be very beneficial but only if you don't become dependant on them. Truck factor is one metric that will highlight your dependencies.
Cross posted from Mike Bowler's Weblog
Posted by Michael Bowler at 06:42 PM | |
What To Do With the Horizon of Predictability
In a previous entry about constant change, the idea of a horizon of predictability was introduced. This concept, along with the agile discipline of amplifying learning, suggest a strategy for addressing problems in a project.
Shorten the length of the iterations you are using. Contract your "planning horizon". The length of your iterations should be motivated by the horizon of predictability for your environment. If your project encounters trouble, particularly of the sort where it looks like you might not accomplish your commitments for an iteration, then shortening the length of iterations will enable you to resolve your problems.
First off, by shortening your iteration length, your opportunities for learning become more frequent.
Secondly, a contracted planning horizon will put you more firmly inside the horizon of predictability... and therefore there will be fewer unexpected changes (on the whole, not in any specific iteration).
A related article is The Pros and Cons of Short Iterations.
Posted by Mishkin Berteig at 06:34 PM | |
May 09, 2005
Reasons for Conflict or Disagreement
As part of the Advanced Scrum Training, Esther Derby presents a section on conflict. One very insightful part of the presentation is a description of four reasons for the existance of conflict or disagreement. They are as follows (adapted from "Advanced Scrum: Collaboration Skills for Scrum Teams" (c)2004-2005 Esther Derby):
1. Lack of Clarity - the communication between parties is missing information or the information is not being communicated in a consistant manner.
2. Position Focus - the parties involved have already each decided on their own solution and are failing to discuss the problem those solutions are addressing.
3. Different Values - the parties are unable to agree because they are holding different sets of values but not articulating those values as part of the discussion.
4. Past History/Personalities - the parties have a previous unresolved conflict that is negatively affecting their ability to work together.
All of these types of conflict are based on either conscious or unconscious failure to be truthful... and therefore the different parties have incompatible perceptions of reality. The Agile Work axiom "Reality is Perceived" then gives us a hint as to how to resolve all of these types of conflict. Find a way to share perception among the parties in conflict so that they have a compatible view of reality.
Posted by Mishkin Berteig at 10:52 PM | | | TrackBack
May 04, 2005
Can dying plan-based projects be recussitated?
We've all seen this. A one-year project in its 13th month, and the Project Manager has been reporting 80% of the tasks at 90% and has been doing so for the last 120 days. There's no end in sight, and the customer is leaking cash every day the product fails to go into production. What can be done? Agile project management principles can help this all-too-frequent scenario.
First, let's look at how this situation comes about. In a typical task-oriented project plan, one is required to decompose the tasks down to a fairly reasonable degree of specificity. The tasks are organized around a dependency graph, estimated, resources are assigned, and the schedule is calculated and/or nudged. (Well, usually the initial estimates don't match the customer's expectation and deadlines, so there's a revision step here where estimates are shortened) But how does change get reflected? If a PM is doing an excellent (if frustrating) job, they are constantly updating the schedule, the plan, and re-conceiving the tasks as new information comes to him.
More often than not, however, this gets so messy and unwieldly that the PM holds fast, and starts to estimate completion based not on a proper decomposition and a completion of each of these tasks, but based on his guess, or his workers' guesses. Complexity of the tasks is hidden, and becomes often quite invisible. Tasks at 80% which suddenly need to be re-decomposed and re-conceived do not, in the main, get moved back down to 40% after certain roadblocks are discovered.
The result? The customer is increasingly afraid, trust between delivery staff and client (and management) are eroded, pressure is increased, mistakes under pressure become more frequent, less sleep is had by all, and 90% complete becomes an asymtote, rather than a milestone.
Now what is to be done with such a project? How can Agile project management approaches help this situation? We can examine this by playing out such a scenario...
We can start by identifying a few key practices of Agile project management, and examine their benefits to the business client.
Timeboxing and Iteration
The first thing we can talk about to the fearful client is timeboxing. Timeboxing caps his investment into small chunks. We look at it and say, OK, you're in deep trouble, but you don't know how deep your trouble is. By timeboxing into a very short timeframe, and making a large project into many little short projects, we can get more visibility into the process - we can see things as they are, rather than how they're reported on a project plan. Speaking of visibility...
Daily Meetings and Iteration Review
Part of the value of iteration and timeboxing is increased visibility, so we really need to have a mechanism for visibility. Already distrustful of project plans, we can tell the client, "Don't believe the paperwork, let's look at what's actually built. At the end of each iteration, we'll review the current situation, and demonstrate existing functionality." His eyes perk up. "Wah?" he asks incredulously? What do you mean? So we say, "we don't want you to guess at how ready this thing is. We'll show you. That way, you can decide if it's ready enough. ...Or your product marketing people, if you prefer. It's up to you, but you get to see it, touch it, and sense for yourself how ready it is, and you won't have development managers and developers waving their hands pretending it's further than it is."
"Oh wierd," says he. "So what are these daily meetings?"
We tell him that the daily meetings have several purposes. Only project members get to speak, and they report on what they did yesterday, what they plan to do today, and what, if anything, is blocking their path. No one else gets to speak, but others who are not on the team itself can listen in. This way, the whole project team is clearly aware of how they're working, what's left to do, and what each other are doing.
"Why do they need this? They have the plan." "True," we reply, "but how often do you have two people who need something, and both do it because they don't know that someone else already did it. With your current project delays, do you want any duplication of effort? Just by way of example?"
Feature Lists, not Task Lists
We also talk to him about working from feature lists, not task lists. The team will be implementing features, and fulfilling requirements. How they do it is up to them. "Why should I care," he'll ask. "You don't," we reply, except to assure him that if people are working against features, then they can choose to re-order their tasks in such a way as to most quickly get to the goal. No wasted effort, we tell him. Sounds good to him.
Customer-Prioritized Features
What's more, we assure him, we will be first working on the highest-priority features. What "needs" to get to market. Everything else slides down the list. Even if the wonderful plan we all love had it earlier. High-value features (as prioritized by the client) and their necessary dependencies. That's it.
"So no features that I didn't ask for?" He looks hopeful.
"That's right," we reassure him, "and you can change your mind."
He faints.
Smelling salts are brought.
"What do you mean, 'I can change my mind?'" he asks.
"If, when we get to the iteration review, and you test drive the thing, Acme corporation has come up with the killer feature that you absolutely must have or the whole thing is useless, you put it at the top priority on the list we use to drive the next iteration."
"And no one will complain that it's out of scope?" he marvels?
"If you put it at the top of the list, it is the scope for the next iteration. We are at your service," we comfort him.
"So when will this be finished?" he asks us desperately.
"When you say it's ready," we reply. He boggles. What does that mean, he thinks.
"What does that mean," he asks.
"It means that we will tie it off, when you think it has enough juice to go to market. If, after you re-prioritize what's left, you find that it's 'ready enough', then we'll roll with it, take a couple of weeks to steady it and ship it, and then we can pick up your next-highest priority things, or anything new you want in it." He looks near fainting again. "And we stop when you feel that spending money on it is no longer bringing you enough value, ideally because we've done all the high-value stuff already, and we're working on less valueable stuff."
"So I have discretion to pull the plug whenever I want to?"
"Yep."
"Please help me..."
Conclusion
This little scenario is fictitous, but in the end, it's consistent with the experiences of many Agile practitioners (particularly Scrum) that I have spoken with. We've only covered a small sampling of Agile practices that may help a project in crisis. Others may help quality, may improve developer productivity - but the above can help a key client or stakeholder begin to see a light at the end of the tunnel. While many people cannot get their heads around it, they may be willing to try new things when they're in enough pain with the existing process.
And it can turn a project failure into an ever-increasing success, because success is defined monthly, re-defined at the desires of the business client, and executed in bite-sized chunks that are digestible and estimable.
Just don't forget that people have the strangest reactions to things that break their expectations... so make sure to bring the smelling salts...
Posted by Christian Gruber at 09:08 PM | | | TrackBack
Agile Practices Applied to Architecture
I have just run across a web site about applying agile practices, specifically from Extreme Programming (XP) to architecture. This site, called "Architectural Practices - Extreme Project Management for Architects" has a great deal of information.
This site represents a set of ideas that have come full-circle. Christopher Alexander wrote a book called "A Pattern Language: Towns, Buildings, Construction (Center for Environmental Structure Series)" about a different way of looking at architecture, interior design, landscape design and city planning. This book made a big impression on some people in the software industry. In particular, it inspired the book "Design Patterns" by Erich Gamma et. al. This book became required reading in the software industry. Many participants in the Agile Software Development movement consider it foundational. And now we see agile practices being adapted back to architecture. Cool.
Posted by Mishkin Berteig at 04:15 PM | |
April 28, 2005
Plan Methods Oppose Agile Work Axioms
Plan driven methodologies which attempt to mechanize the process of doing work are in opposition to the three Agile Work Principles.
We are Creators
A plan methodology attempts to define intermediate and end work products independently of the input and effort of those who perform the work of creating the work products. This disenfranchises people from their work and leads to low morale. It also establishes a heirarchy of value for the people working on an effort where those who create the plans are perceived as more important or valuable than those who execute on the plans.
Change is Natural
This principle is usually acknowledged, but is usually described as a "problem" to be dealt with rather than as a basic principle to be fully embraced. A plan methodology has "change control" or "change management" and "risk management" and puts the whole notion of change in a negative light. This approach also disenfranchises people because they are constantly placed in opposition to reality.
Reality is Perceived
Plans attempt to legislate reality. "Thus and so must the project go" results in a constant struggle between the plan and peoples' perception of reality. Plans marginalize the importance of perception on the belief that reality can be objectively understood. If reality can be objectively understood, then it can be mechanistically manipulated. Thus results can be pre-determined without regard for the perception of those results.
Posted by Mishkin Berteig at 10:54 PM | |
April 26, 2005
Truthfulness and the Three Agile Axioms
A few more words are in order about how Truthfulness is the foundation upon which the three Agile Axioms rest. Taking them one at a time:
1. We are Creators
Truthfulness operates at a very deep level for this agile principle. People typically are not aware of their own need to shape reality, to create things. Developing truthfulness helps a person to uncover this fundamental capacity and bring it to fruition. Truthfulness also helps a person reflect on the results of their creative work. This is essential for learning how to become a more effective creator. In an agile team, truthfulness is also essential in developing the interpersonal trust necessary to know when the process of creation is going aright or awry.
2. Reality is Perceived
This principle is the most obvious domain where Truthfulness plays a role. In this instance, being able to express one's perceptions and share them with others in a truthful manner allows others to develop empathy and formulate an honest response. If Truthfulness is missing, then perception will be based on something other than reality (well,... technically it will be based on the reality of an untruth). In an agile environment, where the value of communication and collaboration is extremely high, truthfulness helps to develop a shared perception for a team.
3. Change is Natural
Here the role of Truthfulness is simple: an agile team cannot function if it is not Truthful about change. Recognizing and communicating change so that a team can embrace it and adapt to it depends completely on people being forthright and truthful about change.
Much of my thinking about Truthfulness comes from the study of a passage from the Sacred Scriptures of the Baha'i Faith: "Truthfulness is the foundation of all human virtues." I have also had substantial discussions with other agile coaches about the importance of truthfulness and its key role in developing trust.
Posted by Mishkin Berteig at 11:51 PM | |
April 19, 2005
Considering the Agile Manifesto and the Axioms of Agile Work
The Agile Manifesto, aimed squarely at software development, is inaccurate when considered against the more general target of Agile Work. The Agile Software Manifesto reads in part:
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Having studied Scrum, and attempted to apply agile practices on non-software projects in non-corporate, non-new-product-development efforts, I am painfully aware that the Agile Software Manifesto needs some re-jigging to become the Agile Work Manifesto. My first cut at doing this was to remove the software implications from the above four statements of value. This resulted in the following three statements of value:
Individuals and Interactions are preferred over Processes and Tools
Moving Towards a Valued Goal is preferred over Producing Ephemera
Responding to Change is preferred over Following a Plan
I removed the part about customer collaboration because I felt that it was strongly implied by "Individuals and Interactions" and "Moving Towards a Valued Goal" and because not all efforts have identifiable customers in the sense that a business effort usually does.
Once I got to this point, I started to feel like the statements of value were not really getting to the fundamental assumptions or principles or axioms of Agile Work. I thought quite a bit about each value and what it was really saying at a very basic level. For example, "Individuals and Interactions over Processes and Tools" is partly about the value and power of human beings. What is that power? "Moving Towards a Valued Goal over Producing Ephemera" begs the questions of what is a valued goal? and can ephemera be valued? and for that matter, what exactly constitutes ephemera and its opposite? Finally, "Responding to Change over Following a Plan" is really saying that plans don't tend to work. And why is that? So in order to answer those questsions, I have come up with the following three Agile Work Axioms:
People are Creators
Perception Mediates Reality
Change is Constant
People are creators and that's why its so important for us to improve ourselves and our interactions with others. Perception mediates reality and that's why we must produce results that are perceived as valuable to those who care about our results. Change is constant and that's why following a plan never works... unless you "embrace change" (Beck). But there is still something missing. There is a foundation necessary to make these principles work together in real human environments.
Truthfulness is the Foundation of Agile Work
Posted by Mishkin Berteig at 10:32 PM | |
April 18, 2005
Book Review: "Good to Great" by Jim Collins
Summary: In this well-written, engaging book, author Jim Collins describes six critical factors that he and his research team found common in companies that transformed themselves from a long period of mediocre or bad results to a long period of great financial results. Although focused on publicly-traded companies in the United States, the results of this research can easily be extended to apply to other types of organizations.
Good to Great describes the following six concepts:
- Level 5 Leadership: personal humilty and professional discipline in an organization's leader are the starting point for the transformation.
- First Who... Then What: don't worry about what to do until the right people are in the right positions in an organization.
- Confront the Brutal Facts (Yet Never Lose Faith): careful and honest examination of an organization's current situation is the foundation for finding a path forward.
- The Hedgehog Concept (Simplicity within the Three Circles): discover a simple business concept that people can be passionate about, that works with a single key economic driver and that the organization can be the best in the world.
- A Culture of Discipline: disciplined people removes the need for heirarchy, disciplined thought removes the need for bureaucracy and disciplined action removes the need for excessive controls.
- Technology Accelerators: pioneer the application of carefully selected technologies without relying on technology for transformation.
Some of these concepts are very close to the underlying prinicples of agile work. For example, in the Scrum methodology, the first five principles are all represented. The Scrum Master embodies some of the attributes of level 5 leadership. A self-organizing team gets the right people in the right position. The daily scrum and the sprint planning meetings are designed to help the team confront the brutal facts. The sprint goal embodies at a local level the simplicity of the hedgehog concept. And the practices in general are a manifestation of a culture of discipline.
Who Should Read this Book? Anyone interested in creating a lasting positive transformation in an organization should read this book. In particular, individuals who are coaching teams or organizations should read this book since the concepts also appear to apply at a sub-organizational level.
Posted by Mishkin Berteig at 07:29 PM | |
April 12, 2005
Agile Household Management
Four weeks ago my wife, Melanie, reviewed a paper I was writing about Agile Work. When I talked to her about it, she asked me why we aren't using these practices for managing our household.
So, we decided to try it. First we developed a starting Work Queue. It includes stuff like "eye appointment for Mishkin" and "build boxes for vegetable garden" and "set up expenditures spreadsheet". It covers any activity that is non-repeating, non-business. Things we need to do with the kids, renovations, travel plans, personal finances, community work, social events.
Once we developed our Work Queue, we agreed that Melanie would be the Queue Master, responsible for managing the work and prioritizing it. So, with a small amount of coaching, she prioritized our rather long list of Work Queue items.
We then jointly decided how much to accomplish over the next week. Our iteration length is one week because we already have a natural rhythm of that length due to my travel schedule for work. We took on about 15 items from the queue, and committed to getting them done.
We have now been doing this for a little more than three weeks. Having this process in place has been helpful in a number of ways. First of all, because it is in priority order, our level of stress about what we need to do is decreasing rapidly. Secondly, we are starting to develop a sense of how much work we can accomplish in a one-week period. By being focused, I am finding that we can get more done than I would otherwise expect. Thirdly, by working in weekly iterations, we can quickly adjust to things that come up. Finally, we have a clear set of expectations about what we will accomplish and this reduces feelings of tension around household work.
There are some things that I would like to add to our approach. We have three young children and I would like them to become involved with the process. The eldest is nearly seven, reads at least three years ahead of his age, and is certainly capable of understanding the text-based format of our planning. I would also like to see how we can incorporate acceptance test-driven work into the mix. Because we are such a small team this may be overkill, but at this point we are not explicitly checking our work at all or setting any sort of acceptance criteria. I can see that jumping up to bite us in the future.
I would be interested in hearing from anyone else who has tried applying agile techniques and practices in their own households.
Posted by Mishkin Berteig at 05:15 PM | |