Agile Advice

September 24, 2007

Agile Benefits: Five Essential Reasons to Try Agile

Although there many other benefits of agile, and although those provide us with other reasons to try agile, the five essential benefits of agile are:

Rapid Learning - disciplined application of the scientific method to explore the best ways to deliver valuable results.

Early Return on Investment - opportunity to use the results of work starting with the work delivered at the end of the first iteration.

Satisfied Stakeholders - engagement in the process in a way that allows meaningful contributions from all stakeholders.

Increased Control - mechanisms to track/measure and therefore steer the direction that work is going so that it meets goals.

Responsiveness to Change - processes, tools, roles and principles that allow a team and an organization to embrace change rather than reject, control or suffer from change.

These reasons are sufficient and apply to operations work, project work and open-ended research work, whenever humans are involved. The above links take you to more detailed descriptions of each of these benefits.

What are some of the other benefits of agile?

Posted by Mishkin Berteig at 10:47 PM | |

September 20, 2007

Agile Benefits: Increased Control

One of the reasons that agile provides stakeholder satisfaction is because of increased control. This increased control is a benefit in its own right. The word "control" must be used carefully... some agile approaches and practitioners are justly wary of the word and its common uses. This particular benefit of agile is often a little surprising to people how have believed the myths around agile that it is a chaotic-anything-goes type method. Let's look at how an agile approach provides more control.

Increased Control

The person responsible for prioritizing the work (the Product Owner, Queue Master, Product Manager) is the locus of business control in an agile environment. This person exercises this control in a number of ways. First and foremost is through the constantly evolving queue of features or results to be delivered by the agile team. Here the Queue Master has final authority based on influence, knowledge, experience and formal appointment. The Queue Master can control the immediate and long-term work of the team through the use of this prioritized, never-frozen list of work. Every iteration, the team will accept the highest priority items according to their capacity (or velocity) and complete those. This fine-grained direction of the team is impossible in a traditional phase-based approach to working where often the "change control" process is a bureaucratic nightmare whose end result is the team preventing the business from getting what it needs.

Clearly there is more control. At what cost? Well, the previous three benefits of agile mentioned indicate that there is no financial cost (in fact there is a benefit), there is no intellectual cost (again there is a benefit), and there is no decrease in satisfaction (yet again, there is a benefit). So what is the cost? There is, unfortunately, one very deep and difficult cost: the team doing the work loses control over deciding what is valuable (cost-benefit). I know that in many IT organizations, this is a terrible political cost to pay for IT management and it is a terrible "status" cost to pay for technical folks.

Fortunately, even these losses are offset by other aspects of the agile process and its benefits... again, around the question of control.

In exchange for taking complete control over value determination, the Queue Master must grant complete control over solution discovery and implementation to the team. Of course, ideally there is close collaboration. However, if there is disagreement, this is the standard dividing line of responsibility in an agile environment. So now, to the benefit of those doing the work, they are given complete control over how to do the work and this can be an incredible benefit: it allows the full run of creativity and problem-solving skills to come into play.

---

Agile Benefits: Rapid Learning
Agile Benefits: Early Return on Investment
Agile Benefits: Satisfied Stakeholders
Agile Benefits: Increased Control
Agile Benefits: Responsiveness to Change
Agile Benefits: Summary Article

Posted by Mishkin Berteig at 09:17 PM | |

August 30, 2007

Agile Retrospectives and the Plan of Action

Bas Vodde has published a good article about making goal oriented action plans for agile projects. It is a nice piece of the puzzle on how to do effective retrospectives. It also nicely ties into the "Learning Circle" Reflection/Learning/Planning/Action steps.

Posted by Mishkin Berteig at 04:59 PM | |

August 18, 2007

Agile Management Tools

Ah, tools...

For agile training and consulting, I always recommend starting with the standard physical tools of note cards on a wall plus a whiteboard and digital camera. However, I am still interested in what tools people have used when they have found that note cards are not sufficient. Here is a list of the tools that I am aware of, as well as a link to a survey...

This list is in alphabetical order:

CardMeeting

Custom In-House Tool

Defect/Enhancement Tracker (e.g. Jira)

ProjectCards

RallyDev

ScrumWorks

Spreadsheet

VersionOne

Wiki (e.g. Fitnesse)

XPlanner

---

The survey is extremely short (four questions) and will be closed on Sept. 15th:
Click Here to take the Agile Management Tools survey

I will publish the results here.

---

If you know of other tools (open source, products, etc.) please let me know in the comments!

---

Here are more tools for managing agile projects/requirements/planning that have been mentioned by folks in the comments:

Tackle

Trac

Silver Catalyst

Target Process

Acunote

Mingle

Thanks to everyone for including the urls in the comments!!!

Posted by Mishkin Berteig at 02:36 AM | |

August 09, 2007

A Cautionary Tale - Delaying Agile Adoption

What happens when you delay adopting agile? Well, one large client I have worked with has found out...

Because this story is a story of failure, and because most people don't like to have their failure's exposed, I have anonymized this story and changed some details to protect the innocent (and they are innocent! Failure is a learning opportunity, not something that's bad).

I started working with BigCom about two years ago. They got a whole bunch of training from me; in total about 60 people received some in-depth training in agile methods. They also decided, wisely, to get me to help coach them organizationally and for some of their Process Facilitators. What they didn't do is heed some good advice: start iterating.

They decided that they needed to do some important up-front architectural research on their huge product re-write/porting project. They had an existing product used by many clients and built over the course of several years. This product was implemented in a legacy programming language and in order to make it easy to add features and keep it up to date, BigCom decided to re-write it in a more modern programming language... but they couldn't decide which one: Java or .NET. Because it was a pure porting job with little if any functional change, they already had a good detailed knowledge of the functional requirements. But this one question of the platform to build upon stalled them. Their research committee studied this questions (and some other less important ones) for four months.

I had advised my client that they should feel free to go ahead and do the research, but that they would be well served by starting to iterate and build the new version of the product, even if it meant building it on two platforms simultaneously. The actual experience of trying to implement real functionality in both Java and .NET might seem like a waste, but actually would provide valuable information for the research effort and help speed the decision-making.

Perhaps my advice was too scary. Or perhaps there was some legitimate reason for many capable people to sit idle for four months while the committee researched, deliberated and failed to make this important decision. In any case, product management and dev management finally got tired of waiting, and they started up the work.

Of course, without the decision made, the work initially was a real mess: developers were left to choose when to use Java and when to use .NET. Sometimes functionality was duplicated in both languages. Sometimes it made connecting functionality extremely difficult. It wasn't systematic. But it was experimental, and it did help clarify the issues quite quickly. After only four weeks of doing actual work on the functionality of the product, it became clear how to use these platforms (for the curious: Java on the UI and .NET on the back end).

The research committee had very little to do with the final decision other than to rubber-stamp it.

Time passed. BigCom continued the development work, features were implemented, and eventually the deadline for releasing the product loomed. And then it passed. Six weeks after the deadline, the product was finally released. Nine months after the start of iterations. Thirteen months after I recommended that they "just start".

Four months wasted... and a deadline missed by six weeks. Was it worth it? Might have been nice to distribute some bonuses for coming in ten weeks early instead. Oh well!

Please, just start iterating!

Posted by Mishkin Berteig at 09:55 AM | |

August 01, 2007

Agile Estimation and Pairing

I just read a recent article on PMHut called "Schedule Questions: Pair Programming and the PNR Curve". There is much in this article that is important for agile project management... and much that should be avoided at all costs!

Estimating Projects

The bulk of the article is an explanation of the PNR curve (Putnam Norden Rayleigh curve). The explanation is sufficient and can be checked against other web articles about software estimation. The basic idea of the PNR curve is to find an ideal number of people with which to staff a project based on an estimate of overall project size (e.g. 72 man-months).

Take a moment to go read Fantasy Estimation (the comments here provide good additional material).

It should be clear now that any initial sizing estimate based on units such as man-months is ridiculous. So what is our alternative? What about an agile approach to estimation and planning?

Agile Estimation

Well, at a minimum, in an agile approach to estimation, we need to resolve some of the problems identified in "Fantasy Estimation". So, we:

1. Get the whole team that will be doing the work to do the estimation
2. We weight estimates based on team maturity, knowledge of tools, knowledge of the business domain and the physical proximity of team members to each other
3. We adjust our estimates of work remaining based on our results for work completed

The great thing about an agile approach is that it allows all these things to be done and to be done well. Scrum has a specific set of "Drag Factors" that can be used to weight the team's estimates. And using a team's past estimates vs. "actuals" is possible on an iteration by iteration basis... and this is possible because every iteration the team is doing the "same thing": delivering an increment of valuable results (working software, edited video, organizational change, whatever is contributing to the project goals).

The other thing that agile methods allow is to make a distinction between estimation for planning and estimation for commitment.

So. It's cool to learn about the PNR Staffing curve and the thinking behind it. It's a little scary to see it being applied in an agile context when we have tools that are much better.

Pairing and Estimation

At the end of the PMHut article, the author, Mike asks a question:

What about pair programming? When using pairs on a team we could either:

1. Continue as is, use the model to determine optimal team size and then encourage pairing to increase efficiency.
2. Treat a pair as one hyper-effective person, so count pairs not individuals and increase team sizes accordingly.

This second option seems counter intuitive to agile, we strive for small teams to reduce communication channels, so I’m not convinced by this idea. I would be very interested to hear people’s thoughts on agile staffing curves. So, lots more questions than answers in this post, please let me know if you have any research or thoughts on the matter.

I wrote in response:

Mike, I feel like the options you present in your question at the end actually miss the true point of pairing… which has to do with communication. I have never seen pairing done in such a way that two people always work together as a pair. Rather, pairing is promiscuous: people switch pairs frequently throughout a day, iteration or project. This switching has two communication effects: 1) the human interaction and gradual diffusion of information as people switch pairs 2) helping everyone understand all parts of the work as a result of frequently working on many different things. From an estimation standpoint, I expect that neither of your two options is quite right. Rather, the third option is to continue as is with existing estimation and then encourage pairing to increase communication. Increased communication shows up in a number of different ways, not just efficiency: risk mitigation, accelerated individual and team learning etc.

I would like to expand on this just a little. To me, pair programming or pair work of any kind has always been able to provide a number of benefits to projects. Problem avoidance is one benefit (lower defect rates, better code quality through constant inspection, spotting each other). Better communication is another benefit. Increasing the Truck Factor is yet another benefit.

None of these benefits have any direct bearing on estimating a project at the start, particularly since most projects sacrifice quality for schedule. From an agile perspective, since planning estimation is not commitment, it actually doesn't even really matter! Get a conservative estimate for you project using whatever method you like, then start working and get a commitment velocity and see what the team can really do.

Posted by Mishkin Berteig at 10:04 AM | |

July 25, 2007

Time is Not Negotiable

The Project Management Institute refers to three variables that can be negotiated or constrained for a given project: scope, resources and schedule. Schedule is an interesting "variable" in that we have no choice about how time flows. We can control how much scope to ask for, we can control how much money to put towards the work, but we cannot actually "buy" more time than, say, our competitors. This has important implications which deeply challenge the PMI's PMBoK model of project management.

In a related article "Quality is Not Negotiable" I wrote about the strategic important of maintaining a high level of quality, and that allowing defects into your work is like taking on a debt with an unknown interest rate. With time, the problem is a little different.

The idea of sunk costs, also know as "don't throw good money after bad" is difficult for most people at a psychological level. We often feel like making just a little more investment of time, work or cash will turn an ailing effort around. Unfortunately, a traditional waterfall or phased-based approach to project work increases the psychological pressure to continue working, even when the project is "bad".

We know from the recent Chaos report statistics that two thirds of all IT projects are still either failing or challenged. We know that the bigger the project, the more likely it is to fail. We know that in other industries such as construction and economic development projects also have low success rates. How is it that we put all this money and time into these projects without knowing earlier if there are problems?

The problem is, of course, time. We can't predict the future. The farther out we need to see, the more uncertain the view. Not only that, but we can't "go back" to correct mistakes. We can only correct past mistakes by changing our future behavior... by using up more time.

One organization I worked with wanted to do a small web-based project. They had been "thinking" about this project for three years. At various times they had done requirements gathering and analysis with various stakeholders... and then failed to execute on the project. Finally they got around to starting the project. Using an agile approach, the first release of the project was completed in about eight weeks. How does this relate to time?

Well, first off, the project success was measured only in terms of the eight weeks of labor, not the time value of the sunk costs over the past three years. Now of course, the reasoning behind not including sunk costs is exactly because they are non-recoverable. Unfortunately, that doesn't give a very good picture of the investment made in the project. The fact that the costs are non-recoverable is not because of money, scope or quality (the other project variables) but because time is not negotiable: you can't get it back, you can't get more of it, you can't store it up, and you can't transfer it to other people. Sunk costs are really better thought of as sunk time.

A project is a temporary and one-time endeavor undertaken to create a unique product or service, which brings about beneficial change or added value. [Wikipedia]

This helps to explain why large projects have such low success rates. The larger a project, the larger the sunk time. It's hard to re-start projects, and the larger they are the harder they are to re-start. And yet, the proper approach to sunk costs (time) is to ignore what has gone past and treat your work as if you were starting from scratch. A project approach specifically does not allow you to do this. Hence the fiction of "earned value" (which is a whole other topic).

Only an agile or operational approach allows for low sunk costs. By doing work in extremely small pieces, each of which actually delivers value, an agile approach is best suited for the reality that time is not negotiable. Agile methods acknowledge that the best way to avoid sunk time is to not do much work until you deliver results. As well, since we can't go back to change our mistakes, agile methods allow us to check our work frequently and learn as we go.

In what other ways is time a different sort of variable than scope or cost? In what other ways does an agile approach take this into account?

Posted by Mishkin Berteig at 10:05 AM | |

March 05, 2007

A Better Iteration Structure

In my coaching work, I have often been asked a question about the planning process for iterations, that until just a few days ago, I would actually brush off!!! I didn't even realize I was doing this, it is only in retrospect that I see this. This question is simple: "how does a team plan for the improvement efforts that come out of the retrospective when they are supposed to be working at maximum velocity when implementing tasks directly related to the items in the work queue?" Or, more simply, "we don't have time for process improvements."

The source of the problem shows up in the normal* structure of an iteration. This structure is as follows:

Step 1: set a goal by selecting work from the top of the work queue.
Step 2: plan the execution of that work by doing a task breakdown and task-level estimation.
Step 3: do the work according to the tasks (this is the bulk of the time in the iteration).
Step 4: do a demo to review the work - use this demo to adjust the work queue if necessary.
Step 5: do a retrospective to review the "how" and come up with action items to make the work more effective next iteration.

Since iterations follow one after another, this structure when rolled up can be re-written in the following manner:

Action (step 3)
Reflection, Learning (step 4)
Reflection, Learning, Planning (step 5)
Planning (step 1 and 2)

Now if we look at the purpose of the steps in comparison to the type of learning activity taking place, we see that there is a big disconnect between the planning that happens in step 5 (the retrospective) and the planning that happens in steps 1 and 2 at the start of the next iteration. Not only that, but reflection and learning are separated into "what" and "how" so are done twice for each iteration.

Frankly, this causes a lot of confusion: I see it in my coaching when teams try to accommodate the two types of planning, and I see it in my training when I teach the structure of the iteration.

---

So what is to be done? Simplify the structure of the iteration. Instead of thinking of the iteration as a linear block of time that starts with step 1 and ends with step 5, think of it as a cycle that is dominated by Action, and punctuated regularly by Reflection, Learning and Planning. Now, instead of four separate meetings, there is a single meeting every iteration that combines the wrap-up of the previous iteration with the start of the next. Let's call this the Iteration Transition Meeting or the RLP Meeting or the Adapt Meeting... (suggestions welcome!)

The agenda for this meeting is very simple and contains the essential elements that exist in the "old" style. It starts with Reflection where the facts of the Action are examined: what did we build? how did we build it? how did we feel about what we built? etc. This moves fairly quickly to the Learning component where we try to understand what the facts are telling us: what went well? what needs improvement? what insights can we glean? what new questions are open to us? and of course, what did we learn? Now we are prepared to treat Planning holistically, with no "hidden" activities. We examine our Work Queue, re-arrange it, put new items on it, etc. and do our normal task breakdown and estimation. The difference this time is that both direct business tasks and process improvement tasks are included in the same planning activity. This makes task level (commitment) velocity more truthful!

I will admit that this is not something that I have tried yet. I have seen it done accidentally when people start to ignore the action items from the retrospective in the old iteration structure and instead include them in the task planning. When it happened, I thought it was "messy" because it broke the separation between "what" and "how" oriented activities. Now I think that this might be the best way to do things.

Anyone out there willing to give this a try? If so, I would love to spend some time with you working on it - helping, experimenting, and learning from it.

I should mention a couple important points that didn't fit in the above narrative. First, this insight happened as a result of discussions over the past week with my father, Garry Berteig who is using agile methods and the learning circle in his classroom environment. Second, Garry has been doing what I just described in his classroom environment; the main difference is that Garry provides a great deal of the Planning to his students by way of the progression of assignments/activities.

---

One other cool thing: I'm in Frankfurt as I write this on my way to Chennai to deliver a three-day Agile Project Management / ScrumMaster Certification course. I haven't done international travel since I went to Beijing last summer, and this is the first time I am going outside of North America for work! Fun!

---

* Normal here means Scrum and XP for the most part. Lean doesn't have this structure and as for other agile methods, I don't know them well enough to say if this is normal or not.

Posted by Mishkin Berteig at 04:16 AM | |

February 22, 2007

Strategic Plan

Okay, this is only marginally related to agile, but I thought it was interesting nevertheless: How to Write a Detailed Strategic Plan. The main connection to Agile Work, is that you need to have a clear performance goal in mind towards which you are working. This may be a great way to clarify your thoughts about such a goal.

Posted by Mishkin Berteig at 11:43 AM | |

January 27, 2007

The Freedom of Limited Capacity

Something that I would have thought impossible has happened. By understanding how incredibly limited my capacity to do work is, I am getting a greater and greater sense of freedom and contentment.

My "experiement" to run my business using Agile Work practices is starting to bear some fruit. The trouble is, that so far, that fruit is simple better knowledge of my extremely limited capacity to get things done during a single week-long iteration. I can do 3 or 4 items from my Work Queue which works out to about 15 or 20 tasks. This seems ridiculous to me! Surely one hard-working guy can get more than that done!

But I can't. I'm getting to the end of my fourth iteration and it's the same thing again!

I know some of the reasons for this: interruptions, unexpected work, unremembered work, and my general nature of being easily distracted by shiny things!

In some ways this has been quite frustrating. I have a huge queue of work I want to get done for my business. But the other side of the coin, the one that looks different than I expected, is that knowing just how limited my capacity is... is liberating! I don't feel nearly as stressed as I did four weeks ago. I have the courage to commit at an appropriate level. I can start to believe my plans. I can see - realistically - just how much I need help to achieve my goals. I can see how soon I will be able to do cost/benefit analysis on hiring vs. doing the work myself (can't do that yet). I can even start to believe that I might be able to reach my original goal of reducing my working hours from 80+/week to 50 or less... as long as I make sure that my Work Queue is properly prioritized with that goal in mind.

---

I have worked with a number of teams that have gotten to this point of certainty about their capacity. I've seen this happen in others... and now I recognize it for what it is: relief. One team I worked with after only two iterations had a very clear idea of their capacity, and you could feel the comfort level of everyone with this new agile thing skyrocket. Another team I worked with got a clear sense of its capacity after only three iterations. This team settled in and then started to work to increase their capacity and did a fabulous job by using automation tools, eliminating various organizational bottlenecks, etc. They wow'ed the rest of the organization with their incredible, consistent delivery of value. One of my former students, Dmitri Zimine, gave a presentation at the XPToronto group where he talked about the incredible level of trust that developed between his team and the rest of the organization after consistently, iteration after iteration after iteration meeting their commitments.

All of this is only possible by a rigorous application of timeboxing, careful and consistent task breakdown, and good, honest tracking of results. I wrote a previous article called Seventeen Tips for Iteration Planning. Look it over and see what new ideas you can apply in your situation. If you are frustrated by a team continually over-committing, you will find a lot of valuable advice there.

---

As a parting shot: consider how you feel right now. Are you stressed out? Over committed? Failing to meet your commitments? Feeling guilty? Working insane hours? Not spending enough time with your family? Killing yourself!?

What about the team or organization you work in? Is it constantly being bombarded by problems it can't handle? Surviving on pure heroics?

I can't tell you that following a careful iterative planning approach will fix your problems... but I can tell you that it will reduce your stress levels, help you (or your organization) make appropriate commitments, and let you see the reality of your situation (good or bad).

Posted by Mishkin Berteig at 03:07 AM | |

January 22, 2007

Cancelled Iteration

Last week went totally wonko for Berteig Consulting. My planning was bad, bad, bad!

For planning, I totally forgot about three very important scheduling things:

- a ScrumMaster certification course on Thursday and Friday which required not just time in class, but also three hours of commuting each day.

- my birthday for which I had planned a very informal party on Saturday (normally a very productive work day) which consumed the whole day from morning to late at night.

- the departure of my brother Alexei and his wife Ma Jin so that Sunday would be my last day of visiting, again, taking up my whole day from morning to late at night.

Suffice it to say, a 3-day work week was not what I was expecting when I did my planning. Sometime on Thursday I gave up.

I decided to delay starting a new iteration until I could recover a bit. Over the course of four days I get nearly 200 emails that I actually need to pay attention to at some level or another. So, today has been a recovery day and at the end of the day I will plan my next iteration. Since I did not have a delivery meeting (since I cancelled the iteration), I can only do an informal review of the accomplishments:

- some progress on development work for my client
- some progress on invoicing my client
- some progress on automation of my course listings (Selenium is great!)
- no progress on providing a statement to my other client

The causes of this massive failure are obvious. It should be noted that by doing the ScrumMaster training, I did actually deliver value to Berteig Consulting... it just wasn't the value that I had planned!

Posted by Mishkin Berteig at 05:58 PM | |

January 15, 2007

End of Iteration 2 ("Capacity") and Start of Iteration 3 ("Automation")

This second iteration when much better than the first. I committed to an amount of work that was much closer to my real capacity, and I stayed more focused on that work. Here are the results of my demo, retrospective and planning for Iteration 3 which I am calling "Automation" for reasons which will be described below.

Demo

I committed to the following items from my Work Queue:

I also added these in the middle of the iteration (which is really a "no-no", but I did it anyway to my regret).

What did I actually get done? Here are the stats:

For Iteration 002 - "Capacity":
# Work Queue Items At Start: 5
# Work Queue Items Completed: 3
# Tasks at Start: 32
# Tasks Remaining: 17

What is scary is that my velocity is exactly the same in terms of number of tasks: I really need to start my iterations with only 15 tasks.

Anyway, what I need to do for my next iteration is schedule my "demo" with my other stakeholder (my wife Melanie). Demo'ing to myself in the wee hours of the morning is not really practical in terms of my overall goal for this project (running my business at a sustainable pace).

Retrospective

Here are the things that went well:

- Came much closer to getting everything done that was in my iteration. This was great - I continued to use Tiddly Wiki to track my Work Queue and Tasks.
- Felt less stress. Although I still worked very hard, I was not worried nearly as much about the things still on my Work Queue that weren't in this iteration. I also decided on certain boundaries to work: generally work 4 to 6 hours during the day and another few hours after lights out (usually after 11pm).
- The prioritization worked well - didn't feel impelled to do too much that was _not_ in the iteration. Althought I did take a couple more items in, it was more because of opportunity than anything else. My brother Alexei wanted to help me with my book (for example). Next time around, perhaps I should be more explicit about negotiating with myself about taking something out of the iteration if I bring something in.

Here are the things that need improvment:

- Get even better at judging capacity and committing to an appropriate amount of work. The numbers speak loudly! This next iteration I have to limit myself to 3 items from the Work Queue and about 15 tasks. I have to remind myself that I can always bring more work in if I complete my work early and I still have time.
- Care with long term scheduling - made a mistake in my scheduling with a Client - hopefully it will be okay! This is a bit of a nit, but I need to keep my daybook up-to-date even with long-term stuff.
- Spending time with my family - e.g. playing with kids, helping around house, assisting with homeschooling. This was better than I was doing in Dec., but I need to continue to improve here. I would really like to work with Justice and Haifa on learning Chinese. We've got tons of materials to do it, just need the discipline to get through it together.

Planning

Okay, maybe I'm a glutton for punishment, but I've decided on four items from my Work Queue:

Client2 Invoice
Client2 Work
Automate Listing of Scheduled Courses on 3 Sites
Client3 Statement of Account

The key one is the automation since that will be an investment to save me time later on. In my task breakdown it looks like this (tasks with an "x" are already done from the previous iteration):

So, that's 4 items from the Work Queue and 17 tasks (the ones with "--" in front of them are sub-tasks... not sure how to handle those, but for now I'm not counting them separately.

I'm really excited about this. It sucks to see that I get so little done in a single week, but on the other hand, knowing my capacity allows me to understand just how much work I have on my plate and how badly I need two things: automation and assistance! I'm going to start with automation, but I'm also looking for assistance. The trouble is that I can't affort to hire someone on a salary... (if you know anyone who would be interested in working on commission/revenue-sharing basis, I would love to talk!).

---

One final note: I did manage to include my wife Melanie in my planning for this iteration. She seemed pleased :-)

Posted by Mishkin Berteig at 03:09 PM | |

January 08, 2007

Planning Iteration 0002 "Capacity"

I've completed my iteration planning for my second iteration. As a reminder, I'm doing this because I want to be working only 50 hours per week by July 2007. My sole improvement item from last iteration was to use get better at committing to an amount of work within my capacity. Here's what I have planned:

I have included my complete list of work to be done broken down into tasks. It is anonymized:

Client1 Proposal Writing
- 1 day intermediate team course
- phone consult details
- QA/tools coaching
- email proposal
Client2 Invoice
- get template from them
- collect hours
- collect receipts
- image receipts
- currency convert receipts if needed
- type up hours
- type up receipts
- email/submit
Client2 work
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
- 4 hour task
India Training
- ping Contact1 "V"
- ping Contact2 "N"
- ping Contact3 "M"
- ping Contact4 "S"
- ping Contact5 "V"
- air reservations
- prepare visa application
- submit visa application
- hotel reservations
Client3 Statement of Account
- equipment
- invoice 1
- invoice 2

As you can see, I have committed to only five items off the Work Queue for this iteration and they add up to thirty-two tasks. This is more than I indicated last night in my retrospective results. I'm nervous about this in two ways. One, I think that I am under-committing and that I'll get into huge trouble if I don't do more than this. Two, I think that I'm over-committing and that with my brother visiting, even doing this much smaller amount of work will be challenging. Still, I have tried to make both the items from the Work Queue and the tasks smaller than in my previous iteration. I guess I'll see in a week!

Posted by Mishkin Berteig at 11:01 AM | |

First Interation Ending

My first iteration using Agile Work for my business development has come to a close. Here is what I did for a "demo" and retrospective.

Demo

This was simple. I looked at my selected Work Queue items and my tasks and check to see what I had accomplished (finished or partially finished). Here are the statistics (grim and discouraging as they are):

For Iteration 001 - "Beginnings":
# Work Queue Items At Start: 11
# Work Queue Items Completed: 2
# Tasks at Start: 43
# Tasks Remaining: 28

From these numbers, for my next iteration I should probably be hesitant about committing to any more than 2 items from the Work Queue and any more than 15 tasks.

Considering how many work hours I have in a week, 43 tasks is absolutely ridiculous... unless they are really small, which many of them weren't. For example, I had one task as "Integrate part one of Alexei's feedback into my eBook" which when I did it actually took about three hours. Its just one sample, but 3 x 43 is 129 hours and is just not possible for a single human being in 7 days.

Aside from the value-oriented items from the Work Queue, I also did the work to finish a more complete Work Queue. It now has 37 items including those which remain un-finished from this first iteration. It is poorly prioritized, but that will improve over time. As well, I suspect there are quite a few things missing that I forgetting. I should probably get my wife, Melanie, to look over it and make some suggestions. I also have some old "to do" lists for my business which might remind me of some things to add onto it near the bottom.

Retrospective

I did a little mini-retrospective to find the three things that went well this iteration, and the three things that need improvement. I also decided on a clear action item to take to try to improve my next iteration. (Another clear reason to use Card Meeting - worked great for my retrospective :-) .)

Three Things that Went Well:
- Referred to my iteration tasks frequently. I kept my TiddlyWiki up in my browser and checked it several times a day.
- Spent time with my family and friends despite huge workload. We had guests and family visiting and I got some good time in with them. I also spent a good amount of time with Melanie and my kids. I had to make a conscious effort to do this, but since this is the goal of my using Agile Work (to have a good life/work balance), I figured I might as well start making some progress on it right away!
- Had courage to do some development work which required changes to other developers' work. I ran into a problem with some of the work I am doing as a contract. I dug around and realized that any fix I made would be felt by other parts of the codebase as API's changed. I made the fix and then tracked down the dependencies and fixed everything up. Fun!

Three Things that Need Improvement:
- Pay more attention to my schedule / calendar so as to reduce problems with work/family boundaries. Still a loooooong way to go on this one!
- Spend more work time "on-task", rather than on things not on my list. I ended up doing a bunch of work stuff that wasn't in my iteration plan. Partly I did it because I got bored of what I had planned, and partly becuase I forgot to focus on my plans. It would probably help if I had a Process Facilitator breathing down my neck :-)
- Make realistic plans given my capacity for work. This is the biggie! It is going to be very hard for me to keep to a small commitment. But considering the my brother is visiting from Beijing with his wife this week and next, I really have to try to get this right.

It is that last point that I will use as the basis for my next improvement. For my next iteration planning meeting (to be held in the morning after a far-to-brief sleep), I will commit to a maximum of 3 items from the Work Queue. This may mean breaking the items down into smaller bits. I'll talk with the Queue Master (myself) tomorrow morning about it :-)

Posted by Mishkin Berteig at 04:37 AM | |

January 03, 2007

My First Challenge

Wednesday is nearly done and I'm looking at my list of tasks and cringing! I've only done a few out of the forty for this week. What's going on?!

Well, for one thing, I'm over-committed, and I know it. This is why my goal is to eventually time-box my effort to 50 hours per week. As it is, I expect I'll have a few more short nights this week in order to get even a reasonable amount of my commitment done.

Also, I've been cherry-picking. I'm doing the easy/nice/comfortable tasks first. I'm avoiding the paperwork tasks. I'm avoiding the administrative tasks. These are the ones that I hate and I always procrastinate about. (Hmm... writing this is actually procrastination too... just a few more words and then I'll get back to work.)

Finally, I've got some obstacles which I'm not sure I will overcome this iteration. First of all, I'm spending less time on work than I anticipated. Secondly, I'm getting distracted by tasks that are not directly related to my work items, but which seem to be pre-conditions. For example, I need to do a bunch of administrative follow-up for recent courses I have delivered. Part of that is to send of images of the attendees license forms. I've got the images on the SD card in my digital camera. I've also got a lot of other images on there. So, instead of focusing purely on the task at hand, I am actually cleaning up the whole card. Which is getting rid of technical/administrative debt, but it is slowing me down. Hmm... "administrative debt"... that's gotta be a common theme for the procrastinator!

Anyone out there want to write an article for Agile Advice about administrative debt?

Posted by Mishkin Berteig at 05:16 PM | |

January 02, 2007

Top Ten Most Popular Entries from 2006

If you are new to Agile Advice, these are not just some of the most popular articles, they are also some of the best! Take a look around; you will find ideas to inspire you, challenge your thinking and maybe that one little thing that will make the difference in applying agile methods in your situation.

1. How Two Hours Can Waste Two Weeks - 25,617 unique views
This little hypothetical story by Dmitri Zimine was very popular on Reddit, and Joel on Software ranted a bit about it.

2. The Case for Context Switching - 2,936 unique views
My rebuttal to Joel's rant. Goes well with Dmitri's article. Emphasizes the idea of building trust in an organization.

3. The Qualities of an Ideal Test - 1,579 unique views
Six qualities that will help make your tests as valuable as can be. Similar to the ACID properties of databases or the INVEST properties of user stories.

4. The Pros and Cons of Short Iterations - 1,555 unique views
A few words that will help you decide how long to make your iteration length. This is an important decision, and most teams and organizations don't know the factors involved.

5. Five Signs of Trouble in an Iteration - 1,489 unique views
A good howto on using burndown charts to discover problems in an iteration.

6. Seventeen Tips for Iteration Planning - 1,427 unique views
A good list of hints and tips that can make the difference between struggling with iteration planning and having it go smoothly and effectively. This is a key part of the Agile Work process, so getting good at it is a high priority for any team new to Agile Work.

7. Change is Natural - "Embrace Change" - 1,397 unique views
A short riff on the universality of change. Also introduces the idea of the "horizon of predictability".

8. The Art of Obstacle Removal - 1,287 unique views
This is a good reference article on types of obstacles and methods for removing them... a critical practice for Process Facilitators.

9. The Seven Core Practices of Agile Work - 1,285 unique views
Agile Work is really quite simple. This is a concise list of the practices that make up this effective methodology.

10. Interview with Alistair Cockburn - Agile and House Renovations - 902 unique views
Applying agile methods to home and garden renovations! Learn a bit about how this luminary of the agile world has taken agile practices out of the software world and into the hardware world.

Posted by Mishkin Berteig at 06:32 PM | |

My First Iteration Planning

I've done my iteration planning for my first iteration called "Beginnings". The length of my iterations is one week (including weekends). Here is the list of backlog items that I have committed to (some detail removed):

Catch up on Scrum course follow-up work
Advertize future courses
Finish eBook 2nd Draft
- email request for feedback return
- go through feedback
- finish references
- finish diagrams
Invoice
Finalize India Trip
Interview (Tuesday)
Meet up with and co. (Sunday)
Continue development work for

I have broken all of these into tasks, but only put here the ones for finishing my eBook since the rest get into lots of private detail.

I'm currently using TiddlyWiki to track my Work Queue, my tasks and my recurring/scheduled duties. I create a new wikiword for each iteration and copy items from the Work Queue into that section. I also have a menu that has links to my daily, weekly and monthly tasks.

I'm not doing any estimation on either Work Queue items or tasks. This may become a problem, but for now I'm going to try doing the work without the estimation overhead.

I also spoke with my wife, Melanie who is my business partner about all this. She agrees that using Agile Work is a good step to take and looks forward to all the benefits of my being more organized :-)

Posted by Mishkin Berteig at 12:05 PM | |

August 29, 2006

Seventeen Tips for Iteration Planning

Agile Work requires a team to take work items from a prioritized list, break those items into small tasks and then execute those tasks inside of the timebox of an iteration. When first trying agile, many teams have trouble with the task breakdown done in the iteration planning meeting. Here are some hints and tips for making this critical part of the agile process more effective.

1. Work items are, among their other qualities, valuable to the customer or stakeholders. Think of these work items as being built out of many smaller pieces. You can imagine a toy model made out of Lego bricks. Each brick, by itself, is of no value. It is only when they are put together that they become useful. The tasks you create for each work item are often small enough that they do not have any value on their own.

2. The word "task" implies activity... but the tasks you create for your work items do not need to be activity based. Tasks can be effective if they are written as components or pieces of the work item you are building. Here is an article about creating good tasks.

3. Sometimes if the work item is not well understood, you might find that a "research" or "experiment" task is a good starting place. Try to be as specific as possible. When writing the task description, spell out what exactly is the goal of your research, and possibly list what options you are going to research.

4. When first starting out with Agile Work, many teams find it difficult to do a good job of iteration planning in the fixed amount of time allocated to it. Consider shortening your iteration length so you can practice this skill more frequently. (Remember to make the planning meeting shorter too!)

5. Make sure that everyone has index cards and a pen to write with! Team members shouldn't have to wait for a "scribe" to write down a task. In many ways this is a brainstorming session. The Process Facilitator can collect all the cards after the meeting is finished if they need to be recorded more formally.

6. Do a first pass by creating "big" tasks... then break them up into smaller tasks if you have time. Since this meeting is timeboxed, it is better to get all the work broken down into big chunks than to break down only a small part of the work into very fine chunks.

7. If the same task keeps showing up for all your work items, it probably shouldn't be a task... instead it should probably be a process step or constraint or condition of satisfaction for the work you are doing. For example, if you always have to write a document that follows a template to record what you have done for each work item, then writing that document can be shown as part of your task board.

8. It's okay for tasks to be _very_ small.

9. Share your tasks! If you write a task down without telling the rest of your team, they can't use your idea to generate more tasks, nor can they improve on your idea.

10. Generating tasks in the iteration planning meeting is a problem solving and creative process. This is where you do a lot of your analysis and design work. This is where you struggle through options and choose _how_ to build/do your work.

11. Consider creativity technique such as light-weight brainstorming for generating lots of ideas quickly. Any technique you use should be streamlined for quick results.

12. Don't worry about administrative stuff while you are generating tasks. For example, if you normally put task cards up on a wall, wait until after the meeting is over to do this. Likewise, if you normally enter them into an electronic tools, wait until the meeting is over to do this.

13. Make sure you have scrap paper or a good whiteboard convenient for notes and drawings so that you can quickly model your solution for the work item. (Check out Agile Modelling for a discussion of this in the software realm.)

14. Remember that it's okay if you end the meeting with an imperfect list of tasks. You will make corrections throughout the iteration. It is more important to maintain the discipline of the timeboxed meeting length, than to get the tasks right up front.

15. The whole team must participate: everyone's experience, skill, expertise and insight are needed to do the best job of generating tasks. Just because it won't be a perfect list doesn't mean you can do a shoddy job of it!

16. You need quick access to information about the work items you are planning. You also need quick access to other relevent information. A computer with a web browser open to Google is a great tool to have at hand.

17. If you don't have anyone on your team who has lots of diverse experience and expertise, then consider inviting someone like this as a guest to help you out. It is much more difficult to do the necessary problem solving if you new to the medium in which you are doing the solving. Such a guest would need some time before the meeting to be prepared.

Posted by Mishkin Berteig at 10:44 PM | |

June 14, 2006

Agile Work Artifacts - Tasks

I recently described the Work Item List. The second type of artifact used in an Agile Work environment is the Task. At its most basic, a Task is a simple description of how to do some bit of work towards completing an item in the Work Item List. However, there are some important things to remember when using Tasks.

What is a Task

Normally, an item from the Work Item List is broken down into multiple Tasks. These Tasks are all the things that need to be done or built to get the item into a deliverable state.

In general, the process of creating a bundle of Tasks for a given Work Item is a design or analysis process. It is a problem-solving process. The Tasks themselves represent the solution: the building blocks of the structure of the Work Item. Tasks do not normally represent the problem solving or analysis process.

Here are two contrasting (somewhat contrived) examples of Tasks generated for a "Home Office Addition" to make this clear:

Example 1 Bad:

• determine requirements for home office
• design home office layout
• check home office layout with client
• choose furniture and decor
• build home office
• install furniture and decor
• unpack books and files into home office

Example 2 Good:

• 300 sq. ft. maple hardwood
• 4' x 7' custom desk
• Aeron chair
• red leather sofa
• 16' custom bookshelves
• 22" double French doors
• 8' 4-panel vinyl triple glaze bay window
• 36 2x4x10' metal studs
• ... drywall
• ... other building materials and furniture and decor

Generating Tasks

The whole team works together to come up with the Tasks for every Work Item being worked upon during the iteration. The majority of the Tasks are determined during the timeboxed iteration planning meeting. Sometimes, the iteration planning meeting is not long enough to get all the Tasks defined. Sometimes new Tasks are discovered during the course of the iteration. Either way, it is fine to make changes to the bundle of Tasks as the work proceeds.

Special Tasks

Due to the timeboxed nature of the iteration planning meeting, a Team will sometimes find itself knowing that there is more analysis or problem-solving needed for a given Work Item. In this case, a special Task can be created to indicate that this analysis work needs to be completed. For example, the task might be labled "Finish analysis for Work Item 62", or it might say "Team meeting to finish Task generation for Work Item 62". Other ways of writing these Tasks include "Research...", "Solve..." or "Discuss...". Everyone needs to recognize that these tasks represent risk and uncertainty about how to get the work of the iteration complete.

Sometimes work needs to be done that is outside of the control of the Team. The Team should always try to find creative solutions to avoid this situation, but it is almost inevitable that Tasks will come up for which the Team does not have the expertise or the authority. These Tasks need to be treated very carefully since they can seriously hinder a Team's forward progress. The Process Facilitator should usually consider these Tasks as obstacles to be removed.

Working on Tasks

Typically, a single person takes responsibility for a single Task at a time. A person may end up working on other tasks for two reasons. First, if the Task he/she is responsible for has an unavoidable wait, then that person may go find someone else to assist until the Task is ready to be worked on again. Secondly, another person may have an urgent need for assistance and call other people away from their current Tasks.

Tracking the Work

In their most basic form, Tasks are assumed to all be roughly the same size or effort and small enough that they are only done or not-done. Therefore, tracking the work on the Tasks throughout the course of the iteration is kept very simple: how many tasks are remaining to complete in the remaining time of the iteration? There is no ambiguity: Team members report on completion of tasks to the Process Facilitator who updates a burndown chart to show the quantity of remaining work.

Gotcha's

Tasks must be written to avoid including wait time. For example, a Task that says "Get the parts delivered" actually includes three steps: make the request for the parts, wait for the parts to be delivered, receive the parts. Since it is often hard to predict how long a "wait" will be, it is important to break this down into two separate tasks: "order the parts" and "receive the parts" and then treat the wait time as an obstacle to be removed/minimized.

Posted by Mishkin Berteig at 09:34 AM | |

June 13, 2006

Fantasy Estimation

In software development (and in many other types of projects), there is a typical non-agile approach to estimation of project size. This method starts with a high-level understanding of the work to be done, the requirements, and uses that to make an initial estimate of the project size. This estimate is often stated in units such as man-months. There is a very important piece missing here that makes this estimate completely useless... that makes it pure fantasy.

The Team

Any estimate made by anyone other than the team doing the work is useless. For any sort of project, software or otherwise, estimates made by a project manager or leader on behalf of an existing, or, even worse, a non-existant team, must be subject to the highest degree of scepticism.

Team Maturity

Not only does the team need to make the estimates for its own work, but the team must be mature! If the team is newly-formed, the team members will have no sense of the team's capacity other than their own individual capacity.

Work History

Not only must the team be mature to make reasonable estimates, it must have a recorded history of their own work capacity in order to be able to estimate their capacity to do work in the future. If there is no record of their capacity, then the usual factors of optimism will be unaccounted for, and the team will almost always over-estimate its capacity.

Knowledge of Domain and Tools

Not only must the team use a recorded measure of their capacity, but they also must be experienced with and knowledgeable of both the subject of the work and the tools they will do to perform the work in order to come up with a reasonable estimate of a project's size. If they need to learn about the project and how to execute it, then again, the team will almost always over-estimate its capacity.

Fantasy

If someone gives you an estimate for project duration, ask them the following questions:

1. Was this estimate made by the team which will actually be doing the work?
2. How long has this team worked together?
3. Has this team measured their own actual capacity and used that measurement for this estimate?
4. How well does the team know the domain and tools to be used?

If the answers to these questions are unsatisfactory, then the estimate is pure fantasy, it is a lie.

Posted by Mishkin Berteig at 09:06 AM | |

June 06, 2006

Agile Work Artifacts - the Work Queue

Agile Work requires only a very small number of simple "artifacts". The most basic is the Work Queue. This is very similar to the Scrum Product Backlog but there are some differences too. The Work Queue is like a carefully managed To-Do list. This article details the use of the Work Queue.

Basic Description

The Work Queue is a list of work to be done by a person, team, organization or community. The list consists of brief descriptions of what to deliver, not how to do it. The list is prioritized so that the item on the top of the list is the single most important thing to get done, with items below that being successively lower priority. All items on the Work Queue contribute directly to an overall goal. The person holding the Product Owner role manages the list.

Place in the Process

The place of the Work Queue in the process is very simple. The Work Queue is initially created after the overall goal is determined. Ideally, it is created before work towards the goal is started, but often Agile Work will be adopted for an ongoing effort, in which case work has already started. The Work Queue is constantly maintained by the Queue Master so that it maintains the above mentioned qualities. One or more of the items from the top of the Work Queue are selected to be worked on. As an item on the list is completed, it is removed from the Work Queue. The number of items on the list that are being worked on simultaneously will depend on the capacity of the people doing the work. If iterations are being used to plan work, then the Work Queue is updated every iteration.

How is it Created/Managed

The Product Owner is responsible for creating and managing the Work Queue.

The initial creation of this Work Queue is a simple process: based on the goals to be attained, and based on an other factors that are relevent, the Product Owner drafts an initial version of the Work Queue. The time spent on creating this initial version should be minimized using three techniques:

1. Keep the Work Queue small by recording only high priority items that need to be done immediately. Don't add items that are uncertain or will be done far in the future. (See "The Horizon of Predictability" for more information about this.)

2. Keep the Work Queue simple by focusing on point-form high-level descriptions of the items in the list. Don't worry about any details about how the item will be accomplished, who will do it, when it will be done, dependencies with other items, or technical or conditional details.

3. In advance, allocate a fixed amount of time to draft the Work Queue... and don't spend any more time on it than that. Get the assistance of the Process Facilitator to keep on track. The list doesn't have to be perfect or complete... you only need enough on the list to allow the work to start.

Ongoing management of the Work Queue consists of removing completed items or items that are no longer needed, adding new items as they are discovered, merging or splitting items as more is learned about them, and reprioritizing items. All of these tasks can be performed at any time. As well, the Work Queue should always be in a state of readiness such that the top items are ready to be selected for work.

The Product Owner is responsible for answering any questions that may arise about the work as it is progressing. If someone doesn't know what is meant by an item, then work on that item should halt until the Product Owner can clarify.

At no time is the Work Queue "frozen" or finalized. It is changing throughout the entire time that people are working towards the goal. Items that are completed are removed from the list, and do not need to be archived or recorded anywhere else, nor is it necessary to save versions of the list as it is changed.

Gating/Queuing

The Work Queue is a queue of work in process (WIP) that is built up in front of the people who are doing the work. As such, it is advisable to keep the list short. The Product Owner acts as a gate to the list: only those things that reasonably contribute to the goal and which are relatively immanent in implementation should be added to the Work Queue. If someone suggests something be added to the Work Queue, the Product Owner must, of course, take that under advisement and collaborate with the suggestor, but is not obligated to add the suggestion to the list.

The Product Owner can manage the size of the Work Queue by deciding how many iterations of future work are allowed on it. In other words, the number of items on the Work Queue is limited to how many can be accomplished in a fixed number of iterations. As the Team takes a number of items off the Work Queue, space is freed to add an equivalent amount of work back onto the Work Queue. The specific number of iterations chosen for this will vary depending on your circumstances.

Differences from the Scrum Product Backlog

The Work Queue does not require two things that are part of the Scrum Product Backlog: item estimates and item values. Adding this information to the Work Queue can be useful in many situations, but is not an essential part of the list.

As well, the management of the Work Queue is different from the Scrum Product Backlog in that the Product Owner is not obligated to put every suggestion that comes their way onto the list.

Advanced Use of the Work Queue

Consider adding the following information to each item on the list:

1. An estimate of effort. Make sure that these estimates are created by the whole group of people who will be doing the work.

2. An estimate of value. The Product Owner needs to be responsible for determining value, but the whole group of people doing the work and all the other stakeholders need to understand how that value was determined.

3. Cycle time tracking. When an item is added to the list, record the date/time it is added, then record the the date/time it is completed and removed from the list. The use of cycle time can assist in making a work process more efficient.

Posted by Mishkin Berteig at 04:24 PM | |

May 31, 2006

The Human Touch

If you are given a problem to solve, how much does the presentation of that problem matter to your ability to solve it? Imagine that it's a simple problem... imagine that it is presented in two different ways, both of them simple. It turns out that presentation differences can still make a huge difference. In fact, there is a way to present problems that makes them substantially easiers to solve: make them people problems.

In The Tipping Point: How Little Things Can Make a Big Difference, by Malcolm Gladwell, we are given a very concrete and suprising example of this. Here it is quoted in its entirety:

Consider the following brain teaser. Suppose I give you four cards labeled with the letters A and D and the numberals 3 and 6. The rule of the game is that a card with a vowel on it always has an even number on the other side. Which of the cards would you have to turn over to prove this rule to be true?

Go ahead and take a few moments to figure that out before continuing on to the answer, and keep track of how long you work on it...

The answer is two: the A card and the three card. The overwhelming majority of people given this test, though, don't get it right. They tend to answer just the A card, or the A and the six. It's a hard question. But now let me pose another question. Suppose four people are drinking in a bar. One is drinking Coke. One is sixteen. One is drinking beer and one is twenty-five. Given the rule that no one under twenty -one is allowed to drink beer, which of those people's IDs do we have to check to make sure the law is being observed?

How long does it take you to figure that out?

Now the answer is easy. In fact, I'm sure that almost everyone will get it right: the beer drinker and the sixteen-year-old. But, as the psychologist Leda Cosmides (who dreamt up this example) points out, it is exactly the same puzzle as the A, D, 3, and 6 puzzle. The difference is that it is framed in a way that makes it about people, instead of about numbers, and as human beings we are a lot more sophisticated about each other than we are about the abstract world.

Now unless you had heard about this before, I suspect you were pretty suprised. I know I was! I always considered myself to be a very good abstract thinker/problem solver. In fact, I considered myself to be well above average in that regard for a number of reasons: I was always very good at math without every memorizing a single formula (I always made them up as I went along as long as I remembered the _idea_ of the formula), I was an excellent programmer in a number of different computer languages including assembler, Miranda, Java, Prolog, Pascal, and Objective-C, and finally, I'm always solving problems by moving the problem to a new level of abstraction - solving the meta-problem first.

So what does all this have to do with agile work methods? Quite a few things actually:

1. Obviously, if you can frame a problem as a people problem, it will be easier to solve... and most problems start out this way!

We tend to try to abstract problems, make them more generic or general purpose in the hopes that they can be communicated more precisely and can be solved in a way that will accomodate contingencies we haven't thought of yet. But all the effort we put into abstracting the statement of the problem ends up costing us doubly: in the initial abstraction and in the difficulty of solution that results. So if you have a team that is solving a people problem, make sure to keep it a people problem when you give it to the team!

2. If you have a problem that is given to you in an abstract form, try to convert it to a people problem before trying to solve it.

In all likelihood, the moment you do the conversion, you will quickly see the solution. It may even feel like the process of de-abstraction is a problem-solving process. You may have to make really odd connections to make the problem a people problem but it will likely be worthwhile.

3. Dealing with people rather than abstractions on a day-to-day basis will always result in a more effective interaction.

Sending printed documents, writing emails, manipulating symbols are all interesting ways to communicate, but fundamentally, you are communicating with other people. If you can make that communication as direct as possible - phone, video conference, in-person - then there will be far less effort involved in understanding the communication, and far more effort can be allocated to high-bandwidth communication. This obviously has special relevence for teams: get people in the same room as much of the time as possible.

---

In the software world, there is one technique that I give teams and that is the use of Personas to assist in solving a software problem. The place I first encountered Personas is in the excellent book The Inmates Are Running the Asylum by Alan Cooper. This book presents some of the basics of the Interaction Design discipline.

The bare essense of the Persona is to create a fictional person who represents a user or actor or stakeholder or customer of whatever it is you are building. This fictional person should have a name and all conversation about the thing being built should be couched in the personal language of these Persona's names. A Persona should also have a short history, a photo and some description of their needs, goals or desires. All of this helps to frame everything about a software project as a people problem... and thus makes it much easier to discuss and solve.

Posted by Mishkin Berteig at 11:12 PM | |

September 26, 2005

Wideband Delphi Estimation Technique

Here's an excellent introductory article on the wideband delphi estimation technique. Typically wideband delphi is used to estimate software development efforts, but can be used in almost any domain of work. This method might be applied to estimating effort for items in a work list at either a project level or tasks in an iteration work list. The way it is described, it sounds fairly heavy-weight, potentially taking several hours for a relatively small list of work items. However, it is worthwhile for process facilitators and product owners to be aware of these sorts of methods if problems with estimating occur in a project team. The wideband delphi model is related to the Delphi method.

[Update 2007/05/24]

Agile Work and Scrum use a very different method of estimating work that has one very important and incredible property: people don't have to get better at estimating for them to get better at committing to work!

The basic idea can be thought of as "commitment velocity". The method here is to use an arbitrary unit of effort that the team applies to estimating all tasks relative to each other. At the start of an iteration, the team can use any collaborative method (including wideband delphi) to come up with estimates for tasks. The team sums up the estimates for all the work of the iteration. Then, at the end of the iteration the team looks at the estimates for all the remaining undone work (if any) and subtracts that from the start of iteration sum. This is the Commitment Velocity. Now on their next iteration, they do their estimation work again, relative to the tasks in the previous iteration. If the sum of the estimates is larger than the Commitment Velocity, then the team needs to de-scope or find more efficient solutions to their work. This process usually converges after only three iterations (assuming: constant team composition, constant iteration length).

This method is taught in detail in my Agile Project Management courses, and there is a little bit more information here: Planning vs. Commitment. The agile estimation method of Commitment Velocity is an application of the Central Limit Theorem from statistics.

[Update: 2007/09/11]

Wikipedia now has an article on Wideband Delphi. The article points out some interesting potential problems with the wideband delphi method including corruption of the group doing the estimation and suppression of minority opinions. I know that in my experience as a coach, I have seen numerous occasions of both types of problem with estimation processes. The fact that wideband delphi is susceptible to these problems is more a factor of the human side of things than the method itself.

Interestingly, the wideband delphi method of estimation is very similar to the "Planning Poker" method of estimation used in many agile and scrum project environments. This method is also covered in the Agile Project Management courses.

Other links to wideband delphi:

Wideband Delphi Estimation Process

The ProjectInitiation.com website has a wideband delphi worksheet (MS Word) available.

Project Estimation: Wideband Delphi

Posted by Mishkin Berteig at 12:40 AM | |

May 17, 2005

The Qualities of an Ideal Test

These six qualities of tests describe how to make a test as effective and as useful as possible. The qualities are similar in style to the INVEST qualities of user stories - but they don't form a nice acronym.

Decisive: The test contains all the information required to automatically determine success or failure. Manual inspection of test results is not necessary to determine success or failure. The test is expressed in a way that produces a pass/fail answer rather than a numerical or qualitative result. Decisive tests are often expressed as assertions.

Valid: The test produces a result that matches the intention for the work artifact under test. Test failure indicates failure in the artifact under test and test success indicates correct operation or configuration of the artifact under test. Simply put: the result of a test should be true.

Complete: The test contains all the information it needs to run correctly with a given test harness and work artifact under test. The test performs all activities and provides all the data necessary. The test does not require any input outside of itself in order to run.

Repeatable: The test always gives the same results if the test harness and the artifact under test are the same. The test is created in such a way that the result is deterministic. Even if the system under test is not deterministic, the test should be created to account for that (possibly with statistical analysis) and produce a deterministic result.

Isolated: The test result is not affected by other tests run before it nor does a test affect the results of tests run after it. The test and the test harness work together to clean up after every test is run. A collection of tests can be run in any order and always produce the same results. Any test that depends upon the results or side-effects of a previous test is not isolated.

Automated: The test requires only a start signal in order to run to completion in a finite amount of time. No manual intervention is required after the test is started. Tests that are automated can be put together into a test suite and run one after another without human intervention. Automation of tests requires the creation of a test harness system.

---

Update 20060106

I've added a little bit more detail to the above qualities. I also wanted to say a few words about my experience with testing:

I first started doing test-driven development almost seven years ago after reading the book Refactoring by Martin Fowler. I picked it up in Windsor Ontario while I was driving to San Francisco for my first contract position as an independent. I had to stay the night there due to the immigration office being closed so I read a lot of the book in my motel room that night. By time I got to California four days later, I was totally convinced that I should be using JUnit and refactoring for everything I did. Fortunately my project was ammenable to this approach. Over the next four months I built a Java JMS layer on top of Tibco Rendezvous. In the process I discovered methods for doing unit tests for asynchronous multi-threaded distributed functionality. And my tests satisfied all the above qualities. I delivered code with 100% test coverage and zero defects discovered until more than two years later when a very rare and obscure deadlock issue was found. Suffice it to say, I was convinced from a quality perspective.

But there is more. To build tests that follow all the above qualities, you need code that is testable. I have come to believe that testability is the most important architectural attribute of code for most software. The implications of having testable code include code that is easily changed, that is verifiable, and that is easy to understand. Refactoring plays a big role here too.

For getting a basic but very practical sense of test driven development, I strongly recomment Test Driven Development: By Example by Kent Beck.

---

I have also written a couple of articles recently about quality:

Quality is Not Negotiable in which I talk about how to handle defects - by always treating them as top priority work items.

and

Technical Debt in which I expand on this common analogy between lack of testing and debt to show that technical debt is even worse than financial debt.

Posted by Mishkin Berteig at 06:48 PM | |

May 16, 2005

Iterative Delivery

Work can often be divided up so that the smaller pieces are valuable on their own. By dividing work this way, a team can deliver value incrementally. The team can choose a short period of time called an iteration and select a small amount of work to complete in that time. This work should be valuable on its own. For example, if a team is building something, then at the end of each iteration whatever is built is usable as it is. This means that each iteration includes all the planning and design as well as construction or creation necessary to deliver a final product or result.

For example, a volunteer group may desire to attract new members. A non-agile approach would have the group plan their membership campaign completely before actually executing on it. An agile approach using iterative delivery would have the group plan a small piece of work that will attract some small number of new members, execute it, and then start a new iteration. One iteration may cover the creation of and delivery of a door-to-door flyer in a neighborhood. Another iteration may cover the design, creation and publishing of a small advertisement in a local newspaper. Each iteration includes all the steps necessary to produce a furthering of the group's goal of attracting new members.

In a business environment, iterative delivery allows for a much faster return on investment. The following diagram compares delivering value iteratively with a non-agile project delivery where results are delivered only at the end of the project:

One can see clearly from the diagrams that the non-agile delivery of value at the end of a project is also extremely risk prone and suseptible to change. If the project is cancelled just before it delivers, then a fairly substantial amount of effort is wasted. In the agile iterative delivery situation, an endeavor can be cancelled at almost any time and it is likely that substantial value has already been delivered.

Even if the work cannot actually be delivered incrementally, it almost always can be divided in a way so that it can be inspected in stages. Either method of dividing work allows us to do the work in iterations.

Iterations are fixed and consistent units of time during which work is performed and between which planning, inspection and adjustment is done. The empowered team will decide on the length of iterations for their work. As a rule of thumb iterations should be shorter than the horizon of predictability. Generally, iterations should never be longer than one month, no matter what the endeavor.

At the end of each iteration, a demonstration of the work completed is given to the stakeholders in order to amplify learning and feedback. Between iterations, the stakeholders collaborate with the team to prioritize the remaining work and choose what will be worked on during the next iteration. During the iteration, the stakeholders need to be accessible for questions and clarifications.

Iterative and incremental delivery is used to allow for the early discovery and correction of mistakes and the incorporation of learning and feedback while at the same time delivering value early.

Posted by Mishkin Berteig at 10:27 PM | |

May 15, 2005

Truck Factor

Truck Factor (definition): "The number of people on your team who have to be hit with a truck before the project is in serious trouble"

Clearly "hit by a truck" is an extreme thought however you could easily substitute "take vacation at the same time" to get the same idea. If any part of your project has a truck factor of one then you are in a particularly fragile situation. If that one person leaves or is unable to work on the project, you will suffer the consequences.

Over time, anyone can be replaced. Truck factor is an indication of how expensive it will be to replace specific people.

In an ideal situation, everyone on the team will know all parts of the system so that the loss of any one person would have minimal impact. In reality, many projects rely on one or more "heros" who are the only one who understand certain critical parts of the system. When these heros leave (and you should assume they will), you must be prepared to recover.

If you have a hero on your team, the best thing you can do is reassign that person to a different part of the system. This will allow the replacement to ramp up while the hero is still available for support. If you wait until the hero has left then the ramp-up will be significantly more expensive.

An added benefit to reassigning the hero is that this person will now have the opportunity to work on something different. Since the hero's tend to be the most technically competent members of the team, this will usually mean that the new area will improve once the hero has worked on it for a while.

Truck factor is a quick metric that will highlight potential problems in your project. Having hero's on your team can be very beneficial but only if you don't become dependant on them. Truck factor is one metric that will highlight your dependencies.

Cross posted from Mike Bowler's Weblog

Posted by Michael Bowler at 06:42 PM | |

What To Do With the Horizon of Predictability

In a previous entry about constant change, the idea of a horizon of predictability was introduced. This concept, along with the agile discipline of amplifying learning, suggest a strategy for addressing problems in a project.

Shorten the length of the iterations you are using. Contract your "planning horizon". The length of your iterations should be motivated by the horizon of predictability for your environment. If your project encounters trouble, particularly of the sort where it looks like you might not accomplish your commitments for an iteration, then shortening the length of iterations will enable you to resolve your problems.

First off, by shortening your iteration length, your opportunities for learning become more frequent.

Secondly, a contracted planning horizon will put you more firmly inside the horizon of predictability... and therefore there will be fewer unexpected changes (on the whole, not in any specific iteration).

A related article is The Pros and Cons of Short Iterations.

Posted by Mishkin Berteig at 06:34 PM | |

May 11, 2005

Appropriate Metrics

At the Advanced ScrumMaster Training, Ken Schwaber presented a substantial amount of thinking about metrics used with Scrum. The main driver for thinking about metrics has come from implementing Scrum in enterprise situations. Management expects metrics to be used in order to provide visibility into the progress of the Scrum implementation.

While this driver has some legitimacy, there are three main concerns to prescribing metrics for use with Scrum.

1. Planned/Engineering Approach - metrics such as "ideal person days" smell like the bad old way of treating people as resources that can be swapped in and out of a project.

2. Normative vs. Empirical - metrics are often used to set a standard for reasons such as prediction, and expectation. Scrum is about discovery and improvement not prediction and planning.

3. Is Something in Scrum not Working? Is adoption being hindered? Are too many Scrum implementations failing? Are metrics a critical success factor?

Keep these concerns in mind while considerig the uses of metrics.

What are Metrics for?

1. Self-Evaluation over Time - use a metric to track the progress of a group. A measurement is taken at a certain point in time, and then taken repeatedly over intervals. Example: the velocity at which a team completes work can be used to identify problems and opportunities for a team.

2. Control - use a metric to legislate the qualities of some process or attribute of work. A goal or standard is set for a measurement. The size of a queue of projects waiting to be worked upon by a team can be controlled in order to limit the size of work in progress and therefore project inventory.

3. Prediction - use a metric with values collected over time in order to predict future values of that metric. By implication, the metric is used to predict the performance of a system. The number of additional tasks discovered inside an iteration can be tracked and used to determine future expectations on extra tasks discovered.

4. Performance Measurement - use a metric in order to determine rewards and/or punishments and/or adjustments to behavior. An individual on a team may be evaluated based on their productivity as measured by lines of code completed and rewarded or penalized on that basis (not recommended!).

5. Behavior Motivation - use a metric to guide behavior by setting a context for thinking, action and reflection. Focus on an important measure in order to draw attention to improving the attribute with which the metric is associated. Measuring the time elapsed from conception of a project idea to the time it actually delivers valuable results can draw attention to improving speed.

The Lesson from Good to Great

Good to Great: Why Some Companies Make the Leap... and Others Don't by Jim Collins discusses the attributes and behavior that are common and unique to companies that have gone from a long history of mediocre results to a long run of great results. One identified aspect is referred to as the "Hedgehog Principle". In this principle, three questions are answered: "what can we be the best in the world at?", "what can we be passionate about?", "what is my one economic driver?"

The economic driver is a metric. For example, at Walgreens, their driver is profit per customer visit. Other large institutions have other metrics. But all good to great companies have a single economic driver metric that guides all their decision making.

See also: A Metric Leading to Success.

Posted by Mishkin Berteig at 05:21 PM | |

May 10, 2005

Steps in Making a Decision

In her workshop "Advanced Scrum: Collaboration Skills for Scrum Teams", Esther Derby includes a brief discussion on the five parts of a decision.

1. Define the Problem
2. Establish Focus and Boundaries
3. Identify Options
4. Choose Among Options
5. Implement

At each step, it is instructive to examine who is "responsible" or involved. In an agile team where team empowerment and self-organization are considered critical success factors, the answer to "who?" fore each step should usually be "the team".

There are however, some situations where decisions are outside the realm of a team's empowerment. As well, some decisions are so trivial that it is wasteful to have the whole team involved. In these trivial decisions, usually another person can take responsibility for all the steps of a decision as a service to the team.

Over time, a team and the organization in which a team operates can evolve a set of standards that describe who acts in each step of a decision under what circumstances.

Many thinking tools described by Edward de Bono in his various books (such as Six Thinking Hats, Lateral Thinking : Creativity Step by Step (Perennial Library), Textbook of Wisdom etc.) can be used at various steps in the decision making process.

Posted by Mishkin Berteig at 11:48 PM | |

May 05, 2005

Change is Natural - "Embrace Change"

Kent Beck's book "Extreme Programming Explained : Embrace Change" provides a good introduction to how software development can embrace the constant change that affects our world. Some of the practices he introduces are very software-specific. However, the overall basic message is sound and provides a foundational principle for all agile work. (By the way, the book is excellent.)

Change really does occur everywhere. Change is constant. A google search for "embrace change" or "change is constant" will both turn up an incredible variety of articles, papers, discussions, books and viewpoints that all affirm the constant nature of change and the need to embrace it.

Nevertheless, it is sometimes difficult to accomodate change when we also have a legitimate and deep desire to know what is coming next.

For many teams, the environment in which they work is constantly changing. This change can be caused by competition between organizations, scientific or technological advances, fads and cultural shifts, major events in people's personal lives or even just a change of opinion with a stakeholder. Any change, even small change, can invalidate a planned course of action. However, goals (as distinct from plans) are more stable and often survive even major environmental changes. Therefore, rather than trying to plan the future, an agile team can focus on being able to respond to change while still reaching a goal.

Nevertheless, a team needs some sense of what it will do in the near future. A team can work with a “horizon of predictability”. This is the distance into the future which a team can be reasonably certain that plans will be stable. Depending on the environment, this may be as little as a few minutes, or as long as a month. It is rarely longer. The horizon of predictability is not a precise demarcation, rather, expect change with a probability based on the horizon of predictability. Then, plan to respond to change. Be detached from the concrete details of a plan, particularly if they occur outside the horizon of predictability.

Responding to change requires a major mental shift for many people that is difficult and takes time and environmental support. People are often penalized socially or formally for being flexible or adaptable. This quality can appear to be “wishy-washy”, uncertain, indecisive, uncommitted or even rebellious.

The terms “agility” or “agile work” refer to this principle of embracing constant change since it is the most visible of the principles. However, the ability to respond to change relies on the establishment of agile work disciplines and practices.

Posted by Mishkin Berteig at 06:57 PM | |

May 04, 2005

Can dying plan-based projects be recussitated?

We've all seen this. A one-year project in its 13th month, and the Project Manager has been reporting 80% of the tasks at 90% and has been doing so for the last 120 days. There's no end in sight, and the customer is leaking cash every day the product fails to go into production. What can be done? Agile project management principles can help this all-too-frequent scenario.

First, let's look at how this situation comes about. In a typical task-oriented project plan, one is required to decompose the tasks down to a fairly reasonable degree of specificity. The tasks are organized around a dependency graph, estimated, resources are assigned, and the schedule is calculated and/or nudged. (Well, usually the initial estimates don't match the customer's expectation and deadlines, so there's a revision step here where estimates are shortened) But how does change get reflected? If a PM is doing an excellent (if frustrating) job, they are constantly updating the schedule, the plan, and re-conceiving the tasks as new information comes to him.

More often than not, however, this gets so messy and unwieldly that the PM holds fast, and starts to estimate completion based not on a proper decomposition and a completion of each of these tasks, but based on his guess, or his workers' guesses. Complexity of the tasks is hidden, and becomes often quite invisible. Tasks at 80% which suddenly need to be re-decomposed and re-conceived do not, in the main, get moved back down to 40% after certain roadblocks are discovered.

The result? The customer is increasingly afraid, trust between delivery staff and client (and management) are eroded, pressure is increased, mistakes under pressure become more frequent, less sleep is had by all, and 90% complete becomes an asymtote, rather than a milestone.

Now what is to be done with such a project? How can Agile project management approaches help this situation? We can examine this by playing out such a scenario...

We can start by identifying a few key practices of Agile project management, and examine their benefits to the business client.

Timeboxing and Iteration

The first thing we can talk about to the fearful client is timeboxing. Timeboxing caps his investment into small chunks. We look at it and say, OK, you're in deep trouble, but you don't know how deep your trouble is. By timeboxing into a very short timeframe, and making a large project into many little short projects, we can get more visibility into the process - we can see things as they are, rather than how they're reported on a project plan. Speaking of visibility...

Daily Meetings and Iteration Review

Part of the value of iteration and timeboxing is increased visibility, so we really need to have a mechanism for visibility. Already distrustful of project plans, we can tell the client, "Don't believe the paperwork, let's look at what's actually built. At the end of each iteration, we'll review the current situation, and demonstrate existing functionality." His eyes perk up. "Wah?" he asks incredulously? What do you mean? So we say, "we don't want you to guess at how ready this thing is. We'll show you. That way, you can decide if it's ready enough. ...Or your product marketing people, if you prefer. It's up to you, but you get to see it, touch it, and sense for yourself how ready it is, and you won't have development managers and developers waving their hands pretending it's further than it is."

"Oh wierd," says he. "So what are these daily meetings?"

We tell him that the daily meetings have several purposes. Only project members get to speak, and they report on what they did yesterday, what they plan to do today, and what, if anything, is blocking their path. No one else gets to speak, but others who are not on the team itself can listen in. This way, the whole project team is clearly aware of how they're working, what's left to do, and what each other are doing.

"Why do they need this? They have the plan." "True," we reply, "but how often do you have two people who need something, and both do it because they don't know that someone else already did it. With your current project delays, do you want any duplication of effort? Just by way of example?"

Feature Lists, not Task Lists

We also talk to him about working from feature lists, not task lists. The team will be implementing features, and fulfilling requirements. How they do it is up to them. "Why should I care," he'll ask. "You don't," we reply, except to assure him that if people are working against features, then they can choose to re-order their tasks in such a way as to most quickly get to the goal. No wasted effort, we tell him. Sounds good to him.

Customer-Prioritized Features

What's more, we assure him, we will be first working on the highest-priority features. What "needs" to get to market. Everything else slides down the list. Even if the wonderful plan we all love had it earlier. High-value features (as prioritized by the client) and their necessary dependencies. That's it.

"So no features that I didn't ask for?" He looks hopeful.

"That's right," we reassure him, "and you can change your mind."

He faints.

Smelling salts are brought.

"What do you mean, 'I can change my mind?'" he asks.

"If, when we get to the iteration review, and you test drive the thing, Acme corporation has come up with the killer feature that you absolutely must have or the whole thing is useless, you put it at the top priority on the list we use to drive the next iteration."

"And no one will complain that it's out of scope?" he marvels?

"If you put it at the top of the list, it is the scope for the next iteration. We are at your service," we comfort him.

"So when will this be finished?" he asks us desperately.

"When you say it's ready," we reply. He boggles. What does that mean, he thinks.

"What does that mean," he asks.

"It means that we will tie it off, when you think it has enough juice to go to market. If, after you re-prioritize what's left, you find that it's 'ready enough', then we'll roll with it, take a couple of weeks to steady it and ship it, and then we can pick up your next-highest priority things, or anything new you want in it." He looks near fainting again. "And we stop when you feel that spending money on it is no longer bringing you enough value, ideally because we've done all the high-value stuff already, and we're working on less valueable stuff."

"So I have discretion to pull the plug whenever I want to?"

"Yep."

"Please help me..."

Conclusion

This little scenario is fictitous, but in the end, it's consistent with the experiences of many Agile practitioners (particularly Scrum) that I have spoken with. We've only covered a small sampling of Agile practices that may help a project in crisis. Others may help quality, may improve developer productivity - but the above can help a key client or stakeholder begin to see a light at the end of the tunnel. While many people cannot get their heads around it, they may be willing to try new things when they're in enough pain with the existing process.

And it can turn a project failure into an ever-increasing success, because success is defined monthly, re-defined at the desires of the business client, and executed in bite-sized chunks that are digestible and estimable.

Just don't forget that people have the strangest reactions to things that break their expectations... so make sure to bring the smelling salts...

Posted by Christian Gruber at 09:08 PM | | | TrackBack

April 28, 2005

Plan Methods Oppose Agile Work Axioms

Plan driven methodologies which attempt to mechanize the process of doing work are in opposition to the three Agile Work Principles.

We are Creators
A plan methodology attempts to define intermediate and end work products independently of the input and effort of those who perform the work of creating the work products. This disenfranchises people from their work and leads to low morale. It also establishes a heirarchy of value for the people working on an effort where those who create the plans are perceived as more important or valuable than those who execute on the plans.

Change is Natural
This principle is usually acknowledged, but is usually described as a "problem" to be dealt with rather than as a basic principle to be fully embraced. A plan methodology has "change control" or "change management" and "risk management" and puts the whole notion of change in a negative light. This approach also disenfranchises people because they are constantly placed in opposition to reality.

Reality is Perceived
Plans attempt to legislate reality. "Thus and so must the project go" results in a constant struggle between the plan and peoples' perception of reality. Plans marginalize the importance of perception on the belief that reality can be objectively understood. If reality can be objectively understood, then it can be mechanistically manipulated. Thus results can be pre-determined without regard for the perception of those results.

Posted by Mishkin Berteig at 10:54 PM | |