Alistair Cockburn is the author of several important books in the agile software development literature including Agile Software Development and Crystal Clear : A Human-Powered Methodology for Small Teams (Agile Software Development Series). I had heard that he had a story to tell about using agile methods and principles on a house renovation project. I contacted him by email and he agreed to an email interview.
AA: Could you please provide me with a 3 or 4 sentence blurb about yourself?
Alistair: Armed with a B.S. in Computer Engineering (note the “engineering” in the degree 😉 and M.S. in Computer Science, I spent 8 years designing high-speed, custom computer graphics hardware, largely at Evans & Sutherland, the next 8 years in IBM Research researching the way people specify and test communications protocols, and the next 15 years consulting around object technology software and methodologies, starting at the IBM Consulting Group in 1991.
I got my doctorate from the University of Oslo in 2003 on the basis of my interviews with projects in the early and mid 1990s. It was on “People and Methodologies in Software Development.” I was really proud about getting the word “people” into a doctoral dissertation in the Informatics / Computer Science field.
These experiences led me to help write the Agile Manifesto in 2001 and the PM Declaration of Inter-Dependence in 2005. More about my personal side is exposed in my answers to the infamous “Proust Questionnaire” and on my website, Alistair.Cockburn.us.
AA: How did you apply agile methods to your home renovations?
Alistair: Many people argue about whether house construction it is or isn’t like software development. I have been making extensions to my house for several years. You can imagine that I would like to make a house extension incrementally, using a venture-capital funding model, and in a close collaboration with both the architect and construction contractors, exactly following the agile model.
In our first conversation, the architect suggested creating a “master plan” of all the desired changes, then doing the work incrementally. I declined, because I was pretty sure that we would change too much for the initial plan to have meaning by the half-way point, and it would end up being a waste of my money. As it turned out, I was correct.
We ran each idea as a stand-alone project, but were careful to ask at each key moment, “If we were to extend in [some particular way] next year, how would that affect our decision now?” Surprisingly, we found no cases where the future choice affected the present decision.
Our first project was to put a basement under our house. (I have fun with this … people tell me that it is normal practice to build the basement first; then I get to say that we agile folks like to sequence features in business-priority order, so who knows, the basement may get put in last!). Anyway, it turned out to be easier than expected, because our house is built on short stilts (to provide an insulating air cushion over the ground). This probably wouldn’t have mattered, because the construction people are used to putting basements under existing houses under normal circumstances.
Rather than excavate the whole basement, we decided to excavate only 1/3 of the basement. This gave us the basement entrance we needed, and left open the question whether we would extend the excavation or build a side wing in the next project. We learned that there would be no cost savings for excavating more than we needed now, even if we chose to expand the basement later. The XP community calls this the YAGNI principle, for “You Aren’t Gonna Need It”. We excavated only what we needed. Three years later we still have no plans to extend the floor space, either above or below ground. YAGNI is holding.
The next part of the “agile” story was watching the architect and the contractor work out how to hold up the house while they excavated. The architect looked underneath the house and launched into a dissertation on the various choices. The contractor cut in with, “Why don’t we just run a giant beam right down the center and hold the whole thing up while we excavate?” Note here the application of the 11th principle of the agile manifesto: “Simplicity – the art of maximizing the amount of work not done – is essential.” The architect adopted the suggestion, and after that, the architect, the contractor and the construction crew had excellent conversations that merged their best ideas with. To this day they always trade construction ideas back and forth along with how those might change the design of the house itself.
The next thing is illustrative of the relationship between construction and software development. On the first day of construction, they knocked a hole in the concrete wall where they were going to excavate, peeked in, and discovered that the beams under that part of the house ran perpendicular to those where they had looked under the house. This ruined their plan for the support beams, and showed that once again, civil engineering is ahead of software engineering. I find that it usually takes software teams a week or two to invalidate plans. These people were able to do it within two hours!
As we progressed I quickly noticed that the contractor kept changing his “fixed-price” bid for the project as new information appeared. I finally suggested we drop the “fixed-price” facade, and just work to time and materials. His reply surprised me: “If you are willing to carry the extra risk, that will be fine with us.” I said, “I don’t see any extra risk. You’ll just change the price whenever something changes any-way.” He said, “That’s true but most people don’t recognize that.” In exchange for my accepting the “extra risk,” he lowered his profit margin from 13% to 10%. We each felt we had benefited from this change.
With the new contract in place, his crew was able to do any number of other small projects for us, such as changing the kitchen counters, adding closets, and fixing the fence around the yard. Neither these, nor the other usual surprises, twists and turns on the project worried either of us any more.
I noticed with some interest that he and his crew held a daily stand-up meeting each morning to set their day’s work goals. In this short meeting, they checked that they knew what they were working on, and had the materials to get it done.
The project came to a successful conclusion, and we gave a small bonus to the contractor and the workers. Unbeknownst to me, our prompt payments made us “preferred customers,” which proved useful for the next projects.
We then did a second project, to sculpt the yard, move in stone slabs and do some general landscaping. Although this was considered a “minor” project, the contractor was delighted to help. He sent digging and moving crews to help when we needed it, and even operated machinery when his key men were ill. This was part of us being preferred customers, a roll-forward benefit from the first project.
We’re on a third project, to add a fancy porch to the side of the house. At this point, we are glad not to have done a “master plan,” because what we have in mind for the changes to the front porch are completely different from anything we had originally had in mind.
By now, there is trust and good communications between the architect, the contractor and us. The small wins with reflection, the venture capital funding model, the willingness to dance with the situation, are all paying off. The contractor passed along our preferred customer rating to his suppliers, so we got extremely fast service. He also took it upon himself to see that they give us good rates, so we saved money. Since we contracted by time and materials, it is natural to have the subcontractors do additional projects around the house as we need. The final bill for all this extra work has been much lower than it would have been had we contracted each one separately.
The point of this long story is to illustrate how the lessons from agile software development transfer to a completely different field. Incremental development, architectural changes, daily standups, YAGNI, time-and-material contracts, venture capital funding, customer involvement and steering, small wins, process miniature, and developing collaboration and trust – each of these proved useful.
(The above story is an extract from the forthcoming 2nd edition of “Agile Software Development”, celebrating the 5th anniversary of the writing of the Agile Manifesto.)
AA: You have a great deal of experience using agile methods in a software context. How did you get the idea to apply agile methods to your renovation project?
Alistair: It is more the other way around: After interviewing successful projects for over a decade and seeing how incremental development and close collaboration make such a difference in project outcome, it would be hard for me NOT to apply agile methods to any project at all. I even tried it with conference organization (the Agile Development Conference in 2003 and 2004) â€“ agile techniques didn’t help us much on the first year (a conference is a one-pass, big-bang integration project if there ever was one), but they helped a great deal the second year.
Also, since we were spending our own money, the venture-capital funding model made sense.
The rest I just made up along the way, kind of as an experiment to see if it would work. I was surprised at the way YAGNI held up and pleasantly surprised by the really positive roll-forward of the close collaboration, communication, community, morale issues.
AA: Your family were also stakeholders in this project. How did they react to the lack of long-term planning, the lack of a master plan for the renovations?
Alistair: My wife has a harder time than I do with the notion of “build a little and then let’s live in it and see what we need next” Her difficulty is not with completing one project at a time, as we did in the house, but in the garden, where I want to make steps and paths out of plywood and dirt and “feel how we use them” before committing to a design.
On the other hand, she is at least as good as me at taking last-instant advantage of serendipitous information. For example, we had an excavator ready to flatten a section of the yard, and he asked where he should put the dirt. I suddenly had the bright idea of building a mound in the yard. She saw it immediately, and in the next 30 seconds the two of us sketched out a first-cut redesign of that section of the yard based on the presence of a mound, sufficient for the bobcat operator to start moving dirt. (She then went and made a sculpture of the yard out of play-doh to serve as his spec!)
AA: Many agile methods include the concept of a timebox for planning and delivery which is used to provide formal feedback and adjustment points with stakeholders. How did you accomplish this feedback on your renovation project?
Alistair: Timeboxing didn’t work for us, because we were subject to the really varying hours of the contractors â€“ lots accomplished one week, nothing the next. We’re small fish and they have bigger contracts to fulfill. So we had to / have to sometimes make do with leftover scraps of time.
However, we have an “on-site customer,” so we have no trouble giving feedback. And since Deanna and I are willing to make adjustments in design really quickly, we can change direction in minutes. There were several times when they ran into some sort of obstacle, and we ran around for minutes or days (if the architect wasn’t on-site at the time) drawing different drawings, checking prices and materials and designs, and then changing directions.
They had ordered tan planks of the Trex material for the porch, and it was sitting on the ground ready to be nailed in place. Neither Deanna nor I really liked the color. I was online looking for aluminum railings, and ran across a Trex site that listed many more colors than they had told us were available. I printed out the sheets, ran downstairs and showed everyone, and we changed to a burgundy color that day. It delayed the schedule a whole day.
With respect to financing, we were paying time-and-materials, so we could stop or delay the work at any time. (We still haven’t figured out how speed work up, of course!)
AA: What was the biggest challenge you had to overcome in adapting agile practices to this project and how did you overcome the challenge?
Alistair: None come to mind. We used agile practices from A to Z easily and profitably.
AA: What aspects of agile methods did you _not_ apply to this work? What thoughts do you have about applying these missing methods to future work of this sort?
Alistair: Test-first design doesn’t apply, as far as I can tell. Timeboxing didn’t apply, in our case. I can’t think of anything else.
AA: What part did trust and truthfulness play in your renovation project?
Alistair: Lots, all described earlier in the project story. We’re inviting the architect and the contractor to the open-house barbecue, because they are interesting people and we like them. We get preferred treatment from the contractor, who passes that on to his network of specialist contractors.
He sometimes has to warn his specialist contractors that we’re unusually frank. We expect them to counter our ideas with their ideas, and vice versa. We ask for hourly prices up front (and if he thinks those prices are too high, he’ll bargain the rates down for us) and tell them to expect changes in direction. He likes all this and helps the other contractors recognize we’re not the usual buyers that they have to protect themselves from.
AA: How would you sum up the most important things you have learned from trying this?
Alistair: Collaboration, open communication, feedback. Work incrementally, so that the feedback within and from each increment can improve all of those on the next round.
Try to remain open-minded when adverse events drag you out of your good mood.
Keep looking for ways to improve collaboration, communication, feedback, then everyone’s best ideas have the best chance of coming forward. With everyone kicking in their best ideas, you’ll get the best ideas possible for the project.
David Anderson has a brief blog entry called Construction time Again that claims agile doesn’t work with house construction.
James Shore has also weighed in about “That Damned Construction Analogy“.
And I have to admit that even I have written extensively that The Construction Analogy is Broken.
Seems we’re all wrong, but that’s good as far as I’m concerned!