September 06, 2007
Four Methods of Perfecting Agile
Most organizations start doing agile (or Scrum or lean or ...) imperfectly. Someone introduces a few practices or a manager gets a team some training, or a person starts using agile terminology. And things might improve, particularly with the use of iterations. One of the core ideas of agile methods is to have frequent delivery of valuable results. In fact, this core idea can be used to drive the improvement of an agile process. How? Here are four methods of perfecting agile by expanding the definition of done.
Perfecting Agile
Let's suppose you already understand the benefits of agile. With these benefits in mind, you would like to improve the organization's ability to deliver working, valuable results at the end of every iteration so that you can get better at realizing those benefits. The primary way to do this is by expanding the definition of done. You can imagine this like so:
On a regular basis, the team/organization find ways to bring work done in either the preparation stage or the close stage into every iteration of the agile portion of the project. By moving the work from these "bookend" stages into the iterations, you reduce the amount of time spent in those stages and simultaneously create a more complete delivery every iteration. The "definition of done" is now expanded to contain the results or value delivered by the work that was taken out of the startup and shutdown stages of the project. By expanding the definition of done, each iteration delivers a more "complete" increment of value, and there is less work done before or after iterations in order to plan or deliver. This gradual process allows the team to get better at doing agile.
There are four methods for transferring work from the start and end of a project into the iterations of a project.
Expand the Agile Team's Skill Set
In some ways, this is the simplest and most common approach to expanding the definition of done in the agile portion of a project. By training, coaching, mentoring, re-assigning or hiring, a team's capacity to do work is expanded and used to expand the definition of done. As a simple example, a software developer might learn to use an automated unit testing framework and therefore expand the definition of done to include some amount of unit test coverage of delivered code. In general, training, coaching and mentoring existing team members should be preferred over adding people to the team since the addition disrupts the team's development and can increase communication overhead among team members.
Expand the Agile Team's Authority
Sometimes, a team is not able to do part of the preparation work or close up work because they are not authorized to do so. This may be a policy, a unspoken assumption or a bureaucratic procedure. By giving the team (or some person on the team) the authority to do this work, the team can find ways to do it every iteration instead of having to work through another non-team individual. Again, a simple example here is a situation where a technical person is given permission to talk directly to an end user in order to reduce the need for up-front requirements gathering and analysis and reduce the need for end-of-project user acceptance testing. The obvious challenge to do this is the question of trust (or lack thereof).
Automate an Existing Manual Process
Automation is often given far less than its due consideration. This is primarily be cause automating a process is an investment of work in and of itself. Fortunately, it is often easy to measure the ROI or savings involved with automation. In many agile environments, heavy automation is critical and a huge enabler for very short iterations. Automated testing, automated translation, automated build processes, are all common areas of improvement. Agile teams should always be looking for opportunities to automate their own work. In this way, the automation work itself is transformed from a separate project to a responsibility of the team.
Remove Wasteful Processes
There are some parts of the project preparation work and the project close up work that are pure waste! There is no independent value to these activities, nor is there indirect value to them. An excellent example of this sort of thing is an approval process that _always_ grants approval ("rubber stamping"). One insurance organization I worked with as they were converting to an agile approach discovered that their "second stage" approvals always allowed proposed projects to proceed. Since they often incurred a 4-6 week delay for this approval process, it became obvious that they should "get rid of it". Now, what they actually did is made it so that it became a parallel review process rather than a gated approval process; this was so that the true purpose of the activity could still be met: to help stakeholders understand the projects that were being worked upon. Here, there is no need to take this approval process and somehow work it into every iteration. An agile approach tends to increase the visibility of the work anyway, so it may be discovered later on that the review process can also be done away with.
Agile is often implemented in a limited fashion when it is first adopted by most teams and organizations. The four methods of expanding the definition of done can help a team or organization get better at doing agile and therefore reap more of the benefits of agile. These methods are simple: expand the agile team's capacity, their authority, have them automate manual processes and remove wasteful activities from the process.
Posted by Mishkin Berteig at 03:32 AM | |
August 09, 2007
A Cautionary Tale - Delaying Agile Adoption
What happens when you delay adopting agile? Well, one large client I have worked with has found out...
Because this story is a story of failure, and because most people don't like to have their failure's exposed, I have anonymized this story and changed some details to protect the innocent (and they are innocent! Failure is a learning opportunity, not something that's bad).
I started working with BigCom about two years ago. They got a whole bunch of training from me; in total about 60 people received some in-depth training in agile methods. They also decided, wisely, to get me to help coach them organizationally and for some of their Process Facilitators. What they didn't do is heed some good advice: start iterating.
They decided that they needed to do some important up-front architectural research on their huge product re-write/porting project. They had an existing product used by many clients and built over the course of several years. This product was implemented in a legacy programming language and in order to make it easy to add features and keep it up to date, BigCom decided to re-write it in a more modern programming language... but they couldn't decide which one: Java or .NET. Because it was a pure porting job with little if any functional change, they already had a good detailed knowledge of the functional requirements. But this one question of the platform to build upon stalled them. Their research committee studied this questions (and some other less important ones) for four months.
I had advised my client that they should feel free to go ahead and do the research, but that they would be well served by starting to iterate and build the new version of the product, even if it meant building it on two platforms simultaneously. The actual experience of trying to implement real functionality in both Java and .NET might seem like a waste, but actually would provide valuable information for the research effort and help speed the decision-making.
Perhaps my advice was too scary. Or perhaps there was some legitimate reason for many capable people to sit idle for four months while the committee researched, deliberated and failed to make this important decision. In any case, product management and dev management finally got tired of waiting, and they started up the work.
Of course, without the decision made, the work initially was a real mess: developers were left to choose when to use Java and when to use .NET. Sometimes functionality was duplicated in both languages. Sometimes it made connecting functionality extremely difficult. It wasn't systematic. But it was experimental, and it did help clarify the issues quite quickly. After only four weeks of doing actual work on the functionality of the product, it became clear how to use these platforms (for the curious: Java on the UI and .NET on the back end).
The research committee had very little to do with the final decision other than to rubber-stamp it.
Time passed. BigCom continued the development work, features were implemented, and eventually the deadline for releasing the product loomed. And then it passed. Six weeks after the deadline, the product was finally released. Nine months after the start of iterations. Thirteen months after I recommended that they "just start".
Four months wasted... and a deadline missed by six weeks. Was it worth it? Might have been nice to distribute some bonuses for coming in ten weeks early instead. Oh well!
Please, just start iterating!
Posted by Mishkin Berteig at 09:55 AM | |
January 02, 2007
Top Ten Most Popular Entries from 2006
If you are new to Agile Advice, these are not just some of the most popular articles, they are also some of the best! Take a look around; you will find ideas to inspire you, challenge your thinking and maybe that one little thing that will make the difference in applying agile methods in your situation.
1. How Two Hours Can Waste Two Weeks - 25,617 unique views
This little hypothetical story by Dmitri Zimine was very popular on Reddit, and Joel on Software ranted a bit about it.
2. The Case for Context Switching - 2,936 unique views
My rebuttal to Joel's rant. Goes well with Dmitri's article. Emphasizes the idea of building trust in an organization.
3. The Qualities of an Ideal Test - 1,579 unique views
Six qualities that will help make your tests as valuable as can be. Similar to the ACID properties of databases or the INVEST properties of user stories.
4. The Pros and Cons of Short Iterations - 1,555 unique views
A few words that will help you decide how long to make your iteration length. This is an important decision, and most teams and organizations don't know the factors involved.
5. Five Signs of Trouble in an Iteration - 1,489 unique views
A good howto on using burndown charts to discover problems in an iteration.
6. Seventeen Tips for Iteration Planning - 1,427 unique views
A good list of hints and tips that can make the difference between struggling with iteration planning and having it go smoothly and effectively. This is a key part of the Agile Work process, so getting good at it is a high priority for any team new to Agile Work.
7. Change is Natural - "Embrace Change" - 1,397 unique views
A short riff on the universality of change. Also introduces the idea of the "horizon of predictability".
8. The Art of Obstacle Removal - 1,287 unique views
This is a good reference article on types of obstacles and methods for removing them... a critical practice for Process Facilitators.
9. The Seven Core Practices of Agile Work - 1,285 unique views
Agile Work is really quite simple. This is a concise list of the practices that make up this effective methodology.
10. Interview with Alistair Cockburn - Agile and House Renovations - 902 unique views
Applying agile methods to home and garden renovations! Learn a bit about how this luminary of the agile world has taken agile practices out of the software world and into the hardware world.
Posted by Mishkin Berteig at 06:32 PM | |
May 20, 2005
Management and Business as Sources of Waste
Waste can be of several different types. Independent of what the types of wastes are, they can also come from different sources. When attempting agile work, it is essential to identify the underlying causes or sources of waste. Once these sources are identified, efforts can be made to change them so that they cause less waste.
Management is least effective and a source of waste in an agile work environment when it imposes decisions, plans work, and directs people. Management is most effective in an agile work environment as a facilitator, shield and servant.
Business can cause waste by not being focused or by being focused on un-profitable efforts. Paradoxically, business can also cause waste by being too focused on profitable efforts. A balance of investment in short-term, long-term and "cleanup" efforts is ideal for an agile work environment.
Posted by Mishkin Berteig at 07:55 PM | |
May 17, 2005
The Qualities of an Ideal Test
These six qualities of tests describe how to make a test as effective and as useful as possible. The qualities are similar in style to the INVEST qualities of user stories - but they don't form a nice acronym.
Decisive: The test contains all the information required to automatically determine success or failure. Manual inspection of test results is not necessary to determine success or failure. The test is expressed in a way that produces a pass/fail answer rather than a numerical or qualitative result. Decisive tests are often expressed as assertions.
Valid: The test produces a result that matches the intention for the work artifact under test. Test failure indicates failure in the artifact under test and test success indicates correct operation or configuration of the artifact under test. Simply put: the result of a test should be true.
Complete: The test contains all the information it needs to run correctly with a given test harness and work artifact under test. The test performs all activities and provides all the data necessary. The test does not require any input outside of itself in order to run.
Repeatable: The test always gives the same results if the test harness and the artifact under test are the same. The test is created in such a way that the result is deterministic. Even if the system under test is not deterministic, the test should be created to account for that (possibly with statistical analysis) and produce a deterministic result.
Isolated: The test result is not affected by other tests run before it nor does a test affect the results of tests run after it. The test and the test harness work together to clean up after every test is run. A collection of tests can be run in any order and always produce the same results. Any test that depends upon the results or side-effects of a previous test is not isolated.
Automated: The test requires only a start signal in order to run to completion in a finite amount of time. No manual intervention is required after the test is started. Tests that are automated can be put together into a test suite and run one after another without human intervention. Automation of tests requires the creation of a test harness system.
Update 20060106
I've added a little bit more detail to the above qualities. I also wanted to say a few words about my experience with testing:
I first started doing test-driven development almost seven years ago after reading the book Refactoring by Martin Fowler. I picked it up in Windsor Ontario while I was driving to San Francisco for my first contract position as an independent. I had to stay the night there due to the immigration office being closed so I read a lot of the book in my motel room that night. By time I got to California four days later, I was totally convinced that I should be using JUnit and refactoring for everything I did. Fortunately my project was ammenable to this approach. Over the next four months I built a Java JMS layer on top of Tibco Rendezvous. In the process I discovered methods for doing unit tests for asynchronous multi-threaded distributed functionality. And my tests satisfied all the above qualities. I delivered code with 100% test coverage and zero defects discovered until more than two years later when a very rare and obscure deadlock issue was found. Suffice it to say, I was convinced from a quality perspective.
But there is more. To build tests that follow all the above qualities, you need code that is testable. I have come to believe that testability is the most important architectural attribute of code for most software. The implications of having testable code include code that is easily changed, that is verifiable, and that is easy to understand. Refactoring plays a big role here too.
For getting a basic but very practical sense of test driven development, I strongly recomment Test Driven Development: By Example 
I have also written a couple of articles recently about quality:
Quality is Not Negotiable in which I talk about how to handle defects - by always treating them as top priority work items.
and
Technical Debt in which I expand on this common analogy between lack of testing and debt to show that technical debt is even worse than financial debt.
Posted by Mishkin Berteig at 06:48 PM | |
April 27, 2005
Eliminate Waste
Waste is the result of activities or environmental conditions that prevent a team from reaching its goal. The opposite of waste is something that adds value (more, faster or higher quality) to the desired result.
The whole notion of eliminating waste comes from lean manufacturing. More recently, Mary and Tom Poppendieck applied this idea to software in their book "Lean Software Development: An Agile Toolkit for Software Development Managers". In this (excellent) book, the authors list the wastes of manufacturing and the wastes of software. Here I have summarized and generalized these types of wastes so that they apply in any situation:
| The Seven Wastes |
| 1. waiting caused by delays, unreadiness, or simple procrastination |
| 2. partially done work or inventory caused by sub-optimal workflow |
| 3. extra processing or processes caused by poor organization or bureaucracy |
| 4. defects and rework caused by insufficient skill, tools, inspection or filtering |
| 5. movement of people or work caused by physical separation |
| 6. overproduction or extra features caused by working towards speculative goals |
| 7. task switching caused by multiple commitments |
As wastes are eliminated or reduced, a team will function faster and with higher quality. However, not all waste can be eliminated. Sometimes waste is legislated, sometimes waste is an unavoidable byproduct of work, sometimes mistakes are made, and sometimes it takes a great deal of effort to eliminate a waste.
In order to eliminate waste, first waste has to be detected and identified, then the underlying causes of the waste have to be identified, and finally changes to the work environment need to be made to both eliminate the cause of the waste and the waste itself. Many agile work practices help with this process.
Value stream mapping is one particular tool that can be used by a team or organization to identify wasteful activities. The team describes the amount of time that work takes to go through each activity in their overall work process. Next, the team determines if each activity adds value or does not add value to the end goal. All activities are subject to speed improvements, and activities that do not add value are subject to elimination.
In order to determine the causes of waste, special attention should be paid to incentives and motivations. Wasteful behavior often exists because there is some incentive for people to do it. Sometimes these incentives are explicit, but sometimes they are the side-effects of other things going on in the team's environment. Changing the incentives can be an effective way of reducing waste.
By eliminating waste, the team will find it has reduced frustrations, and enabled greater productivity and creativity. The team will also increase its speed and delivery of value, and at the same time reduce defects.
Posted by Mishkin Berteig at 07:40 PM | |
Eliminate Waste
Waste is the result of activities or environmental conditions that prevent a team from reaching its goal. The opposite of waste is something that adds value (more, faster or higher quality) to the desired result.
The whole notion of eliminating waste comes from lean manufacturing. More recently, Mary and Tom Poppendieck applied this idea to software in their book "Lean Software Development: An Agile Toolkit for Software Development Managers". In this (excellent) book, the authors list the wastes of manufacturing and the wastes of software. Here I have summarized and generalized these types of wastes so that they apply in any situation:
| The Seven Wastes |
| 1. waiting caused by delays, unreadiness, or simple procrastination |
| 2. partially done work or inventory caused by sub-optimal workflow |
| 3. extra processing or processes caused by poor organization or bureaucracy |
| 4. defects and rework caused by insufficient skill, tools, inspection or filtering |
| 5. movement of people or work caused by physical separation |
| 6. overproduction or extra features caused by working towards speculative goals |
| 7. task switching caused by multiple commitments |
As wastes are eliminated or reduced, a team will function faster and with higher quality. However, not all waste can be eliminated. Sometimes waste is legislated, sometimes waste is an unavoidable byproduct of work, sometimes mistakes are made, and sometimes it takes a great deal of effort to eliminate a waste.
In order to eliminate waste, first waste has to be detected and identified, then the underlying causes of the waste have to be identified, and finally changes to the work environment need to be made to both eliminate the cause of the waste and the waste itself. Many agile work practices help with this process.
Value stream mapping is one particular tool that can be used by a team or organization to identify wasteful activities. The team describes the amount of time that work takes to go through each activity in their overall work process. Next, the team determines if each activity adds value or does not add value to the end goal. All activities are subject to speed improvements, and activities that do not add value are subject to elimination.
In order to determine the causes of waste, special attention should be paid to incentives and motivations. Wasteful behavior often exists because there is some incentive for people to do it. Sometimes these incentives are explicit, but sometimes they are the side-effects of other things going on in the team's environment. Changing the incentives can be an effective way of reducing waste.
By eliminating waste, the team will find it has reduced frustrations, and enabled greater productivity and creativity. The team will also increase its speed and delivery of value, and at the same time reduce defects.
Posted by Mishkin Berteig at 07:40 PM | |