- retiring Oracle data warehouse licenses / servers,
- retiring disk space / hardware,
- and saving CPU time with new hardware
What is Agile?
In 2001, 17 people got together for a world-changing discussion about software development. They tried to find the common values and principles by which people could do better at the work of software development, which was in a terrible crisis (and still, to some extent is). They were successful in that they created a list of 4 values and 12 principles to guide people trying to find better ways of developing software: the Agile Manifesto was born.
Now, nearly 14 years later, Agile software development has become well-known (if not well-practiced) throughout the business world. In fact, the concepts of Agile software development have been extended through to many other fields as diverse as mining, church management, personal time management, and general corporate management. In the process, there has also been a growing recognition of the relationship between Agile values and principles and those of Lean thinking.
It is time to think about the concepts behind the values and principles. To acknowledge that the Agile Manifesto (for software development) can be re-stated at a much deeper level. To abstract Agile software development to Agile work in general. This is my goal over a series of essays about the Agile Manifesto.
Let’s start with an analysis of the values of the Agile Manifesto in relationship to the concept of “value”.
The Agile Manifesto Values
In the Agile Manifesto we can read the four values:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
While a few of these already are quite general, let’s dig a bit deeper by starting with the second value, “working software over comprehensive documentation”. What does this value really refer to? Why do we care about software over documentation. Why “working” vs. “comprehensive”? This is where Lean thinking can help us. The notion of “customer value-added” or just value added work is any work that changes the form, fit or function of that which you are delivering and simultaneously is work that a customer would pay for independently of all the other activities and results you may be spending time on. In this specific example of software and documentation, we can try to imagine what it would be like to say to a potential customer, “we can give you documentation for our software, but we won’t be able to give you the actual software itself… but we’d still like to be paid.” I’m sure it will be clear that it would be a very unusual customer who would agree to such a proposal. Thus, we see that the value of software development activities is in producing the software itself, and the documentation is by necessity of secondary importance.
But what if we are writing a book of fiction? Surely this is documentation! But, it is not. To make the analogy to the type of documentation mentioned in the Agile Manifesto, we would sell a book not just with the story itself, but also with in-depth instructions on how to use a book, how to read, how to interpret our feelings as we read, etc. And not just that, but we would also provide a set of notes to the publisher about exactly how we wrote the book: the time and place of each paragraph written, our original outlines, our research including much that was thrown away, all our conversations with people as we struggled to sort out various plot, character and setting elements, and possibly even all our edits that we had thrown away. This is the documentation to which, by analogy, the Agile Manifesto is referring.
Perhaps now we can look at a connection between the first value and the second value of the Manifesto: that documentation, tools and processes are all much of the same thing. They all belong to the same abstract category, namely, the means used to achieve a particular end. Of course, we all know that both the means and the ends are important, although we may not all agree on their relative importance. Nevertheless, we can probably agree on some extreme outliers that will help us come to the point I wish to make. For example, we can agree that killing someone merely to get to the front of the grocery store line and save a minute or two of time is an extreme case where the end clearly does not justify the means. Likewise, we can also agree that refusing honestly given help out of a desire for independence when it ends with the death of our children by starvation is putting means too much in the fore. Balance is required, therefore. The Agile Manifesto acknowledges this balance by its epilogue to the values, “That is, while there is value in the items on the right, we value the items on the left more.” The other two values of the Agile Manifesto which mention contract negotiation and following a plan are similarly pointing out activities of the means that are non-value added, in Lean terms.
The Agile Manifesto, is stating four values, is, quite directly, pointing to those things which in life are fundamentally of value as ends, not just as means. Individuals and interactions, working software, customer collaboration and responding to change, are all valuable. In order to abstract away from software, then, and create a more general statement of the nature of Agility, we need to explore the idea of value.
The Idea of Value
If we are in business, determining value, while possibly complicated, is not usually too obscure an effort. We look at exchanges of money to see where the “market” agrees there is value. The price of a product or service is only representative of value if someone will actually pay. Therefore, businesses look at return on investment, profit margins and the like to determine value. Similarly, in other types of markets such as the stock market, value is determined by an exchange of money. But underlying this exchange of money is a decision made by an individual human being (or several, or many) to refuse all the other potential uses of that money for the one specific use of making a particular purchase. This choice is based on all sorts of factors. In economics, we talk about these factors mostly in rational selfish terms: what sort of benefit does the purchaser get from the purchase. Factors such as risk, short-term vs. long-term, net present value, trade-offs, etc. certainly can play a role in such decisions. But in business (and in particular marketing and sales), we know that there are also lots of non-rational forces at work in deciding to spend money on a particular something. Value, therefore, has a great deal to do with how a particular person both feels and understands the current proposed “investment” opportunity.
Feeling and understanding arise from many specific factors, as discussed, but what can we say generally about feeling and understanding? They are internal states of a person’s mind (or heart, if you like). Those internal states have been the subject of much discussion in the realm of psychology, sociology, economics, and philosophy. But quite simply, those internal states are greatly determined by perception. How a person perceives a situation is the immediate general factor that determines those internal states. Of course, perception is a general term that includes sensory perception, but also the kinds of prejudices we have, the categories into which we place things conceptually, the internal language we use to describe things, and our existing emotional and mental constructs. So, fundamentally, value is perceived depending on all these perceptual factors.
The Agile Manifesto authors, therefore, had (and perhaps still have) a perception of value which places individuals and interactions, working software, customer collaboration and responding to change all in a position of more value than those other items. But this perception of value may not be in alignment with other people’s perception of value. Still, we can see already in 13 short years that there is broad, if not universal, agreement on these statements of value. Why is this? Why has Agile become so popular over a relatively sustained length of time with a trajectory that still seems to have it growing in popularity for years to come?
Agile values address a deep need in people in the software development discipline and indeed, by analogy, into much of the work world.
In my next essay on the Agile Manifesto, we will take a look in more depth at the first value of the Agile Manifesto: Individuals and Interactions over Processes and Tools.
Some links to commentary on the values of the Agile Manifesto while you wait for my next essay instalment!…
“Truthfulness is the foundation of all the virtues of the world of humanity”
Many people can see some validity or value in this statement, but it may seem strange to them to incorporate this component into business practices or corporate culture. After all much of what is common practice does not reward or encourage those who choose to be truthful.
But as Bob Dylan so aptly put it, “the times they are a-changin’”.
The environment, our capacities as human beings, and our tools to interact with the world are constantly evolving and growing. Yet much of what we do today is based on assumptions about human nature arrived at hundreds or even thousands of years ago when we had less knowledge and understanding about the world and ourselves. Along with the rest of the universe we are evolving as a human species, as such it only makes sense that our higher understanding and knowledge inform our decisions and practices, so we can keep progressing forward.
OpenAgile recognizes the true nature of humanity and how it can work to create a remarkable world in every endeavour. Scientific discovery is revealing this truth about our nature as well, as the video below so wonderfully illustrates.
Be Open, Be Agile, Be Free.
As a Team Member, it is my job to figure out how to solve a problem or request that is stated by a Product Backlog Item (PBI), with the help of my team. It is the responsibility of the Product Owner to give us the vision of the product and decide how much scope is to be done to satisfy the PBI. One simple way to think about this concept is that the Product Owner is responsible for the “what” and “why” and the Scrum Team is responsible for the “how” and “who”. If the Team Members decide on the product vision by themselves, they run the risk of misinterpreting features, moving down a path that is not valuable or even creating work disconnected from the needs of those who will be using the software. If the Team Members choose their own scope they may do much less than is needed or much more than is required. There is a balance in the Product Owner providing vision and scope, and the Scrum Team providing knowledge and experience in its execution.
The Definition of “Done” for a Scrum Team makes transparent how close the team’s work is coming to being shippable at the end of every Sprint. Expanding the Definition of “Done” until the team is able to ship their product increment every Sprint is a process that every Team Member helps advance. Team Members expand the Definition of “Done” by learning new skills, developing trust and gaining authority to do work, automating repetitive activities, and finding and eliminating wasteful activities. When every Team Member is systematically expanding the Definition of “Done”, the team builds its capacity to satisfy business needs without relying on outside people, groups or resources. If Team Members are not actively working on this, then many of the obstacles to becoming a high-performance team will not be discovered.
Product Backlog Items are brief descriptions of a feature or function. Usually they are short enough that they could be hand-written on a small note card. This brevity is meant to be just enough information so that the Product Owner and the team can use the PBIs as invitations to conversations. The resulting conversation (ongoing, evolving and involving the whole team and stakeholders) and the shared understanding that comes from that conversation is where the real value of the PBI resides. Part of the conversation occurs when the Product Owner initially writes the PBI so that the team can estimate the effort of building it. Part of the conversation is the actual work being done during a Sprint. Another part of the conversation is during the Sprint Review when stakeholders see the results of the team building the PBIs. If one creates PBIs as detailed specifications then we are essentially handcuffing the team into a set path and a prescriptive solution. The reason we hire qualified people onto our Scrum Team is for their knowledge, experience, and problem solving abilities. If we lock them into a set path, then we are literally turning them into cogs in a machine to spit out specific code. PBIs that are invitations to conversations allow them the flexibility to figure out how to solve the problem by engaging in a conversation on what is needed.
Product Backlog Items are ordered into a sequence in the Product Backlog in such a way that the Product Owner is able to maximize the return on investment (ROI) in the team. The very first PBI in the Product Backlog should be the one with the highest expected value considering the effort to build the PBI. There are many ways to calculate this expected value including Return on Investment (ROI), Net Income After Taxes (NIAT), Net Present Value (NPV), etc. The Scrum Team members should be free to ask why one PBI is prioritized higher than another, and the Product Owner should be able to give a reasonable answer. Since the entire Scrum Team is accountable for its work, it is in the best interest of all members of the team to use expected value, so that both the Scrum team and the customer will be committed to the work that is currently being worked on and the upcoming work in the future Sprints. If we don’t order the PBIs by expected value, then the Product Owner is likely to prioritize them based on dates, feelings, urgency, or other less valuable methods. These other prioritization methods will diminish the trust of the team in the Product Owner and may lead to morale problems.
Interesting: The Business Value Game. If you have tried it out with clients or with a team, please let us know in the comments!
I’m going to Edmonton to do a presentation on the Value of Retrospectives for the Agile Edmonton user Group. Should be a lot of fun! I’ve done many CSM classes in Edmonton (and Calgary). If you are going to be in Edmonton on April 1st, consider coming out to the meeting! See you there! – Mishkin.
Agile Work consists of seven core practices. These practices form a solid starting point for any person, team or community that wishes to follow the Middle Way to Excellence.
Any group of people that wish to be an Agile Team need to take the initiative to determine for themselves how they are going to work (process) and how they are going to do the work (product). The term “team” really applies quite broadly to any size group of people that are working together towards a common goal.
Teams go through stages of development as they perform their work. The most important result of team development is the team itself, and not the specific skills and abilities that the individuals learn.
If the team is part of a broader organization, that organization must give the team the authority, space and safety to learn to be self-organizing. The organization’s leadership is responsible for determining the “why?”, some constraints on “how?”, and then letting the team respond to the need as best as it can.
Also Known As: Whole Team (Extreme Programming), Cross-Functional Team (business management).
Agile Work uses short fixed periods of time to frame the process of delivering something of value. Each of these iterations or timeboxes is structured so that the team or group actually finishes a piece of work and delivers it to stakeholders. Then, the team builds on what has previously been delivered to do it again in the same short amount of time.
The sooner that valuable results can be delivered, the more value can be obtained from those results. This extra value is derived from opportunities such as earlier sales, competitive advantage, early feedback, and risk reduction.
There is an explicit tradeoff: the shorter the time to delivery, the smaller the piece of value will be. But, like investing in one’s retirement account, the earlier you start, even with small amounts of money, the better off you are in the long run.
Also Known As: Sprint (Scrum), Iteration (Extreme Programming), Timeboxing (generic), Time Value of Money (accounting).
Plan to Learn
Every type of work is governed by a Horizon of Predictability. Any plan that extends beyond this horizon of predictability is bound to fail. Agile work uses an explicit learning cycle tied in with the planning of work to accomodate this inevitable change.
First, a goal is required. This goal can be long-term. Teams using Agile Work then create a queue of work items to be done in order to reach this goal. Each iteration, some of these items are selected, finished and then the queue is adjusted. The changes in the work queue are based on external factors, and learning that the team does as it goes.
One of the most effective methods for the team to learn about how it is doing its work is the retrospective. After each delivery of results, the team holds a retrospective to examine how it can improve.
Also Known As: Inspect and Adapt (Scrum), Kaizen (Lean), Adaptive Planning (generic).
A team needs to have effective means of communicating, both amongst team members and also to stakeholders. To Communicate Powerfully, a team needs to prefer in-person communication over distributed communication. Synchronous over asynchronous communication. High-bandwidth over low-bandwidth communication. Multi-mode communication over single-mode communication.
The results of failing to communicate powerfully include wasted time for waiting, misunderstandings leading to defects or re-work, slower development of trust, slower team-building, and ultimately a failure to align perceptions of reality.
The single most effective means to communicate powerfully, is to put all the team in a room together where they can do their work, every day for the majority of the work time.
Some types of work do not lend themselves to this approach (e.g. creating a documentary video), but every effort should be made to improve communication.
Also Known As: Visibility (Scrum), Whole Team and Team Room (Extreme Programming), War Room (business management).
Defects are one of the most critical types of waste to eliminate from a work process. By testing everything, by driving all the work of a team by creating test cases to check the work, a team can reach extremely high quality levels. This ability to prevent defects is so important that only an executive level decision should be considered sufficient to allow defects into a work process. Quality is not negotiable.
In Agile Work, removing a defect is the only type of work that takes priority over any new features/functionality/production. If the end result desired is to maximize value, then removing defects is an important means to that end.
A team has an ethical duty to discover new ways to effectively test their work. This can be through the use of tools, various feedback mechanisms, automation, and good old problem-solving abilities.
Also Known As: Canary in the Coal Mine (Scrum), Test-Driven Development (Extreme Programming), Defects per Opportunities (Six-Sigma).
Since Reality is Perceived, it is important for an agile team and organization to have a clear method of describing and perceiving what is important for the organization. Measuring value is a critical method for describing and perceiving what is important.
A single metric can be used to drive all the measurement and goal-setting and rewards in an organization. All other measurements are secondary and must be treated as such: limited in use and temporary.
There are many things which are easier to measure than value. It is often easy to measure cost, or hours worked, or defects found, or estimate vs. actual… etc. However, all of these other measurements either implicitly or explicitly drive sub-optimal behavior.
Also Known As: Measuring Results (Scrum), ROI (business management), Economic Driver (Good to Great), Running Tested Features (Extreme Programming).
Clear the Path
Everyone in an organization using Agile Work takes responsibility for clearing the path, removing the obstacles that prevent work from being done effectively. Clearing the Path doesn’t just mean expedient, quick fixes to problems, but rather taking the time to look at an obstacle and do the best possible to remove it permanently so that it never blocks the path again.
In the Agile Work method, the Process Facilitator is the person who is responsible for tracking obstacles and ensuring that the path is cleared. To do this, the Process Facilitator maintains a Record of Obstacles.
Clearing the Path is sometimes painful work that exposes things we would rather not deal with. As a result, it is critical that people build their capacity for truthfulness and work to develop trust amongst each other. Building a capacity for truthfulness is not something that can be done by using an explicit process.
Also Known As: Removing Obstacles (Scrum), Stopping the Line (Lean).
Remember also, that these practices must always be viewed and implemented in the context of the Agile Axioms. These axioms provide a check to ensure that the practices are not being applied blindly, but rather applied appropriately to the given situation.
Work can often be divided up so that the smaller pieces are valuable on their own. By dividing work this way, a team can deliver value incrementally. The team can choose a short period of time called an iteration and select a small amount of work to complete in that time. This work should be valuable on its own. For example, if a team is building something, then at the end of each iteration whatever is built is usable as it is. This means that each iteration includes all the planning and design as well as construction or creation necessary to deliver a final product or result.
For example, a volunteer group may desire to attract new members. A non-agile approach would have the group plan their membership campaign completely before actually executing on it. An agile approach using iterative delivery would have the group plan a small piece of work that will attract some small number of new members, execute it, and then start a new iteration. One iteration may cover the creation of and delivery of a door-to-door flyer in a neighborhood. Another iteration may cover the design, creation and publishing of a small advertisement in a local newspaper. Each iteration includes all the steps necessary to produce a furthering of the group’s goal of attracting new members.
In a business environment, iterative delivery allows for a much faster return on investment. The following diagram compares delivering value iteratively with a non-agile project delivery where results are delivered only at the end of the project:
One can see clearly from the diagrams that the non-agile delivery of value at the end of a project is also extremely risk prone and suseptible to change. If the project is cancelled just before it delivers, then a fairly substantial amount of effort is wasted. In the agile iterative delivery situation, an endeavor can be cancelled at almost any time and it is likely that substantial value has already been delivered.
Even if the work cannot actually be delivered incrementally, it almost always can be divided in a way so that it can be inspected in stages. Either method of dividing work allows us to do the work in iterations.
Iterations are fixed and consistent units of time during which work is performed and between which planning, inspection and adjustment is done. The empowered team will decide on the length of iterations for their work. As a rule of thumb iterations should be shorter than the horizon of predictability. Generally, iterations should never be longer than one month, no matter what the endeavor.
At the end of each iteration, a demonstration of the work completed is given to the stakeholders in order to amplify learning and feedback. Between iterations, the stakeholders collaborate with the team to prioritize the remaining work and choose what will be worked on during the next iteration. During the iteration, the stakeholders need to be accessible for questions and clarifications.
Iterative and incremental delivery is used to allow for the early discovery and correction of mistakes and the incorporation of learning and feedback while at the same time delivering value early.