Great technique described by Shahin Sheidaei on his blog: Challenge, Estimate or Override (CEO) Game for Effective Estimations. It is much quicker than the Planning Game, and probably a bit slower than the Bucket System.
Great technique described by Shahin Sheidaei on his blog: Challenge, Estimate or Override (CEO) Game for Effective Estimations. It is much quicker than the Planning Game, and probably a bit slower than the Bucket System.
Agile methods such as Scrum, Kanban and OpenAgile do not require long-term up-front plans. However, many teams desire a long-term plan. This can be thought of as a roadmap or schedule or a release plan. Agile planning helps us build and maintain long-term plans.
The steps to do this are actually very simple:
Agile planning allows a team to update estimates at any time. Therefore, the techniques used above should not be thought of as a strict sequence. Instead, as the team and the business people learn, the estimates and long-term plan can be easily updated. Scrum refers to this ongoing process at “Product Backlog Refinement”.
In order to use Agile planning effectively, people must be aware of and support the principles of Agile planning:
The team decides on how much work it will do in a Sprint. No one should bring pressure on the team to over-commit. This simply builds resentment, distrust and encourages low-quality work. That said, of course teams can be inspired by challenging overall project or product goals. A stretch goal for a Sprint is just a way to 100% guarantee failure. Even the team should not set its own stretch goals.
There are a few interesting principles that apply here. For example, the Agile Manifesto mentions sustainability:
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
The Agile Manifesto also points out the importance of trust:
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
Stretch goals are incompatible with both of these principles from the Agile Manifesto.
There are two types of stretch goals. The first type are those assigned by outsiders to the team. The second type are those which a team sets for itself. Both types are bad.
The worst extreme of this type of stretch goal is also the most common! This is the fixed-scope-fixed-date project deadline. In this type of stretch goal, the project team, doing Scrum or not, is forced to work backwards from the deadline to figure out how to get the work done. If the team can’t figure this out, managers often say things like “re-estimate” or “just get it done.” (Note: another thing that managers do in this situation is even worse: adding people to the project! Check out “The Mythical Man-Month” by F. Brooks for a great analysis of this problem.)
My anecdotal experience with this sort of thing is simple: quality suffers or sustainability suffers. I once worked with three other people on a mission critical project to help two banks with their merger. There was a regulatory deadline for completing the integration of the two existing systems for things like trading, etc. Fixed-scope-fixed-date. Coffee and sleepless nights were our solution since we tried not to sacrifice quality. We actually ended up working in my home for the last few 24-hour stretches so that we would have access to a shower. Suffice it to say, there’s no way we could have sustained that pace. It’s anti-Agile.
A quick search for ideas and opinions about stretch goals makes it very clear that there is no commonly agreed “correct” answer. However, from an Agile perspective stretch goals assigned by outsiders are clearly against the principles of the Agile Manifesto.
The Scrum Guide states:
The number of items selected from the Product Backlog for the Sprint is solely up to the Development Team. Only the Development Team can assess what it can accomplish over the upcoming Sprint.
For emphasis: what it can accomplish – not what it (the Development Team) wants to accomplish, or what it should accomplish, or what it could accomplish if everything goes perfectly. A Development Team should be accomplishing their Sprint plan successfully (all Product Backlog Items done) on a regular basis. Of course, exceptional circumstances may intervene from time to time, but the team should be building trust with stakeholders. Here’s another story:
I had a good friend. We would always go out for coffee together. We just hung out – chatted about life, projects, relationships. Of course, from time-to-time one or the other of us would cancel our plans. That’s just life too. But there came a time when my friend started cancelling more often than not. There was always a good excuse: I’m sick, unexpected visitors, work emergency, whatever. After a little while of this I started to think that cancelling would be the default. I even got to the point where I was making alternative plans even if my friend and I had plans. I got to the point where I no longer trusted my friend. It didn’t matter that the excuses were always good. Trust was broken.
It doesn’t matter why a team fails to meet a goal. It reduces trust. It doesn’t matter why a team succeeds in meeting a goal. It builds trust. Even among team members. A team setting stretch goals is setting itself up for regular failure. Even if the team doesn’t share those stretch goals with outsiders.
Stretch goals destroy trust within the team.
Think about that. When a team fails to meet its own stretch goal, team members will start to look for someone to blame. People look for explanations, for stories. The team will create its own narrative about why a stretch goal was missed. If it happens over and over, that narrative will start to become doubt about the team’s own capacity either by pin-pointing an individual or in a gestalt team sense.
The importance of trust cannot be over-stated. In order for individuals to work effectively together, they must trust each other. How much trust? Well, the Agile Manifesto directly addresses trust:
Build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.
Here is my recent YouTube video about stretch goals… check it out and subscribe to our channel!
When asked to provide metrics to assess “how well” an Agile transformation is going, re-frame the discussion around measuring changes in the impact the IT organization is having (or not) on it’s Business environment, and define a small set of “fitness for purpose” metrics.
Sooner or later, as an IT organization embarks on a transformation towards Agile mindset and practices, someone will be asked to provide “hard evidence” that the effort is paying off, and the conclusion will be that metrics is the vehicle to satisfy that request. What are your Agile transformation metrics?
It’s been my experience that this request usually leads to a discussion about measuring the specific Agile initiatives the IT organization has launched. In organizations where the emphasis has been around engineering disciplines, such metrics might be things like unit test code coverage, or integration build times. If the focus was around teams and process, then counting number of teams “converted” to Scrum, or people sent to Scrum Master training may appear as the choice.
While those measurement might be useful indicators in some context, they have two problems. First, they are akin to measuring the performance of the car engine without looking outside the window; the engine might be performing well, but if the car doesn’t have the wheels attached, we’re going nowhere. More importantly, though, these figures are usually meaningless for Business stakeholders, who are the ones usually asking for them in the first place. Agile transformation metrics need to be meaningful to the Business.
Agile transformation efforts can be very costly exercises, therefore it is legitimate to ask about the results of such endeavour. The important thing to realize, though, is that this question is really equivalent to another question: “is the IT organization improving its impact on its Business environment.” Another way to put it is, borrowing from the terminology used by the Kanban community: “is the IT organization becoming more and more fit for purpose?” Answering this question, of course, requires a clear understanding of what is that the Business expects from its interactions with IT.
The IT organization can be seen as providing various services to customers. Arguably, if IT has decided to “transform” in some way (perhaps by moving towards an Agile mindset), it’s doing so to improve its impact on those customers, so this is what needs to be measured to know “how the transformation” is going.
Some of those customers are different areas of the organization (like Finance, or HR.) But it doesn’t stop there, because the Business’ engagement with IT doesn’t have value for its own sake. Ultimately, the Business is using IT as a way to optimize its operations so that it can provide external customers with more effective products and services. Moreover, IT is these days the direct channel through which those products and services are delivered to external customers (for example, through web sites and mobile applications.) Therefore, the concept of “fitness for purpose” of the IT organization can be extended to consider the fitness for purpose of the Business respect the external customers it intends to serve.
Measuring “agile transformation success” really means measuring the success of the exchanges between IT and the Business, and between the Business and its external customers. Measuring the internal processes and practices that IT puts in place as part of that “transformation” is beside the point. This implies starting with a careful definition of the boundaries that delineate the exchanges to be measured. There might be more to external customer fitness for purpose than IT operations, for example, and that needs to be considered when defining Agile transformation metrics, especially if we’re later going to be drawing causation conclusions.
Defining Agile transformation metrics will be, of course, a highly contextual exercise because every business organization is different. But we can, however, draw again from the Kanban community for some general guidelines on what to look for. Their thought leaders talk about classifying metrics into 3 categories: fitness for purpose metrics, health indicators and improvement drivers. Using this framework, when talking about “agile transformation metrics” we are referring mainly to the first category, and perhaps a bit to the second. Based on those, improvement initiatives can be put in place, and perhaps driven with metrics belonging to the third category.
A fitness for purpose metric (also known as KPI) is an indicator of something a customer will care about. This is a key distinction: if the metric is not easily recognizable and meaningful for the customer, then it’s not a KPI. Another key characteristic is that a minimum threshold for its value can be defined: if the metric goes below the threshold, the Business is putting the relation with its customers at risk (perhaps they will walk away, initiate legal actions, etc.). In other words, the Business is no longer “fit for purpose”. We can then measure the effectiveness of the “agile transformation” by analyzing how KPI values over time compare to their respective thresholds. A typical KPI is delivery time, measured from the moment a customer request is accepted and committed to, until the moment it’s delivered to production. This is usually a good Agile transformation metric.
Health indicators are metrics that are inwards facing. Customers don’t really care about them (or even understand), but they indicate how a given aspect of the system is operating. The key characteristic is that they are not directly actionable; they only provide information that needs to be analyzed and put in context. As the value of a health indicator changes, we can draw some conclusions about how the system works, or explain why something is happening (or not), but it doesn’t necessarily leads to concrete action. Defect count is an example of this. Customers will certainly care about quality of the product, and we can make inferences about that quality by looking at how many defects we have, but the absolute number of defects will not necessarily make the product more or less fit for purpose. It may happen that customers consider the current quality to be “good enough”, irrespective of the number of defects.
Finally, improvement driver metrics are metrics put in place to influence behaviour towards a particular change. Their key characteristic is that they are temporary: we set a target on them and once the target is achieved, the metric is no longer necessary. The reason for this is related to the unintended behaviours that a metric might encourage in people, which may lead to locally optimizing the metric at the expense of other aspects, leading to global sub-optimization of the system. An example is unit testing code coverage: if we have determined that a given service is not fit for purpose and the cause is related to poor unit test coverage, then establishing a target for minimum coverage may influence developers to work on adding tests to reverse the situation.
Even though the concept of self-organizing teams has been around for a long time, still some people think that a project manager or team lead should be assigning tasks to team members. Don’t do this!!! It is better to wait for someone to step up than to “take over” and start assigning tasks.
Assigning tasks can be overt or subtle. Therefore, avoid even suggestions that could be taken as assigning tasks. For example, no one should ever tell a Scrum Team member “hey! You’re not doing any work – go take a task!” (overt) or “This task really needs to get done – why don’t you do it?” (semi-overt) or “Would you consider working on this with me?” (subtle). Instead, any reference to tasks should be to the team at large. For example it would be okay for a team member to say “I’m working on this and I would like some help – would anyone help me?”
In the Scrum Guide, a partial definition of self-organizing is given:
Scrum Teams are self-organizing….. Self-organizing teams choose how best to accomplish their work, rather than being directed by others outside the team.
A more formal definition of the concept of “self-organizing” can be found here:
Self-organisation is a process where some form of global order or coordination arises out of the local interactions between the components of an initially disordered system. This process is spontaneous: it is not directed or controlled by any agent or subsystem inside or outside of the system; however, the laws followed by the process and its initial conditions may have been chosen or caused by an agent.
The key here is that there is no single point of authority, even temporarily, in a self-organizing team. Every individual member of the team volunteers for tasks within the framework of “the laws followed by the process” – namely Scrum. Scrum does define some constraints on individual behaviour, particularly for the Product Owner and the ScrumMaster. People in those two roles have specific duties which usually prevent them from being able to volunteer for any task. But all the other team members (the Development Team) have complete freedom to individually and collectively figure out how they will do the work at hand.
People who are managers are often worried about this. What if there is one person on the team who just doesn’t seem to be doing any work? Isn’t assigning tasks to this person a good thing? Scrum will usually make this bad behaviour really obvious. Let’s say that Alice hasn’t completed any tasks in the last four days (but she does have a task that she volunteered for at the start of the Sprint). Raj notices that Alice hasn’t finished that initial task. An acceptable solution to this problem is for Raj to volunteer to help Alice. Alice can’t refuse the help since Raj is self-organizing too. They might sit together to work on it.
Of course, that might not solve the problem. So another technique to use that respects self-organization is to bring it up in the Sprint Retrospective. The ScrumMaster of the team, Sylvie, chooses a retrospective technique that is designed to help the team address the problem. In a retrospective, it is perfectly acceptable for people on the team to be direct with each other. Retrospectives need to be safe so that this kind of discussion doesn’t lead to animosity between team members.
Remember: everyone goes through ups and downs in productivity. Sometimes a person is overwhelmed by other aspects of life. Sometimes a person is de-motivated temporarily. On the other hand, sometimes people become extremely engaged and deliver exceptional results. Make sure that in your team, you give people a little bit of space for these ups and downs. Assigning tasks doesn’t make a person more productive.
Dig deep and find out why no one wants to do it. This problem is usually because the task itself is worthless, frustrating, repetitive, or imposed from outside without a clear reason. If no one wants to do a task, the first question should always be: what happens if it doesn’t get done? And if the answer is “nothing bad”… then don’t do it!!!
There are, unfortunately, tasks that are important that still are not exciting or pleasant to do. In this situation, it is perfectly acceptable to ask the team “how can we solve this problem creatively?” Often these kinds of tasks can be addressed in new ways that make them more interesting. Maybe your team can automate something. Maybe a team member can learn new skills to address the task. Maybe there is a way to do the task so it never has to be done again. A self-organizing Scrum Team can use innovation, problem-solving and creativity skills to try to over come this type of problem.
And, of course, there’s always the Sprint Retrospective!
Autonomy is one of the greatest motivators there is for people doing creative and problem-solving types of work. The ability to choose your own direction instead of being treated like a mushy, weak, unreliable robot. Motivation, in turn, is one of the keys to creating a high-performance state in individuals and teams. The greatest outcome of good self-organization is a high-performance team that delivers great work results and where everyone loves the work environment.
Assigning tasks to people is an implicit claim that you (the assigner) know better than them (the assignees). Even if this is true, it is still easy for a person to take offence. However, most of the time it is not true. People know themselves best. People are best at assigning tasks to themselves. And therefore, having one person assigning tasks to other people almost always leads to sub-optimal work distribution among the members of a team.
The ScrumMaster plays an important role in Scrum. Part of this role is to encourage self-organization on a team. The ScrumMaster should never be assigning tasks to team members under any circumstances. And, the ScrumMaster should be protecting the team from anyone else who is assigning tasks. If someone within the team is assigning tasks to another team member, the ScrumMaster should be intervening. The ScrumMaster needs to be constantly aware of the activity on his or her team.
I have added a video to YouTube that you might consider sharing with ScrumMasters you know about this topic:
This article is a follow-up article to the 24 Common Scrum Pitfalls written back in 2011.
Although subtle, having a public retrospective can do terrible damage to a Scrum team. In this video I explain what I mean by “public”, why it is so bad, and what you should do instead. This is part of a video series on the Myths of Scrum that is meant to respond to some of the most common mis-information (myths) about Scrum and Scrum practices. I will follow-up this video in several weeks with a written article complimentary to this video. Feel free to let me know in the comments if you have any topics that you would like me to cover in my video series!
My colleague and friend Mike Caspar has posted another insightful article on his blog: Do not create unnecessary fear and animosity with Development Managers. From the article:
As teams grow, they build confidence and seem to become self-contained, and almost insular to some. This is normal and is not a sign of dysfunction. This simply indicates that the team is starting to have self identity. They are starting to act as a team. This is a good sign…. As the teams become more effective, the Development Manager feels more loss.
I had the privilege of attending Scrum.org‘s 2-day seminar on Scaled Professional Scrum. The Nexus, a connection or series of connections linking two or more things (direct translation from Latin a binding together), is the recommended scaling framework. The purpose of the Nexus is to manage dependencies between 3-9 Scrum Teams towards “reification”, to make an abstract idea real or concrete. This is ensured mostly through a single Product Owner, single Product Backlog, integrated (Nexus) Sprint Planning, Review and Retrospective and the addition of a Nexus Integration Team whose membership is made up mostly of Scrum team members internal to the Nexus, but often also includes other support personnel. The structure is very similar to LeSS, but perhaps even less prescriptive and is certainly much less prescriptive than SAFe. This is probably my favourite thing about the Nexus – the fact that it has just enough structure to be a model for scaling Scrum, but is light and flexible enough to accommodate all of the nuances that “just depend” on your situation. Like the other two above-mentioned scaling models, it places emphasis on the need for strong technical practices, continuous integration and the synchronization of events to facilitate integration. There is flexibility around synchronization, in that if the Nexus Sprint is 4 weeks in duration and teams within the Nexus want to do 2 or even 1 week Sprints, the model accommodates – as long as all of the teams’ work is combined into a fully integrated (reified) increment of potentially shippable product by the end of the Nexus Sprint.
How much documentation does it take to run a project with ten people working for six months? For some organizations it takes way too much:
This binder (about 3 or 4 inches thick) is all the documentation associated with such a project. In looking carefully at the project, creating the documentation took far more time than the time spent on designing, writing and testing the software. Yet, the documentation does not produce any value. Only the software produces value. The Agile Manifesto, asks us to focus on the outcome (working software) and to make tradeoffs to minimize the means (comprehensive documentation).
The Agile Manifesto asks us to challenge our assumptions about documentation. In many work environments, documentation is an attempt to address some interesting and important needs:
Documentation is usually heavier (more comprehensive) the more the following circumstances exist in an organization:
What if the software itself could address the needs that often documentation is used to address? Let’s look at them in turn:
In my recent training programs as research for this article, I have surveyed over 100 people on one aspect of documentation – code documentation. Every individual surveyed is either currently coding or has a coding background, and every single person had the same answer to a simple scenario question:
Imagine that you have just joined a new organization and you are about to start working as a software developer. One of the existing team members comes up to you and introduces himself. He has with him a piece of paper with a complicated-looking diagram and a full binder that looks to be holding about 250 pages. He asks you, “you need to get up to speed quickly on our existing system – we’re starting you coding tomorrow – would you prefer to go over the architecture diagram with me or would you prefer to review the detailed specifications and design documents.” He indicates the one-page diagram and the binder respectively. Which would you prefer?
(I’ve put up a Survey Monkey one-question survey: Code Documentation Preference to extend the reach of this question. It should take you all of 60 seconds to do it. I’ll post results when I write the next Agile Manifesto essay in a month or two.)
The fact that everyone answers the same way is interesting. What is even more interesting to me is that if you think through this scenario, it is actually almost the worst-case scenario where you might want documentation for your developers. That means that in “better” cases where documentation for developers may not be as urgent or necessary, then the approach of just going to talk with someone is a lot better.
Documentation and Maps
The problem with documentation is the same problem we have with maps: “the map is not the territory” (quote from the wisdom of my father, Garry Berteig). We sometimes forget this simple idea. When we look at, say, Google Maps, we always have in the back of our consciousness that the map is just a guide and it is not a guarantee. We know that if we arrive at a place, we will see the richness of the real world, not the simplified lines and colours of a map. We don’t consider maps as legally binding contracts (usually). We use maps to orient ourselves… as we look around at our reality. We can share directions using maps, but we don’t share purpose or problems with maps. And finally, maps assume that physical reality is changing relatively slowly (even Google Maps).
Many times when we create documentation in organizations, however, we get confused about the map versus the territory.
Agility and Documentation
Of course, code is a funny thing: all code is documentation too. The code is not the behaviour. But in software, code (e.g. Java, ASM, Scheme, Prolog, Python, etc.) is as close as possible to the perfect map. Software is (mostly) deterministic. Software (mostly) doesn’t change itself. Software (mostly) runs in a state absent from in-place human changes to that software. Software (mostly) runs on a system (virtual or physical) that has stable characteristics. The code we write is a map. From this perspective, documentation becomes even less important if we have people that already understand the language(s)/platform(s) deeply.
This essay is a continuation of my series on the Agile Manifesto. The previous two essays are “Value and Values” and “Individuals and Interactions over Processes and Tools“.
From the full article “5 Things Agile Leaders Must Do“:
Consider enrolling in our Scrum and Enterprise Agile for Executives ½ day seminar in Toronto.
Pair Programming Economics by Olaf Lewitz describes three activities in programming: typing, problem-solving and reading code. How does pair programming help? By making the balance between those three activities better.
This value is the hardest to do well.
In IT and high-tech, there is a “natural” prevailing culture that makes this first value incredibly difficult. This difficulty is rooted in traditional “scientific management“, but made even more so by a critical additional factor that is mostly invisible: techies solve problems with tools.
Management wants to define processes with clearly described activities, clear inputs and outputs, and clear sources and recipients of the activity (see the description of SIPOC for an explanation of this thinking). Techies build tools to automate these well-defined processes to improve their efficiency, quality and reliability.
Management creates organizational roles with detailed descriptions, detailed goals and detailed performance measurements (see the description of RACI for an explanation of this thinking). Techies build tools to carefully constrain people to these detailed roles to improve efficiency, quality and reliability.
Management has money. Techies want some of that money. So they build the tools to help management get what they really want: a completely automated organization of computers, machines and robots.
The culture of technology is to solve problems with individuals and interactions by introducing processes and tools. The culture of technology is (almost) inherently anti-Agile.
The culture of technology is to solve problems with individuals and interactions by introducing processes and tools. The culture of technology is (almost) inherently anti-Agile.
Individuals and Interactions
Let’s look at the first part of this value in a bit more depth. When we think about work, most of us work with other people. We bring our unique skills, personality and interests to work, and we work with other people who also bring unique skills, personality and interests. In a high-bureaucracy, high-technology work environment, it is easy to forget about all this uniqueness and instead objectify people. When people sense they are being objectified, mostly they feel bad about it. We want to be acknowledged as thinking, feeling, unique beings with agency. Objectification, no matter the source or the rationale, is depressing and de-humanizing. The Agile Manifesto implicitly recognizes this concept and asks us who follow the Manifesto to try to shift our value-focus.
There are many aspects to this concept of humanizing work. Some things that come to mind immediately include recognizing and encouraging people’s capacity for:
Processes and Tools
This side of the value is also interesting. Processes and tools do not have agency. They do not improve on their own. Instead, processes and tools only either remain the same or degrade. Processes and tools are forces for stasis: they encourage maintenance of the status quo. Only humans introduce new processes and tools.
Technologists live in a philosophical double-standard: we build processes and tools for others to use and which we frequently would not like used on ourselves. (We will discuss the cases where me might both build and benefit from processes and tools in a bit.) This is one of the challenges of the type of work we do in technology, but it also applies to many other types of work. So how do we solve this conundrum? I would assert that the principles of the Agile Manifesto and the various Agile methods and techniques are all answers to this question. They show us possible ways to implement this value (and the others) without getting stuck in processes and tools.
Only humans introduce new processes and tools.
What are Processes Good For, What are Tools Good For?
Some processes are good. Some amount of process is good. How do we determine what is good? Well, it largely depends on context. Some examples:
If a close family member is living in a distant location then the advances in communication tools are extremely helpful: the telegraph, the telephone, the cell phone, email, Skype. These tools create connections where otherwise there would be little or none.
If a great deal of data is created while running a marketing campaign and needs to be stored and manipulated, then computers are amazing tools for this. Computers are much much better than human minds and manual record-keeping for this sort of work.
If you create a fantastic new soup, from scratch, for some special occasion and you want to remember how to make and even share how to make it with others, then you document the process in a recipe.
Context, Emphasis and Crisis
Context here is important. The value of Individuals and Interactions over Processes and Tools is basically a statement that given the right circumstances we can use processes and tools, but that our default approach to work and problem-solving should be to focus on individuals and their interactions. Depending on the state of your work environment this is easier or harder.
For example, a startup company founded by three long-time friends who have not yet employed anyone else is almost certainly going to solve most problems that come up through discussion amongst founders and through the development of their skills and capabilities. As a company gets larger, however, there is pressure to adopt more and more processes and tools. This pressure comes from a deep source: lack of trust. At about 12 people, you reach the limit of the number of people you can have and still have anyone do anything (this limit is referred to obliquely in “The Wisdom of Teams” by Katzenbach and Smith). After 12 people, it becomes harder to avoid role specialization and some basic forms of processes and tools. In other words, bureaucracy starts growing as the organization grows. Even at this size, however, it is still relatively easy to have a very strong emphasis on individuals and interactions. There is another important limit: somewhere around 150 to 200 people, any hope of 100% mutual trust among the members of the organization is lost. This is the point at which processes and tools “naturally” start to truly take over. (This transition can happen even in much smaller organizations if the culture does not emphasize trust-based interactions.)
In small trust-based organizations, crisis is usually addressed by the mechanisms of mutual respect, skill development, informal agreements, and strengthening the interactions between people. In a large organization with low trust, crisis is almost always addressed by the creation of new bureaucracy: sign-offs, audits, traceability, procedures, policies, processes and tools.
The true test of the an organization’s commitment to the first value of the Agile Manifesto is, therefore, how it responds to crisis. When someone makes a mistake, can we help them develop the skill and the support networks to avoid the mistake in the future? Or do we put in place even more restrictive constraints on what that person does and how they do it?
In a large organization with low trust, crisis is almost always addressed by the creation of new bureaucracy.
Beyond IT and High-Tech
For now, all that needs to be said is that this particular value of the Agile Manifesto does not in any way directly refer to software or software development. As such, it is pretty easy to see how it could be applied in many other types of work. However, there are some types of work where processes and tools really do take precedence over individuals and interactions. If we want to apply the concepts of Agile universally (or near-universally), we have to examine some of these exceptions. I will leave that for a future essay.
In the next few articles, I will continue to look in-depth at each of the values of the Agile Manifesto. If you missed the first essay in this series, please check it out here: The Agile Manifesto – Essay 1: Value and Values.
What is Agile?
In 2001, 17 people got together for a world-changing discussion about software development. They tried to find the common values and principles by which people could do better at the work of software development, which was in a terrible crisis (and still, to some extent is). They were successful in that they created a list of 4 values and 12 principles to guide people trying to find better ways of developing software: the Agile Manifesto was born.
Now, nearly 14 years later, Agile software development has become well-known (if not well-practiced) throughout the business world. In fact, the concepts of Agile software development have been extended through to many other fields as diverse as mining, church management, personal time management, and general corporate management. In the process, there has also been a growing recognition of the relationship between Agile values and principles and those of Lean thinking.
It is time to think about the concepts behind the values and principles. To acknowledge that the Agile Manifesto (for software development) can be re-stated at a much deeper level. To abstract Agile software development to Agile work in general. This is my goal over a series of essays about the Agile Manifesto.
Let’s start with an analysis of the values of the Agile Manifesto in relationship to the concept of “value”.
The Agile Manifesto Values
In the Agile Manifesto we can read the four values:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
While a few of these already are quite general, let’s dig a bit deeper by starting with the second value, “working software over comprehensive documentation”. What does this value really refer to? Why do we care about software over documentation. Why “working” vs. “comprehensive”? This is where Lean thinking can help us. The notion of “customer value-added” or just value added work is any work that changes the form, fit or function of that which you are delivering and simultaneously is work that a customer would pay for independently of all the other activities and results you may be spending time on. In this specific example of software and documentation, we can try to imagine what it would be like to say to a potential customer, “we can give you documentation for our software, but we won’t be able to give you the actual software itself… but we’d still like to be paid.” I’m sure it will be clear that it would be a very unusual customer who would agree to such a proposal. Thus, we see that the value of software development activities is in producing the software itself, and the documentation is by necessity of secondary importance.
But what if we are writing a book of fiction? Surely this is documentation! But, it is not. To make the analogy to the type of documentation mentioned in the Agile Manifesto, we would sell a book not just with the story itself, but also with in-depth instructions on how to use a book, how to read, how to interpret our feelings as we read, etc. And not just that, but we would also provide a set of notes to the publisher about exactly how we wrote the book: the time and place of each paragraph written, our original outlines, our research including much that was thrown away, all our conversations with people as we struggled to sort out various plot, character and setting elements, and possibly even all our edits that we had thrown away. This is the documentation to which, by analogy, the Agile Manifesto is referring.
Perhaps now we can look at a connection between the first value and the second value of the Manifesto: that documentation, tools and processes are all much of the same thing. They all belong to the same abstract category, namely, the means used to achieve a particular end. Of course, we all know that both the means and the ends are important, although we may not all agree on their relative importance. Nevertheless, we can probably agree on some extreme outliers that will help us come to the point I wish to make. For example, we can agree that killing someone merely to get to the front of the grocery store line and save a minute or two of time is an extreme case where the end clearly does not justify the means. Likewise, we can also agree that refusing honestly given help out of a desire for independence when it ends with the death of our children by starvation is putting means too much in the fore. Balance is required, therefore. The Agile Manifesto acknowledges this balance by its epilogue to the values, “That is, while there is value in the items on the right, we value the items on the left more.” The other two values of the Agile Manifesto which mention contract negotiation and following a plan are similarly pointing out activities of the means that are non-value added, in Lean terms.
The Agile Manifesto, is stating four values, is, quite directly, pointing to those things which in life are fundamentally of value as ends, not just as means. Individuals and interactions, working software, customer collaboration and responding to change, are all valuable. In order to abstract away from software, then, and create a more general statement of the nature of Agility, we need to explore the idea of value.
The Idea of Value
If we are in business, determining value, while possibly complicated, is not usually too obscure an effort. We look at exchanges of money to see where the “market” agrees there is value. The price of a product or service is only representative of value if someone will actually pay. Therefore, businesses look at return on investment, profit margins and the like to determine value. Similarly, in other types of markets such as the stock market, value is determined by an exchange of money. But underlying this exchange of money is a decision made by an individual human being (or several, or many) to refuse all the other potential uses of that money for the one specific use of making a particular purchase. This choice is based on all sorts of factors. In economics, we talk about these factors mostly in rational selfish terms: what sort of benefit does the purchaser get from the purchase. Factors such as risk, short-term vs. long-term, net present value, trade-offs, etc. certainly can play a role in such decisions. But in business (and in particular marketing and sales), we know that there are also lots of non-rational forces at work in deciding to spend money on a particular something. Value, therefore, has a great deal to do with how a particular person both feels and understands the current proposed “investment” opportunity.
Feeling and understanding arise from many specific factors, as discussed, but what can we say generally about feeling and understanding? They are internal states of a person’s mind (or heart, if you like). Those internal states have been the subject of much discussion in the realm of psychology, sociology, economics, and philosophy. But quite simply, those internal states are greatly determined by perception. How a person perceives a situation is the immediate general factor that determines those internal states. Of course, perception is a general term that includes sensory perception, but also the kinds of prejudices we have, the categories into which we place things conceptually, the internal language we use to describe things, and our existing emotional and mental constructs. So, fundamentally, value is perceived depending on all these perceptual factors.
The Agile Manifesto authors, therefore, had (and perhaps still have) a perception of value which places individuals and interactions, working software, customer collaboration and responding to change all in a position of more value than those other items. But this perception of value may not be in alignment with other people’s perception of value. Still, we can see already in 13 short years that there is broad, if not universal, agreement on these statements of value. Why is this? Why has Agile become so popular over a relatively sustained length of time with a trajectory that still seems to have it growing in popularity for years to come?
Agile values address a deep need in people in the software development discipline and indeed, by analogy, into much of the work world.
In my next essay on the Agile Manifesto, we will take a look in more depth at the first value of the Agile Manifesto: Individuals and Interactions over Processes and Tools.
Some links to commentary on the values of the Agile Manifesto while you wait for my next essay instalment!…
Thanks everyone for feedback, comments and suggestions for my new book, Agile Advice – Creating High Performance Teams In Business Organizations. It is available for purchase (only $2.99) on lulu.com (that’s where the link goes). Here is the blurb about the book:
Agile Advice is a collection of brief articles and longer essays about Agile methods and their principles and practices. Agile Advice articles come primarily from the popular AgileAdvice.com blog which has been in the top 50 of Agile blogs since its inception in 2005. The book has three never-before-seen essays including “Agile Mining at a Large Canadian Oil Sands Company”, “Crossing the Knowing-Doing Gap”, and “Becoming a Professional Software Developer”. Agile Advice also has a whole section on choosing an Agile method. The author, Mishkin Berteig, has been working with Agile methods such as Scrum, Kanban, and Extreme Programming since 1996.
Once you have read it, I would love to hear your feedback and reviews here. I will try to publish updates quarterly over the next year to make it even better! Thanks again for your support.
The Real Agility Program is an Enterprise Agile change program to help organizations develop high-performance teams, deliver amazing products, dramatically improve time to market and quality, and create work environments that are awesome for employees.
This article is a written summary of the Executive Briefing presentation available upon request from the Real Agility Program web site. If you obtain the executive briefing, you can follow along with the article below and use it to present Real Agility to your enterprise stakeholders.
At Berteig Consulting we have been working for 10 years to learn how to help organizations transform people, process and culture. The problem is simple to state: there is a huge amount of opportunity waste and process waste in most normal enterprise-scale organizations. If you have more than a couple hundred people in your organization, this almost certainly affects you.
We like to call this problem “the Bureaucratic Beast”. The Bureaucratic Beast is a self-serving monster that seems to grow and grow and grow. As it grows, this Beast makes it progressively more difficult for business leaders to innovate, respond to changes in the market, satisfy existing customers, and retain great employees.
Real Agility, a system to tame the Bureaucratic Beast, comes from our experience working with numerous enterprise Agile adoptions. This experience, in turn, rests on the shoulders of giants like John Kotter (“Leading Change”), Edgar Schein (“The Corporate Culture Survival Guide”), Jim Collins (“Good to Great” and “Built to Last”), Mary Poppendieck (“Lean Software Development”) Jon Katzenbach (“The Wisdom of Teams”) and Frederick Brooks (“The Mythical Man-Month”). Real Agility is designed to tame all the behaviours of the Bureaucratic Beast: inefficiency, dis-engaged staff, poor quality and slow time-to-market.
Studies have proven that Agile methods work in IT. In 2012, the Standish Group observed that 42% of Agile projects succeed vs. just 14% of projects done with traditional “Bureaucratic Beast” methods. Agile and associated techniques aren’t just for IT. There is growing use of these same techniques in non-technoogy environments such as marketing, operations, sales, education, healthcare, and even heavy industry like mining.
Real Agility Basics: Agile + Lean
Real Agility is a combination of Agile and Lean; both systems used harmoniously throughout an enterprise. Real Agility affects delivery processes by taking long-term goals and dividing them into short cycles of work that deliver valuable results rapidly while providing fast feedback on scope, quality and most importantly value. Real Agility affects management processes by finding and eliminating wasteful activities with a system view. And Real Agility affects human resources (people!) by creating “Delivery Teams” which have clear goals, are composed of multi-skilled people who self-organize, and are stable in membership over long periods of time.
There are lots of radical differences between Real Agility and traditional management (that led to the Bureaucratic Beast in the first place). Real Agility prioritizes work by value instead of critical path, encourages self-organizing instead of command-and-control management, a team focus instead of project focus, evolving requirements instead of frozen requirements, skills-based interactions instead of roles-based interaction, continuous learning instead of crisis management, and many others.
Real Agility is built on a rich Agile and Lean ecosystem of values, principles and tools. Examples include the Agile Manifesto, the “Stop the Line” practice, various retrospective techniques, methods and frameworks such as Scrum and OpenAgile, and various thinking tools compatible with the Agile – Lean ecosystem such as those developed by Edward de Bono (“Lateral Thinking”) and Genrich Altshuller (“TRIZ”).
Real Agility acknowledges that there are various approaches to Agile adoption at the enterprise level: Ad Hoc (not usually successful – Nortel tried this), Grassroots (e.g. Yahoo!), Pragmatic (SAFe and DAD fall into this category), Transformative (the best balance of speed of change and risk reduction – this is where the Real Agility Program falls), and Big-Bang (only used in situations of true desperation).
Why Choose Transformative?
One way to think about these five approaches to Agile adoption is to compare the magnitude of actual business results. This is certainly the all-important bottom line. But most businesses also consider risk (or certainty of results). Ad-Hoc approaches to Agile adoption have poor business results and a very high level of risk. Big-Bang approaches (changing a whole enterprise to Agile literally over night) often have truly stunning business results, but are also extremely high risk. Grassroots, where leaders give staff a great deal of choice about how and when to adopt Agile, is a bit better in that the risk is lower, but the business results often take quite a while to manifest themselves. Pragmatic approaches tend to be very low risk because they often accommodate the Bureaucratic Beast, but that also limits their business results to merely “good” and not great. Transformative approaches which systematically address organizational culture are just a bit riskier than Pragmatic approaches, but the business results are generally outstanding.
More specifically, Pragmatic approaches such as SAFe (Scaled Agile Framework) are popular because they are designed to fit in with existing middle management structures (where the Bureaucratic Beast is most often found). As a result, there is slow incremental change that typically has to be driven top-down from leadership. Initial results are good, but modest. And the long term? These techniques haven’t been around long enough to know, but in theory it will take a long time to get to full organizational Agility. Bottom line is that Pragmatic approaches are low risk but the results are modest.
Transformative approaches such as the Real Agility Program (there are others too) are less popular because there is significantly more disruption: the Bureaucratic Beast has to be completely tamed to serve a new master: business leadership! Transformative approaches require top-to-bottom organizational and structural change. They include a change in power relationships to allow for grassroots-driven change that is empowered by servant leaders. Transformative approaches are moderate in some ways: they are systematic and they don’t require all change to be done overnight. Nevertheless, often great business results are obtained relatively quickly. There is a moderate risk that the change won’t deliver the great results, but that moderate risk is usually worth taking.
Regardless of adoption strategy (Transformative or otherwise) there are a few critical success factors. Truthfulness is the foundation because without it, it is impossible to see the whole picture including organizational culture. And love is the strongest driver of change because cultural and behavioural change requires emotional commitment on the part of everyone.
Culture change is often challenging. There are unexpected problems. Two steps forward are often followed by one step back. Some roadblocks to culture change will be surprisingly persistent. Leaders need patience and persistence… and a systematic change program.
The Real Agility Program
The Real Agility Program has four tracks or lines of action (links take you to the Real Agility Program web site):
Structurally an enterprise using Real Agility is organized into Delivery Groups. A Delivery Group is composed of one or more Delivery Teams (up to 150 people) who work together to produce business results. Key roles include a Business leader, a People leader and a Technology leader all of whom become Real Agility Coaches and take the place of traditional functional management. As well, coordination across multiple Delivery Teams within a Delivery Group is done using an organized list of “Value Drivers” maintained by the Business leader and a supporting Business Leadership Group. Cross-team support is handled by a People and Technology Support Group co-led by the People and Technology leaders. Depending on need there may also be a number of communities of practice for Delivery Team members to help spread learning.
At an organizational or enterprise level, the Leadership Team includes top executives from business, finance, technology, HR, operations and any other critical parts of the organization. This Leadership Team communicates the importance of the changes that the Delivery Groups are going through. They lead by example using techniques from Real Agility to execute organizational changes. And, of course, they manage the accountability of the various Delivery Groups throughout the enterprise.
The results of using the Real Agility Program are usually exceptional. Typical results include:
Of course, these results depend on baseline measures and that key risk factors are properly managed by the Leadership Team.
Not every organization needs (or is ready for) the Real Agility Program. Your organization is likely a good candidate if three or more of the following problems are true for your organization:
Consider that list carefully and if you feel like you have enough of the above problems, please contact us at firstname.lastname@example.org. or read more about the Real Agility Program for Enterprise Agility on the website.