This article is adapted from a session proposal to Toronto Agile Conference 2018.
Leadership occurs as conscious choice carried out as actions.
Everyone has the ability to carry out acts leadership. Therefore, everyone is a potential leader.
For leadership to be appropriate and effective, acts of leadership need to be tuned to the receptivity of those whose behaviour the aspiring leader seeks to influence. Tuning leadership requires the ability to perceive and discern meaningful signals from people and, more importantly, the system and environment in which they work.
As leaders, the choices we make and the actions we carry out are organic with our environment. That is, leaders are influenced by their environments (often in ways that are not easily perceived), and on the other hand influence their environments in ways that can have a powerful impact on business performance, organizational structures and the well-being of people. Leaders who are conscious of this bidirectional dynamic can greatly improve their ability to sense and respond to the needs of their customers, their organizations and the people with whom they interact in their work. The following list is one way of describing the set of capabilities that such leaders can develop over time.
Create Identity: Real leaders understand that identity rules. They work with the reality that “Who?” comes first (“Who are we?”), then “Why?” (“Why do we do what we do?”).
Focus on Customers: Real leaders help everyone in their organization focus on understanding and fulfilling the needs of customers. This is, ultimately, how “Why?” is answered.
Cultivate a Service Orientation: Real leaders design and evolve transparent systems for serving the needs of customers. A leader’s effectiveness in this dimension can be gauged both by the degree of customer satisfaction with deliverables and to the extent which those working in the system are able to self-organize around the work.
Limit Work-In-Progress: Real leaders know the limits of the capacity of systems and never allow them to become overburdened. They understand that overburdened systems also mean overburdened people and dissatisfied customers.
Manage Flow: Real leaders leverage transparency and sustainability to manage the flow of customer-recognizable value through the stages of knowledge discovery of their services. The services facilitated by such leaders is populated with work items whose value is easily recognizable by its customers and the delivery capability of the service is timely and predictable (trustworthy).
Let People Self-Organize: As per #3 above, when people doing the work of providing value to customers can be observed as self-organizing, this is a strong indication that there is a real leader doing actions 1-5 (above).
Measure the Fitness of Services (Never People): Real leaders never measure the performance of people, whether individuals, teams or any other organization structure. Rather, real leaders, practicing actions 1-6 (above) understand that the only true metrics are those that provide signals about customers’ purposes and the fitness of services for such purposes. Performance evaluation of people is a management disease that real leaders avoid like the plague.
Foster a Culture of Learning: Once a real leader has established all of the above, people involved in the work no longer need be concerned with “safe boundaries”. They understand the nature of the enterprise and the risks it takes in order to pursue certain rewards. With this understanding and the transparency and clear limits of the system in which they work, they are able to take initiative, run experiments and carry out their own acts of leadership for the benefit of customers, the organization and the people working in it. Fear of failure finds no place in environments cultivated by real leaders. Rather, systematic cycles of learning take shape in which all can participate and contribute. Feedback loop cadences enable organic organizational structures to evolve naturally towards continuous improvement of fitness for purpose.
Encourage Others to Act as Leaders: Perhaps the highest degree of leadership is when other people working with the “real leader” begin to emerge as real leaders themselves. At this level, it can be said that the culture of learning has naturally evolved into a culture of leadership.
Stay Humble: Real leaders never think that they have it all figured out or that they have reached some higher state of consciousness that somehow makes them superior to others in any way. They are open and receptive to the contributions of others and always seek ways to improve themselves. Such humility also protects them from the inevitable manipulations of charlatans who will, form time to time, present them with mechanical formulas, magic potions, palm readings and crystal ball predictions. Real leaders keep both feet on the ground and are not susceptible to the stroking of their egos.
If you live in Toronto, and you would like to join a group of people who are thinking together about these ideas, please feel welcome to join the KanbanTO Meetup.
Register here for a LeanKanban University accredited leadership class with Travis.
Over the years I have done a number of talks for local chapters of the Project Management Institute. They have covered a range of topics, but one common theme that comes up over and over is that Scrum is not the best Agile method for delivering an IT Project. I even published a short video on the topic:
Several years ago, I also published a short article describing what Scrum is good for:
So… if Scrum isn’t so good for IT project work, then what can bring real agility to IT projects?
IT Project Attributes
Most of my work experience prior to running my business was in IT projects in banking, capital markets, insurance and a bit in government and healthcare. I mention that merely to indicate that my discussion of this isn’t just theoretical: I’ve seen good projects and bad projects. I’ve been on death-march projects, small projects and massive projects ($1b+). I’ve dealt with regulatory issues, vendor issues, offshoring issues, telecommuting issues, architectural issues, political issues, and seen enough problems to understand the complexity of reality.
IT projects have some common characteristics:
Like any project, there’s a deadline and a scope of work and a budget. These things don’t work well with Scrum. It’s possible to force them to fit together, but you lose a lot of what makes Scrum effective.
IT (as opposed to, say, tech startups) tends to use more mature technology platforms. Scrum is neutral about technology, but there are other Agile methods that address this type of technology more effectively.
IT Projects are often not the only thing going on in the technology organization. In particular, operations and user support add to IT project complexity, and require different “classes of service” than Scrum provides.
The issues that I mentioned above such as regulation, vendors, offshoring etc. are also common attributes of IT projects. Scrum makes harsh demands on an organization that challenge the approach to dealing with these issues. The change required to accommodate Scrum may not be worth it.
The Bad News about IT Project Agility
The whole project orientation to IT work is questionable. It’s just not a good fit. In most mid- to large-size organizations, IT does two things: it provides technology services to the rest of the organization, and it provides technical product development capacity to lines of business. For example, upgrading the office wi-fi routers and adding a new payment type to the online customer portal, respectively. The work of the IT department, therefore, falls into several different categories:
New artifacts that need to be created. Usually this is the stuff like coding algorithms and other business logic, creating new databases, configuring purchased systems, etc.
Repetitive activities that need to be sustained for a period or indefinitely, or which occur on-demand but at irregular times. For example, running a nightly batch process or deploying an update to a production environment.
Quality problems that need to be fixed. Defects and production problems are the obvious categories here, but also quality problems that are causing user confusion or time wastage.
Obstacles to work that need to be overcome. Often obstacles come from outside the project team in the form of interruptions. Other forms of obstacles can be unexpected bureaucracy, shifting funding, problems with a vendor, etc.
Calendar events that need to be accommodated. Milestones in the project, particularly regulatory milestones are crucial in IT project work, there are many other types such as all-hands meetings, statutory holidays, hiring or contract end dates, etc.
Of these, only repetitive activities and calendar events fit well into a project perspective. The others typically have a level of uncertainty… complexity… that makes it very difficult to approach with the project perspective of fixed deadlines and scope.
On the other hand, Scrum only really handles new artifacts and obstacles directly, and quality problems indirectly. These are the kinds of activities that are the focus of product development. Repetitive activities and calendar events are anathema to the core Scrum framework. If I think about this from a scoring perspective, Scrum supports these kinds of work as follows (-5 means totally counter, 0 means no impact, +5 means total support):
Scrum Support for IT Project Work Types:
New artifacts: +5
Repetitive activities: -2
Quality problems: 0
Obstacles: +4
Calendar events: -5
SCORE: +2 – barely positive impact on IT project work!!!
The bad news, therefore: neither a project orientation nor Scrum really cover all the needs of an IT project environment.
There are many, but these are my three favourite alternatives: Extreme Programming, Kanban and OpenAgile. All three of them cover the five types of work more effectively than Scrum. All three of them are oriented to more generic types of work. After describing each briefly, I’ll also mention which one is my top choice for IT project work.
Extreme Programming for IT Project Work
Historically, Extreme Programming (XP) emerged in an IT Project context: the famous C3 project at Chrysler. This approach to IT project work has many things in common with other approaches to agility (which are described in the Agile Manifesto). XP allows the five types of work as follows:
New artifacts are the core of XP and are usually expressed as User Stories. This is common to Scrum and many other Agile methods. These are typically the features and functionality of a system… the scope of the project work. XP does not make any strong assertions about the size or stability of the backlog of new artifacts and as such can accommodate the project orientation in IT with relatively fixed scope.
Repetitive activities are not explicitly addressed in XP, but nor is there anything in XP which would cause problems if an XP team is required to do operational or support work which is the source of most repetitive activities in an IT environment.
Quality problems are addressed directly with both preventative and reactive measures. Specifically, Test-Driven Development, Acceptance Test-Driven Development are preventative, and Refactoring and Continuous Integration are reactive. XP has a deep focus on quality.
Obstacles are not directly addressed in XP, but indirectly through the XP value of courage. Implicitly, then, obstacles would be overcome (or attempted) with courage.
Calendar events are not addressed directly for the most part with the exception of release planning for a release date. However, the stuff related to other calendar activities is not directly handled. XP is less antagonistic to such things than Scrum, but only by implication: Scrum would often put calendar events in the category of obstacles to be removed to help a team focus.
XP Support for IT Project Work Types:
New artifacts: +5
Repetitive activities: 0
Quality problems: +5
Obstacles: +2
Calendar events: +1
SCORE: +13 – moderate to strong positive impact on IT project work!
Summary: much better than Scrum, but still with some weaknesses.
Kanban for IT Project Work
Kanban is different from most other approaches to agility in that it is a “continuous flow” method, rather than an iterative/incremental method. This distinction basically means that we move packages of work through a process based on capacity instead of based on a fixed cadence. Kanban asks that we visualize the current state of all work packages, limit the amount of work in progress at any stage in our delivery process, and use cadences only for iterative and incremental improvement of our process (not our work products).
Kanban is much gentler than Scrum or Extreme Programming in that it does not require leader-led reorganization of staff into cross-functional team units. Instead, we identify a service delivery value stream and leaders manage that stream as it currently operates.
New artifacts in Kanban are supported, and certainly welcome, but Kanban does not seem to acknowledge the problem of formal complexity (creativity, problem-solving, human dynamics) in the creation of new artifacts. There are good attempts to apply statistical methods to the management of new artifacts, but their fundamentally unknowable cost/end (undecidable problem) is not really effectively addressed.
Repetitive activities are handled extremely well in Kanban including different classes of service. Repetitive activities are handled well partly as a result of the history of Kanban as a signalling system in manufacturing environments.
Quality problems are handled similarly to new artifacts: supported, welcome, and even possibly addressed in the cadences of continuous improvement that Kanban supports. However, quality problems are another area where technical complexity makes proper analysis of these activities difficult.
Kanban relegates the handling of obstacles to the manager of service delivery. There is no explicit support for strong organizational change efforts. In fact, Kanban discourages “transformative” change which is sometimes required given the problem of Nash equilibriums.
Kanban works well with Calendar events by treating them as activities with a particular class of service required.
Kanban Support for IT Project Work Types:
New artifacts: +3
Repetitive activities: +5
Quality problems: +3
Obstacles: 0
Calendar events: +5
SCORE: +16 – strong positive impact on IT project work!!
Summary: even better than XP, easier to adopt. (Actually, almost anything is easier to adopt than XP!!!)
OpenAgile for IT Project Work
OpenAgile is an obscure non-technology-oriented method based on the work I and a few others did about 10 years ago. The OpenAgile Primer is the current reference on the core of the OpenAgile framework. OpenAgile has been applied to general management, small business startups, sales management, mining project management, emergency services IT, and many other areas of work. I’m partial to it because I helped to create it!
OpenAgile emerged from consulting work I did at CapitalOne in 2004 and 2005 and work I did with my own business in 2006 and 2007. A great deal of the older articles on this blog are forerunners of OpenAgile as it was being developed. See, for example, Seven Core Practices of Agile Work.
The types of work listed above, are indeed the core types of work described in OpenAgile. As such, OpenAgile fully supports (nearly) all five types of activities found in IT projects. However, OpenAgile is not just a work delivery method. It is also a continuous improvement system (like Kanban and Scrum) and so it also assumes that a team or organization using OpenAgile must also support learning. This support for learning means that OpenAgile does not over-specify or give precise definitions on how to handle all five types of work. Thus, my scores below are not all +5’s…
OpenAgile Support for IT Project Work Types:
New artifacts: +4
Repetitive activities: +4
Quality problems: +4
Obstacles: +4
Calendar events: +4
SCORE: +20 – very strong positive impact on IT project work!!!
Summary: OpenAgile is the best approach I know of for general IT project environments.
Conclusion
Regrettably, I wouldn’t always recommend OpenAgile – there are just too few people who really understand it or know how to help an organization adopt it effectively. If you are interested, I’d be happy to help, and we can certainly arrange private training and consulting, but mostly I would recommend Kanban to people interested in taking the next step in effectiveness in IT projects. Please check out or Kanban learning events and consider registering for one or asking for us to come to your organization to deliver training, coaching or consulting privately.
The Agile Manifesto was signed and made public in 2001. It begins with short, pithy statements regarding what should be the priorities of software developers, followed by Twelve Principles. In this article I want to call attention to the fifth principle in the Agile Manifesto, which is:
“Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.”
Although it appears to be a very simple statement, I suggest that it is jam-packed with profitable guidance, and is essential to, and at the heart of, real Agility. Human qualities must be considered.
Motivation
The first part of the principle urges us to build projects around motivated individuals. What does this imply?
The idea of “building a project” makes it a process, not necessarily a fait accompli. It can change and be altered as one works toward it. There may be a structural roadmap, but many details and aspects can change in the “building.”
The second part of the statement describes motivated individuals. The verb “motivate”is an action word, meaning to actuate, propel, move or incite. Thus, in this line, is the “project” the thing which will “move or incite” those being asked to carry it out?
Or do we understand this to imply that the individuals are already “motivated” in themselves, which is an emotional condition of individuals? Is this motivation already there prior to starting a project?
The topic of motivation is rich. How does motivation occur? Is it the culture and environment of the company, lived and exemplified by it’s leaders, which motivates? Or is motivation an intrinsic quality of the individual? It may be both. (Daniel Pink, author of “Drive,” uses science to demonstrate that the best motivators are autonomy, mastery and purposeful-ness – ideas which are inherent in the Agile Manifesto.)
In any case, the line itself suggests that the project may be a) interesting to pertinent (perhaps already motivated) individuals, b) do-able by those same individuals, and c) contains enough challenges to test the mastery and creativity of the individuals. In other words, it’s going to be a project that the individuals in your company care about for more than one reason.
Environment
The second line from the fifth Principlehas two distinct parts to it. The first part, “Give them the environment and support they need” puts a great deal of responsibility on whoever is assigning the project. Let’s look at the idea of environment first.
In a simple way, we can understand environment as the physical place which influences a person or a group. It can be any space or room; it can refer to the lighting, the colours, the furniture, the vegetation, the walls, whether water or coffee is available – physical elementswhich will certainly affect the actions of people and teams. For example, creating face-to-face collaboration environments is also part of the Agile Manifesto.
But we must remember that environment also entails the non-physical ie, the intellectual, emotional, or even the spiritual. Is the environment friendly or not? Cheerful or not? Encouraging or not? Affirming or not? We can think of many non-physical attributes that make up an environment.
Support
These attributes allude to the second part of what’s to be given by an owner or manager: “…and support they need.” This idea of support pertains not just to helping someone out with tools and responding to needs, but that the environment is supportive in every way –physically, intellectually, emotionally and spiritually. This may be a more holistic way of considering this Agile principle.
The last part of the statement is of great importance as well: and trust them to get the job done.
If you as product owner, or manager have created motivation, environment and support, then the last crucial requirement of trust becomes easier to fulfill. There is nothing more off-putting than being micromanaged, supervised or controlled with excessive attention to small details. Trust means you have confidence in the capacity of your team and its individual members. It also implies that they will communicate with transparency and honesty with you, and you with them, about the project.
Context
The principles of Agile do not exist in a vacuum, because, of course, other principles such as the following, are relevant to this discussion:
“The best architectures, requirements, and designs emerge from self-organizing teams.”
“At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.”
This fifth principle has application far beyond IT projects. I wanted to reflect on it because it speaks to human qualities, which must be recognized as a key factor in happy work places, and in any high-performance team.
Valerie Senyk is a Customer Service agent and Agile Team Developer with BERTEIG.
In reality, Kanban isn’t actually saving Agile nor is it intended to, nor is any thoughtful and responsible Kanban practitioner motivated by this agenda. What I’m really trying to convey is how human thinking about the business of professional services (including software development) has evolved since “Agile” as many of us know it was conceived around 20 or so years ago. The manifesto is the collective statement of a group of software development thought leaders that captured some of their ideas at the time about how the software industry needed to improve. Essentially, it was about the iterative and incremental delivery of high-quality software products. For 2001, this was pretty heady stuff. You could even say that it spawned a movement.
Since the publication of the manifesto in 2001, a lot of other people have had a lot of other good ideas about how the business of delivering professional services can improve. This has been well documented in well known sources too numerous to mention for the scope of this article.
Substantial contributions to the discourse have been generated by and through the LeanKanban community. The aim of Kanban is to foster environments in which knowledge workers can thrive and create innovative, valuable and viable solutions for improving the world. Kanban has three agendas: survivability (primarily but not exclusively for the business executives), service-orientation (primarily but not exclusively for managers) and sustainability (primarily but not exclusively for knowledge workers). Kanban provides pragmatic, actionable, evidence-based guidance for improving along these three agendas.
Evolutionary Theory is one of the key conceptual underpinnings of the Kanban Method, most notably the dynamic of punctuated equilibrium. Evolution is natural, perpetual and fundamental to life. Long periods of equilibrium are punctuated by relatively short periods of “transformation”—apparent total and irreversible change. An extinction event is a kind of punctuation, so too is the rapid explosion of new forms. Evolutionary theory is not only a scientifically proven body of knowledge for understanding the nature of life. It can be also applied to the way we think about ideas, methods and movements.
For example, science has more or less established that the extinction of the dinosaurs, triggered by a meteor impact and subsequent dramatic atmospheric and climate change, was in fact a key punctuation point in the evolution of birds. In other words, dinosaurs didn’t become extinct, rather they evolved into birds. That is, something along the lines of the small dinosaurs with large feathers hanging around after Armageddon learned to fly over generations in order to escape predators, find food and raise their young. Dinosaurs evolved into birds. Birds saved the dinosaurs.
There has been a lot of social media chatter and buzz lately about how Agile is dead. It is a movement that has run its course, or so the narrative goes. After all, 20 years is more or less the established pattern for the rise and fall of management fads. But too much emphasis on the rise and fall of fads can blind us to larger, broader (deeper) over-arching trends.
The agile movement historically has been about high-performing teams. More recently, market demand has lead to the profusion of “scaling” approaches and frameworks. Scaling emerged out of the reality of systemic interdependence in which most Agile teams find themselves. Most agile teams are responsible for aspects of workflows—stages of value creation—as contributors to the delivery of a service or multiple services. Agile teams capable of independently taking requests directly from and delivering directly to customers are extremely rare. For the rest, classical Agile or Scrum is not enough. The feathers just aren’t big enough. Agile teams attempting to function independently (pure Scrum) in an interdependent environment are vulnerable to the antibodies of the system, especially when such interdependencies are merely denounced as impediments to agility.
Some organizations find themselves in a state of evolutionary punctuation (the proverbial sky is falling) that can trigger rapid adaptations and the emergence of local conditions in which independent service delivery teams can thrive. Most large, established organizations seem to be more or less in a state of equilibrium. Whether real or imagined, this is what change agents have to work with. However, more often than not, the typical Agile change agent seems adamant that the sky is always falling and that everyone accepting that the sky is falling is the first step to real and meaningful change. This is not an attitude held by Agile change agents alone. This is a standard feature of traditional 20th Century change management methods, the key selling point for change management consulting.
Naturally, most self-identifying “Agilists” see themselves as change agents. Many of them find themselves in the position of change management consultants. But the motivation for change can quickly become misaligned: Change needs to happen in order for Agile to work. If you are passionate about Agile, you will seek to bring about the environmental changes that will allow for Agile to thrive. We don’t need to follow this path too far until Agile becomes an end in itself. It is understandable then that for some, Agile appears to be a dead end, or just dead.
But if there is a larger, over-arching historical process playing out, what might that be? Perhaps it has something to do with the evolution of human organization. Perhaps we are living in a period of punctuation.
Many organizations are attempting to use Agile methods. Banks, telecom companies, government agencies, and all manner of mid-size and small organizations. Most of these attempts are limited in that they think of Agile as a solution instead of as a culture. In this video, I explore some of the conditions for creating Real Agility.
This is the first video in a series of eleven that is oriented towards what managers need to know to create Real Agility in their organizations. The final two videos in the series are going to be content exclusively available to subscribers to our Real Agility Newsletter. Those final two videos are about “Dealing with Crisis” and “The Knowing-Doing Gap”. (Our newsletter also includes other great content including interviews – we are featuring an interview with Mary and Tom Poppendieck in just a few weeks!)
The team decides on how much work it will do in a Sprint. No one should bring pressure on the team to over-commit. This simply builds resentment, distrust and encourages low-quality work. That said, of course teams can be inspired by challenging overall project or product goals. A stretch goal for a Sprint is just a way to 100% guarantee failure. Even the team should not set its own stretch goals.
There are a few interesting principles that apply here. For example, the Agile Manifesto mentions sustainability:
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
The Agile Manifesto also points out the importance of trust:
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
Stretch goals are incompatible with both of these principles from the Agile Manifesto.
There are two types of stretch goals. The first type are those assigned by outsiders to the team. The second type are those which a team sets for itself. Both types are bad.
Stretch Goals Assigned by Outsiders
The worst extreme of this type of stretch goal is also the most common! This is the fixed-scope-fixed-date project deadline. In this type of stretch goal, the project team, doing Scrum or not, is forced to work backwards from the deadline to figure out how to get the work done. If the team can’t figure this out, managers often say things like “re-estimate” or “just get it done.” (Note: another thing that managers do in this situation is even worse: adding people to the project! Check out “The Mythical Man-Month” by F. Brooks for a great analysis of this problem.)
My anecdotal experience with this sort of thing is simple: quality suffers or sustainability suffers. I once worked with three other people on a mission critical project to help two banks with their merger. There was a regulatory deadline for completing the integration of the two existing systems for things like trading, etc. Fixed-scope-fixed-date. Coffee and sleepless nights were our solution since we tried not to sacrifice quality. We actually ended up working in my home for the last few 24-hour stretches so that we would have access to a shower. Suffice it to say, there’s no way we could have sustained that pace. It’s anti-Agile.
A quick search for ideas and opinions about stretch goals makes it very clear that there is no commonly agreed “correct” answer. However, from an Agile perspective stretch goals assigned by outsiders are clearly against the principles of the Agile Manifesto.
The number of items selected from the Product Backlog for the Sprint is solely up to the Development Team. Only the Development Team can assess what it can accomplish over the upcoming Sprint.
For emphasis: what it can accomplish – not what it (the Development Team) wants to accomplish, or what it should accomplish, or what it could accomplish if everything goes perfectly. A Development Team should be accomplishing their Sprint plan successfully (all Product Backlog Items done) on a regular basis. Of course, exceptional circumstances may intervene from time to time, but the team should be building trust with stakeholders. Here’s another story:
I had a good friend. We would always go out for coffee together. We just hung out – chatted about life, projects, relationships. Of course, from time-to-time one or the other of us would cancel our plans. That’s just life too. But there came a time when my friend started cancelling more often than not. There was always a good excuse: I’m sick, unexpected visitors, work emergency, whatever. After a little while of this I started to think that cancelling would be the default. I even got to the point where I was making alternative plans even if my friend and I had plans. I got to the point where I no longer trusted my friend. It didn’t matter that the excuses were always good. Trust was broken.
It doesn’t matter why a team fails to meet a goal. It reduces trust. It doesn’t matter why a team succeeds in meeting a goal. It builds trust. Even among team members. A team setting stretch goals is setting itself up for regular failure. Even if the team doesn’t share those stretch goals with outsiders.
Stretch goals destroy trust within the team.
Think about that. When a team fails to meet its own stretch goal, team members will start to look for someone to blame. People look for explanations, for stories. The team will create its own narrative about why a stretch goal was missed. If it happens over and over, that narrative will start to become doubt about the team’s own capacity either by pin-pointing an individual or in a gestalt team sense.
Trust and Agility
The importance of trust cannot be over-stated. In order for individuals to work effectively together, they must trust each other. How much trust? Well, the Agile Manifesto directly addresses trust:
Build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.
Here is my recent YouTube video about stretch goals… check it out and subscribe to our channel!
Regular big up-front planning is not necessary with Scrum. Instead, a team can just get started and use constant feedback in the Sprint Review to adjust it’s plans. Even the Product Backlog can be created after the first Sprint has started. All that is really necessary to get started is a Scrum Team, a product vision, and a decision on Sprint length. In this extreme case, the Scrum Team itself would decide what to build in its first Sprint and use the time of the Sprint to also prepare some initial Product Backlog Items. Then, the first Sprint Review would allow stakeholders to provide feedback and further develop the Product Backlog. The empirical nature of Scrum could even allow the Product Owner to emerge from the business stakeholders, rather than being assigned to the team right from the start.
Starting a Sprint without a Product Backlog is not easy, but it can be done. The team has to know at least a little about the business, and there should be some (possibly informal) project or product charter that they are aware of. The team uses this super basic information and decides on their own what to build in their first Sprint. Again, the focus should be on getting something that can be demoed (and potentially shippable). The team is likely to build some good stuff and some things that are completely wrong… but the point is to get the Inspect and Adapt cycle started as quickly as possible. Which means of course that they need to have stakeholders (customers, users) actually attend the demo at the end of the Sprint. The Product Owner may or may not even be involved in this first Sprint.
One important reason this is sometimes a good approach is the culture of “analysis paralysis” that exists in some organizations. In this situation, an organization is unable to do anything because they are so concerned about getting things right. Scrum is a framework for inspect and adapt and that can (and does) include the Product Backlog. Is it better for a team to sit idle while someone tries to do sufficient preparation? Or is it better to get started and inspect and adapt? This is actually a philosophical question (as well as a practical question). The mindset and philosophy of the Agile Manifesto and Scrum is that trying to produce valuable software is more important that documentation… that individuals and how they work together is more important than rigidly following a process or tool. I will agree that in many cases it is acceptable to do some up-front work, but it should be minimized, particularly when it is preventing people from starting to deliver value. The case of a team getting started without a product backlog is rare… but it can be a great way for a team to help an organization overcome analysis paralysis.
The Agile Manifesto is very clear: “The BEST architectures, requirements and designs emerge out of self-organizing teams.” [Emphasis added.]
Hugely memorable for me is the story that Ken Schwaber told in the CSM course that I took from him in 2003. This is a paraphrase of that story:
I [Ken Schwaber] was talking to the CIO of a large IT organization. The CIO told me that his projects last twelve to eighteen months and at the end, he doesn’t get what he needs. I told him, “Scrum can give you what you don’t need in a month.”
I experienced this myself in a profound way just a couple years into my career as an Agile coach and trainer. I was working with a department of a large technology organization. They had over one hundred people who had been working on Agile pilot projects. The department was responsible for a major product and executive management had approved a complete re-write. The product managers and Product Owners had done a lot of work to prepare a product backlog (about 400 items!) that represented all the existing functionality of the product that needed to be re-written. But, the big question, “what new technology platform do we use for the re-write?” had not yet been resolved. The small team of architects were tasked with making this decision. But they got stuck. They got stuck for three months. Finally, the director of the department, who had learned to trust my advice in other circumstances, asked me, “does Scrum have any techniques for making these kind of architectural decisions?”
I said, “yes, but you probably won’t like what Scrum recommends!”
She said, “actually, we’re pretty desperate. I’ve got over a hundred people effectively sitting idle for the last three months. What does Scrum recommend?”
“Just start. Let the teams figure out the platform as they try to implement functionality.”
She thought for a few seconds. Finally she said, “okay. Come by this Monday and help me launch our first Sprint.”
The amazing thing was that the teams didn’t lynch me when on Monday she announced that “our Agile consultant says we don’t need to know our platform in order to get started.”
The first Sprint (two weeks long) was pretty chaotic. But, with some coaching and active support of management, they actually delivered a working increment of their product. And decided on the platform to use for the rest of the two-year project.
You must trust your team.
If your organization is spending more than a few days preparing for the start of a project, it is probably suffering from this pitfall. This is the source of great waste and lost opportunity. Use Scrum to rapidly converge on the correct solutions to your business problems instead of wasting person-years of time on analysis and planning. We can help with training and coaching to give you the tools to start fast using Scrum and to fix your Scrum implementation.
How much documentation does it take to run a project with ten people working for six months? For some organizations it takes way too much:
This binder (about 3 or 4 inches thick) is all the documentation associated with such a project. In looking carefully at the project, creating the documentation took far more time than the time spent on designing, writing and testing the software. Yet, the documentation does not produce any value. Only the software produces value. The Agile Manifesto, asks us to focus on the outcome (working software) and to make tradeoffs to minimize the means (comprehensive documentation).
The Agile Manifesto asks us to challenge our assumptions about documentation. In many work environments, documentation is an attempt to address some interesting and important needs:
Knowledge sharing among stakeholders and the people working on a project.
Knowledge sharing across time as people come in and out of a project.
Verification and traceability for contracts or other compliance needs.
Decision-making and analysis for business and technical problems.
Management oversight and control.
Various aspects of individual accountability.
Documentation is usually heavier (more comprehensive) the more the following circumstances exist in an organization:
Geographical distribution of people.
Lack of trust between people, departments or organizations.
Regulated work environments.
Depth of management hierarchy.
Number of people directly and indirectly involved.
Knowledge and skill sets highly segregated between people.
Culture of respect for written texts.
Working Software
What if the software itself could address the needs that often documentation is used to address? Let’s look at them in turn:
Knowledge sharing among stakeholders and the people working on a project.
If the software is functional at all stages, as supported by Agile methods such as Scrum and Extreme Programming, then the software becomes an effective representation of the knowledge of all the people who have participated in building it.
Knowledge sharing across time as people come in and out of a project.
Software that is technically excellent is often easier to understand for people who are new to it. For example, excellence in user experience and design means new users can get up to speed on software faster. Use of good design patterns and automated testing allows new developers to understand existing software easily.
Verification and traceability for contracts or other compliance needs.
Test-driven development (code) and specification by example (scripting and code) are forms of traceable, executable documentation that easily stay in-sync with the underlying software system.
Decision-making and analysis for business and technical problems.
In particular, diagrams can help a great deal here. However, electronic tools for creating such diagrams can be slow and awkward. Consider the practice of Agile Modelling (basically using a whiteboard and taking photos) as a good alternative to precise technical diagramming if you are doing problem-solving.
Management oversight and control.
Reports and metrics drive much of the traditional documentation in an organization. Simplifying reports and metrics often leads to a clearer picture of what is going on, reduces the opportunities to “game” the system, and always results in lower levels of documentation. As well, some reports and metrics can be generated 100% through automated means. All that said, the fundamental premise in the Agile manifesto is that management should base decisions on what is actually built – the “Working software” by looking at it and using it.
Various aspects of individual accountability.
If you really need this, a good version control system can give you the information for this. Sign-offs and other types of accountability documentation are typically just waste that doesn’t actually help in process improvement. Most people who are in high-compliance environments already have licenses and/or security clearances that provide this accountability. If you software is working, however, then this isn’t even a concern as trust is built and bureaucracy can be reduced.
In my recent training programs as research for this article, I have surveyed over 100 people on one aspect of documentation – code documentation. Every individual surveyed is either currently coding or has a coding background, and every single person had the same answer to a simple scenario question:
Imagine that you have just joined a new organization and you are about to start working as a software developer. One of the existing team members comes up to you and introduces himself. He has with him a piece of paper with a complicated-looking diagram and a full binder that looks to be holding about 250 pages. He asks you, “you need to get up to speed quickly on our existing system – we’re starting you coding tomorrow – would you prefer to go over the architecture diagram with me or would you prefer to review the detailed specifications and design documents.” He indicates the one-page diagram and the binder respectively. Which would you prefer?
(I’ve put up a Survey Monkey one-question survey: Code Documentation Preference to extend the reach of this question. It should take you all of 60 seconds to do it. I’ll post results when I write the next Agile Manifesto essay in a month or two.)
The fact that everyone answers the same way is interesting. What is even more interesting to me is that if you think through this scenario, it is actually almost the worst-case scenario where you might want documentation for your developers. That means that in “better” cases where documentation for developers may not be as urgent or necessary, then the approach of just going to talk with someone is a lot better.
Documentation and Maps
The problem with documentation is the same problem we have with maps: “the map is not the territory” (quote from the wisdom of my father, Garry Berteig). We sometimes forget this simple idea. When we look at, say, Google Maps, we always have in the back of our consciousness that the map is just a guide and it is not a guarantee. We know that if we arrive at a place, we will see the richness of the real world, not the simplified lines and colours of a map. We don’t consider maps as legally binding contracts (usually). We use maps to orient ourselves… as we look around at our reality. We can share directions using maps, but we don’t share purpose or problems with maps. And finally, maps assume that physical reality is changing relatively slowly (even Google Maps).
Many times when we create documentation in organizations, however, we get confused about the map versus the territory.
Agility and Documentation
Of course, code is a funny thing: all code is documentation too. The code is not the behaviour. But in software, code (e.g. Java, ASM, Scheme, Prolog, Python, etc.) is as close as possible to the perfect map. Software is (mostly) deterministic. Software (mostly) doesn’t change itself. Software (mostly) runs in a state absent from in-place human changes to that software. Software (mostly) runs on a system (virtual or physical) that has stable characteristics. The code we write is a map. From this perspective, documentation becomes even less important if we have people that already understand the language(s)/platform(s) deeply.
May people have concerns about the possibility of using Scrum or other Agile methods on large projects that don’t directly involve software development. Data warehousing projects are commonly brought up as examples where, just maybe, Scrum wouldn’t work.
I have worked as a coach on a couple of such projects. Here is a brief description of how it worked (both the good and the bad) on one such project:
The project was a data warehouse migration from Oracle to Teradata. The organization had about 30 people allocated to the project. Before adopting Scrum, they had done a bunch of up-front analysis work. This analysis work resulted in a dependency map among approximately 25,000 tables, views and ETL scripts. The dependency map was stored in an MS Access DB (!). When I arrived as the coach, there was an expectation that the work would be done according to dependencies and that the “team” would just follow that sequence.
I learned about this all in the first week as I was doing boot-camp style training on Scrum and Agile with the team and helping them to prepare for their first Sprint.
I decided to challenge the assumption about working based on dependencies. I spoke with the Product Owner about the possible ways to order the work based on value. We spoke about a few factors including:
retiring Oracle data warehouse licenses / servers,
retiring disk space / hardware,
and saving CPU time with new hardware
The Product Owner started to work on getting metrics for these three factors. He was able to find that the data was available through some instrumentation that could be implemented quickly so we did this. It took about a week to get initial data from the instrumentation.
In the meantime, the Scrum teams (4 of them) started their Sprints working on the basis of the dependency analysis. I “fought” with them to address the technical challenges of allowing the Product Owner to work on the migration in order based more on value – to break the dependencies with a technical solution. We discussed the underlying technologies for the ETL which included bash scripts, AbInitio and a few other technologies. We also worked on problems related to deploying every Sprint including getting approval from the organization’s architectural review board on a Sprint-by-Sprint basis. We also had the teams moved a few times until an ideal team workspace was found.
After the Product Owner found the data, we sorted (ordered) the MS Access DB by business value. This involved a fairly simple calculation based primarily on disk space and CPU time associated with each item in the DB. This database of 25000 items became the Product Backlog. I started to insist to the teams that they work based on this order, but there was extreme resistance from the technical leads. This led to a few weeks of arguing around whiteboards about the underlying data warehouse ETL technology. Fundamentally, I wanted to the teams to treat the data warehouse tables as the PBIs and have both Oracle and Teradata running simultaneously (in production) with updates every Sprint for migrating data between the two platforms. The Technical team kept insisting this was impossible. I didn’t believe them. Frankly, I rarely believe a technical team when they claim “technical dependencies” as a reason for doing things in a particular order.
Finally, after a total of 4 Sprints of 3 weeks each, we finally had a breakthrough. In a one-on-one meeting, the most senior tech lead admitted to me that what I was arguing was actually possible, but that the technical people didn’t want to do it that way because it would require them to touch many of the ETL scripts multiple times – they wanted to avoid re-work. I was (internally) furious due to the wasted time, but I controlled my feelings and asked if it would be okay if I brought the Product Owner into the discussion. The tech lead allowed it and we had the conversation again with the PO present. The tech lead admitted that breaking the dependencies was possible and explained how it could lead to the teams touching ETL scripts more than once. The PO basically said: “awesome! Next Sprint we’re doing tables ordered by business value.”
A couple Sprints later, the first of 5 Oracle licenses was retired, and the 2-year $20M project was a success, with nearly every Sprint going into production and with Oracle and Teradata running simultaneously until the last Oracle license was retired. Although I don’t remember the financial details anymore, the savings were huge due to the early delivery of value. The apprentice coach there went on to become a well-known coach at this organization and still is a huge Agile advocate 10 years later!
Last night I had the honour of giving a talk at the PMI-SWOC. It seemed well received and I really enjoyed the opportunity. The slides from the talk are attached to this post.
There were quite a few people in attendance who were new to Agile and I spent a bit of time talking about the Agile Framework before really getting into the slides of my talk.
On Tuesday Dec. 2, Mishkin Berteig will be speaking about The Agile Enterprise and the five different approaches to implementing Agile at the enterprise level. The talk will also include some details about two frameworks used at the enterprise level: SAFe (Scaled Agile Framework) and RAP (Real Agility Program).
In reality, Kanban isn’t actually saving Agile nor is it intended to, nor is any thoughtful and responsible Kanban practitioner motivated by this agenda. What I’m really trying to convey is how human thinking about the business of professional services (including software development) has evolved since “Agile” as many of us know it was conceived around 20 or so years ago. The manifesto is the collective statement of a group of software development thought leaders that captured some of their ideas at the time about how the software industry needed to improve. Essentially, it was about the iterative and incremental delivery of high-quality software products. For 2001, this was pretty heady stuff. You could even say that it spawned a movement.
