From push system to pull system thinking
One of the disconnects holding teams back the most in an organization embarking on an Agile transformation is the lack of will and perhaps understanding of vision on the part of the business. The required shift in thinking is from a “push system” to a “pull system”. Historically and still culturally, most organizations, even those claiming to be ‘Agile’ are very much push systems. The business folks in client services – VPs, Directors, sales people, etc. seem to make time (deadline) commitments to clients on behalf of teams and then the teams are given the deadlines to finish the work. Sometimes, the deadlines are decided on in consultation with particular individuals on the teams and very rarely, if ever, with the actual teams themselves. In any case, the fact that the business is almost entirely deadline-driven is the centre of the push system. Deadlines push or drive everything else. Deadlines are fixed and often considered non-negotiable. Deadlines are a taboo subject – it is considered a waste of time to even discuss them because they just don’t and won’t ever move. The general attitude is that if we try to move deadlines, we put the entire business at risk because our clients will drop us and turn to one of our competitors who claim to be able to promise and keep deadlines. If we lose our clients, we lose business, we lose money and it potentially puts us all out of jobs. What this exposes is not only a push system driven by deadlines, but a culture that is actually driven by fear. The not-so-implicit message is that if you miss a deadline, you might lose your job, so you had better do whatever it takes to not miss the deadline. Or else. Push and pull systems and mentalities are like oil and water – they don’t mix. In Agile, there is no place for fear of failure. Rather, teams must be allowed to fail (miss deadlines) and learn from their failures (plan better).
Why quality, not time/cost or scope is non-negotiable
The “make the deadline or else message” is couched and clouded by other talk. The main excuse is to blame the client, as noted above. “That’s just the way our clients work, the way the market works”. Of course, such an excuse contains a kernel of truth. Without a true understanding and embrace of Agile, the idea of not meeting deadlines and the perceived consequences can be truly scary. Generally, there is an understanding from the business that the productivity of teams may drop somewhat as they progress through the storming stage. What this translates into is a difficult discussion with clients around delayed delivery. It is tolerable in that it is temporary. “Once the teams get up and running, we can go back to meeting our deadlines, and even be able to deliver early because Agile is supposed to be faster.” But the benefits of truly adopting Agile are much more powerful than this.
Understanding the true business value of Agile
What needs to be understood is the true business value for investing in Agile processes and practices – how it may add cost and time to the initial development Cycle, but how it saves both the business and the client tremendously on technical debt and support long-term. This needs to be understood and championed by the business in order for the organization to become liberated from its enslavement to the push system mentality. At the heart of such a mental liberation is the wholehearted adoption and commitment to the Agile/Lean principle that quality is non-negotiable. The investment in Agile processes and practices is essentially an investment not only in quality, but in continuous quality improvements towards the goal of being able to frequently deliver products of increasing relevance and quality (value). The ability to ship frequently allows for sustainable growth. All of this is made impossible by the deadline-driven push system mentality/culture of fear.
The urgent need for slack
One of the first things that a team needs in order to focus on continuous quality improvements is slack so that it can learn to learn. The first goal of the business leadership should be to facilitate scope and deadline slack for the team. This goal should also be fully and visibly championed by the business leadership. In order to develop the capability to facilitate slack, the business needs to gain knowledge around the purpose and importance of Agile processes and practices and be able to articulate a strong business case for them. The business leadership needs to develop the skill of educating the team, management and business leadership on the long-term benefits of an Agile transformation – the transformation from a push system to a pull system. The key stakeholders and business leaderships need to possess the courage to engage in difficult conversations with management and clients who may be upset by the short-term pain of delays and missed deadlines and protect the team from continued attempts to push work into the team. Perhaps above all, the business leadership needs to develop an attitude of learning – a humble learning posture that allows for the setting aside of preconceived notions, fears and prejudices around what it means to be a good business leader. A leader possessed of this posture demonstrates a learning attitude by stressing first and foremost the importance of creating slack for the team to learn to learn. It is a common pitfall for inexperienced business leaderships and stakeholders to expect Agile to provide solutions for their push system woes, woes that include the broken trust of clients from consistently broken (unrealistic, dreamt-up) deadline promises and the crippling effects of technical debt (the fallout of the former – when scope, time and cost are fixed, quality is compromised).
If the business leadership, with the support of the Process Facilitators and the Transformation Team, is able to foster the organizational will to create slack for the teams, then the teams will have the space they need to truly focus on continuous quality improvements. This is a critical milestone on the path to realizing the true, measurable benefits of Agile. Although the support of others is needed, the business leadership is in a unique position benefitting from an intimate relationship with both the needs of the business as well as the daily life of the team.
Why the business leadership needs to own the process
The first way that the business leadership creates slack for the teams is by championing the process. In OpenAgile, like all other Agile methodologies, there are key features of the process the purpose of which are to give space for new teams to begin to make the often seemingly inconsequential, yet ultimately critical first steps towards continuous quality improvements. One of the most obvious of these features is the Agile team meetings. In the early stages of team development, organizational understanding and will, the OpenAgile meetings (particularly the Reflection and Learning aspects of the Engagement Meetings in OpenAgile) can easily be discounted as an obstacle preventing the team from getting the “real work” done. What is often forgotten under the pressure of deadlines is the fact that in order for a team to be able to learn to make continuous quality improvements, it needs to develop the capability of systematic (frequent & regular) inspection and adaptation of the way that it works. It is easy to save on the short term pain of perceived non-negotiable deadlines (meeting deadlines at all cost = success) by compromising on investing in the process, especially when the team is still learning to learn and the effectiveness of the meetings is not yet apparent. When the team and the organization have an expectation of Agile as something that fits into the push system and allows for a team to function better within such a system, it can be hard to understand how spending time in a kind of meeting that the team doesn’t seem good at yet can be of any value. This is where the business leadership needs to stand firmly behind the process. The team needs the meetings – the space to reflect, learn and plan – in order to learn to become more effective at making continuous quality improvements – a critical feature of an effective pull system. Without the meetings, the team will never develop this critical capability and as a result, will never become an Agile team. Instead, the team will revert back to getting the “real work” done with all of the quality problems crippling the organization and which led to the decision to adopt an Agile framework in the first place.
Why the business leadership should care about burn-down
Another key feature of the process for the business leadership to understand and champion is the concept of burn-down as represented by the burn-down chart of an Agile team. Agile doesn’t care about how much work the team gets done. It assumes that the team is doing valuable work and getting things done – in other words, that the team is managing itself and working towards its goals and commitments. There are no tools in Agile for an individual, least of all the business leadership, to measure and manage how much work the team is getting done. Agile acknowledges the reality that real (sustainable) productivity cannot be forced on any team. Instead, a team grows its productivity at a sustainable pace when it is given enough slack to do so. The team makes a plan at the beginning of the Cycle based on what it understands about its capacity, works towards that goal throughout the Cycle and hopefully delivers valuable results at the end of the Cycle. By learning to apply the process of continuous improvement, quality and productivity go up hand in hand. That is the essence of the pull model. Through all of this, the team manages “how” it gets work done. The productivity of a team can be measured, but the burn-down chart is neither an appropriate nor effective tool for measuring the productivity of a team. Instead, burn-down provides one specific measurement and ONLY this one measurement: WORK REMAINING (in order to achieve the goal/commitment of the current Cycle). It does not and cannot tell you how much the team got done and even less so the whys and hows of the output and productivity of the team during the Cycle.
So what is the purpose of burn-down and why should the business leadership even care? If it can’t be used as a tool to measure the productivity of the team (in other words, if it can’t be used to manage the team) then what importance can it possibly have? These are typical questions of teams and individuals that are coming from a traditional project management, i.e. command & control, i.e. “push” system mentality. Understanding the purpose of burn-down depends on the ability to make the shift from the push system paradigm to the pull system paradigm. In a push system, burn-down is nice but somewhat irrelevant. For an organization committed to an Agile transformation (towards a pull system of self-managed teams) it is an invaluable launch pad for powerful conversations that live at the heart of continuous quality improvements.
Commitment to the business requirements come from the Agile teams
When a team decides on a plan for a Cycle of work, the plan is a commitment. This is a critical step in the Agile process. It is only after a unanimous commitment from the whole team that the team begins to work on the plan. If any individual team member feels hesitant about the work in the plan, tasks and potentially even Value Drivers should be removed until everyone is comfortable making a commitment. When the business leadership is telling a team what the plan is, then it is not the team’s plan and therefore it cannot be a team commitment. This is not only an inappropriate use of authority, it is also breaking the Agile process. Moreover, when a plan and therefore a commitment is forced onto a team, the team cannot be held accountable for failure. Worse yet, the team will never learn to plan. If a team is not able to plan, then it is not able to make commitments. If the team is overwhelmed by an overly-ambitious, management-forced plan, it will not learn to evaluate its capacity and apply that knowledge to long-term planning and project estimates. It will not learn to make meaningful quality improvements and reflect on its progress. It will not learn to self-manage or self-organize. The purpose of burn-down is to establish commitment velocity. In other words, the amount of work that the team can truthfully expect to complete during the Cycle when it is making the Plan. The difference between the number of tasks in the Cycle Plan and the number of tasks remaining at the end of the Cycle gives the team its commitment velocity. Commitment velocity is always based on minimum historical velocity. The team uses commitment velocity to make a Cycle Plan containing no more than the number of tasks represented by its commitment velocity. This allows the team to spend just the right amount of effort and time on planning and allows the team slack to focus on the truly Agile work of learning and continuous quality and process improvements. Over-planning, especially when it is wedded to over-committing or even worse, non-committing (a common push system mentality pitfall forced onto teams by the business leadership) leaves the team in a state of dependent on daily micro-management and can completely halt the progress of a team. Not to mention that this is a flagrant violation of Agile values (truthfulness, responding to change over following a plan) and principles (sustainable development). Such compromises to foundational Agile values, principles and processes may produce desired results in the very short-term, but the long-term costs can be crippling to teams and organizations. The wasteful activity associated with team dependency on micro-management is what often leads organizations to the accumulation of technical debt that places them in dire competitive disadvantage and desperate need for Agile transformation in the first place. If an organization misses out on this golden opportunity, teams can become demoralized and innocuous to the Agile practices and the promise of an Agile transformation can quickly erode.
Affiliated Promotions:
Please share!
Hi Travis,
Thank you for writing this. I have a few questions for you about Agile, based on some of the things you’ve written here. And some observations based on my own experiences. I would greatly appreciate your feedback.
Firstly, I understand you’re advocating for Agile because you believe in it. I also understand that you see it as having many positive benefits. However, your article doesn’t seem to leave much room for other approaches. Is there room in your beliefs about development approach for non-micromanaged, traditional waterfall or hybrid approaches?
One of the challenges in moving from push to pull (in my expereince) is demanding that one method or approach is the silver bullet. For instance seeing Agile and the pull development, “environment” as the best and only way to execute successfully in the long term. For instance, our management team sets, “challenge” dilivery dates for very good reasons; they have to meet real world business needs. For example, releasing software in sync with physical product. This is a real world dependancy which your article recognizes but dissmisses as secondary to using the Agile approach. Perhaps you have some clarifications you can offer on this?
To be more direct, I’d say that people need to own the process not the other way around. Perhaps we’d agree in principle on that point. In my experience as both a software developer and manager of development teams I agree in theory with the statement, “Agile acknowledges the reality that real (sustainable) productivity cannot be forced on any team.”, however, the reality is that a programmer, manager, BA and PM all have jobs to do. We measure the effectiveness of individuals, and by extension, the companies they work for through performance. Measuring that performance and knowing where the project is at in the lifecycle is important to business, in any construct. Losing site of this fact is not realistic, even if it’s rebranded as, “Agile”.
People need to deliver in their jobs regardless of what they do. It’s what we get paid for. I try to use a hybrid approach that recognizes Mangements business realities because ultimately their realities are our realities. We trust our developers to do a good job because we enable them by involving them. For example our developers give estimates on work rather then setting a deadline based on our own expectations. We course correct during the life of the project and we do our very best to protect the intgerity of quality while keeping our commitments to delivery.
You’re statement, “…in other words, that the team is managing itself and working towards its goals and commitments.”, fails to recognize the important role that effective leadership (on every level) can play in helping to enable the teams, remove opbstacles and solve problems. Excellent leadership which is committed to buidling great software enables its people, but also recognizes its serious role and responsibility in meeting the obligations of the company. In other words, understanding the importance your role as a leader has on the livelihoods of others.
From my experience, a truly agile approach is one that recognizes there are many paths to success and really effective people know how to adapt for the specific challenge. In setting goals and challenges for our developers which recognize the critical importance of quality and the realities of delivery we accept and embrace the challenge of continuous improvement. We don’t create an environment of fear to acheive this, we enable and expect the best from our teams. We are as committed to quality as we are speed, we just don’t see them as mutually exclusive.
I hope you’ll appreciate that mine is just one viewpoint and in no way is intended to be critical of your ideas about Agile. I’m still learning and just trying to reconcile my own experiences with some of the ideas you’ve written about assoicated with Agile.
Thanks much.
Martin, I very much appreciate the significant effort that you put into your response to my piece. Thank you for the questions you have asked and the openness with which you have shared your own views.
Indeed, I am advocating Agile because I believe in it and have experienced its benefits. If my piece seems to not leave much room for other approaches, it may be because what I am actually trying to emphasize are the fundamental differences between Agile and traditional Waterfall approaches. Making the shift from a push system to a pull system is difficult and it is also, for many organizations, worth it. And the closer these organizations get to Agile, the better their results.
Much of the focus of my work has been with organizations that want to fully adopt Agile approaches and derive the benefits of an Agile transformation. Partial adoption can also be a good thing. Invariably, however, partial adoption at best yields partial benefits.
Many organizations say that they want the full benefits of Agile and at the same time the leadership not willing or able to make the fundamental shift that is required. This shift often requires a degree of leadership will and courage that I have rarely seen. It also almost always requires a lot of external help, at least initially. What I and many other Agile consultants I know have found is that most of the obstacles for full transformation are cultural and are enforced by management. In order for an Agile transformation to succeed, fundamental changes need to occur at all levels of an organization.
Part of what I am addressing in my piece are some of the challenges I’ve seen faced by business leadership in organizations that are trying to transform their culture and become an Agile organization. Often in such organizations different parts of the business start out with conflicting priorities which are often highly political and involve vested interests like bonuses, career ambition, influence and the like. Often these factors become drivers that push work into the teams. Clear priorities, goals, quality and performance metrics are set aside for quick fixes and the ability to “yes” to every unreasonable customer demand. This in turn creates a culture of individual heroes, excessive overtime, low staff morale, high turn-over, distrust and communication breakdown between the business and technical folks, disengaged staff, lack of ownership and poor quality. Over time, these conditions can cripple an organization. This is the disaster that Waterfall has allowed and Agile for which has been created to overcome.
Agile requires that all business opportunities are made visible to all stakeholders from which an ordered list is created for the teams to deliver on. It is from this set of deliverables that the team pulls according to its own capacity to deliver work. When new work comes in and jumps the queue, it is visible to all stakeholders, who can then have a conversation to ensure that indeed the new ordering is in the best interest of the whole organization.
My intention was not to offer Agile as a silver bullet. Indeed, I do not believe in magic. However, as I have mentioned above, there are certain fundamental aspects of Agile that when fully implemented make possible the realization of the potential benefits of high-performance teams. One of these fundamentals is the imperative of self-organizing teams that decide on their own how much product they will deliver in a single iteration (one month or less). For teams to do this properly requires a lot of hard work, including the very difficult work for some managers to overcome the need to maintain an illusion of control. There are often many other mental barriers that many people need to overcome in order to make this work. It takes a long time, and lot of patience and courage. I often like to quote Ken Schwaber who said that Agile is a test of character for an organization. It is indeed a test for everyone involved!
People come to own a process by recognizing it’s value and voluntarily contributing to its success, without fear of being punished for failure. This is something Daniel Pink describes well in his book “Drive”. The carrot and stick approach of traditional management lulls people into doing exactly what they are told. It rewards heroes and punishes mistakes, which is essentially an inhumane abuse of power that dates back far into our primitive past. It demoralizes people and kills creativity. What Agile offers is a way for people to work together in a creative and collaborative team environment to find better and better ways to do work and deliver value to society. It is not only a revolutionary methodology, it is a social movement.
Losing site of measuring performance is not part of Agile. Rather, measuring team performance is very important. Moreover, it is important for teams to learn how to measure their own performance. Their are many ways of doing this. The “Wisdom of Teams” by Katsenbach and Smith serves as a good introduction to the world of high-performance teams. In Agile, individual performance is taken care of within the team and with the help of management at the request of the team when necessary. Teams deliver value to the organization and measure their own performance accordingly. Teams give estimates on work. Teams course correct during the life of the project. Teams protect the integrity of quality while keeping commitments to delivery. The main difference I see here is the aspect of self-organizing teams.
I don’t see how a team managing itself and working towards its goals and commitments in any way fails to recognize the important role that effective leadership can play in helping to enable teams, remove obstacles and solve problems. On the contrary, what I have actually experienced is that by allowing teams to manage themselves (and pull their work), leadership actually becomes capable of leading because they are less occupied with trying to control; this frees up a huge amount of leadership capacity for mentoring, coaching, guiding and empowering. Agile has allowed the leaders I have worked with to better understand the importance of their role as leaders and has liberated them to better fulfil that role.
You have recognized that there are many paths to success and that really effective people know how to adapt for specific challenges; that you have recognized the critical importance of quality and continuous improvement; that you enable and expect the best from your people and that you recognize speed and quality as being mutually inclusive; all of these are highly commendable values for any leader of any organization. If there are any statements in my piece that give the impression that I hold or espouse contrary views (or that I claim to be representing Agile in such a way) I hope that those discrepancies can be overlooked. Certainly if what you are doing is working for you, please do not change it!
At the same time, there are certain methods and approaches that have, over time, proven to be more effective than others. In general, self-organizing Agile teams of highly effective people adapt much better to specific challenges than managed groups of individuals; they also tend to produce much higher quality product that improves more regularly and more often; they are much more enabled to perform at their best and improve their performance over time; and the false dichotomy of speed versus quality is quickly overcome by Agile teams, (in the cases of technical teams, especially those that develop strong Agile engineering practices).
High-performance Agile teams—in other words teams that set their own high-performance goals, measure their own performance, pull work and allow individual team members to volunteer for tasks, teams that collaborate and care about one another—are able to realize far greater levels of performance than groups of workers who are assigned work and measured on their performance as individuals by their managers. And by “far greater”, I mean organizations that invest in full Agile transformation should expect to see a 100% productivity increase in the first 3-6 months and a 400% productivity increase within 1-2 years.
This sort of transformation is not easy. Few companies have achieved it. But when I refer to Agile, this is what I’m talking about.
All that said, I very much appreciate your comments and I would be very happy to continue this conversation in this thread or otherwise.
If you are interested in a more direct conversation, please feel free to contact me by email.
travis@berteigconsulting.com