« April 2005 | Main | June 2005 »
May 31, 2005
Scrum and its Relationship to Other Management Techniques
(On relationships of Scrum,TOC, BPR, Systems’ Thinking, Knowledge Management, TQM, Complexity, Manufacturing, New Product Development, etc.)
Although there are great synergies between TOC and Scrum, there are also vast differences. Both provide patterns to resolve systemic problems akin to those described in Systems’ Thinking by Senge, but TOC resolves systemic problems that are more akin to process improvement ala TQM, rather than complete overhauls as in BPR, which I argue, is closer to what Scrum does – it reengineers the process from scratch, by implementing high performance patterns altogether.
For the most part, TOC, and The Goal, assume a somewhat repeatable business process, where the process of removing constraints, exploiting constraints, finding new ones, etc.; is an ongoing process in a somewhat stable, repeatable process.
Scrum, on the other hand, generalizes this technique in the absence of a repeatable process, it simply removes *any* constraints constantly, with no assurance that the constraints removed will remain so the next day, and that the particular constraint optimized
“has been removed”, and that we can move on to new ones at that point. It is usually the case, however.
Another important difference is that a Scrum team is already an organizational structure that avoids process anti-patterns like handoffs, iteration and re-work (among different organizational structures, between an “analysis” team and a “design team” for example, or between an organizational structure building component A, communicating with another one building component B, in a Conway’s Law configuration i.e. where Organization Follows Architecture.
In that sense, a Scrum team is a true “Case Team”, as termed in Michael Hammer’s BPR, i.e. an organization structure that is *completely responsible* for the success of a process or project, where the ScrumMaster is the “Case Manager* ala Hammer, that responds to the Product Owner.
From the Knowledge Management perspective, TOC optimizes the process and therefore, more knowledge of the processes themselves are more valued, while in the Scrum (Case Team) perspective, the individual contributor’s knowledge is more valued.
On the other, and to make things slightly more complicated, Scrum sits atop a number of processes, that as more applications of the same kind are built, tend to be more repeatable, like configuration management, testing of certain components, vertical slice development or new applications on top of reusable architectural services.
The interesting thing to note, is that in Scrum, smaller reusable processes develop underneath as a base of knowledge but they don’t drive development, while the Scrum process, acts as a “work generator/work management” “process envelope” (a superprocess, or meta-process, if you prefer), that drives the work. (In Complexity terms, it is the Maxwell Demon that provides the autocatalytic cycles to drive self-organization, ala Kaufman’s “Origins of Order”.
Curiously, manufacturing is turning more and more to the original BPR patterns to build things. However, New Product Development always requires one meta-level more of work organization to handle the variability of “building something new every time”. In software development this difference evolves into a more continuum spectrum. The first application in a domain is New Product development, but as new ones get added, only a percentage is New Product development, the other percentage is more like manufacturing, and therefore more repeatable.
But don’t worry – no one needs to understand all the above gibberish for Scrum to work. On the other hand, it is nice to understand these things to properly understand Scrum in the context of other management techniques,
- Mike Beedle
Posted by Mike Beedle at 02:54 PM | |
Doctor, it hurts when I do this...
Patient: Doctor, it hurts when I do this...
Doctor: Then stop doing it...
A wonderful definition of insanity is "doing the same thing repeatedly, and expecting different results." Yet, for some strange reason, we persist in using methods that are not working. On several projects at a past employer, I was hearing reports of our corporate-branded custom methodology resulting in late delivery, incorrect delivery, and reduced features, etc. The argument given was always "this extraneous factor happened," or "the customer kept changing their minds," or "the customer wasn't implementing the methodology properly."
What was the solution? Why, to do the same thing again, only harder! This I hear from many of my colleagues quite frequently. When all is lost, and the methodology is failing, cling more heavily to its rules and structures. Now sometimes this is valid. If, in fact, the methodology is being poorly implemented, and if the methodology is supportive of the environment and culture and circumstance of the project, then by all means, tighten up the implementation. Sadly, however, seldom is a proper analysis done of the fitness of the methodology to the needs of the project.
One of the very nice things about Scrum, for example, is that it is a short-cycle iterative feedback system. It is not a large methodology with lots of process. In a sense, it is a process for uncovering the work that needs doing, and for structuring that work in a highly compartmentalized way. Because of this, it is often quite resilient to external factors. Also, Scrum assumes that outward conditions will change, and assumes further that many of these changes are entirely out of the project's control. Therefore Scrum is organized to find these externals irrelevant to its measures of success. It's classic lateral thinking.
Why mention the above? Because the most common failure of a methodology is its inability to handle fundamental change. It requires a certain number of assumptions. If these assumptions change, then the whole project needs to be re-conceived. If you have a project lifecycle that lasts about 2 years, this is a very expensive re-conception. In this context, my above paraphrase could be re-stated to: "Insanity is doing the same thing, in a different context, and expecting the same results."
With Scrum and other empirical processes, you re-formulate the project on a cyclical basis (say, a month). Thus, all new information can be assumed for the next cycle safely, and everyone is secure in the knowledge that all things can be re-examined next cycle. A problem is turned into a strength.
I'm not saying that Scrum can't be misapplied, or that people can't get into trouble there... but the fundamentals of Scrum and empirical processes are such that, if reasonably applied, you shouldn't need to bang your head into the wall month after month. After all, in the end, if it's that bad, it's much cheaper to cancel a scrum project than a traditional plan-based project, because you will tend to know sooner that it needs to be cancelled.
Posted by Christian Gruber at 01:18 PM | |
May 30, 2005
Agile Household Management - Update
My wife and I have been doing weekly iterations to deal with our household management stuff. As we go along, we are definitely getting through the high-priority items on our household backlog. Along the way, we have encountered a couple of glitches.
1. We missed doing our weekly planning meeting one week. Basically, it was a really busy weekend with way too much stuff going on, both expected and unexpected. We simply failed to remember to do our planning meeting until it was too late to fit it in.
2. Most weeks we miss a number of items that we have selected to complete that week. It can be anywhere from 10% to 70% of the total number of tasks. Usually we get the "larger" tasks done and it is just smaller stuff that we miss.
3. I don't have nearly the same visibility into the work as my wife does. I travel and am away from home four days a week. As a result, I don't get to see the state of the house or participate in daily planning except for the three days that I am home.
So, we've discussed these things and decided that one thing that will help is to create an information radiator. We will be putting up our weekly selection of tasks on yellow sticky notes on a prominent wall in the hallway between our bedrooms and the rest of the house. The location is a compromise between visibility and privacy. We don't want the tasks to be in a public portion of the house.
We hope that having the tasks up will allow me to be a little more conscious of the work that needs to be done. As well, it will be a frequent reminder of what is left to accomplish. I hope that this will be a fairly easy and valuable addition to our agile household management process.
Posted by Mishkin Berteig at 08:33 PM | |
May 28, 2005
Interesting Article by Scott Berkun
Why smart people defend bad ideas by Scott Berkun looks at a fascinating, and fairly common, problem. Interestingly, this problem of smart people doing dumb things / definding bad ideas, is often related to perception. Agile principles and practices are about aligning perception among all the stakeholders, and in particular, between an execution team and the rest of the stakeholders. By aligning perception, the best environment is created for doing the right thing. In the case of a business, your project team are the experts in "how" to do stuff, and your business team are the experts in "why" to do stuff. This combination of "how" and "why" is done as efficiently as possible using agile work methods.
Posted by Mishkin Berteig at 10:49 PM | |
People are Creators - Effectiveness
Our effectiveness as creators depends greatly on our knowledge and our skills. Less obviously, our effectiveness as creators also depends on our emotional and mental state, and even what some would call our spiritual state. Finally, our ability to create can be greatly magnified or greatly reduced by our environment, particularly the other people who are around us. Which is interesting, because our creative nature affects our environment.
Posted by Mishkin Berteig at 08:54 AM | |
May 27, 2005
Reality is Perceived
Our senses and the devices we create to extend our senses, are filters through which we perceive reality. We don't get to sense all of reality. Then, our minds take in the streams of information from our senses, and filter them further. Our attention or focus, our preconceptions, our mood, all determine the way we filter information from our senses. If we are feeling guilty, we may be more likely to hear other peoples' conversation as criticism. If we are feeling proud, we may be more likely to hear those same conversations as praise.
If our perceptions are wrong then no amount of logical excellence will give the right answer. So it is a pity that almost the whole of our traditional intellectual effort has been directed at logic and so little at perception.
Logic will not change emotions and feelings. Perception will.” (Edward de Bono, Textbook of Wisdom, 1996)
Posted by Mishkin Berteig at 10:33 AM | |
May 26, 2005
About People, Tools and Processes
Experienced, smart individuals who work together effectively will always perform better than junior untalented people thrown together at random. The experienced effective group will build its own tools and create its own processes. The random junior group will be unable to effectively utilize tools given to them, nor will they be able to effectively follow a process.
When a team needs improvement, don't impose a process or throw tools at them. Instead, concentrate on improving the team and the individuals within it. Technical, personal and team training and coaching will always be time and money well-spent. Spending money on processes and tools before an excellent team is in place can be very risky and wasteful.
Individualism and competition have no place in an agile work environment. Instead, the agile environment supports and fosters teamwork, collaboration and consultation. In turn, teamwork, collaboration and consultation depend on trust and truthfulness. “Truthfulness is the foundation of all human virtues.”
Nevertheless, processes and tools still have some importance. Great people with a great flexible process and great flexible tools will be hyper-productive. A junior group may need training on tools that will help them be more productive. Just be sure to never let processes and tools get in the way of the team.
In many ways, improving people is a sufficient practice for agile work. All the other principles, disciplines and practices would eventually arise out of this one practice. However, the depth of individual and group improvement needed for this practice to stand on its own is very great. Therefore we make the other principles, disciplines and practices explicit.
Posted by Mishkin Berteig at 09:57 AM | |
May 25, 2005
Agile Corporate Culture Change
In "The Corporate Culture Survival Guide", Edgar H. Schein describes a method for implementing culture change in corporations. He says:
As the change team works on the ideal state and the present state [of the corporate culture], it probably has to periodically redefine the change problem in terms of the gap or gaps identified. In other words, though the process is a set of steps undertaken in sequence, there are many feedback loops that force going back to earlier steps to guarantee clear thinking. . . .
As various gaps are identified in concrete form, it becomes apparent where cultural assumptions aid or hinder the change agenda. For example, having sales teams work together on big accounts may sound simple until it is discovered that the organization's individualistic culture prevents changing the incentive system to a group-based compensation program. The change program then has to shift to examining how to change some of the individualistic assumptions; this might entail an entirely new change program not previously thought about at all.
In other words, expect and embrace changes in understanding brought about by learning from work performed. Deliver corporate culture change work iteratively. Plan corporate change adaptively.
Posted by Mishkin Berteig at 03:46 PM | |
May 24, 2005
People are Creators
People are creators: we willfully change our environment, we imagine new things and through our actions, manifest those things in the world. All people do this. Human beings enjoy having an effect on their environment and seeing the results of that effect. We all have this ability. We all derive satisfaction from the exercise of this ability. And we can all learn to improve this ability.
In any endeavor we take on, we are attempting to create something new. A new system, a new object, a new pattern of behavior, or a new way of thinking. We are attempting to create change. This is one of three fundamental axioms of Agile Work.
Posted by Mishkin Berteig at 08:12 PM | |
May 20, 2005
Management and Business as Sources of Waste
Waste can be of several different types. Independent of what the types of wastes are, they can also come from different sources. When attempting agile work, it is essential to identify the underlying causes or sources of waste. Once these sources are identified, efforts can be made to change them so that they cause less waste.
Management is least effective and a source of waste in an agile work environment when it imposes decisions, plans work, and directs people. Management is most effective in an agile work environment as a facilitator, shield and servant.
Business can cause waste by not being focused or by being focused on un-profitable efforts. Paradoxically, business can also cause waste by being too focused on profitable efforts. A balance of investment in short-term, long-term and "cleanup" efforts is ideal for an agile work environment.
Posted by Mishkin Berteig at 07:55 PM | |
May 19, 2005
Test-Driven Work
In an agile environment, all work done needs to be directly related to the needs of stakeholders. Stakeholders request or “pull” work from the team, and they do this by defining prioritized work packages. The team needs some way to know when they have completed a work package, so both work packages and iteration tasks need to have testable acceptance or success criteria. The team collaborates with the stakeholders to determine what needs to be done to successfully complete a work package.
Based on the acceptance criteria, tests are described or created. Ideally, these tests are created before or simultaneously to any work that is done on the work package or task. Any work done is done only to make the tests succeed – no speculative (wasteful) work should be done. The team members should carefully avoid the belief that they can predict work that needs to be done if there is no test for that work.
Tests can be informal, formal or even automated depending on the environment and the type of work being done. Tests can be questions or measurements and their expected results. A test can also be a procedure to follow and the results of that procedure. If the environment supports it, automating tests can be an excellent investment for reducing waste. In an ideal environment and work domain, tests can fullfil all the attributes of an ideal test.
Test driven work has two solid benefits: it helps ensure close collaboration between the team and the stakeholders, and it helps eliminate the waste of unnecessary work. Thus it supports the three agile work disciplines of Empower the Team, Amplify Learning and Eliminate Waste.
In software development, where Test-Driven Work is very sophisticated, there are a number of excellent testing tools and resources.
Posted by Mishkin Berteig at 06:01 PM | |
May 18, 2005
Making Friends Sure Beats Making Enemies
I heard a story about a situation where someone was refused career advancement because she had made an enemy a long time ago.
It made me think. Why do we make enemies? Is it because we don't really know how to make friends? In Agile Work, where communication, collaboration, teamwork and truthfulness are so important, making enemies is the worst thing a person can do. (It might not be such a big problem in mechanistic environments.)
The golden rule is a good starting place for learning how to make friends. Esther derby has a great course on teamwork that could help. An of course there's the old classic: How to Win Friends & Influence People which really is very good (if a little dated). If anyone knows some other good books or resources about learning to make friends, please reply in the comments!
Posted by Mishkin Berteig at 03:26 PM | |
May 17, 2005
The Qualities of an Ideal Test
These six qualities of tests describe how to make a test as effective and as useful as possible. The qualities are similar in style to the INVEST qualities of user stories - but they don't form a nice acronym.
Decisive: The test contains all the information required to automatically determine success or failure. Manual inspection of test results is not necessary to determine success or failure. The test is expressed in a way that produces a pass/fail answer rather than a numerical or qualitative result. Decisive tests are often expressed as assertions.
Valid: The test produces a result that matches the intention for the work artifact under test. Test failure indicates failure in the artifact under test and test success indicates correct operation or configuration of the artifact under test. Simply put: the result of a test should be true.
Complete: The test contains all the information it needs to run correctly with a given test harness and work artifact under test. The test performs all activities and provides all the data necessary. The test does not require any input outside of itself in order to run.
Repeatable: The test always gives the same results if the test harness and the artifact under test are the same. The test is created in such a way that the result is deterministic. Even if the system under test is not deterministic, the test should be created to account for that (possibly with statistical analysis) and produce a deterministic result.
Isolated: The test result is not affected by other tests run before it nor does a test affect the results of tests run after it. The test and the test harness work together to clean up after every test is run. A collection of tests can be run in any order and always produce the same results. Any test that depends upon the results or side-effects of a previous test is not isolated.
Automated: The test requires only a start signal in order to run to completion in a finite amount of time. No manual intervention is required after the test is started. Tests that are automated can be put together into a test suite and run one after another without human intervention. Automation of tests requires the creation of a test harness system.
Update 20060106
I've added a little bit more detail to the above qualities. I also wanted to say a few words about my experience with testing:
I first started doing test-driven development almost seven years ago after reading the book Refactoring by Martin Fowler. I picked it up in Windsor Ontario while I was driving to San Francisco for my first contract position as an independent. I had to stay the night there due to the immigration office being closed so I read a lot of the book in my motel room that night. By time I got to California four days later, I was totally convinced that I should be using JUnit and refactoring for everything I did. Fortunately my project was ammenable to this approach. Over the next four months I built a Java JMS layer on top of Tibco Rendezvous. In the process I discovered methods for doing unit tests for asynchronous multi-threaded distributed functionality. And my tests satisfied all the above qualities. I delivered code with 100% test coverage and zero defects discovered until more than two years later when a very rare and obscure deadlock issue was found. Suffice it to say, I was convinced from a quality perspective.
But there is more. To build tests that follow all the above qualities, you need code that is testable. I have come to believe that testability is the most important architectural attribute of code for most software. The implications of having testable code include code that is easily changed, that is verifiable, and that is easy to understand. Refactoring plays a big role here too.
For getting a basic but very practical sense of test driven development, I strongly recomment Test Driven Development: By Example 
I have also written a couple of articles recently about quality:
Quality is Not Negotiable in which I talk about how to handle defects - by always treating them as top priority work items.
and
Technical Debt in which I expand on this common analogy between lack of testing and debt to show that technical debt is even worse than financial debt.
Posted by Mishkin Berteig at 06:48 PM | |
Entry for Agile Work on Wikipedia
Can be found here. It is very bare-bones and will eventually be fleshed out. Presents the outline of the basic axioms, disciplines and practices of working agile. Please feel free to contribute.
Posted by Mishkin Berteig at 06:21 PM | |
May 16, 2005
Iterative Delivery
Work can often be divided up so that the smaller pieces are valuable on their own. By dividing work this way, a team can deliver value incrementally. The team can choose a short period of time called an iteration and select a small amount of work to complete in that time. This work should be valuable on its own. For example, if a team is building something, then at the end of each iteration whatever is built is usable as it is. This means that each iteration includes all the planning and design as well as construction or creation necessary to deliver a final product or result.
For example, a volunteer group may desire to attract new members. A non-agile approach would have the group plan their membership campaign completely before actually executing on it. An agile approach using iterative delivery would have the group plan a small piece of work that will attract some small number of new members, execute it, and then start a new iteration. One iteration may cover the creation of and delivery of a door-to-door flyer in a neighborhood. Another iteration may cover the design, creation and publishing of a small advertisement in a local newspaper. Each iteration includes all the steps necessary to produce a furthering of the group's goal of attracting new members.
In a business environment, iterative delivery allows for a much faster return on investment. The following diagram compares delivering value iteratively with a non-agile project delivery where results are delivered only at the end of the project:
One can see clearly from the diagrams that the non-agile delivery of value at the end of a project is also extremely risk prone and suseptible to change. If the project is cancelled just before it delivers, then a fairly substantial amount of effort is wasted. In the agile iterative delivery situation, an endeavor can be cancelled at almost any time and it is likely that substantial value has already been delivered.
Even if the work cannot actually be delivered incrementally, it almost always can be divided in a way so that it can be inspected in stages. Either method of dividing work allows us to do the work in iterations.
Iterations are fixed and consistent units of time during which work is performed and between which planning, inspection and adjustment is done. The empowered team will decide on the length of iterations for their work. As a rule of thumb iterations should be shorter than the horizon of predictability. Generally, iterations should never be longer than one month, no matter what the endeavor.
At the end of each iteration, a demonstration of the work completed is given to the stakeholders in order to amplify learning and feedback. Between iterations, the stakeholders collaborate with the team to prioritize the remaining work and choose what will be worked on during the next iteration. During the iteration, the stakeholders need to be accessible for questions and clarifications.
Iterative and incremental delivery is used to allow for the early discovery and correction of mistakes and the incorporation of learning and feedback while at the same time delivering value early.
Posted by Mishkin Berteig at 10:27 PM | |
May 15, 2005
Truck Factor
Truck Factor (definition): "The number of people on your team who have to be hit with a truck before the project is in serious trouble"
Clearly "hit by a truck" is an extreme thought however you could easily substitute "take vacation at the same time" to get the same idea. If any part of your project has a truck factor of one then you are in a particularly fragile situation. If that one person leaves or is unable to work on the project, you will suffer the consequences.
Over time, anyone can be replaced. Truck factor is an indication of how expensive it will be to replace specific people.
In an ideal situation, everyone on the team will know all parts of the system so that the loss of any one person would have minimal impact. In reality, many projects rely on one or more "heros" who are the only one who understand certain critical parts of the system. When these heros leave (and you should assume they will), you must be prepared to recover.
If you have a hero on your team, the best thing you can do is reassign that person to a different part of the system. This will allow the replacement to ramp up while the hero is still available for support. If you wait until the hero has left then the ramp-up will be significantly more expensive.
An added benefit to reassigning the hero is that this person will now have the opportunity to work on something different. Since the hero's tend to be the most technically competent members of the team, this will usually mean that the new area will improve once the hero has worked on it for a while.
Truck factor is a quick metric that will highlight potential problems in your project. Having hero's on your team can be very beneficial but only if you don't become dependant on them. Truck factor is one metric that will highlight your dependencies.
Cross posted from Mike Bowler's Weblog
Posted by Michael Bowler at 06:42 PM | |
What To Do With the Horizon of Predictability
In a previous entry about constant change, the idea of a horizon of predictability was introduced. This concept, along with the agile discipline of amplifying learning, suggest a strategy for addressing problems in a project.
Shorten the length of the iterations you are using. Contract your "planning horizon". The length of your iterations should be motivated by the horizon of predictability for your environment. If your project encounters trouble, particularly of the sort where it looks like you might not accomplish your commitments for an iteration, then shortening the length of iterations will enable you to resolve your problems.
First off, by shortening your iteration length, your opportunities for learning become more frequent.
Secondly, a contracted planning horizon will put you more firmly inside the horizon of predictability... and therefore there will be fewer unexpected changes (on the whole, not in any specific iteration).
A related article is The Pros and Cons of Short Iterations.
Posted by Mishkin Berteig at 06:34 PM | |
May 13, 2005
Self Organizing Systems FAQ
Just a link to the Self Organizing System FAQ - glanced through it, it looks amazing.
Posted by Mishkin Berteig at 10:29 PM | |
Amplify Learning
Learning is the result of both encountering new experiences and deliberate experimentation. Learning creates new knowledge, increased volition and improved action.
Some of people's best learning comes from “failure”. An essential component of learning is feedback.
Learning and feedback can be amplified in several ways. Provide opportunities for learning through books, training courses, coaching, deliberate exposure to diverse work, and deliberate experimentation. Frequently ask the questions “why?” and “how?” and answer the honestly and deeply. Increase the level and quality of communication among the stakeholders and team members. Inspect work in progress frequently or even continuously. Learning accelerates greatly if a culture of learning is created where individuals feel free to experiment and take initiative even on critical aspects of the work.
Learning and feedback support all three agile principles. People become more effective creators as they learn. People are better able to adapt to and embrace change as they learn. And a person's span of perception increases as they learn. Any increase in learning or feedback leads to an increase in the manifestation of the principles.
Learning and feedback can be amplified rapidly, but an empowered team is necessary to effectively take advantage of this amplification. If a team is not empowered, then rapid learning can lead to frustration. Amplified learning and feedback result in excitement, enthusiasm and playfulness, rapid problem solving, high quality work results and satisfied stakeholders.
An excellent analysis of learning at a social or group level is presented in Progress and Its Problems by Larry Laudan. In this very well written book, Laudan takes a look at the history and philosophy of science and develops a model for learning that is applicable to the development of agile methods.
Posted by Mishkin Berteig at 01:12 PM | |
May 12, 2005
Appropriate Metrics - Continued
Lean Metrics... are Lean
In the book Lean Software Development by Mary and Tom Poppendieck, there are a small number of references to metrics: process cycle time, process cycle efficiency, business models. Lean practices focus very much on diagnostic metrics that are used to help people find and eliminate waste and improve value and quality. The other aspect of metrics is to measure up for purposes of motivation and performance measurement.
Conclusions for Scrum and Agile in General
Don't prescribe specific metrics!
Agile work is about an empirical process where a team responds to change, learns, and self-regulates. An agile team has the power to choose their own metrics for their own self-inspection. For upper management, the single economic driver that is part of the "Good to Great" process should be sufficient.
See also: Appropriate Metrics and A Metric Leading to Success.
Posted by Mishkin Berteig at 07:14 PM | |
Scrum Gathering May 2005 in Boston - Rough Notes
Here are my rough notes from the May 2005 Scrum Gathering in Boston. Regrettably I was not in the room for most of Mike Cohn's presentation on User Stories... but his book (User Stories Applied : For Agile Software Development) is excellent :-)
The notes in this entry include predictions from Ken Schwaber, a presentation from Bob Schatz formerly of Primavera on their enterprise-wide implementation of Scrum, a panel discussion with Tim Bacon, Jeff McKenna, and Diana Larsen, moderated by Esther Derby. In the afternoon we heard from Pete Deemer about Yahoo!'s enterprise adoption of Scrum, Mike Cohn about User Stories, and to close the day we had an energetic presentation from Tim Dorsey of WildCard Systems about their enterprise implementation of Scrum.
Scrum Gathering 20050511
Companies using scrum: Microsoft, Sun, Siemens, State Farm, IBM, Federal Reserve Bank, Yahoo
Purpose: implement scrum
http://www.controlchaos.com/playbook.pdf
Predictions by Ken Schwaber:
33% of orgs implementing Scrum get most of the benefits – triple the initial productivity
66% will largely fail – software not that important for these orgs
Current software workload will be done by 25% of the staff
Developer will become a umbrella title for software developers – no longer specialized titles
Scrum and XP will be seen as two complimentary approaches that will start merging – may drop these names over five years
At Wildcard Systems, transition from fixed price to time and materials for clients
Bob Schatz – implemented Scrum at Primavera
Recently left Primavera now at Solstice Software
Enterprise Agile – what does it really mean?
“Enterprise”
multiple teams, same projects
multiple teams, multiple projects
implementing across an enterprise
building enterprise applications
building systems to support an enterprise
implementing across geography and cultures
Enterprise Agile is all of these and more
Implementing agile is a culture changed
culture changes are extremely difficult
transitions are different for everyone
Everyone likes change... as long as it doesn't affect them
Two factors that facilitate faster changes
pain of doing it the old way is greater than pain of chang
new opportunity where the old way simply won't work
Other than that
it takes strong leadership, determination, and TRUST
Learn to crawl before you think about walking
(Primavera did it all in a big chunk – WHAT SITUATIONS MADE THIS WORK?)
- amount of pain – about to start a new project – very challenging, big technical hurdles – asked about doing it with just one team – what's the point? - whole org new the pain – let's just get this done – didn't want to compare apples to oranges (agile to waterfall) – decided to switch pains – bob had a management team that he spent a lot of time talking with – management team was absolutely on board – failures have come in places where management is working at cross purposes
Communication
open communications are critical in agile projects
larger, more complex projects require more communication
across organizational lines
frequent customer communications
Coordination
project managers coordinate the efforts of multiple teams
focus on team building and self-managing teams
progress and obstacles must be visible to everyone
learning and knowledge must be shared across teams
Co-location
individual teams ideally should be co-located in team rooms
the reality is that they can't always be
get creative!
Do the best you can and compensate with technology
No silver bullet
enterprise implies complexity
implementing requires leaders, knowledge, skill, and adaptive techniques
one person will not single handedly change a company
a consultant will not change your culture for you
buying a tool will not make you instantly agile
not everyone will read the stack of books you buy them
executive sponsorship and support is needed (DISAGREE? ANYONE HAVE EXPERIENCE WITH THIS NOT BEING REQUIRED?)
Keys to Success at Primavera (100 people in dev org)
sufficient motivation to change (pain)
good coaching from outside
team rooms
town hall project meetings – like daily scrum but from reps for each team
project manager role transition
information radiators
no OT/weekend working – change perspective of managers that OT is good
test driven development
“science fair” sprint reviews
build process
rotating “Scrum Master” responsibilities (after doing scrum for 1.5 years started seeing little hierarchies in teams) – certified ScrumMasters became cross-team coaches
best team performance rewards @ end of each sprint related to performance, scrum compliance, etc.
team-based bonus component (two components focused on scrum plus teamwork)
feature budgeting – work on feature only budgeted time, if over, negotiate with PO (ESTIMATES DON'T WORK regardless of tools)
sprint defect limits (tradeoff in favor of quality)
customer Webex sprint reviews
commitment to learning!
Combination of these things reduced defects by 75% (year 1 1200 to 600 with scrum, year 2 600 to 300 with XP)
What to Expect
resistance – always resistance to change
setbacks
problems
obstacles
damaged egos
unrealistic expectations (kept implementation of Scrum quiet)
difficult personalities
lack of trust
“The Hurricane's” System!!
Ken -> Hurricane -> disaster/destruction -> rioting -> calm -> Ken
Ken tells it like it is
people get freaked out
What you will get
well-respected motivated focused workforce
creative energetic learning environment
everyone focused on achieving goals
ability to meet commitments and be flexible
high value products
a disciplined repeatable approach to building software
cross functional teams that understand technology and business value
involved satisfied “customers”
Academia does not model cross-functional collaborative team work (e.g. No collaboration with business school)
What can you do?
have a vision and sell it
identify pain points and don't be afraid to try something different
openly discuss behaviors and challenges
look for the “real” issues, not just what people talk about
identify champions and leverage their power (people who influence others – make sure they're on board)
educate people outside of development and involve them in the process
brush up on your influence and negotiation skills
use the language and ceremony of agile
reward good behaviors, celebrate success
work the bad attitudes out of the system
don't sweep them under the carpet
be patient, persistent... don't give up!
What would you do differently
everyone makes mistakes
communication as a leader could have been better
How did you set expectations?
upper management already had low expectations
middle management was told that there were no expectations for the first few sprints
Add process/environment/team improvements to the backlog – someone has to pay the price for it. Everything goes on the backlog! Nothing is ever perfected – always changes and improvement.
Made mistake – Scrum of Scrums was institutionalized with responsibilities – but this caused power and conflict issues – eliminated this and move to town hall meetings (Scrum of Scrums as a meeting rather than an organizational structure)
Times for management feedback
sprint planning and review
special meetings (maybe triggered by observation in daily scrum)
Core of stubbornness, courage or stupidity.
Esther Derby – Panel Discussion of Agile Change: Building the Self-Organizing Enterprise
Tim Bacon – from UK – ScrumMaster and Coach
Jeff McKenna – Original Builder of Scrum
Diana Larson
Tim:
transition to XP since 2000
deal with personal and small group level
culture about how people interact – congruence between thoughts and action – values
Jeff:
1991 Scrum
Agile before that
primarily with small teams but up to 80 people
difficulty – getting people engaged – people don't have responsibility for the things they do – power to make changes in working lives – very difficult
Diana:
culture is the stories we tell ourselves about ourselves
changing language
systems level of change - “freedom's just another word for nothing left to lose” J. Joplin
Deb Hartmann
are there places where we shouldn't bother trying to change?
Yes, but depends on how you like to spend your energy.
Jeff:
if entire org is saying “no” then listen, but sometimes people say “no” without knowing what they are saying “no” to
as a consultant: there are times when you say “I don't want to do this anymore”
if motivation is there “I want to change” then it is easier
Diana:
if some say “yes”
“see where the buffalos of change have roamed before and see what kind of a trail they left” - indicators of where things might go in the future
ego involvement – significant influences who have lots of investment in the way things are can prevent change – or need someone who can move them out of the way
How do you make org change stick?
You don't
orgs keep changing – not really a new status quo, just a breather
Tim:
you can always change starting with yourself
personal integrity and desire to go to work
sometimes need to change other people to keep integrity
Jeff:
find out what people mean by “being agile”
explore what they mean
Going from waterfall to scrum – is there a smooth transition?
Diana:
define “smooth”?
People go through change in different ways – lots of models of change
resistance can be characterized as wanting to hold onto something of value
people need to see new/better value in the destination
may be emotional
the transition in people changing is chaotic and it is hard to do smoothly
some people are not set up psychologically to make a smooth transition
can't completely make transition roughness go away completely
Jeff:
some orgs are easier than others
some orgs have a culture of change
2nd order effect
need some success with change
gain some power to make lives better
rank and file are often pessimistic
Tim:
uncover the pain that is already there
uncover the good things
Ken Schwaber:
a few years ago Martin Fowler: not going to try help unless the org is a train wreck, without hope
now seeing large orgs investigating going agile because of agile's successes
sitting with CEO of corp that want's to go agile
how can I determine if this is even worth doing?
What are some yes/no questions?
Diana:
what types of change and how did they go?
What is it like when someone needs a change from facilities?
How work with HR when needs to be an adjustment?
How do the support systems support the org?
Is the org capable of making a container?
Jeff:
find out where the pain is? What is agile going to address? How is he/she thinking about it?
V.P. Of large company – 2 hours
6 what if's suggested
kept saying couldn't do it
left (maybe someone else could do it)
Tim:
litmus test
looking for intellectual curiosity
actively asking questions
seeing possibilities as you provide suggestions, description
Esther:
what are you willing to do to support this change?
Craig Larman:
ask CEO what do you know about agile? Pair programming
Customer in Scandinavia
successful scrum project compared to other projects
head of project leaving to go to another company
other PMs didn't know about this successes
How do you sell internal success?
Diana:
retrospectives at the end of all projects
what are the root causes of success, not just failure
how do we spread the word?
Risk associated with having only one person lead something like Scrum
need to make explicit the job of knowledge and skills transfer to as many people as possible
e.g. Language “pairing” vs. “shadowing”
use newsletters, brown bags, etc.
Tim:
a big part of all our jobs is to sell what we do
all heard of failures from the inside, but seen as successes from the outside
need to make sure successes are seen as success from the outside
use networks of people
if we don't have the networks, find the “connectors” (The Tipping Point)
secretaries
long-timers
bar, cafeteria
Jeff;
Guerrilla techniques
information radiators
celebrate when work gets done: use a bell
team loves the bell and gets done
other teams ask – what's going on?
Can this be done from a grassroots level? Overallocated people, management confused. Really struggling. Wholesale change is impossible. How?
Tim:
yes
Jeff:
Pete from Yahoo! Will talk about this
warning management
team may have self-starting ability
may bring someone in – should start working on management immediately
technical side can improve team very quickly
constraint moves to management and planning
management will stop work if not ready
management can no longer blame dev for failures
can go higher in management hierarchy
Diana:
start with one team and work from there
team has to manage their boundary between themselves and the rest of the organization
Tim:
we tend to lack courage when implementing from the bottom up
courage is the most important
“be the change you want to see” - Gandhi
there is a lot of fear about change upsetting the status quo
Mike (?):
works outside of development
what are the upside risks? Success can kill you – what happens when you are successful and the org starts to think about institutionalizing Scrum? How do you take this to senior management? Expectation management? Transition and scaling? Lots of cutting of admin work – don't need a lot of middle management to get stuff done.
Diana:
doing self-org teams for 20 years (*********************)
come up against: managers take two stances:
get worried and mess about
abdicate
BUT
role needs to change
what support do these teams need?
Put on workshops about how leadership shifts in this environment
planning going away
resource management, championing, boundary, facilitating all becomes even more important
requires professional development on part of managers
Jeff:
managers who pay attention know they have to learn and improve
always opportunities for improvement
what goes through the membrane between group and rest of org
sole job to manage membrane – 5 to 10 visitors a day
org context
boundaries change and move
new container forming outside of original group
Keep aware of org context and work with that
Tim:
Bob Schatz – managers have an identity problem
transition to leadership – have courage to have vision and lead
subtle
invite senior people to see the teams and join them
always work for these people to do in the team
Bottom up implementation – management happy, later ask about working more hours and weekends to get more work done? What about managers that want to squeeze their people?
Bob Schatz:
OT/weekends lowers quality – period.
Use that quality price to drive behavior
productivity drops because of quality problems
Tim:
find these managers and get them to propose this to the team
team that has successfully transitioned will make reasons for not doing this very very clear
team will supply questions to the manager
Jeff:
just ignore them – worked well
manager broke the team up into two pieces
long hours team velocity went down
normal hours team maintained velocity
Diana:
if question comes up because underlying belief not working enough hours
could be something else going on there – and underlying personal dynamic
could suggest other experiments
Not necessarily that somethings not working. Question about increase productivity based on hours.
Diana:
recruit someone to be team champion
that sort of question can be directed to champion
Jeff:
medical community research about long hours being bad for productivity
Tim:
analogy:
mechanistic thinking
people are machines?
Factories don't run machines at 100% utilization
Lean Software movement
Theory of Constraints
people aren't machines
put it on the backlog as a problem:
how to increase productivity by 25% without increasing manpower
Tim (?): manager may be asking for something realistic
..
Brian (?): what about external customer who says we're not doing agile. How do we help our customers to adopt this? E.g. A client that has contracted for work.
Jeff:
what do you mean by customer?
Is your business taking an external stance on agile?
Boundary is the organization
can have waterfall relationship to client and work agile inside
may have waterfall embedded in law
Diana:
are you asking about influencing the culture of an industry?
Invite them in to be involved with the process as much as they can stand
might not be very much at first
someone internal must become a domain expert and act as customer proxy
Jeff:
start by inviting to monthly demos (relatively easy)
invite them to come more frequently
like teaching children
go visit customer
Tim:
changing relationship to customer is sometimes about asking them to give something up
have to articulate what they have to gain
Ian: how to reward agile teams to get the best out of them? Reward according to influence not control.
Tim:
“Punished by Rewards” - carrots can be just as damaging as sticks
fun work can be its own reward
agile makes work fun again
extrinsic rewards may be an indicator of other problems
Diana:
frequent small rewards better than large infrequent rewards
large cumulative effect
Esther:
worked with team that received huge financial bonus
bonus was nice BUT
want recognition and notice from managers at a personal level
Jeff:
rewards for goals set by someone else is a problem
e.g. Sales quota had no connection to work in field
?: Starting from top down? What are the top three pushbacks from dev org? How do you overcome them?
Diana:
manage your own expectations
imposing change is very difficult task
frame it as an invitation
Jeff:
“yet another change”
Diana:
varies from org to org depending on culture
e.g. “flavor of month – wait it out” passive resistance
Jeff:
keep working the way I always have
?:
working cross functionally is resisted very strongly
want to protect specialization
iterative may seem less efficient locally
Solve: team building, coaching, build on initial successes
First line management team has to buy in
some people will have to move out of the organization
Jeff:
do lots of educational work
find pain points before imposing solution
Ken Schwaber:
very uncomfortable
teams for X-functional
lots of releases
management questions about jobs
power prestige
get people to talk about their feelings and facilitate solutions
notice the problem and facilitate talking about problems
Tim:
fear comes from responsibility for things that were formerly hidden
now get to own solutions
?:
org change
need to paint vision of new organization
resistance comes from what needs to be given up?
Self-image might need to be given up
Diana:
territorialism
prestige
relationships
Jeff:
paradigm shift – whoop de doo
10-20% of people will not change
some people will have to leave the team
?:
Executives have reposed the question:
how do I facilitate a bottom-up
Mishkin:
key driver metric – top down
Tim:
initial response will be emotional
Diana:
Fast Company article “change or die”
Jeff:
limbic system seat of emotion
decisions are made by this system not the rational side
allow people to use much more of themselves
Roger:
inhibitor to change: the old plaster syndrome
things go wrong, put in a solution, solution is institutionalized
have to deal with all the injuries of the past
What about Some Good References for Org Culture and Change? Self-promotion is okay!
-
Tim: Fearless Change: Patterns for Introducing New Ideas
Jeff: Leading Change , The Heart of Change: Real-Life Stories of How People Change Their Organizations
Bob Schatz: Managing Transitions: Making the Most of Change
Diana: Surviving Transition
?: more about cultural changes for creating team rules.
Jeff:
take over conference rooms
build a relationship with facilities
give team both common area and privacy – gradually spend more time in common
Tim:
might be about owning tools
Jeff:
keep separate tools for email etc.
dev machines don't even have personal logins
Brent:
Q&A session to HR people
dealing with emotional level
some who were most concerned went to HR directly
Esther: one sentence of advice:
Jeff:
Pay attention, watch observer, get awareness up
Tim:
be yourself and allow others to be self
Diana:
be patient, positive and don't give up
Pete Deemer From Yahoo!
History @ yahoo! - founded in 1995
Show up at work and just build stuff
No cubes
Until late 2000
Can't spend anymore
Impose control
Hired consulting firm to impose control
The waterfall for 4 years
Last September – Tobias Mayer – organized brown bag by Jeff Sutherland
Email went out
Pete attended along with 100 engineers
Two hours – blew off meetings
We all have “muscle memory” of small nimble projects that just worked
Pete invited Jeff S. to executive team meeting
The execs often do know what is going on
Potentially revolutionary
Exec team said “ya!” - method to madness that we used to have
December groundswell of engineers and executives – never happened before
Six project teams wanted to start
Esther Derby, Mike Cohn, Paul Hodgets brought in to inform
Four teams left
Deal: support in every way needed in exchange for completed training, coaching, one sprint followed religiously then choose
Scrum pilot program started in early Feb.
Tried bribing Ken, he recommended Paul Hodgets
Paul is light touch, close to teams
Mike Cohn – like a football coach
End of first sprint did survey (anonymous)
Team members and managers
90% response rate
Amazed
First sprint really tough...
but there's something here
How much done, cooperation, quality of work life all had substantial improvement
Didn't want to stop
Quote from manager who was sceptical “I don't know how they did it but what those guys accomplished was remarkable.”
Word spread
More teams wanting to do this
Brought in Gabby Benefield
Head of Scrum practices
Now we have 8 teams + more in the queue
From six months so far -> 15 observations for big companies
1.start with the teams that want to use Scrum – scattered seeds of knowledge widely and watched for the seedlings – everyone on the team must be at least open to it working – skepticism is healthy
2.call it a pilot program – we're not migrating to anything – creates air cover for the messiness – purpose is to learn the lessons – never stop being a pilot – stop when no more teams raising their hands – then examine why – teams that raise their hands have the most pain – Yahoo is classic matrix organization with functional silos – what about interaction between agile and non-agile: non-agile teams become enthusiastic about moving to agile
3.change is scary to many people, Scrum is really scary – resistance is “pre-conscious” - presented as common-sense practices that teams choose to take on – not management forcing it – all of this is about people – people change at different speeds – Scrum is like Indian food: some will love it right away, some will need to try it a few times – some of the biggest skeptics have evolved into biggest supporters – actively managed this – ground the exercise in honesty, openness and realism – gives people room to change their mind – don't create us vs. them
4.Patience is a Virtue – err on the side of rolling out fewer teams and spending time getting it right – every team has had issues – don't risk over-extending and having it collapse – at start many more evangelists for failure and news of failure spreads quickly – over budgeted for working through systemic issues that are made visible in teams – what is us, Us and Scrum
5.Find the middle path philosophically – scrum pragmatists – revolutionaries tend to be the purists – incredible passion – but also they can feel anxiety as transfer from guerrilla warfare against the system to being the system – how to keep the purists involved – revolutions eventually become mundane reality – people are messy, scrum is messy – no utopia where things work perfectly – pragmatists are very effective at carrying this out – but prone to compromise and corner-cutting – creative tension between purists and pragmatists
6.Set a high bar and set low expectations – force teams to take personal cost of training – very hard work, very expensive, may not work for you – people know when they are being sold – lots of pain and work moving a team to scrum
7.Scrum is hard – surfaces lots of nasty stuff – every pilot has had to deal with some big nasty issue – make sure people are prepared – emphasize this pain is scrum working not the opposite – was this problem there before, and will fixing it make things better – never had a team that said Scrum created a problem – be ready to go on a rescue mission – sometimes teams need help, can't always solve things on their own – what kind of help? - more facilitative help but can include direct suggestions – teams need to build their muscles
8.get experienced help – mechanics simple, but still help is useful – but outside perspective can be very positive calm reassurance – fear of looking like an “ass” and being the “Scrum guy” as a leader – spot the small stuff that can be very telling – comes with experience
9.your enemy is your friend – spend the most time with the people who like scrum the least – the detractors can have much more impact in early stages – make them informally part of the team – take their problems and ask them to help fix them – some of them might have some good points – figure it out early – don't let battle lines get drawn
10.be prepared to use guerrilla tactics to get things done – some policy/strategy stuff – but many are just little bumps in the road – e.g. Conference room assigned but had a big conference table so couldn't stand up – tried to get table moved by facilities – got email “thanks for getting table moved out” - table disappeared one weekend – someone came in and tipped over table and moved it out – don't know where it went
11.make good information more accessible than bad information – rumor mill will go into overdrive – email updates, brownbags, top 20 myths, FAQ – continuous flow of good info
12.find your evangelists – build a network of scrum proponents in every group and level of the organization – knowledge of reality both good and bad – able to speak with candor – including bad news – anchor since day one – if scrum is as good as it needs to be, it doesn't really need me to defend it – individuals using it should inform the organization about the reality
1.Senior executives – need to have an advocate but have to be careful with it – with a single statement that person can set the tone – danger in getting exec too excited – can lead to top down imposition
1.Story about “Let's Go” travel guides – every student reporter is fired every year – publisher has no power because temp work – used very scrum-like process
2.Something connected to a prior experience
13.Make the result visible – distributed results of survey to the rest of the team – this is the good, this is the bad – made people feel comfortable to enter the conversation – this is Scrum fixing itself
14.The urge to tinker is great – everyone has a way to improve scrum – some are necessary adaptations – a lot of them aren't – e.g. We're going to split out the eng, design and qa sprints (subtle reversion to waterfall), want SM and PO to be same person, want to use some practices – say “That's fine” but reflect and compare – make very clear what is and what is not Scrum – say “using agile practices” - don't let Scrum's name get sullied by experiments that aren't scrum – but experiments are still okay
15.Scrum will always be messy – Scrum is not about cleanliness, its about reality – people are insensitive and inconsistent – make mistakes and bad choices – Scrum makes the mess visible – if it feels too clean it may not be working – even the final product is messy – there is no utopia we will arrive at
Biggest problem:
first question: what does it stand for? What is the acronym
propose making it an acronym
Mad About You – Society for the Complete Ruination of Universal Mankind
Screwing Customers Royally with Methodology
How did Yahoo! get bamboozled by waterfall?
Accenture as experts in a time of trouble
Mike Cohn Filling your Backlog with User Stories
Tim Dorsey Wildcard Systems from Florida
SVP of performance improvement @ Wildcard
6 sigma and Lean
Brian Stallings from Citibank IT
change is often compared to a step into the dark
took four weeks to see results were much better – CEO took whole org to scrum immediately
What is the light at the end of the tunnel? Train? End of tunnel?
You need a leadership that will be the light
Vital stats
founded in 1997
move from cash to electronic payments – whitebox of gift cards
prepaid payment card services
2004 revenues 57mil 40-50% growth per yearsglobal
2000 – 700 million in transactions
2004 – 3.45 billion in transactions – 6 second response time
Now have 50 clients
12 million cards issued
Flexible Solutions – Managed for Client Success
Want to do good – but kept tripping over selves
System was a catastrophe waiting to happen – growth faster than expected
No fallback position because of growth
9 months scrumming last july 27 dev 6 qa 160 dev 60 qa now
Client feedback and employee morale – before scrum – not friendly to deal with, products always late, lots of bugs (but still best in business) – 82% of employees said they didn't like coming to work – tonnes of overtime, multiple projects
Speaking executive language is very important to going enterprise wide
Executive support – let go SVP of Sales cause he couldn't get with the program
Scrum:
1.process
2.technology
3.scrum teams (operational) everyone using it
4.people
5.org and infrastructure
6.client involvement
7.culture, behavior, morale
8.competitive advantage
Enterprise -wide implementation of Scrum:
1.Strategic Alignment
2.Organizational development – team building
3.Organizational analysis – metrics what is really going on?
4.Implement for Results
1.Symptom -> leap of faith -> solution
Took a baseline
Strategic Alignment (setting the course)
vision values and mission
process boundaries and accountability
goals indicators and targets
participation, buy-in and ownership
communications: structure and process
Lessos learned:
+strong executive support and review
+professional coaching and guidance
+consistent scrum practices
-WCS Culture: “get it done no matter what”
-recreate what we just threw out
-good is the enemy of great
Organizational Development (preparing the people)
team building and dynamics
roster roles and ground rules
expectations and concerns
interpersonal skills and meeting effectiveness
communications: metrics and alignment
Lessons Learned:
+strong review process
+learning organization (releases) – raising the bar means controlled release of the process improvements
+time boxed meetings
+team selection (teams initially had a team lead with 6months, a totally new scrummaster and 4 contractors)
-PO wanted to “own” the team
-lack of shared accountability (lack of collective knowledge of state of tasks)
-we're different, “our client expects...” - clients are exposed to “dirty laundry” through the daily scrum as chickens and they love it
Organizational Analysis (looking in the mirror)
selection criteria & ROI
process and tools
clients, suppliers, products and services
resources and portfolio management
communications: available and consistent – e.g. Mandate on use of Primavera for communication
Lessons Learned:
+Utilized Primavera for Scrum Template
+Staffing Models Reflect Backlogs
+financial benefits – able to move from $150/hr for dev to $60/hr only ask 2 weeks notice to shut team down
+client involvement
- ROI responsibility
-poor forecasting of Start and Due Dates
-valuable early weeks wasted waiting for client decisions
Implementing for Results (making it happen)
scrum planning and launch (2 weeks)
product backlog and estimates
sprint planning decomposition and review
agile programming tools
communications: daily scrum meetings and burndown charts
Lessons Learned
+introduced launch checklist
+consistent backlog and decomposition format
+application code collisions
+operational bottlenecks exposed
-overall sizing estimates were poor
-didn't require burndown charts immediately
-financial impact of a scrum team
Scrum Assessment: Radar Chart
Two week sprints all ending on the same day across the organization
What happens in two week ramp up after team formation develop backlog, interview clients, interview systems people
Stick to strictly normal working hours – teams are allowed to choose to work on the weekend but if it starts being a regular occurance then look at why
Metrics:
focus on process metrics
Posted by Mishkin Berteig at 10:35 AM | |
May 11, 2005
Appropriate Metrics
At the Advanced ScrumMaster Training, Ken Schwaber presented a substantial amount of thinking about metrics used with Scrum. The main driver for thinking about metrics has come from implementing Scrum in enterprise situations. Management expects metrics to be used in order to provide visibility into the progress of the Scrum implementation.
While this driver has some legitimacy, there are three main concerns to prescribing metrics for use with Scrum.
1. Planned/Engineering Approach - metrics such as "ideal person days" smell like the bad old way of treating people as resources that can be swapped in and out of a project.
2. Normative vs. Empirical - metrics are often used to set a standard for reasons such as prediction, and expectation. Scrum is about discovery and improvement not prediction and planning.
3. Is Something in Scrum not Working? Is adoption being hindered? Are too many Scrum implementations failing? Are metrics a critical success factor?
Keep these concerns in mind while considerig the uses of metrics.
What are Metrics for?
1. Self-Evaluation over Time - use a metric to track the progress of a group. A measurement is taken at a certain point in time, and then taken repeatedly over intervals. Example: the velocity at which a team completes work can be used to identify problems and opportunities for a team.
2. Control - use a metric to legislate the qualities of some process or attribute of work. A goal or standard is set for a measurement. The size of a queue of projects waiting to be worked upon by a team can be controlled in order to limit the size of work in progress and therefore project inventory.
3. Prediction - use a metric with values collected over time in order to predict future values of that metric. By implication, the metric is used to predict the performance of a system. The number of additional tasks discovered inside an iteration can be tracked and used to determine future expectations on extra tasks discovered.
4. Performance Measurement - use a metric in order to determine rewards and/or punishments and/or adjustments to behavior. An individual on a team may be evaluated based on their productivity as measured by lines of code completed and rewarded or penalized on that basis (not recommended!).
5. Behavior Motivation - use a metric to guide behavior by setting a context for thinking, action and reflection. Focus on an important measure in order to draw attention to improving the attribute with which the metric is associated. Measuring the time elapsed from conception of a project idea to the time it actually delivers valuable results can draw attention to improving speed.
The Lesson from Good to Great
Good to Great: Why Some Companies Make the Leap... and Others Don't by Jim Collins discusses the attributes and behavior that are common and unique to companies that have gone from a long history of mediocre results to a long run of great results. One identified aspect is referred to as the "Hedgehog Principle". In this principle, three questions are answered: "what can we be the best in the world at?", "what can we be passionate about?", "what is my one economic driver?"
The economic driver is a metric. For example, at Walgreens, their driver is profit per customer visit. Other large institutions have other metrics. But all good to great companies have a single economic driver metric that guides all their decision making.
See also: A Metric Leading to Success.
Posted by Mishkin Berteig at 05:21 PM | |
May 10, 2005
Steps in Making a Decision
In her workshop "Advanced Scrum: Collaboration Skills for Scrum Teams", Esther Derby includes a brief discussion on the five parts of a decision.
1. Define the Problem
2. Establish Focus and Boundaries
3. Identify Options
4. Choose Among Options
5. Implement
At each step, it is instructive to examine who is "responsible" or involved. In an agile team where team empowerment and self-organization are considered critical success factors, the answer to "who?" fore each step should usually be "the team".
There are however, some situations where decisions are outside the realm of a team's empowerment. As well, some decisions are so trivial that it is wasteful to have the whole team involved. In these trivial decisions, usually another person can take responsibility for all the steps of a decision as a service to the team.
Over time, a team and the organization in which a team operates can evolve a set of standards that describe who acts in each step of a decision under what circumstances.
Many thinking tools described by Edward de Bono in his various books (such as Six Thinking Hats, Lateral Thinking : Creativity Step by Step (Perennial Library), Textbook of Wisdom etc.) can be used at various steps in the decision making process.
Posted by Mishkin Berteig at 11:48 PM | |
Empower the Team
Empowerment is the ability of a team to make decisions about how to do their work and execute on those decisions without outside interference. If a team is empowered, then it will be more capable of responding to change, and it will be able to focus on manifesting the members' creative potential. Empowerment comes from a combination of several factors:
1. members of the team have a deep sense of self-worth that includes nobility, and contribution to the progress of humanity
2. tacit or explicit authority and responsibility for results as a team and as individuals
3. a team environment which is honest, trusting and allows for mistakes
4. the absence of personal attacks against individuals on the team and in particular a total lack of gossip and backbiting
There are several ways that team members will demonstrate their empowerment. People will derive joy from their work. Team members will be dedicated to the work and the team. Individuals on the team will frequently take individual initiative to accomplish tasks, share insights, and develop improvements. Spontaneous leadership will become common. Individuals will step out of comfort zones or areas of specialization in order to assist the team.
Empowering a team is a process that can sometimes take a great deal of time and effort. In order to start on this process, the team members should carefully listen to each other and ask many questions. More mature individuals should lead and teach by example. And all the team members can start to question and challenge the rules and procedures of an environment that are preventing effective work. If the team is in an organizational environment where team members have management to report to, then management must be aware of this opportunity for empowerment and support it.
An empowered team can gradually understand and internalize the agile work principles (People are Creators, Change is Constant, Perception mediates Reality). By internalizing these principles, a team can move beyond specific agile work practices and become a high performing team setting their own work practices.
Jeff Sutherland has a very brief blurb about the progress of teams as they evolve in their use of Scrum.
Future entries here will discuss the methods of creating empowered teams.
Posted by Mishkin Berteig at 11:04 PM | |
Information Radiators
An information radiator is a large display of critical team information that is continuously updated and located in a spot where the team can see it constantly. The term "information radiator" was introduced extensively with a solid theoretical framework in Agile Software Development by Alistair Cockburn.
Information radiators are typically used to display the state of work packages, the condition of tests or the progress of the team. Team members are usually free to update the information radiator. Some information radiators may have rules about how they are updated.
Whiteboards, flip charts, poster boards or large electronic displays can all be used as the base media for an information radiator. For teams new to adopting agile work practices the best medium is usually a poster board on the wall with index cards and push pins. The index cards have a small amount of information on each of them and the push pins allow them to be moved around.
Information radiators help amplify feedback, empower teams and focus a team on work results. Too many information radiators become confusing to understand and cumbersome to maintain. If an information radiator is not being updated it should be reconsidered and either changed or discarded.
Here is an information radiator used to quickly display the status of multiple projects to an agile Project Management Office:
As the team used an information radiator, it can adapt the display of information to become more appropriate to the way the team is operating and the information they are really concerned about. For example, on the above IR for a PMO, the group may decide that they wish to put red dots on projects that are in trouble in some way.
Some very interesting examples of the effective visual display of information can be found in The Visual Display of Quantitative Information.
Posted by Mishkin Berteig at 06:25 PM | |
May 09, 2005
Reasons for Conflict or Disagreement
As part of the Advanced Scrum Training, Esther Derby presents a section on conflict. One very insightful part of the presentation is a description of four reasons for the existance of conflict or disagreement. They are as follows (adapted from "Advanced Scrum: Collaboration Skills for Scrum Teams" (c)2004-2005 Esther Derby):
1. Lack of Clarity - the communication between parties is missing information or the information is not being communicated in a consistant manner.
2. Position Focus - the parties involved have already each decided on their own solution and are failing to discuss the problem those solutions are addressing.
3. Different Values - the parties are unable to agree because they are holding different sets of values but not articulating those values as part of the discussion.
4. Past History/Personalities - the parties have a previous unresolved conflict that is negatively affecting their ability to work together.
All of these types of conflict are based on either conscious or unconscious failure to be truthful... and therefore the different parties have incompatible perceptions of reality. The Agile Work axiom "Reality is Perceived" then gives us a hint as to how to resolve all of these types of conflict. Find a way to share perception among the parties in conflict so that they have a compatible view of reality.
Posted by Mishkin Berteig at 10:52 PM | | | TrackBack
May 06, 2005
Agile Readiness - When Can Your Organization Adopt Agile?
I've come up with a short quiz about agile readiness. It has never been tested anywhere. Rather, it represents my observations about what conditions have been in place in organizations where agile has taken root and flourished vs. places where attempts at agile have failed.
Agile Readiness Assessment Worksheet
Empowered Agile Process Facilitator
This person is responsible for the success of the agile process used on your project. They help your teams and your organization deal with the obstacles to becoming agile. They facilitate an explicit learning process. They avoid "command and control" approaches to managing work. An excellent source of such people is those who have taken the training available from Agile Masters.
Your situation is best described by:
Score: 0 - No one available for this role
Score: 3 - Someone is available and empowered by management, but he/she is inexperienced
Score: 3 - Someone is available and experienced, but not empowered by management
Score: 6 - Someone is available who is both experienced and empowered
Your Score: _______
Management Support
Management support provides protection from the “old way” of doing things, and enables a team to self-organize, cut through red tape, and establish a “new way”. Initially, this is often done with a pilot project. It can also be done with special task force teams. In the long term, it requires upper management to understand and support with clear goals and measures the change needed.
Score: 0 – Management insists on full compliance with existing internal project process
Score: 2 – Management does not know that the team is doing Agile
Score: 4 – Management accepts Agile, but is ambivalent about methodology
Score: 8 – Management is excited about doing Agile
Score: 8 - Management is unconcerned about method and cares about results only
Your Score: _______
Team Readiness
A team must be ready to take on work. If the team is already familiar with the work to be done, and the team has worked together previously, then the team is ready to take on the new challenge of agile work practices.
Score: 0 – no team has been created for the project
Score: 2 – team has been created but has never worked on this domain
Score: 4 – team has been created and members have worked in the domain previously
Score: 6 – team has been created and members have worked in your organization in the domain previously
Score: 8 – team has been created and worked together previously in your organization in the domain
Your Score: _______
Team Experience with Agile
Past experience with agile practices can have a substantial positive effect on the outcome of the project launch.
Score: 1 – no experience and not interested in Agile
Score: 4 – no experience but very interested in Agile
Score: 6 – some experience with Agile and still interested in it
Score: 8 – very experienced with Agile
Your Score: _______
Project Readiness
The project must be funded (or otherwise materially supported) in order to be launched. A strong champion can often ensure funding is in place before project launch commences. It is also critical that the project is satisfying a real need in the organization.
Score: 0 – Project has no champion nor has it been funded by the organization
Score: 1 – Project has a strong champion but no funding yet
Score: 5 – Project has no single strong champion, but it has funding
Score: 9 – Project has both a strong champion and confirmed funding
Your Score: _______
Scoring:
Sum up the scores for each aspect of readiness.
0 – 12, or 0 on any single aspect: Definitely NOT ready to launch an agile project. You need to do some basic work to prepare.
13 – 27: Your organization is ready to take on the launch of an agile project... but with assistance! Consider hiring a coach.
28 – 39: Go for it! You have enough experience/support/excitement to have a good chance of doing it on your own successfully.
(NOTE: this assessment is an indicator not a guarantee.)
Posted by Mishkin Berteig at 04:44 PM | |
May 05, 2005
Change is Natural - "Embrace Change"
Kent Beck's book "Extreme Programming Explained : Embrace Change" provides a good introduction to how software development can embrace the constant change that affects our world. Some of the practices he introduces are very software-specific. However, the overall basic message is sound and provides a foundational principle for all agile work. (By the way, the book is excellent.)
Change really does occur everywhere. Change is constant. A google search for "embrace change" or "change is constant" will both turn up an incredible variety of articles, papers, discussions, books and viewpoints that all affirm the constant nature of change and the need to embrace it.
Nevertheless, it is sometimes difficult to accomodate change when we also have a legitimate and deep desire to know what is coming next.
For many teams, the environment in which they work is constantly changing. This change can be caused by competition between organizations, scientific or technological advances, fads and cultural shifts, major events in people's personal lives or even just a change of opinion with a stakeholder. Any change, even small change, can invalidate a planned course of action. However, goals (as distinct from plans) are more stable and often survive even major environmental changes. Therefore, rather than trying to plan the future, an agile team can focus on being able to respond to change while still reaching a goal.
Nevertheless, a team needs some sense of what it will do in the near future. A team can work with a “horizon of predictability”. This is the distance into the future which a team can be reasonably certain that plans will be stable. Depending on the environment, this may be as little as a few minutes, or as long as a month. It is rarely longer. The horizon of predictability is not a precise demarcation, rather, expect change with a probability based on the horizon of predictability. Then, plan to respond to change. Be detached from the concrete details of a plan, particularly if they occur outside the horizon of predictability.
Responding to change requires a major mental shift for many people that is difficult and takes time and environmental support. People are often penalized socially or formally for being flexible or adaptable. This quality can appear to be “wishy-washy”, uncertain, indecisive, uncommitted or even rebellious.
The terms “agility” or “agile work” refer to this principle of embracing constant change since it is the most visible of the principles. However, the ability to respond to change relies on the establishment of agile work disciplines and practices.
Posted by Mishkin Berteig at 06:57 PM | |
May 04, 2005
Can dying plan-based projects be recussitated?
We've all seen this. A one-year project in its 13th month, and the Project Manager has been reporting 80% of the tasks at 90% and has been doing so for the last 120 days. There's no end in sight, and the customer is leaking cash every day the product fails to go into production. What can be done? Agile project management principles can help this all-too-frequent scenario.
First, let's look at how this situation comes about. In a typical task-oriented project plan, one is required to decompose the tasks down to a fairly reasonable degree of specificity. The tasks are organized around a dependency graph, estimated, resources are assigned, and the schedule is calculated and/or nudged. (Well, usually the initial estimates don't match the customer's expectation and deadlines, so there's a revision step here where estimates are shortened) But how does change get reflected? If a PM is doing an excellent (if frustrating) job, they are constantly updating the schedule, the plan, and re-conceiving the tasks as new information comes to him.
More often than not, however, this gets so messy and unwieldly that the PM holds fast, and starts to estimate completion based not on a proper decomposition and a completion of each of these tasks, but based on his guess, or his workers' guesses. Complexity of the tasks is hidden, and becomes often quite invisible. Tasks at 80% which suddenly need to be re-decomposed and re-conceived do not, in the main, get moved back down to 40% after certain roadblocks are discovered.
The result? The customer is increasingly afraid, trust between delivery staff and client (and management) are eroded, pressure is increased, mistakes under pressure become more frequent, less sleep is had by all, and 90% complete becomes an asymtote, rather than a milestone.
Now what is to be done with such a project? How can Agile project management approaches help this situation? We can examine this by playing out such a scenario...
We can start by identifying a few key practices of Agile project management, and examine their benefits to the business client.
Timeboxing and Iteration
The first thing we can talk about to the fearful client is timeboxing. Timeboxing caps his investment into small chunks. We look at it and say, OK, you're in deep trouble, but you don't know how deep your trouble is. By timeboxing into a very short timeframe, and making a large project into many little short projects, we can get more visibility into the process - we can see things as they are, rather than how they're reported on a project plan. Speaking of visibility...
Daily Meetings and Iteration Review
Part of the value of iteration and timeboxing is increased visibility, so we really need to have a mechanism for visibility. Already distrustful of project plans, we can tell the client, "Don't believe the paperwork, let's look at what's actually built. At the end of each iteration, we'll review the current situation, and demonstrate existing functionality." His eyes perk up. "Wah?" he asks incredulously? What do you mean? So we say, "we don't want you to guess at how ready this thing is. We'll show you. That way, you can decide if it's ready enough. ...Or your product marketing people, if you prefer. It's up to you, but you get to see it, touch it, and sense for yourself how ready it is, and you won't have development managers and developers waving their hands pretending it's further than it is."
"Oh wierd," says he. "So what are these daily meetings?"
We tell him that the daily meetings have several purposes. Only project members get to speak, and they report on what they did yesterday, what they plan to do today, and what, if anything, is blocking their path. No one else gets to speak, but others who are not on the team itself can listen in. This way, the whole project team is clearly aware of how they're working, what's left to do, and what each other are doing.
"Why do they need this? They have the plan." "True," we reply, "but how often do you have two people who need something, and both do it because they don't know that someone else already did it. With your current project delays, do you want any duplication of effort? Just by way of example?"
Feature Lists, not Task Lists
We also talk to him about working from feature lists, not task lists. The team will be implementing features, and fulfilling requirements. How they do it is up to them. "Why should I care," he'll ask. "You don't," we reply, except to assure him that if people are working against features, then they can choose to re-order their tasks in such a way as to most quickly get to the goal. No wasted effort, we tell him. Sounds good to him.
Customer-Prioritized Features
What's more, we assure him, we will be first working on the highest-priority features. What "needs" to get to market. Everything else slides down the list. Even if the wonderful plan we all love had it earlier. High-value features (as prioritized by the client) and their necessary dependencies. That's it.
"So no features that I didn't ask for?" He looks hopeful.
"That's right," we reassure him, "and you can change your mind."
He faints.
Smelling salts are brought.
"What do you mean, 'I can change my mind?'" he asks.
"If, when we get to the iteration review, and you test drive the thing, Acme corporation has come up with the killer feature that you absolutely must have or the whole thing is useless, you put it at the top priority on the list we use to drive the next iteration."
"And no one will complain that it's out of scope?" he marvels?
"If you put it at the top of the list, it is the scope for the next iteration. We are at your service," we comfort him.
"So when will this be finished?" he asks us desperately.
"When you say it's ready," we reply. He boggles. What does that mean, he thinks.
"What does that mean," he asks.
"It means that we will tie it off, when you think it has enough juice to go to market. If, after you re-prioritize what's left, you find that it's 'ready enough', then we'll roll with it, take a couple of weeks to steady it and ship it, and then we can pick up your next-highest priority things, or anything new you want in it." He looks near fainting again. "And we stop when you feel that spending money on it is no longer bringing you enough value, ideally because we've done all the high-value stuff already, and we're working on less valueable stuff."
"So I have discretion to pull the plug whenever I want to?"
"Yep."
"Please help me..."
Conclusion
This little scenario is fictitous, but in the end, it's consistent with the experiences of many Agile practitioners (particularly Scrum) that I have spoken with. We've only covered a small sampling of Agile practices that may help a project in crisis. Others may help quality, may improve developer productivity - but the above can help a key client or stakeholder begin to see a light at the end of the tunnel. While many people cannot get their heads around it, they may be willing to try new things when they're in enough pain with the existing process.
And it can turn a project failure into an ever-increasing success, because success is defined monthly, re-defined at the desires of the business client, and executed in bite-sized chunks that are digestible and estimable.
Just don't forget that people have the strangest reactions to things that break their expectations... so make sure to bring the smelling salts...
Posted by Christian Gruber at 09:08 PM | | | TrackBack
Agile Practices Applied to Architecture
I have just run across a web site about applying agile practices, specifically from Extreme Programming (XP) to architecture. This site, called "Architectural Practices - Extreme Project Management for Architects" has a great deal of information.
This site represents a set of ideas that have come full-circle. Christopher Alexander wrote a book called "A Pattern Language: Towns, Buildings, Construction (Center for Environmental Structure Series)" about a different way of looking at architecture, interior design, landscape design and city planning. This book made a big impression on some people in the software industry. In particular, it inspired the book "Design Patterns" by Erich Gamma et. al. This book became required reading in the software industry. Many participants in the Agile Software Development movement consider it foundational. And now we see agile practices being adapted back to architecture. Cool.
Posted by Mishkin Berteig at 04:15 PM | |
Asymmetry of Knowledge and Abuse of Agile Practice
I read an article in Wired yesterday that was modified from a book "Freakonomics
Doctors have a similar motivation. Statistically, when times are tighter (fewer patients), the article pointed to the proportion of interventions going up (referring here to internists and surgeons). The article isn't crying conspiracy. Rather, it's talking about the natural incentives, and the consequences of the asymmetry of knowledge. The doctor knows more, so if they (consciously or subconsciously) recommend more intervention than is necessary, the patient has no way of knowing in order to assess bias and accomodate. Likewise with Real-estate agents.
With alternative practices or new sciences there is an even larger knowledge gap, because even popular wisdom hasn't filtered down to the masses in digestible CNN-sound-bite chunks. Also, there is a lot of "naive money" in new fields, as well as a lot of genuine people who are just trying to help. Unfortunately, this means that there are a lot of wolves among the sheep, and they're wearing the costume of a sheep-dog.
I think, in some ways, that was at the basis for Ken Schwaber's concern on the Scrum Developer's list about a scrum practitioner who was not really teaching scrum, but was (in Ken's paraphrased view) bilking the customer and ego-tripping. Scrum will have a similar dynamic as one finds in, say, alternative health and nutrition, or other early-stage professional arenas. There will be idealists and opportunists and then some who try to apply balance. One has to be very careful of both the idealists and the opportunists. Sometimes it is very hard to tell, and this has a high chance of painting the whole Agile community with the same brush. One of my associates had a very very bad experience with Scrum for that reason. An unscrupulous person who used the position of Scrum Master to aggrandize himself.
Posted by Christian Gruber at 01:07 PM | |
May 03, 2005
Index Cards as an Agile Work Tool
Index cards are an excellent tool to use to optimize communication. There are two primary types of use for index cards.
1. Dynamic Collaboration
Many types of meetings can be made more collaborative and therefore more effective by the use of index cards. Cards are written on to record ideas. By writing on cards, all the participants in the meeting can see the ideas.
As an example, suppose that a team is examining two competing ideas of some complexity. The team uses two colors of 5x7 index cards and a pile of Sharpie(R) markers. The different card colors represent the competing ideas. Members of the team take cards and write down their comments as they think of them. The team member then reads their card aloud and passes it to a facilitator. The facilitator takes the cards and puts them up on the wall under various categories. The team uses a thinking tool such as PMI (Pluses, Minuses, Interesting) from CoRT to organize the cards.
Contrast this method with someone taking notes. Using the cards allows all the team members to see all the information at the same time. It also allows team members to come up to the wall and either add notes to a card or move cards around.
2. Dynamic Tracking
Teams are more focused when they are constantly aware of the status of their work. Index cards can be used to improve the team's awareness of status information by forming them into an "information radiator".
For example, a team might be working from a list of tasks. Each task can be written on an index card. The task cards can then be put on a wall and organized into groupings based on their completion state. A typical set of states might include "Not Started", "In Progress", "Waiting for Test", "Tested" and "Approved". As team members work on tasks, they move them from group to group. In this way, the team can easily see how much work they have done, and have yet to start. As work progresses, the team gets immediate positive feedback as the number of cards in the "Approved" state grows.
This method can be compared to an online project plan status updated by a project manager. The index card information radiator is participatory and always visible. It requires no effort to check status