Tag Archives: project management

Pitfall of Scrum: Stretch Goals

The team decides on how much work it will do in a Sprint. No one should bring pressure on the team to over-commit. This simply builds resentment, distrust and encourages low-quality work. That said, of course teams can be inspired by challenging overall project or product goals. A stretch goal for a Sprint is just a way to 100% guarantee failure. Even the team should not set its own stretch goals.

There are a few interesting principles that apply here. For example, the Agile Manifesto mentions sustainability:

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

The Agile Manifesto also points out the importance of trust:

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Stretch goals are incompatible with both of these principles from the Agile Manifesto.

There are two types of stretch goals. The first type are those assigned by outsiders to the team. The second type are those which a team sets for itself. Both types are bad.

Stretch Goals Assigned by Outsiders

The worst extreme of this type of stretch goal is also the most common! This is the fixed-scope-fixed-date project deadline. In this type of stretch goal, the project team, doing Scrum or not, is forced to work backwards from the deadline to figure out how to get the work done. If the team can’t figure this out, managers often say things like “re-estimate” or “just get it done.” (Note: another thing that managers do in this situation is even worse: adding people to the project! Check out “The Mythical Man-Month” by F. Brooks for a great analysis of this problem.)

My anecdotal experience with this sort of thing is simple: quality suffers or sustainability suffers. I once worked with three other people on a mission critical project to help two banks with their merger. There was a regulatory deadline for completing the integration of the two existing systems for things like trading, etc. Fixed-scope-fixed-date. Coffee and sleepless nights were our solution since we tried not to sacrifice quality. We actually ended up working in my home for the last few 24-hour stretches so that we would have access to a shower. Suffice it to say, there’s no way we could have sustained that pace. It’s anti-Agile.

A quick search for ideas and opinions about stretch goals makes it very clear that there is no commonly agreed “correct” answer. However, from an Agile perspective stretch goals assigned by outsiders are clearly against the principles of the Agile Manifesto.

Stretch Goals Set by the Scrum Team

The Scrum Guide states:

The number of items selected from the Product Backlog for the Sprint is solely up to the Development Team. Only the Development Team can assess what it can accomplish over the upcoming Sprint.

For emphasis: what it can accomplish – not what it (the Development Team) wants to accomplish, or what it should accomplish, or what it could accomplish if everything goes perfectly. A Development Team should be accomplishing their Sprint plan successfully (all Product Backlog Items done) on a regular basis. Of course, exceptional circumstances may intervene from time to time, but the team should be building trust with stakeholders. Here’s another story:

I had a good friend. We would always go out for coffee together. We just hung out – chatted about life, projects, relationships. Of course, from time-to-time one or the other of us would cancel our plans. That’s just life too. But there came a time when my friend started cancelling more often than not. There was always a good excuse: I’m sick, unexpected visitors, work emergency, whatever. After a little while of this I started to think that cancelling would be the default. I even got to the point where I was making alternative plans even if my friend and I had plans. I got to the point where I no longer trusted my friend. It didn’t matter that the excuses were always good. Trust was broken.

It doesn’t matter why a team fails to meet a goal. It reduces trust. It doesn’t matter why a team succeeds in meeting a goal. It builds trust. Even among team members. A team setting stretch goals is setting itself up for regular failure. Even if the team doesn’t share those stretch goals with outsiders.

Stretch goals destroy trust within the team.

Think about that. When a team fails to meet its own stretch goal, team members will start to look for someone to blame. People look for explanations, for stories. The team will create its own narrative about why a stretch goal was missed. If it happens over and over, that narrative will start to become doubt about the team’s own capacity either by pin-pointing an individual or in a gestalt team sense.

Trust and Agility

The importance of trust cannot be over-stated. In order for individuals to work effectively together, they must trust each other. How much trust? Well, the Agile Manifesto directly addresses trust:

Build projects around motivated individuals. Give them the environment and support they need and trust them to get the job done.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

Video: The Myths of Scrum – Scrum is for IT Projects

Subscribe to our BERTEIG / WorldMindware YouTube channel – we have over 20 more videos planned on the Myths of Scrum.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

Pitfall of Scrum: Excessive Preparation/Planning

Regular big up-front planning is not necessary with Scrum. Instead, a team can just get started and use constant feedback in the Sprint Review to adjust it’s plans. Even the Product Backlog can be created after the first Sprint has started. All that is really necessary to get started is a Scrum Team, a product vision, and a decision on Sprint length. In this extreme case, the Scrum Team itself would decide what to build in its first Sprint and use the time of the Sprint to also prepare some initial Product Backlog Items. Then, the first Sprint Review would allow stakeholders to provide feedback and further develop the Product Backlog. The empirical nature of Scrum could even allow the Product Owner to emerge from the business stakeholders, rather than being assigned to the team right from the start.

Starting a Sprint without a Product Backlog is not easy, but it can be done. The team has to know at least a little about the business, and there should be some (possibly informal) project or product charter that they are aware of. The team uses this super basic information and decides on their own what to build in their first Sprint. Again, the focus should be on getting something that can be demoed (and potentially shippable). The team is likely to build some good stuff and some things that are completely wrong… but the point is to get the Inspect and Adapt cycle started as quickly as possible. Which means of course that they need to have stakeholders (customers, users) actually attend the demo at the end of the Sprint. The Product Owner may or may not even be involved in this first Sprint.

One important reason this is sometimes a good approach is the culture of “analysis paralysis” that exists in some organizations. In this situation, an organization is unable to do anything because they are so concerned about getting things right. Scrum is a framework for inspect and adapt and that can (and does) include the Product Backlog. Is it better for a team to sit idle while someone tries to do sufficient preparation? Or is it better to get started and inspect and adapt? This is actually a philosophical question (as well as a practical question). The mindset and philosophy of the Agile Manifesto and Scrum is that trying to produce valuable software is more important that documentation… that individuals and how they work together is more important than rigidly following a process or tool. I will agree that in many cases it is acceptable to do some up-front work, but it should be minimized, particularly when it is preventing people from starting to deliver value. The case of a team getting started without a product backlog is rare… but it can be a great way for a team to help an organization overcome analysis paralysis.

The Agile Manifesto is very clear: “The BEST architectures, requirements and designs emerge out of self-organizing teams.” [Emphasis added.]

Hugely memorable for me is the story that Ken Schwaber told in the CSM course that I took from him in 2003.  This is a paraphrase of that story:

I [Ken Schwaber] was talking to the CIO of a large IT organization.  The CIO told me that his projects last twelve to eighteen months and at the end, he doesn’t get what he needs.  I told him, “Scrum can give you what you don’t need in a month.”

I experienced this myself in a profound way just a couple years into my career as an Agile coach and trainer.  I was working with a department of a large technology organization.  They had over one hundred people who had been working on Agile pilot projects.  The department was responsible for a major product and executive management had approved a complete re-write.  The product managers and Product Owners had done a lot of work to prepare a product backlog (about 400 items!) that represented all the existing functionality of the product that needed to be re-written.  But, the big question, “what new technology platform do we use for the re-write?” had not yet been resolved.  The small team of architects were tasked with making this decision.  But they got stuck.  They got stuck for three months.  Finally, the director of the department, who had learned to trust my advice in other circumstances, asked me, “does Scrum have any techniques for making these kind of architectural decisions?”

I said, “yes, but you probably won’t like what Scrum recommends!”

She said, “actually, we’re pretty desperate.  I’ve got over a hundred people effectively sitting idle for the last three months.  What does Scrum recommend?”

“Just start.  Let the teams figure out the platform as they try to implement functionality.”

She thought for a few seconds.  Finally she said, “okay.  Come by this Monday and help me launch our first Sprint.”

The amazing thing was that the teams didn’t lynch me when on Monday she announced that “our Agile consultant says we don’t need to know our platform in order to get started.”

The first Sprint (two weeks long) was pretty chaotic.  But, with some coaching and active support of management, they actually delivered a working increment of their product.  And decided on the platform to use for the rest of the two-year project.

You must trust your team.

If your organization is spending more than a few days preparing for the start of a project, it is probably suffering from this pitfall.  This is the source of great waste and lost opportunity.  Use Scrum to rapidly converge on the correct solutions to your business problems instead of wasting person-years of time on analysis and planning.  We can help with training and coaching to give you the tools to start fast using Scrum and to fix your Scrum implementation.

This article is a follow-up article to the 24 Common Scrum Pitfalls written back in 2011.

[UPDATE: 2015/08/19] I’ve just added a video to the “Myths of Scrum” YouTube series that adds a bit to this:

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

The Agile Manifesto – Essay 3: Working Software over Comprehensive Documentation

How much documentation does it take to run a project with ten people working for six months?  For some organizations it takes way too much:

Photo of heavy documentation for software project

This binder (about 3 or 4 inches thick) is all the documentation associated with such a project.  In looking carefully at the project, creating the documentation took far more time than the time spent on designing, writing and testing the software.  Yet, the documentation does not produce any value.  Only the software produces value.  The Agile Manifesto, asks us to focus on the outcome (working software) and to make tradeoffs to minimize the means (comprehensive documentation).

The Agile Manifesto asks us to challenge our assumptions about documentation.  In many work environments, documentation is an attempt to address some interesting and important needs:

  • Knowledge sharing among stakeholders and the people working on a project.
  • Knowledge sharing across time as people come in and out of a project.
  • Verification and traceability for contracts or other compliance needs.
  • Decision-making and analysis for business and technical problems.
  • Management oversight and control.
  • Various aspects of individual accountability.

Documentation is usually heavier (more comprehensive) the more the following circumstances exist in an organization:

  • Geographical distribution of people.
  • Lack of trust between people, departments or organizations.
  • Regulated work environments.
  • Depth of management hierarchy.
  • Number of people directly and indirectly involved.
  • Knowledge and skill sets highly segregated between people.
  • Culture of respect for written texts.

Working Software

What if the software itself could address the needs that often documentation is used to address?  Let’s look at them in turn:

  • Knowledge sharing among stakeholders and the people working on a project.
    If the software is functional at all stages, as supported by Agile methods such as Scrum and Extreme Programming, then the software becomes an effective representation of the knowledge of all the people who have participated in building it.
  • Knowledge sharing across time as people come in and out of a project.
    Software that is technically excellent is often easier to understand for people who are new to it.  For example, excellence in user experience and design means new users can get up to speed on software faster.  Use of good design patterns and automated testing allows new developers to understand existing software easily.
  • Verification and traceability for contracts or other compliance needs.
    Test-driven development (code) and specification by example (scripting and code) are forms of traceable, executable documentation that easily stay in-sync with the underlying software system.
  • Decision-making and analysis for business and technical problems.
    In particular, diagrams can help a great deal here.  However, electronic tools for creating such diagrams can be slow and awkward.  Consider the practice of Agile Modelling (basically using a whiteboard and taking photos) as a good alternative to precise technical diagramming if you are doing problem-solving.
  • Management oversight and control.
    Reports and metrics drive much of the traditional documentation in an organization.  Simplifying reports and metrics often leads to a clearer picture of what is going on, reduces the opportunities to “game” the system, and always results in lower levels of documentation.  As well, some reports and metrics can be generated 100% through automated means.  All that said, the fundamental premise in the Agile manifesto is that management should base decisions on what is actually built – the “Working software” by looking at it and using it.
  • Various aspects of individual accountability.
    If you really need this, a good version control system can give you the information for this.  Sign-offs and other types of accountability documentation are typically just waste that doesn’t actually help in process improvement.  Most people who are in high-compliance environments already have licenses and/or security clearances that provide this accountability.  If you software is working, however, then this isn’t even a concern as trust is built and bureaucracy can be reduced.

In my recent training programs as research for this article, I have surveyed over 100 people on one aspect of documentation – code documentation.  Every individual surveyed is either currently coding or has a coding background, and every single person had the same answer to a simple scenario question:

Imagine that you have just joined a new organization and you are about to start working as a software developer.  One of the existing team members comes up to you and introduces himself.  He has with him a piece of paper with a complicated-looking diagram and a full binder that looks to be holding about 250 pages.  He asks you, “you need to get up to speed quickly on our existing system – we’re starting you coding tomorrow – would you prefer to go over the architecture diagram with me or would you prefer to review the detailed specifications and design documents.” He indicates the one-page diagram and the binder respectively.  Which would you prefer?

(I’ve put up a Survey Monkey one-question survey: Code Documentation Preference to extend the reach of this question.  It should take you all of 60 seconds to do it.  I’ll post results when I write the next Agile Manifesto essay in a month or two.)

The fact that everyone answers the same way is interesting.  What is even more interesting to me is that if you think through this scenario, it is actually almost the worst-case scenario where you might want documentation for your developers.  That means that in “better” cases where documentation for developers may not be as urgent or necessary, then the approach of just going to talk with someone is a lot better.

Documentation and Maps

The problem with documentation is the same problem we have with maps: “the map is not the territory” (quote from the wisdom of my father, Garry Berteig).  We sometimes forget this simple idea.  When we look at, say, Google Maps, we always have in the back of our consciousness that the map is just a guide and it is not a guarantee.  We know that if we arrive at a place, we will see the richness of the real world, not the simplified lines and colours of a map.  We don’t consider maps as legally binding contracts (usually).  We use maps to orient ourselves… as we look around at our reality.  We can share directions using maps, but we don’t share purpose or problems with maps.  And finally, maps assume that physical reality is changing relatively slowly (even Google Maps).

Many times when we create documentation in organizations, however, we get confused about the map versus the territory.

Agility and Documentation

Of course, code is a funny thing: all code is documentation too.  The code is not the behaviour.  But in software, code (e.g. Java, ASM, Scheme, Prolog, Python, etc.) is as close as possible to the perfect map.  Software is (mostly) deterministic.  Software (mostly) doesn’t change itself.  Software (mostly) runs in a state absent from in-place human changes to that software.  Software (mostly) runs on a system (virtual or physical) that has stable characteristics.  The code we write is a map.  From this perspective, documentation becomes even less important if we have people that already understand the language(s)/platform(s) deeply.


This essay is a continuation of my series on the Agile Manifesto.  The previous two essays are “Value and Values” and “Individuals and Interactions over Processes and Tools“.

 

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

An Evolution of the Scrum of Scrums

This is the story of how the Scrum of Scrums has evolved for a large program I’m helping out with at one of our clients.

Scrum of Scrum photo Continue reading An Evolution of the Scrum of Scrums

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

Scrum Data Warehouse Project

May people have concerns about the possibility of using Scrum or other Agile methods on large projects that don’t directly involve software development.  Data warehousing projects are commonly brought up as examples where, just maybe, Scrum wouldn’t work.
I have worked as a coach on a couple of such projects.  Here is a brief description of how it worked (both the good and the bad) on one such project:
The project was a data warehouse migration from Oracle to Teradata.  The organization had about 30 people allocated to the project.  Before adopting Scrum, they had done a bunch of up-front analysis work.  This analysis work resulted in a dependency map among approximately 25,000 tables, views and ETL scripts.  The dependency map was stored in an MS Access DB (!).  When I arrived as the coach, there was an expectation that the work would be done according to dependencies and that the “team” would just follow that sequence.
I learned about this all in the first week as I was doing boot-camp style training on Scrum and Agile with the team and helping them to prepare for their first Sprint.
I decided to challenge the assumption about working based on dependencies.  I spoke with the Product Owner about the possible ways to order the work based on value.  We spoke about a few factors including:
  • retiring Oracle data warehouse licenses / servers,
  • retiring disk space / hardware,
  • and saving CPU time with new hardware
The Product Owner started to work on getting metrics for these three factors.  He was able to find that the data was available through some instrumentation that could be implemented quickly so we did this.  It took about a week to get initial data from the instrumentation.
In the meantime, the Scrum teams (4 of them) started their Sprints working on the basis of the dependency analysis.  I “fought” with them to address the technical challenges of allowing the Product Owner to work on the migration in order based more on value – to break the dependencies with a technical solution.  We discussed the underlying technologies for the ETL which included bash scripts, AbInitio and a few other technologies.  We also worked on problems related to deploying every Sprint including getting approval from the organization’s architectural review board on a Sprint-by-Sprint basis.  We also had the teams moved a few times until an ideal team workspace was found.
After the Product Owner found the data, we sorted (ordered) the MS Access DB by business value.  This involved a fairly simple calculation based primarily on disk space and CPU time associated with each item in the DB.  This database of 25000 items became the Product Backlog.  I started to insist to the teams that they work based on this order, but there was extreme resistance from the technical leads.  This led to a few weeks of arguing around whiteboards about the underlying data warehouse ETL technology.  Fundamentally, I wanted to the teams to treat the data warehouse tables as the PBIs and have both Oracle and Teradata running simultaneously (in production) with updates every Sprint for migrating data between the two platforms.  The Technical team kept insisting this was impossible.  I didn’t believe them.  Frankly, I rarely believe a technical team when they claim “technical dependencies” as a reason for doing things in a particular order.
Finally, after a total of 4 Sprints of 3 weeks each, we finally had a breakthrough.  In a one-on-one meeting, the most senior tech lead admitted to me that what I was arguing was actually possible, but that the technical people didn’t want to do it that way because it would require them to touch many of the ETL scripts multiple times – they wanted to avoid re-work.  I was (internally) furious due to the wasted time, but I controlled my feelings and asked if it would be okay if I brought the Product Owner into the discussion.  The tech lead allowed it and we had the conversation again with the PO present.  The tech lead admitted that breaking the dependencies was possible and explained how it could lead to the teams touching ETL scripts more than once.  The PO basically said: “awesome!  Next Sprint we’re doing tables ordered by business value.”
A couple Sprints later, the first of 5 Oracle licenses was retired, and the 2-year $20M project was a success, with nearly every Sprint going into production and with Oracle and Teradata running simultaneously until the last Oracle license was retired.  Although I don’t remember the financial details anymore, the savings were huge due to the early delivery of value.  The apprentice coach there went on to become a well-known coach at this organization and still is a huge Agile advocate 10 years later!
Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

Enterprise Agility – Pragmatic or Transformative – Presentation to PMI South Western Ontario Chapter

Last night I had the honour of giving a talk at the PMI-SWOC. It seemed well received and I really enjoyed the opportunity. The slides from the talk are attached to this post.

20141202 PMI SWO Chapter – The Agile Enterprise [PDF]

There were quite a few people in attendance who were new to Agile and I spent a bit of time talking about the Agile Framework before really getting into the slides of my talk.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

Announcement: PMI Chapter Talk – The Agile Enterprise

On Tuesday Dec. 2, Mishkin Berteig will be speaking about The Agile Enterprise and the five different approaches to implementing Agile at the enterprise level.  The talk will also include some details about two frameworks used at the enterprise level: SAFe (Scaled Agile Framework) and RAP (Real Agility Program).

This talk is hosted by the South Western Ontario chapter of the PMI.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

The Three Fundamental Principles of Agile Estimation – The Third One Will Surprise You!

You probably already use an Agile Estimation technique such as the Planning Game or the Bucket System, but surprisingly few people understand the underlying principles of Agile Estimation.  This lack of understanding often causes confusion or stress for the people who try to use Agile Estimation techniques.  The discrepancy between traditional estimation techniques and Agile techniques is large and it is hard to bridge that mental gap without understanding the principles involved.  There are three fundamental principles of Agile estimation:

Principle One: Collaborative

Agile estimation methods are collaborative.  This means that multiple people work together to arrive at estimates for work in an Agile project or product development effort.  Traditional estimation techniques (such as those related to bottom-up or top-down) tend to focus on individuals estimating the work that they are responsible for doing and “trusting” those individual estimates.  Collaborative estimation means that most estimation is done by people in formally facilitated meetings where people are present in-person.

Collaborative techniques are generally used where there is some expectation that multiple minds are better than a single mind in discovering some new knowledge or solving a problem.  Teamwork and groupwork are based on this concept.  This idea of collaboration for problem solving is also applied to Agile Estimation and it has some interesting ramifications.

The most radical consequence of collaborative estimation methods is that there is no possibility to trace a particular estimate for a particular item to a particular individual person.  This lack of traceability is important to create a sense of safety on the part of participants in the estimation effort.  This safety is necessary to allow participants to be fully honest about estimates even if those estimates conflict with expectations of powerful stakeholders.  Another way of stating this principle is that no individual can be punished for a bad estimate.

Many Agile estimation techniques take this principle beyond just mere collaboration to the level of consensus-building techniques where everyone in a group doing estimation work must agree on the final estimate for each and every item being estimated.  This strengthens the idea of safety to the point where no participant in an estimation effort can ever say “I didn’t agree with that” and thereby leave other participants “on the hook” for a bad estimate.

Principle Two: Relative Estimation

Imagine you are shown a glass bottle with some water in it.  You can see the water sloshing around.  Someone asks you, “how much water is in the bottle?”  You might, at first, think about the overall size of the bottle and respond by saying “it’s 1/3 full.”  Or, if asked to provide a measure in units like millilitres or fluid ounces, you might mentally compare what you see in front of you to some container (e.g. a measuring cup) where you know the quantity.  In both cases, you are doing a relative estimate of the amount of water in the bottle.  You are comparing the amount of water to a known quantity.

Imagine a counter-example: someone walks up to you with a red pen ask asks you “what is the wavelength of the light being reflected from this red ink?”  If you are like most people, you have probably forgotten (if you ever knew) the wavelengths of light in the visible spectrum.  You have no basis for comparison.  You might take a wild guess, but it is just that.  Going back to our relative measure, you might be able to easily say if it is darker or lighter than another red colour, or you might even be able to tell what hue of red it is.  But those cases are, again, relying on our inherent ability to see relative differences.

So instead of ignoring this capacity, in Agile estimation techniques, we leverage it.  When estimating effort, we start by setting a clear baseline for what we are comparing: another piece of work.  The baseline piece of work is often given an “estimate” that is arbitrary and in some non-standard units.  For example, it is common to use “points” when estimating the effort for Product Backlog Items.

When doing relative estimates it is very important to ensure we are comparing “apples to apples”.  Both the piece of work to be estimated and the comparison piece should both be work items that are not yet done!  If you have already completed one of the pieces of work, you have prior knowledge that you don’t have for the work to be estimated.

This last point is subtle, but important.  If you have already done something, you know much more about it.  If you try to compare to something you haven’t yet done, you will be tempted to assume that the two things will be more similar than they may be when you actually get to work on it.  By comparing two pieces of work that you have not yet done, you become much more conscious of the risks of comparing, and that consciousness will help you make better relative estimates.

It is important to note that one side advantage of using relative units for estimation is that it makes it much more difficult to use estimates as a baseline for either measuring performance or for tracking schedule variance, both of which are essentially meaningless in a good Agile environment (which should be almost entirely results-oriented).

Principle Three: Fast

In Agile estimation we don’t care (!!!) about accuracy nor about precision of estimates.  Whoa!  Why is that?  Because estimation is waste.  You can’t sell estimates, and estimates don’t affect the “form, fit or function” of the thing you’re building.  Therefore, both Agile and Lean concur: do your utmost to eliminate that waste.  There are actually lots of Agile practitioners who think estimates are evil (and they have some good arguments!)

In order to do Agile estimation in a maximally non-evil way, we need to make estimation fast!  Really fast!!!  Many of the Agile estimation techniques allow you to estimate a product release schedule lasting as much as a year in just a few hours given a reasonably well-crafted Product Backlog.

There are really only two modestly good reasons for doing estimation in an Agile project or product:

  1. Estimates provide simplified information to the Product Owner to allow him/her to make sure the Product Backlog is ready for the next Sprint (ordered, refined).
  2. Estimates allow stakeholders, including the team doing the work, to generate high-level common understanding and expectations without dwelling on details.

As a business stakeholder, one can do a simple mental exercise.  Ask yourself, “how much money would I be willing to spend to accomplish those two objectives?”  Whatever your answer, I hope that it is a very small amount compared to what you are willing to spend on getting results.  If not, perhaps you haven’t really embraced the Agile mindset yet where “the primary measure of progress is working software” (the Agile Manifesto).

Bonus Principle: We Suck at Estimating

Most people doing estimation in traditional project management try to measure in units like person-days or dollars.  There is no doubt that these units are useful if you can get good estimates.  However, most of the people doing estimation are fundamentally and irredeemably bad at it.  How do I know?  Because they are not wealthy… and have thereby proven that they cannot predict the future.  If you can predict the future, even just in limited circumstances (like estimating effort or revenue), you can leverage that to generate almost untold wealth.  Given that, it is fruitless and wasteful to try to get better at estimating.  Instead, the principles of Agile estimation help us focus our attention on the right things: collaboration, comparative estimates and doing them fast so we can get to the real work, and most importantly: delivering valuable results now.  Understanding these principles helps teams overcome many of the struggles they have in using Agile estimation techniques.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

What is Scrum good for?

I have worked with a lot of people, teams and organizations over the last 8 years helping them to adopt Scrum and I have seen some interesting patterns about where Scrum works well and where it doesn’t work so well. I wanted to share my observations to see if they correlate with what other people are experiencing.

So first off, I want to describe what I mean by Scrum working well:

  1. Teams using Scrum are obviously high-performance teams whose business results are at least 4x that of normal teams.
  2. The organization in which Scrum is being used experiences a change in culture to become more team oriented, more value oriented, and more customer oriented.

So now I can describe where I have observed Scrum to work really well:

  1. When an organization (or team) is in deep trouble and willing to admit it adopting Scrum seems to be a catalyst for creating a new culture, process and team environment where getting out of trouble is possible. This is Scrum for Crisis. The “willing to admit it” part is extremely important as I have worked with two organizations where the “deep trouble” part was obvious to me as an external person, but in both cases management and staff did not seem willing to admit the depth of their crisis and in both cases Scrum failed to act as a catalyst to get them out of trouble. In this use of Scrum, sometimes resolving the crisis then leads back to complacency and Scrum fades away.
  2. Small growing organizations that have no existing formal processes for development can use Scrum as an effective way to maintain their high-performance without getting burdened in bureaucracy. In this case, it is important to note that they are _already_ in a high performance state and their struggle is to maintain that while at the same time growing. I’ve worked with quite a number of small organizations where all they need is the CSM (plus maybe one or two days of coaching) to adopt and maintain Scrum. I have also worked with small organizations that were _not_ already high performance and Scrum has not typically worked to bump them up to a high-performance state.
  3. Pure new product development where a single strong Product Owner can be identified who has the authority to make product decisions independently of anyone else (including product budget decisions). By “pure new product development” I mean that neither the individual team members nor the team as a whole have any responsibilities outside of the product work – there is no “fractional allocation” or “resource levelling” across projects or products. The strong Product Owner is critical to success with Scrum and must understand the principles of Scrum as well as the mechanics of being a Product Owner.

I have also seen Scrum be inappropriate and not lead to the results I mentioned above:

  1. Management teams. It seems like Scrum could or should work for management teams, but it appears that managers have too much of the following problems to be able to use Scrum:
    – operational responsibilities (non-creative, non-problem-solving work)
    – urgent, legitimate interruptions (e.g. an escalated customer issue)
    – real commitments to events or projects that are calendar based (e.g. a management off-site)
    – ego: they don’t want to follow an apparently rigid process or they are always happy to make exceptions for themselves
    Again, one might imagine that Scrum _should_ work to help resolve these issues, but unfortunately I have never seen it able to do so in this context.
  2. Small teams/projects. Scrum is too heavy for teams less than 5 people or for projects shorter than 2 months in total duration. Those numbers aren’t meant to be hard and fast, but when I’ve seen small teams/projects attempt to do Scrum they _always_ end up breaking lots of the rules partly because they can and partly because they must. That said, some folks have created “Personal Scrum” and other variants. I’m not sure if we as the Scrum Alliance officially recognize/endorse those variants.
  3. Purely operational work. There just isn’t enough creativity/problem-solving to make the Sprint an appropriate process element, nor the Product Backlog an appropriate organizing mechanism. I have seen some operational environments get some benefit from doing regular retrospectives, but just doing retrospectives is not Scrum in my book. My experience with Kanban is still a little limited, but it seems to be an appropriate approach for these environments.
  4. Organizations where there is very little need to change. I’ve spent some time working with big profitable banks to adopt agile and without exception, they just can’t wrap their minds around the need for Scrum… because they are already so successful as a business. The general attitude is that Scrum is popular therefore we will call what we are doing “Scrum”, but it really isn’t. It’s Scrummerfall and Scrum-Butt wrapped up in the terminology of Scrum. They will adopt some Agile practices and get very modest benefits. I have seen minor improvements in team morale and minor improvements in quality and productivity, but certainly not anything near to what is possible for improvements. When we do assessments in this type of environment, we see Value Stream maps with waste at the 80-90% level so there is huge room for improvement… but it just doesn’t happen.

Scrum can definitely transform the world of product development. It can definitely act as a catalyst to get teams and organizations out of crisis. But that isn’t the whole world of work. I’m also concerned about the idea of using Scrum for general project management. There might be some good practices that are part of Scrum that would also be valuable in general project management (e.g. regular retrospectives, daily team meetings) but that doesn’t make Scrum a general project management framework.

I don’t claim that any of the above observations are “correct”. That’s partly why I am sharing – I would love to have a good discussion here about this because I think it is critical for us as Agilists to be able to answer this question well when we are asked: “what is Scrum good for?” – particularly since Scrum is by far the most popular Agile method.

I would love to hear other people’s observations about where Scrum works well (as I have defined “well” above) vs. where it either is only a modest improvement to existing approaches or where it might even not work at all.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

AgilePM + Scrum(CSM) + OpenAgile + Kanban training in Markham Sept. 7 & 8

We have an upcoming three-day agile training seminar in Markham September 7 & 8, 2011.

In this unique seminar, we will be offering a practical view of three important Agile methods: OpenAgile – used for general agile project management and agile teamwork including projects and organizations doing any kind of work. Scrum – used for software new product development and IT project management. Kanban – used for teams doing operational work.

This seminar contributes towards three certification programs: the Scrum Alliance’s Certified ScrumMaster program, the OpenAgile Team Member level and the IPMA/PMAC Agile Project Management certification.

To register: http://www.berteigconsulting.com/UpcomingAgileScrumOpenAgileSeminars

Proudly delivered by Berteig Consulting, a Canadian organization since 2004.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

AgilePM+Scrum(CSM)+OpenAgile+Kanban training in Toronto, August 22-24

We have an upcoming three-day agile training seminar in Toronto, August 22-24, 2011.

In this unique seminar, we will be offering a practical view of three important Agile methods: OpenAgile – used for general agile project management and agile teamwork including projects and organizations doing any kind of work. Scrum – used for software new product development and IT project management. Kanban – used for teams doing operational work.

This seminar contributes towards three certification programs: the Scrum Alliance’s Certified ScrumMaster program, the OpenAgile Team Member level and the IPMA/PMAC Agile Project Management certification.

To register: http://www.berteigconsulting.com/UpcomingAgileScrumOpenAgileSeminars

Proudly delivered by Berteig Consulting, a Canadian organization since 2004.

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

AgilePM+Scrum(CSM)+OpenAgile+Kanban training in London, Sept. 7-9

This 3-day training covers 3 Agile methods: ScrumMaster (the most popular Agile method), OpenAgile (the most widely applicable Agile method), and Kanban (a method that can be used together with other Agile methods).

To register: http://www.berteigconsulting.com/UpcomingAgileScrumOpenAgileSeminars

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

The BIG Difference Between Agile Teams and Project Teams

Last week I ran an Agile Coach Training session in-house for a large Canadian organization. It was just myself and five other participants. We were discussing possible things to do if there is a person on an agile team who is not able to work effectively in that sort of environment. One “intervention” we discussed was to “Assign Work”…

Pause

Now hopefully everyone who just read that phrase “Assign Work” had all sorts of alarm bells go off in their head!!!!

As an Agile coach, I would only do this under extreme circumstances. And I would always be fully aware of the consequence of assigning work. I would be removing that person from the team by Assigning Work. A person is not in an Agile (self-organizing) team if they need to be assigned work.

Guess what?!? THAT is the BIG difference between Agile Teams and Project (or Functional) Teams.

On a project, the Project Manager gets someone onto the team by assigning them work!!!!

On an Agile Team, a person is removed from the team by assigning them work.

Have you seen this happen? How did the assignee feel?

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail

Report average velocity and fail 50% of the time

The question of “expected velocity” and long-term planning has come up at more than one client. A recent client conversation got me thinking, however, questioning how to interpret velocity when estimating and plotting a roadmap based on a current backlog of features. Assume, for a moment, a backlog of story-pointed features, and 10 good iterations (consistent team, no odd occurrences that would affect velocity). Mathematically average velocity (well, a mean really) is a 50/50 proposition for any subsequent iteration. Some organizations don’t find this level of confidence acceptable. What velocity should be reported as expected for iteration/sprint planning and roadmap forecasting, and how should it be used?

Context

Interpreting velocity, before anything else, requires some context. An agile organization that sees estimates as hypothetical might find this article is of less use. In fact, a good question is whether estimation is even a value-added activity. For this post assume an organization that sees strong value in estimation and planning.

Culture

The biggest piece of context is to know the organizational culture. This is important in two respects, and both of these cultural factors are important because they impact how Velocity is understood within the organization.

What is Failure?

First is the meaning of failure in the organization. Is failure to deliver what was committed to by the planned date considered a failure of the team, or is it simply a fact to be understood and accounted for in future planning? Even in Agile organizations, the former is often true and a hard habit to break. If not delivering to expectations is considered failure and has negative consequences, then that means that estimation is being treated not as estimation, but as prediction and contract. Velocity is therefore a commitment, and should therefore be used conservatively.

Consistency or Speed?

The second item to know is whether consistency and predictability of delivery is of a higher strategic value than the actual rate of delivery. This is often un-stated. Usually people want fast and consistent delivery. The truth is that you can get consistent, or fast software development, or a balance between the two. Lack of trust is usually a strong motivation to encourage consistency over speed, or a history of quality problems, etc. In this case, as well, Velocity is more of a boundary than an indicator.

Emotional Loading in Estimation (or why not Low-ball?)

If estimation is seen as binding, contractual, or limiting, then additional emotions get overloaded. Trust, promise, and betrayal are words used in such organizational cultures. Distrust is usually a strong factor, especially between silos (business vs. technology, company vs. project management vs. customer, etc.). So when people are asked to give estimates, even using agile-friendly mechanisms such as story points, there is usually a process of cementing that estimate into a part of an accountability model, so estimates start to get conservative. People are then accused of low-balling, others are accused of irrational expectations… we’ve all seen this. The language clearly becomes one of contention and blame. Even the term low-balling is often an outright pejorative term for estimating too conservatively.

This doesn’t happen only in agile environments, and project managers in traditional PMBOK frameworks have long factored risk into “contingency budgets”. Interestingly, however, if a Project Manager were to factor risk into the task estimates, they’d be “low-balling capacity,” yet if they were to factor it out and layer it on top of the project work, it’s “contingency budgeting” (At least in a few experiences I’ve had). Either way, someone’s adding a factor for uncertainty, based on the need to predict conservatively or liberally or somewhere in between.

That’s the point of the article: how can Agile projects use velocity to estimate as conservatively (or liberally) as is appropriate?

An average is a 50% chance to succeed (or fail)

Velocity is not a constant. It’s a set of instantaneous values on a curve, with instances being iterations. That means that it varies, and is therefore only meaningful statistically. So how do you reasonably use velocity statistically, and improve confidence? One way is to stop delivering against “average” velocity.

A lot of coaches use average velocity over the previous N iterations. This is not helpful for all sorts of reasons, if estimation is a commitment. By definition, average (well, actually a mean, but they’re close) is a 50/50 proposition. If you report the average team velocity (assuming it’s accurate), then about half the time the team will be under and about half the time the team will be over, statistically. So basically an average is a crap shoot, when taken in any given instance. It’s can only be good in the long run. For this to work, the long-haul has to include permission to fail and a lot of trust. Teams need to be able to go miss dates but will sometimes exceed dates and it should all wash out in the end. In organizations such as I’m describing, that trust isn’t there, so. Additionally, if the language of commitment is around meeting instantaneous iteration commitments (as opposed to delivering high-quality customer value as quickly as is sustain-ably possible) then you aren’t playing the long-game, you’re playing a very short-game.

Simulate Velocity, not work

In a PMI training course I took when I was at Sun Microsystems, we were nicely informed that two point estimates of tasks are a perfect way to fail half the time, per the above logic. One point estimates are just idiotic. Three point estimates were better. We simulated with a monte-carlo algorithm and found a curve and a distribution, and then determined a confidence level yadda yadda. Well, we’re trying to avoid wasting a lot of time estimating up-front, but one way to start representing velocity properly is to do the same kind of statistical modelling done in traditional product management, only simulate velocity, not work items.

In this approach, you take the last N iterations (say 10). Determine the maximum velocity (optimistic) and the minimum velocity (pessimistic), and then the mode (the velocity value that seems to occur most frequently). Then you do monte-carlo simulation so you get a statistical pattern. Now, you actually can determine an answer based on confidence. If you want to be right with an 80% confidence, you pick a velocity where 80% of the simulated runs were successful. (Note – there are a paucity of excel templates to do this math automatically, and often they are for sale. It would be nice to have a few functions with arbitrary distributions based on min-max-mode to help this along.)

It’s not perfect, and it’s a potentially huge amount of administrative overhead. Elsewhere I’ve referenced blogs that entirely oppose any estimation at all, but if you are gong to, then working statistically with simulation is the only way to take small sample numbers meaningful.

Commitment Velocity: Low-Ball as a policy.

Another approach, one perhaps controversial, but taught by some Scrum trainers is to pick the lowest historical delivered velocity. This is a commitment-based approach, on the assumption that building trust around consistent delivery is critical to building sound relationships where product owners and teams can safely state their needs and get things done with a minimum of contractual behaviour. By taking the minimum, you force a low-ball capacity, which means you can have high-confidence of success after a few iterations. You have, likely, after a while, some spare time on your hands. Teams can then choose to pull more work in (without adjusting their commitment velocity), work on “technical debt”, improve their skills, etc. A team could raise their commitment velocity in certain inflection points in the project. A new team member is added that provides a necessary skill not previously available, and after a few iterations the team is consistently hitting a higher number, but this is a careful process to ensure that they are committing, and if they don’t make their new number, it goes down to what they got accomplished.

Indemnify teams’ learning

An arguably healthier option, if you have built enough trust, is to simply indemnify a team from failing to meet the estimate. Since you’re doing mathematics on actuals to generate an expected future number, everyone can acknowledge that past behaviour is no guarantee of future behaviour, and simply use it for capacity planning. In this case, estimation is actually estimation, not commitment or contract. The team is expected to be ahead sometimes, and behind sometimes. The upside of this is that a lot of extra time isn’t spent playing with fictional numbers. Teams are spending their efforts on delivery as quickly-yet-sustain-ably as they can, and the organization treats them as trusted professionals in this. The temptation to assume you can predict the future is seen as folly, and the estimates are used to guide overall direction, not to make outward customer commitments.

Don’t be mindless

There may be other approaches, I’m sure. The agile community is certainly not short of people who love this topic and can talk for hours on “proper” estimation. The point of this post is merely to point out some options, and ask you to look at your organizational culture, team culture, customer culture, the meaning of terms like commitment, failure, success, consistency, speed, etc. As you understand the culture, balance consistency vs. speed, trust, and other factors to choose a method of estimation that meets your goals. Don’t do estimation based on your own, internal cultural assumptions, as you may have developed or been taught techniques that are useful when and where they were taught, but may no longer be so. Or maybe they weren’t so useful then either. Regardless, this because estimation cuts at the heart of the dialogue between producer and consumer, and establishes parameters for that discussion, it’s critical that you think your choice through.

[Christian also blogs at http://www.geekinasuit.com/]

Try out our Virtual Scrum Coach with the Scrum Team Assessment tool - just $500 for a team to get targeted advice and great how-to information

Please share!
facebooktwittergoogle_plusredditpinterestlinkedinmailfacebooktwittergoogle_plusredditpinterestlinkedinmail