On many occasions, I have observed “Scrum Masters” and even “Product Owners” attempting to drive what they understand to be the Daily Scrum. Just this morning, I witnessed a “Daily Scrum” in which a “Product Owner” gave the team a bunch of program updates and made sure that each team member had tasks to work on for the day. Then, the PO “wrapped up” the meeting and left the team to get to the work. I then stayed and observed what the team did next. They actually stayed together to discuss the work and figure out how they were going to organize themselves for the day. I then went over to the Product Owner and whispered in her ear that the team was now doing the real Daily Scrum. She said “Oh,” and promptly walked over to find out what was going on. I then observed her from a distance nodding her head several times while appearing to understand what the team was talking about. I’m not sure if she understood or not, but that’s irrelevant. The point is that the Daily Scrum is for the Development Team to inspect and adapt its progress towards the Sprint Goal and decide how it will self-organize for the coming day. If the Development Team decides as a result of the Daily Scrum that it needs to re-negotiate any previously forcasted functionality with the Product Owner, then that conversation can certainly happen at that time.
Regular big up-front planning is not necessary with Scrum. Instead, a team can just get started and use constant feedback in the Sprint Review to adjust it’s plans. Even the Product Backlog can be created after the first Sprint has started. All that is really necessary to get started is a Scrum Team, a product vision, and a decision on Sprint length. In this extreme case, the Scrum Team itself would decide what to build in its first Sprint and use the time of the Sprint to also prepare some initial Product Backlog Items. Then, the first Sprint Review would allow stakeholders to provide feedback and further develop the Product Backlog. The empirical nature of Scrum could even allow the Product Owner to emerge from the business stakeholders, rather than being assigned to the team right from the start.
Starting a Sprint without a Product Backlog is not easy, but it can be done. The team has to know at least a little about the business, and there should be some (possibly informal) project or product charter that they are aware of. The team uses this super basic information and decides on their own what to build in their first Sprint. Again, the focus should be on getting something that can be demoed (and potentially shippable). The team is likely to build some good stuff and some things that are completely wrong… but the point is to get the Inspect and Adapt cycle started as quickly as possible. Which means of course that they need to have stakeholders (customers, users) actually attend the demo at the end of the Sprint. The Product Owner may or may not even be involved in this first Sprint.
One important reason this is sometimes a good approach is the culture of “analysis paralysis” that exists in some organizations. In this situation, an organization is unable to do anything because they are so concerned about getting things right. Scrum is a framework for inspect and adapt and that can (and does) include the Product Backlog. Is it better for a team to sit idle while someone tries to do sufficient preparation? Or is it better to get started and inspect and adapt? This is actually a philosophical question (as well as a practical question). The mindset and philosophy of the Agile Manifesto and Scrum is that trying to produce valuable software is more important that documentation… that individuals and how they work together is more important than rigidly following a process or tool. I will agree that in many cases it is acceptable to do some up-front work, but it should be minimized, particularly when it is preventing people from starting to deliver value. The case of a team getting started without a product backlog is rare… but it can be a great way for a team to help an organization overcome analysis paralysis.
The Agile Manifesto is very clear: “The BEST architectures, requirements and designs emerge out of self-organizing teams.” [Emphasis added.]
Hugely memorable for me is the story that Ken Schwaber told in the CSM course that I took from him in 2003. This is a paraphrase of that story:
I [Ken Schwaber] was talking to the CIO of a large IT organization. The CIO told me that his projects last twelve to eighteen months and at the end, he doesn’t get what he needs. I told him, “Scrum can give you what you don’t need in a month.”
I experienced this myself in a profound way just a couple years into my career as an Agile coach and trainer. I was working with a department of a large technology organization. They had over one hundred people who had been working on Agile pilot projects. The department was responsible for a major product and executive management had approved a complete re-write. The product managers and Product Owners had done a lot of work to prepare a product backlog (about 400 items!) that represented all the existing functionality of the product that needed to be re-written. But, the big question, “what new technology platform do we use for the re-write?” had not yet been resolved. The small team of architects were tasked with making this decision. But they got stuck. They got stuck for three months. Finally, the director of the department, who had learned to trust my advice in other circumstances, asked me, “does Scrum have any techniques for making these kind of architectural decisions?”
I said, “yes, but you probably won’t like what Scrum recommends!”
She said, “actually, we’re pretty desperate. I’ve got over a hundred people effectively sitting idle for the last three months. What does Scrum recommend?”
“Just start. Let the teams figure out the platform as they try to implement functionality.”
She thought for a few seconds. Finally she said, “okay. Come by this Monday and help me launch our first Sprint.”
The amazing thing was that the teams didn’t lynch me when on Monday she announced that “our Agile consultant says we don’t need to know our platform in order to get started.”
The first Sprint (two weeks long) was pretty chaotic. But, with some coaching and active support of management, they actually delivered a working increment of their product. And decided on the platform to use for the rest of the two-year project.
You must trust your team.
If your organization is spending more than a few days preparing for the start of a project, it is probably suffering from this pitfall. This is the source of great waste and lost opportunity. Use Scrum to rapidly converge on the correct solutions to your business problems instead of wasting person-years of time on analysis and planning. We can help with training and coaching to give you the tools to start fast using Scrum and to fix your Scrum implementation.
May people have concerns about the possibility of using Scrum or other Agile methods on large projects that don’t directly involve software development. Data warehousing projects are commonly brought up as examples where, just maybe, Scrum wouldn’t work.
I have worked as a coach on a couple of such projects. Here is a brief description of how it worked (both the good and the bad) on one such project:
The project was a data warehouse migration from Oracle to Teradata. The organization had about 30 people allocated to the project. Before adopting Scrum, they had done a bunch of up-front analysis work. This analysis work resulted in a dependency map among approximately 25,000 tables, views and ETL scripts. The dependency map was stored in an MS Access DB (!). When I arrived as the coach, there was an expectation that the work would be done according to dependencies and that the “team” would just follow that sequence.
I learned about this all in the first week as I was doing boot-camp style training on Scrum and Agile with the team and helping them to prepare for their first Sprint.
I decided to challenge the assumption about working based on dependencies. I spoke with the Product Owner about the possible ways to order the work based on value. We spoke about a few factors including:
retiring Oracle data warehouse licenses / servers,
retiring disk space / hardware,
and saving CPU time with new hardware
The Product Owner started to work on getting metrics for these three factors. He was able to find that the data was available through some instrumentation that could be implemented quickly so we did this. It took about a week to get initial data from the instrumentation.
In the meantime, the Scrum teams (4 of them) started their Sprints working on the basis of the dependency analysis. I “fought” with them to address the technical challenges of allowing the Product Owner to work on the migration in order based more on value – to break the dependencies with a technical solution. We discussed the underlying technologies for the ETL which included bash scripts, AbInitio and a few other technologies. We also worked on problems related to deploying every Sprint including getting approval from the organization’s architectural review board on a Sprint-by-Sprint basis. We also had the teams moved a few times until an ideal team workspace was found.
After the Product Owner found the data, we sorted (ordered) the MS Access DB by business value. This involved a fairly simple calculation based primarily on disk space and CPU time associated with each item in the DB. This database of 25000 items became the Product Backlog. I started to insist to the teams that they work based on this order, but there was extreme resistance from the technical leads. This led to a few weeks of arguing around whiteboards about the underlying data warehouse ETL technology. Fundamentally, I wanted to the teams to treat the data warehouse tables as the PBIs and have both Oracle and Teradata running simultaneously (in production) with updates every Sprint for migrating data between the two platforms. The Technical team kept insisting this was impossible. I didn’t believe them. Frankly, I rarely believe a technical team when they claim “technical dependencies” as a reason for doing things in a particular order.
Finally, after a total of 4 Sprints of 3 weeks each, we finally had a breakthrough. In a one-on-one meeting, the most senior tech lead admitted to me that what I was arguing was actually possible, but that the technical people didn’t want to do it that way because it would require them to touch many of the ETL scripts multiple times – they wanted to avoid re-work. I was (internally) furious due to the wasted time, but I controlled my feelings and asked if it would be okay if I brought the Product Owner into the discussion. The tech lead allowed it and we had the conversation again with the PO present. The tech lead admitted that breaking the dependencies was possible and explained how it could lead to the teams touching ETL scripts more than once. The PO basically said: “awesome! Next Sprint we’re doing tables ordered by business value.”
A couple Sprints later, the first of 5 Oracle licenses was retired, and the 2-year $20M project was a success, with nearly every Sprint going into production and with Oracle and Teradata running simultaneously until the last Oracle license was retired. Although I don’t remember the financial details anymore, the savings were huge due to the early delivery of value. The apprentice coach there went on to become a well-known coach at this organization and still is a huge Agile advocate 10 years later!
