Tag Archives: efficiency

Scrum Data Warehouse Project

Learn more about transforming people, process and culture with the Real Agility Program
May people have concerns about the possibility of using Scrum or other Agile methods on large projects that don’t directly involve software development.  Data warehousing projects are commonly brought up as examples where, just maybe, Scrum wouldn’t work.
I have worked as a coach on a couple of such projects.  Here is a brief description of how it worked (both the good and the bad) on one such project:
The project was a data warehouse migration from Oracle to Teradata.  The organization had about 30 people allocated to the project.  Before adopting Scrum, they had done a bunch of up-front analysis work.  This analysis work resulted in a dependency map among approximately 25,000 tables, views and ETL scripts.  The dependency map was stored in an MS Access DB (!).  When I arrived as the coach, there was an expectation that the work would be done according to dependencies and that the “team” would just follow that sequence.
I learned about this all in the first week as I was doing boot-camp style training on Scrum and Agile with the team and helping them to prepare for their first Sprint.
I decided to challenge the assumption about working based on dependencies.  I spoke with the Product Owner about the possible ways to order the work based on value.  We spoke about a few factors including:
  • retiring Oracle data warehouse licenses / servers,
  • retiring disk space / hardware,
  • and saving CPU time with new hardware
The Product Owner started to work on getting metrics for these three factors.  He was able to find that the data was available through some instrumentation that could be implemented quickly so we did this.  It took about a week to get initial data from the instrumentation.
In the meantime, the Scrum teams (4 of them) started their Sprints working on the basis of the dependency analysis.  I “fought” with them to address the technical challenges of allowing the Product Owner to work on the migration in order based more on value – to break the dependencies with a technical solution.  We discussed the underlying technologies for the ETL which included bash scripts, AbInitio and a few other technologies.  We also worked on problems related to deploying every Sprint including getting approval from the organization’s architectural review board on a Sprint-by-Sprint basis.  We also had the teams moved a few times until an ideal team workspace was found.
After the Product Owner found the data, we sorted (ordered) the MS Access DB by business value.  This involved a fairly simple calculation based primarily on disk space and CPU time associated with each item in the DB.  This database of 25000 items became the Product Backlog.  I started to insist to the teams that they work based on this order, but there was extreme resistance from the technical leads.  This led to a few weeks of arguing around whiteboards about the underlying data warehouse ETL technology.  Fundamentally, I wanted to the teams to treat the data warehouse tables as the PBIs and have both Oracle and Teradata running simultaneously (in production) with updates every Sprint for migrating data between the two platforms.  The Technical team kept insisting this was impossible.  I didn’t believe them.  Frankly, I rarely believe a technical team when they claim “technical dependencies” as a reason for doing things in a particular order.
Finally, after a total of 4 Sprints of 3 weeks each, we finally had a breakthrough.  In a one-on-one meeting, the most senior tech lead admitted to me that what I was arguing was actually possible, but that the technical people didn’t want to do it that way because it would require them to touch many of the ETL scripts multiple times – they wanted to avoid re-work.  I was (internally) furious due to the wasted time, but I controlled my feelings and asked if it would be okay if I brought the Product Owner into the discussion.  The tech lead allowed it and we had the conversation again with the PO present.  The tech lead admitted that breaking the dependencies was possible and explained how it could lead to the teams touching ETL scripts more than once.  The PO basically said: “awesome!  Next Sprint we’re doing tables ordered by business value.”
A couple Sprints later, the first of 5 Oracle licenses was retired, and the 2-year $20M project was a success, with nearly every Sprint going into production and with Oracle and Teradata running simultaneously until the last Oracle license was retired.  Although I don’t remember the financial details anymore, the savings were huge due to the early delivery of value.  The apprentice coach there went on to become a well-known coach at this organization and still is a huge Agile advocate 10 years later!

Please share!

New Video: Kanban in One Minute – Do It Right

Learn more about transforming people, process and culture with the Real Agility Program

Another short video for your viewing pleasure!  Kanban in One Minute – Do It Right by Michael Badali.


Please share!

Link: Short Article About Pair Programming

Learn more about transforming people, process and culture with the Real Agility Program

Pair Programming Economics by Olaf Lewitz describes three activities in programming: typing, problem-solving and reading code. How does pair programming help? By making the balance between those three activities better.

Please share!

Announcing: The Real Agility Program

Learn more about transforming people, process and culture with the Real Agility Program

Real Agility Program LogoThe Real Agility Program is an Enterprise Agile change program to help organizations develop high-performance teams, deliver amazing products, dramatically improve time to market and quality, and create work environments that are awesome for employees.

This article is a written summary of the Executive Briefing presentation available upon request from the Real Agility Program web site.  If you obtain the executive briefing, you can follow along with the article below and use it to present Real Agility to your enterprise stakeholders.

The Problem

At Berteig Consulting we have been working for 10 years to learn how to help organizations transform people, process and culture.  The problem is simple to state: there is a huge amount of opportunity waste and process waste in most normal enterprise-scale organizations.  If you have more than a couple hundred people in your organization, this almost certainly affects you.

We like to call this problem “the Bureaucratic Beast”.  The Bureaucratic Beast is a self-serving monster that seems to grow and grow and grow.  As it grows, this Beast makes it progressively more difficult for business leaders to innovate, respond to changes in the market, satisfy existing customers, and retain great employees.

Real Agility, a system to tame the Bureaucratic Beast, comes from our experience working with numerous enterprise Agile adoptions.  This experience, in turn, rests on the shoulders of giants like John Kotter (“Leading Change”), Edgar Schein (“The Corporate Culture Survival Guide”), Jim Collins (“Good to Great” and “Built to Last”), Mary Poppendieck (“Lean Software Development”) Jon Katzenbach (“The Wisdom of Teams”) and Frederick Brooks (“The Mythical Man-Month”).  Real Agility is designed to tame all the behaviours of the Bureaucratic Beast: inefficiency, dis-engaged staff, poor quality and slow time-to-market.

Studies have proven that Agile methods work in IT.  In 2012, the Standish Group observed that 42% of Agile projects succeed vs. just 14% of projects done with traditional “Bureaucratic Beast” methods.  Agile and associated techniques aren’t just for IT.  There is growing use of these same techniques in non-technoogy environments such as marketing, operations, sales, education, healthcare, and even heavy industry like mining.

Real Agility Basics: Agile + Lean

Real Agility is a combination of Agile and Lean; both systems used harmoniously throughout an enterprise.  Real Agility affects delivery processes by taking long-term goals and dividing them into short cycles of work that deliver valuable results rapidly while providing fast feedback on scope, quality and most importantly value.  Real Agility affects management processes by finding and eliminating wasteful activities with a system view.  And Real Agility affects human resources (people!) by creating “Delivery Teams” which have clear goals, are composed of multi-skilled people who self-organize, and are stable in membership over long periods of time.

There are lots of radical differences between Real Agility and traditional management (that led to the Bureaucratic Beast in the first place).  Real Agility prioritizes work by value instead of critical path, encourages self-organizing instead of command-and-control management, a team focus instead of project focus, evolving requirements instead of frozen requirements, skills-based interactions instead of roles-based interaction, continuous learning instead of crisis management, and many others.

Real Agility is built on a rich Agile and Lean ecosystem of values, principles and tools.  Examples include the Agile Manifesto, the “Stop the Line” practice, various retrospective techniques, methods and frameworks such as Scrum and OpenAgile, and various thinking tools compatible with the Agile – Lean ecosystem such as those developed by Edward de Bono (“Lateral Thinking”) and Genrich Altshuller (“TRIZ”).

Real Agility acknowledges that there are various approaches to Agile adoption at the enterprise level: Ad Hoc (not usually successful – Nortel tried this), Grassroots (e.g. Yahoo!), Pragmatic (SAFe and DAD fall into this category), Transformative (the best balance of speed of change and risk reduction – this is where the Real Agility Program falls), and Big-Bang (only used in situations of true desperation).

Why Choose Transformative?

One way to think about these five approaches to Agile adoption is to compare the magnitude of actual business results.  This is certainly the all-important bottom line.  But most businesses also consider risk (or certainty of results).  Ad-Hoc approaches to Agile adoption have poor business results and a very high level of risk.  Big-Bang approaches (changing a whole enterprise to Agile literally over night) often have truly stunning business results, but are also extremely high risk.  Grassroots, where leaders give staff a great deal of choice about how and when to adopt Agile, is a bit better in that the risk is lower, but the business results often take quite a while to manifest themselves.  Pragmatic approaches tend to be very low risk because they often accommodate the Bureaucratic Beast, but that also limits their business results to merely “good” and not great.  Transformative approaches which systematically address organizational culture are just a bit riskier than Pragmatic approaches, but the business results are generally outstanding.

More specifically, Pragmatic approaches such as SAFe (Scaled Agile Framework) are popular because they are designed to fit in with existing middle management structures (where the Bureaucratic Beast is most often found).  As a result, there is slow incremental change that typically has to be driven top-down from leadership.  Initial results are good, but modest.  And the long term?  These techniques haven’t been around long enough to know, but in theory it will take a long time to get to full organizational Agility.  Bottom line is that Pragmatic approaches are low risk but the results are modest.

Transformative approaches such as the Real Agility Program (there are others too) are less popular because there is significantly more disruption: the Bureaucratic Beast has to be completely tamed to serve a new master: business leadership!  Transformative approaches require top-to-bottom organizational and structural change.  They include a change in power relationships to allow for grassroots-driven change that is empowered by servant leaders.  Transformative approaches are moderate in some ways: they are systematic and they don’t require all change to be done overnight. Nevertheless, often great business results are obtained relatively quickly.  There is a moderate risk that the change won’t deliver the great results, but that moderate risk is usually worth taking.

Regardless of adoption strategy (Transformative or otherwise) there are a few critical success factors.  Truthfulness is the foundation because without it, it is impossible to see the whole picture including organizational culture.  And love is the strongest driver of change because cultural and behavioural change requires emotional commitment on the part of everyone.

Culture change is often challenging.  There are unexpected problems.  Two steps forward are often followed by one step back.  Some roadblocks to culture change will be surprisingly persistent.  Leaders need patience and persistence… and a systematic change program.

The Real Agility Program

The Real Agility Program has four tracks or lines of action (links take you to the Real Agility Program web site):

  1. Recommendations: consultants assess an organization and create a playbook that customizes the other tracks of the Real Agility Program as well as dealing with any important outliers.
  2. Execution: coaches help to launch project, product and operational Delivery Teams and Delivery Groups that learn the techniques of grassroots-driven continuous improvement.
  3. Accompaniment: trainer/coaches help you develop key staff into in-house Real Agility Coaches that learn to manage Delivery Groups for sustainable long-term efforts such as a product or line of business.
  4. Leadership: coaches help your executive team to drive strategic change for long-term results with an approach that helps executives lead by example for enterprise culture change.

Structurally an enterprise using Real Agility is organized into Delivery Groups.  A Delivery Group is composed of one or more Delivery Teams (up to 150 people) who work together to produce business results.  Key roles include a Business leader, a People leader and a Technology leader all of whom become Real Agility Coaches and take the place of traditional functional management.  As well, coordination across multiple Delivery Teams within a Delivery Group is done using an organized list of “Value Drivers” maintained by the Business leader and a supporting Business Leadership Group. Cross-team support is handled by a People and Technology Support Group co-led by the People and Technology leaders.  Depending on need there may also be a number of communities of practice for Delivery Team members to help spread learning.

At an organizational or enterprise level, the Leadership Team includes top executives from business, finance, technology, HR, operations and any other critical parts of the organization.  This Leadership Team communicates the importance of the changes that the Delivery Groups are going through.  They lead by example using techniques from Real Agility to execute organizational changes.  And, of course, they manage the accountability of the various Delivery Groups throughout the enterprise.

The results of using the Real Agility Program are usually exceptional.  Typical results include:

  • 20x improvement in quality
  • 10x improvement in speed to market
  • 5x improvement in process efficiency
  • and 60% improvement in employee retention.

Of course, these results depend on baseline measures and that key risk factors are properly managed by the Leadership Team.

Your Organization

Not every organization needs (or is ready for) the Real Agility Program.  Your organization is likely a good candidate if three or more of the following problems are true for your organization:

  • high operating costs
  • late project deliveries
  • poor quality in products or services
  • low stakeholder satisfaction
  • managers overworked
  • organizational mis-alignment
  • slow time-to-market
  • low staff morale
  • excessive overtime
  • you need to tame the Bureaucratic Beast

Consider that list carefully and if you feel like you have enough of the above problems, please contact us at tame.the.beast@berteigconsulting.com. or read more about the Real Agility Program for Enterprise Agility on the website.

Please share!

Book List for Enterprise Agile Transformations

Learn more about transforming people, process and culture with the Real Agility Program

Leaders of Agile Transformations for the Enterprise need to have good sources of information, concepts and techniques that will guide and assist them.  This short list of twelve books (yes, books) is what I consider critical reading for any executive, leader or enterprise change agent.  Of course, there are many books that might also belong on this list, so if you have suggestions, please make them in the comments.

I want to be clear about the focus of this list: it is for leaders that need to do a deep and complete change of culture throughout their entire organization.  It is not a list for people who want to do Agile pilot projects and maybe eventually lots of people will use Agile.  It is about urgency and need, and about a recognition that Agile is better than not-Agile.  If you aren’t in that situation, this is not the book list for you.


These books all help you to understand and work with the deeper aspects of corporate behaviour which are rooted in culture.  Becoming aware of culture and learning to work with it is probably the most difficult part of any deep transformation in an organization.

The Corporate Culture Survival Guide – Edgar Schein

Beyond the Culture of Contest – Michael Karlburg

The Heart of Change – John Kotter


This set of books gets a bit more specific: it is the “how” of managing and leading in high-change environments.  These books all touch on culture in various ways, and build on the ideas in the books about culture.  For leaders of an organization, there are dozens of critical, specific, management concepts that often challenge deeply held beliefs and behaviours about the role of management.

Good to Great – Jim Collins

The Leaders’ Guide to Radical Management – Steve Denning

The Mythical Man-Month – Frederick Brooks

Agile at Scale

These books discuss how to get large numbers of people working together effectively. They also start to get a bit technical and definitely assume that you are working in technology or IT. However, they are focused on management, organization and process rather than the technical details of software development. I highly recommend these books even if you have a non-technical background. There will be parts where it may be a bit more difficult to follow along with some examples, but the core concepts will be easily translated into almost any type of work that requires problem-solving and creativity.

Scaling Lean and Agile Development – Bas Vodde, Craig Larman

Scaling Agility – Dean Leffingwell

Lean Software Development – Mary and Tom Poppendieck


These books (including some free online books) are related to some of the key supporting ideas that are part of any good enterprise Agile transformation.

Toyota Talent: Developing Your People the Toyota Way – Jeffrey Liker, David Meier

Agile Retrospectives – Esther Derby

Continuous Delivery – Jez Humble, David Farley

The Scrum Guide – Ken Schwaber, Jeff Sutherland, et. al.

The OpenAgile Primer – Mishkin Berteig, et. al.

Priming Kanban – Jesper Boeg

Please share!

Announcement: The Scrum Team Assessment

Learn more about transforming people, process and culture with the Real Agility Program

Over the past six months we have been working hard on launching a new product: The Scrum Team Assessment.  This tool delivers to you a valuable report full of practical advice on how your team can get better at Scrum… and deliver better results!  It’s like an automated Scrum coach.  All your team members will fill in a comprehensive survey, we collect the results, generate a report – and then we personally review it – and send it back to you.

For more information, please visit our Scrum Team Assessment site.

Please share!

Collocate the Team

Learn more about transforming people, process and culture with the Real Agility Program

Collocation of the team is used to improve communication efficiency and to allow the team to learn to be more collaborative. Perfect collocation would have all stakeholders and work performers in the same work space (e.g. a large room) during all working hours. This level of collocation is not usually possible, so adjustments are made.

Collocation reduces wastes associated with waiting, movement, and inefficient communication. Collocation increases learning and feedback and assists with team empowerment.

Collocation can present challenges to people used to working on their own. For these people, a careful consideration of how to accommodate their working style is important, but more important is helping them to understand the need for and benefits from collocation. As this understanding grows and as the team starts to produce noticeable results, most people start to enjoy the close working environment.

When perfect collocation is not possible, consider part-time collocation, video conferencing, having a decision-making proxy represent the stakeholders, getting rid of closed offices, moving into open or shared work spaces or collocating part of the team.

I typically hear a great deal of positive feedback from teams that were previously not collocated after they come together in a common space. For example: “I don’t have to spend hours dealing with email anymore – it takes a few seconds to lean over and ask a question and get a response.” “Meetings that used to take half an hour to organize on people’s calendars now happen spontaneously.” “I can work much more efficiently because the people who I need to collaborate with are right there. No more emails, phone calls, scheduling, and pestering.”

Please share!