Tag Archives: compliance

The Agile Manifesto – Essay 3: Working Software over Comprehensive Documentation

Learn more about transforming people, process and culture with the Real Agility Program

How much documentation does it take to run a project with ten people working for six months?  For some organizations it takes way too much:

Photo of heavy documentation for software project

This binder (about 3 or 4 inches thick) is all the documentation associated with such a project.  In looking carefully at the project, creating the documentation took far more time than the time spent on designing, writing and testing the software.  Yet, the documentation does not produce any value.  Only the software produces value.  The Agile Manifesto, asks us to focus on the outcome (working software) and to make tradeoffs to minimize the means (comprehensive documentation).

The Agile Manifesto asks us to challenge our assumptions about documentation.  In many work environments, documentation is an attempt to address some interesting and important needs:

  • Knowledge sharing among stakeholders and the people working on a project.
  • Knowledge sharing across time as people come in and out of a project.
  • Verification and traceability for contracts or other compliance needs.
  • Decision-making and analysis for business and technical problems.
  • Management oversight and control.
  • Various aspects of individual accountability.

Documentation is usually heavier (more comprehensive) the more the following circumstances exist in an organization:

  • Geographical distribution of people.
  • Lack of trust between people, departments or organizations.
  • Regulated work environments.
  • Depth of management hierarchy.
  • Number of people directly and indirectly involved.
  • Knowledge and skill sets highly segregated between people.
  • Culture of respect for written texts.

Working Software

What if the software itself could address the needs that often documentation is used to address?  Let’s look at them in turn:

  • Knowledge sharing among stakeholders and the people working on a project.
    If the software is functional at all stages, as supported by Agile methods such as Scrum and Extreme Programming, then the software becomes an effective representation of the knowledge of all the people who have participated in building it.
  • Knowledge sharing across time as people come in and out of a project.
    Software that is technically excellent is often easier to understand for people who are new to it.  For example, excellence in user experience and design means new users can get up to speed on software faster.  Use of good design patterns and automated testing allows new developers to understand existing software easily.
  • Verification and traceability for contracts or other compliance needs.
    Test-driven development (code) and specification by example (scripting and code) are forms of traceable, executable documentation that easily stay in-sync with the underlying software system.
  • Decision-making and analysis for business and technical problems.
    In particular, diagrams can help a great deal here.  However, electronic tools for creating such diagrams can be slow and awkward.  Consider the practice of Agile Modelling (basically using a whiteboard and taking photos) as a good alternative to precise technical diagramming if you are doing problem-solving.
  • Management oversight and control.
    Reports and metrics drive much of the traditional documentation in an organization.  Simplifying reports and metrics often leads to a clearer picture of what is going on, reduces the opportunities to “game” the system, and always results in lower levels of documentation.  As well, some reports and metrics can be generated 100% through automated means.  All that said, the fundamental premise in the Agile manifesto is that management should base decisions on what is actually built – the “Working software” by looking at it and using it.
  • Various aspects of individual accountability.
    If you really need this, a good version control system can give you the information for this.  Sign-offs and other types of accountability documentation are typically just waste that doesn’t actually help in process improvement.  Most people who are in high-compliance environments already have licenses and/or security clearances that provide this accountability.  If you software is working, however, then this isn’t even a concern as trust is built and bureaucracy can be reduced.

In my recent training programs as research for this article, I have surveyed over 100 people on one aspect of documentation – code documentation.  Every individual surveyed is either currently coding or has a coding background, and every single person had the same answer to a simple scenario question:

Imagine that you have just joined a new organization and you are about to start working as a software developer.  One of the existing team members comes up to you and introduces himself.  He has with him a piece of paper with a complicated-looking diagram and a full binder that looks to be holding about 250 pages.  He asks you, “you need to get up to speed quickly on our existing system – we’re starting you coding tomorrow – would you prefer to go over the architecture diagram with me or would you prefer to review the detailed specifications and design documents.” He indicates the one-page diagram and the binder respectively.  Which would you prefer?

(I’ve put up a Survey Monkey one-question survey: Code Documentation Preference to extend the reach of this question.  It should take you all of 60 seconds to do it.  I’ll post results when I write the next Agile Manifesto essay in a month or two.)

The fact that everyone answers the same way is interesting.  What is even more interesting to me is that if you think through this scenario, it is actually almost the worst-case scenario where you might want documentation for your developers.  That means that in “better” cases where documentation for developers may not be as urgent or necessary, then the approach of just going to talk with someone is a lot better.

Documentation and Maps

The problem with documentation is the same problem we have with maps: “the map is not the territory” (quote from the wisdom of my father, Garry Berteig).  We sometimes forget this simple idea.  When we look at, say, Google Maps, we always have in the back of our consciousness that the map is just a guide and it is not a guarantee.  We know that if we arrive at a place, we will see the richness of the real world, not the simplified lines and colours of a map.  We don’t consider maps as legally binding contracts (usually).  We use maps to orient ourselves… as we look around at our reality.  We can share directions using maps, but we don’t share purpose or problems with maps.  And finally, maps assume that physical reality is changing relatively slowly (even Google Maps).

Many times when we create documentation in organizations, however, we get confused about the map versus the territory.

Agility and Documentation

Of course, code is a funny thing: all code is documentation too.  The code is not the behaviour.  But in software, code (e.g. Java, ASM, Scheme, Prolog, Python, etc.) is as close as possible to the perfect map.  Software is (mostly) deterministic.  Software (mostly) doesn’t change itself.  Software (mostly) runs in a state absent from in-place human changes to that software.  Software (mostly) runs on a system (virtual or physical) that has stable characteristics.  The code we write is a map.  From this perspective, documentation becomes even less important if we have people that already understand the language(s)/platform(s) deeply.


This essay is a continuation of my series on the Agile Manifesto.  The previous two essays are “Value and Values” and “Individuals and Interactions over Processes and Tools“.

 


Please share!
Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

SAFe Program Consultant Training – Review

Learn more about transforming people, process and culture with the Real Agility Program

I want to give some perspective on SAFe and the training that I have been attending these last few days.  The training itself is not actually over, but we are very near the end.  Just one day left, but it is dominated by the SPC exam and open Q&A on advanced topics.  In other words, we have covered the essence of SAFe.

Ad Hoc, Pragmatic and Transformative

When I think about organizations or departments trying to become Agile enterprises, I generally categorize those efforts into three approaches.

The “Ad Hoc” approach is typified by a grassroots movement or an executive decreeing “be Agile” with no one really knowing what that means.  A lot of organizations have some teams in this condition – they try Scrum, try some other Agile-ish things, and have modest successes.  When the enterprise is large enough, these ad hoc approaches reach a natural limit of effectiveness before they become severely blocked by organizational considerations.  Then, the leadership of the organization must turn to systematic approaches to becoming an Agile enterprise: the Pragmatic approaches or the Transformative approaches.

The “Pragmatic” approach acknowledges the difficulty of change, particularly for those in middle management.  There is still a deep acknowledgement of the Agile values and principles, but the pragmatic part is to say that the organization will take quite a long time to adopt those values and principles end-to-end, top-to-bottom.  These pragmatic approaches typically have low risk and good results.  SAFe (Scaled Agile Framework) falls into this category along with DAD (Disciplined Agile Delivery) and possibly others that I’m not aware of.

The “Transformative” approach acknowledges the deep nature of Agile as a cultural transformation that can be done quickly when there is urgency to do so.  There is still an acknowledgement that Agile can be difficult for many people as it requires a change in mindset and deep habitual behaviours.  These approaches are transformative because they require all protagonists in the enterprise to be open to this deep and fast change to a new culture.  LeSS (Large Scale Scrum) and RAP (Real Agility Program) are both systematic transformative frameworks.

SAFe, as a pragmatic approach, has a number of excellent features that will help an organization accomplish its business and technology goals.

Scaled Agile Framework – Practical, Pragmatic, and Still Pure Agile

One big concern I had about SAFe, based on other people’s comments, was that it somehow was compromising the values of the Agile Manifesto.  I want to say clearly and unequivocally that SAFe is most certainly true to Agile.  This fact was demonstrated multiple times and in multiple ways throughout the training:

  • Explicit statements that SAFe is based on the Agile Manifesto.  At one point, Dean Leffingwell emphatically repeated several times that “we live or die by the Agile Manifesto!”
  • Clear examples of SAFe implementations making choices based on the values and principles of the Agile Manifesto.  It was common to talk about situations where SAFe ScrumXP teams, Agile Release Trains and the people involved made decisions based on “individuals and interactions”.
  • Practices and guidelines that implement the values and principles of Agile are pervasive throughout SAFe.  The Inspect and Adapt meeting, Program Increments, daily business collaboration with SAFe ScrumXP teams, customer collaboration through various forms of backlogs, reviews and demos, focus on simplicity and technical excellence with Architectural Runway, Test-Driven Development and other Agile engineering practices.
  • The instructors (not just Mr. Leffingwell) often mentioned their own philosophy of being flexible with the SAFe “framework” by making appropriate context-specific changes to the details.
  • Even participants in the class who have already started using SAFe in their organizations shared stories that clearly indicated a strong emphasis on the values and principles of Agility.

At the same time, SAFe manages to create a relatively simple interface with a traditional management organization.  This is critical and what makes it really effective as a pragmatic approach to enterprise agility.  For example, at the Agile Release Train level, there are nine roles identified (e.g. System Architect, Product Management, Business Owners).  The explicit acknowledgement and identification of these roles and how they interact with the SAFe ScrumXP teams through meetings, artifacts and other processes and tools helps an organization to map Agility at the staff level to traditional concepts at the middle-management level.  This interfacing is also pervasive throughout the SAFe framework and occurs at all levels of effort from individual team members up to high level business leaders.

Some people have grumbled about the complicated diagram as “proof” that SAFe can’t be Agile.  But a different way of looking at the diagram is that it is comfort for management.  I really appreciate this.  Back in 2004 and 2005 when I was consulting at Capital One on their first enterprise attempt at Agile, one of the coaches I was working with shared a story with me about the importance of comfort.  The project manager for an important project was very nervous that there was no Gantt chart in Agile.  At a personal level, she needed the comfort of having a Gantt chart to track the work of the team.  The coach for this project told the project manager “please, make your Gantt chart – just make sure that you let the team organize themselves without being disturbed to help you with the Gantt chart.”  Most Agilists are anti-Gantt.  This was a real eye-opener for me.  That project manager went on to gain confidence in the Agile team and was able to eventually discard the Gantt chart.

SAFe isn’t just a framework, it’s actually a scaffolding.  When you build an arch, you need a scaffold to keep everything in place until the keystone is in place.  In creating an Agile enterprise, you use SAFe as a scaffold to get you to Agility.

Lean, Agile and Leadership

This training has also spent a lot of time discussing Lean thinking, Lean product flow and Lean leadership.  SAFe asserts four principles of Agile Leadership:

  1. Take a systems view
  2. Embrace the Agile Manifesto
  3. Implement product development flow
  4. Unlock the intrinsic motivation of knowledge workers

I like this list.  I might change the wording slightly, but in going through the details of what these mean, it is clear that if leaders could adopt these principles, every organization would be a much better place to work.

There is a fair amount of time spent on helping leaders make the shift in thinking from traditional “scientific management” to “Agile leadership”.  There are a lot of good reading references given in these discussions including “Five Dysfunctions of a Team”.  There is also a lot of time spent on value stream thinking including some great discussion exercises.

Organizational Structure in SAFe

SAFe does not define all the structures throughout the whole organization.  By design, it is not end-to-end, top-to-bottom.  It does define a structure for three levels of activity: the team level, the program level and the portfolio level.

At the team level, SAFe relies on a slightly modified version of Scrum and Extreme Programming (XP) that it calls SAFe ScrumXP.  As a Certified Scrum Trainer, I’m confident that the Scrum described is “good enough” to be legitimate Scrum even if there are small variations.  One example is in the idea of commitment.  Scrum espouses the value of Commitment.  In “old” versions of Scrum, the Scrum Team was required to commit to the work of the Sprint (the business scope).  SAFe keeps this concept.  However, if you look in the most recent version of the Scrum Guide, this concept is no longer present.  One thing that I think is absolutely fantastic is that several of the XP technical practices are required practices in SAFe: Test Driven Development, Continuous Integration, Pair Programming, User Stories, Acceptance Test Automation and Refactoring.  I wish that Scrum would get around to officially requiring these practices.  This set of canned answers is sometimes an irritant for Scrum folks, but the fact is that, again, middle managers are often made more comfortable by being provided with concrete answers.  And, in my not-so-humble opinion, SAFe is providing the right answers.  Since all this is at the Team level, middle managers are even more comfortable because they can tell all these staff-level people how to work.

At the program level, SAFe scales the basic concept of a Sprint up to a larger “Program Increment” (PI) concept.  The core concept that holds the program level together is the Agile Release Train which is based on a limit to the number of people who can work effectively in a social network (Dunbar’s number ? 150).  Again, SAFe is quite definitive about process at this level: Sprints are 2 weeks long and PIs are 5 Sprints long (10 weeks).  Timeboxing is explained effectively with the concepts of cadence and synchronization as a way to ensure predictability at the program level.  Unlike the simplicity of the Team level, the Program level in SAFe introduces a number of important connectors to transitional organizations.  This is done through defining several roles that have extremely close analogues to traditional roles (and even use a lot of the same names), and through other artifacts such as vision, roadmap, non-functional requirements, and features.  There are even a number of recommended metrics for evaluating the performance of the program (not the people).

At the Portfolio level, SAFe simplifies again somewhat in that there are no new aspects of cadence or synchronization introduced, and the number of defined roles and artifacts at this level is relatively small.  One important difference at this level compared to the Program and Team levels is the introduction of a Kanban approach used to feed “Program Epics” to the Agile Release Trains at the Program level.  At this level, Kanban is used to drive the flow of value, but there is not as much emphasis on continuous improvement here (although there is when SAFe discusses leadership).  At all three levels, there is a constant emphasis on the lean concept of focusing on value rather than cost.  This comes in many of the details, but may be a bit difficult for middle managers.  Fortunately, the Portfolio level  includes some excellent advice on working with budgets and allocating those budgets to business vs. technical needs and based on the effort required at the Program level with the Agile Release Trains.  SAFe recommends revisiting budgets every six months (I believe this is meant to be every 2 Product Increments) and is the only aspect of cadence and synchronization at the Portfolio level.

The Training

I’ll admit that overall I didn’t particularly enjoy the training.  I love SAFe.  As a trainer myself, I’m too critical perhaps.  Certainly, the training I deliver has evolved over ten years of work with lots and lots of feedback and mentorship.  However, in the Agile community, the overall standard for training has improved greatly over the last 5 years and I would love to see our three trainers who helped with this course improve their delivery.

There are a also some general comments about the training that I would like to make that are about personal preference.

First, I would prefer more small exercises that are experiential.  For example, there was a great deal of time spent on centralized vs. decentralized decision-making and leadership which could have been compressed greatly with a simple exercise like the “Command and Control Walking Simulation” which takes about 5 minutes to drive home the point unequivocally.  The first two days were largely lecture with a couple big exercises (both the lecture and the big exercises were generally good).

Second, the slides.  The slides.  The slides.  The slides… and more slides!!!  Too much by far.  And using the slides for lecture made it very difficult to stay on track for time with lots of slides missed or touched on only very briefly.  This is anxiety-inducing and boredom-inducing for me.  Some people like lots of slides, but most people don’t.

Third, not enough breaks for a 9 to 6 training session.  Usually just one break in the morning and one in the afternoon as well as a short lunch.  Two breaks and a longer lunch would have made it much more tolerable from a personal comfort level.  At one point on the third day I just had to take an extra break and I ended up missing about 30 minutes before I felt ready to come back.

Final Words

I’m happy I invested in this for both myself and for Travis.  We have learned a lot about SAFe, a little about Agile and Lean, and we are both excited about offering SAFe-related services to some of our clients.  At this point I am convinced that it is appropriate and good under some common (but not universal) conditions.

I will probably write several more articles about SAFe as I process the information and start to relate it to more specific aspects of Agile, Lean, organizations, management, leadership, productivity, and, of course, our own Agile Enterprise framework, the Real Agility Program. I’m excited and happy to see that the two frameworks are not competitive or exclusive in any significant way… more about that of course!


Please share!
Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail

The ScrumMaster is Responsible for What Artifacts?

Learn more about transforming people, process and culture with the Real Agility Program

Organizations like to have clear role definitions, clear processes outlined and clear documentation templates.  It’s just in the nature of bureaucracy to want to know every detail, to capture every dotted “i” and crossed “t”, and to use all that information to control, monitor, predict and protect.  ScrumMasters should be anti-bureaucracy.  Not anti-process, not anti-documentation, but constantly on the lookout for process and documentation creep.

To help aspiring ScrumMasters, particularly those who come from a formal Project Management background, I have here a short list of exactly which artifacts the ScrumMaster is responsible for.

REQUIRED:
– None – the ScrumMaster is a facilitator and change agent and is not directly responsible for any of the Scrum artifacts (e.g. Product Backlog) or traditional artifacts (e.g. Gantt Chart).

OPTIONAL/COMMON:
Obstacles or impediments “backlog” – a list of all the problems, obstacles, impediments and challenges that the Scrum Team is facing.  These obstacles can be identified by Team Members at any time, but particularly during the Daily Scrum or the Retrospective.
Definition of “Done” gap report, every Sprint – a comparison of how “done” the Team’s work is during Sprint Review vs. the corporate standards required to actually ship an increment of the Team’s work (e.g. unit testing done every Sprint, but not system testing).
– Sharable retrospective outcomes report, every Sprint – an optional report from the Scrum Team to outside stakeholders including other Scrum Teams.  Current best practice is that the retrospective is a private meeting for the members of the Scrum Team and that in order to create a safe environment, the Scrum Team only shares items from the retrospective if they are unanimously agreed.  Outsiders are not welcome to the retrospective.
– Sprint burndown chart every Sprint – a chart that tracks the amount of work remaining at the end of every day of the Sprint, usually measured in number of tasks.  This chart simply helps a team to see if their progress so far during a Sprint is reasonable for them to complete their work.
– State of Scrum report, every Sprint – possibly using a checklist or tool such as the “Scrum Team Assessment” (shameless plug alert!).

NOT RECOMMENDED (BUT SOMETIMES NEEDED):
– minutes of Scrum meetings
– process compliance audit reports
– project administrative documents (e.g. status reports, time sheets)

NEVER RECOMMENDED:
– project charter (often recommended for the Product Owner, however)
– project plans (this is done by the Product Owner and the Scrum Team with the Product Backlog)
– any sort of up-front technical or design documents

The ScrumMaster is not a project manager, not a technical lead, not a functional manager, and not even a team coach.  There are aspects of all of those roles in the ScrumMaster role, but it is best to think of the role as completely new and focused on two things:
– improving how the team uses Scrum
– helping the team to remove obstacles and impediments to getting their work done.


Please share!
Facebooktwittergoogle_plusredditpinterestlinkedinmailFacebooktwittergoogle_plusredditpinterestlinkedinmail