Mike Caspar has another thought provoking article, this one about self-driving cars and code test coverage.
Personally, I don’t want to let a vehicle drive me where the developers have been pressured (or the developers have decided on their own) to write code without tests…. I am just thinking about my future and I don’t want this topic to come up when it’s far too late for myself, family or friends.
Dear reader: if you have anyone connected to the auto industry and self-driving vehicles, please share Mike’s blog post!
Ken Schwaber and Jeff Sutherland have just announced the new version of the Scrum Guide! The only change is the addition of two paragraphs about the five Scrum values:
When the values of commitment, courage, focus, openness and respect are embodied and lived by the Scrum Team, the Scrum pillars of transparency, inspection, and adaptation come to life and builds trust for everyone. The Scrum Team members learn and explore those values as they work with the Scrum events, roles and artifacts.
Successful use of Scrum depends on people becoming more proficient in living these five values. People personally commit to achieving the goals of the Scrum Team. The Scrum Team members have courage to do the right thing and work on tough problems. Everyone focuses on the work of the Sprint and the goals of the Scrum Team. The Scrum Team and its stakeholders agree to be open about all the work and the challenges with performing the work. Scrum Team members respect each other to be capable, independent people.
The Scrum Guide is the sole and official definition of Scrum.
This post is a follow-up to an earlier article: There Are No Breaks Between Sprints.
Breaks between Sprints indicate a problem. Usually such breaks are filled with planning activities including research, requirements gathering, design & preparation, negotiations & approvals and the problem is threefold:
- Such plans are based on conjecture (risky and not compatible with Scrum) rather than empiricism (less risky and compatible with Scrum). Those activities are most beneficial when diligently performed by skilled inspectors at the point of the work. The four formal events within each Sprint provide the team and stakeholders adequate opportunity for inspection and ensure that decisions are being made in light of the up-to-date product increment and with respect to current user needs and market conditions.
- Breaks between Sprints often include activities which do not add value to the product or are entirely unrelated.
- Breaks between Sprints defer the delivery of value because the work performed does not result in potentially-releasable increment of “Done” product.
To correct this problem it is important to identify whether any of the effort spent between Sprints is adding value to the product — that is, which activities effect the form, fit, or function of the actual product. If determined to not be value adding, stop the activity entirely — it is waste. If determined to be value adding then the work ought to be part of their Sprints and the Scrum Team may decide that either the activity should be represented and ordered in the Product Backlog, or should be represented in the team’s Definition of Done.
BERTEIG offers agile training for Certified ScrumMaster, Certified Scrum Product Owner, Certified Scrum Developer, Leading SAFe, and more. This promotional video gives you a glimpse into the classroom for these fantastic events.
Find out more on our BERTEIG / WorldMindware course registration site.
Note: all prices in Canadian dollars. Most courses are delivered in the Toronto area, but we also come to you to deliver training upon request!
In just a few weeks we will be hosting Craig Larman here in Toronto as he facilitates the first-ever-in-Canada Certified Large Scale Scrum Practitioner training! Large Scale Scrum (LeSS) is about de-scaling. In simple terms, this is about using Scrum to make the best possible use of the creativity, problem-solving and innovation abilities of large numbers of people, rather than getting them stuck in bureaucracy and management overhead.
Here are the details of this unique learning event:
- Date and Time: April 11-13 (3 Days), 2016 – 9am to 6pm all three days
- Location: Courtyard by Marriott Downtown Toronto, 475 Yonge St. Phone: 416-924-0611
- Price: $3990.00 / person (that’s in Canadian Dollars – super great deal if you are coming from the US!)
Here are some quotes from previous attendees:
“It was inspiring to discuss Large-Scale Scrum with Craig Larman. The content of the course was top-notch.” – Steve Alexander
“The delivery was outstanding and the supporting material vast and detailed.” – Simone Zecchi
“The best course I have ever been on. Totally blown away.” – Simon Powers
Toronto is a great place to visit (I know many of our Dear Readers are from the United States) – don’t hesitate to consider coming in for a weekend as well as the course!
Register now! (Goes to our BERTEIG / World Mindware learning event registration site.)
The best architectures, requirements and designs emerge from self-organizing teams.
The quality of our software systems depends on refactoring. In fact, I believe that the only way that an organization can avoid refactoring is by going out of business. Maybe I should explain that.
Refactor or Die
Every software system that we build is inside a dynamic environment. The organization(s) using the software are all in a state of constant change. The people using the software are also constantly changing. Due to this constant change, every software system needs to be adapted to the environment in which it is used. Most of the time, businesses think of this constant change in terms of new features and enhancements – the scope of functionality that a system can handle. Less commonly, businesses think of this change in terms of the obvious external qualities and attributes of the system such as performance or security. But almost never does an organization, from a business perspective, think of the invisible qualities of the software system such as simplicity and technical excellence.
What happens when the business does not recognize those invisible qualities? I’m sure almost every software developer reading this can answer this question easily: the system becomes “crufty”, hard to maintain, bug-prone, costly to change, maze-like, complex. Some people refer to this as legacy code or technical debt.
The longer this state is allowed to continue, the more it costs to add new features – the stuff that the business really cares about. It is pretty easy to see how this works – for someone who has a technical background. But for those without a technical background it can be hard to understand. Here is a little analogy to help out.
Imagine that you set up a system for giving allowance to your kids. In this system, every week your kids have to fill out a simple form that has their name, the amount that they are requesting, and their signature. After a few weeks of doing this, you realize that it would be helpful to have the date on the form. You do this so that you can enter their allowance payments in your personal bookkeeping records. Then you decide that you need to add a spot for you to counter-sign so that the paper becomes a legal record of the allowance payment. Then your kids want extra allowance for a special outing. So you add some things on the form to allow them to make these special requests. Your accountant tells you that some portions of your kids allowance might be good to track for tax purposes. So, the form gets expanded to have fields for the several different possible uses that are beneficial to your taxes. Your form is getting quite complex by this point. Your kids start making other requests like to be paid by cheque or direct-deposit instead of in cash or to be paid advances against future allowances. Every new situation adds complexity to the form. The form expands over multiple pages. Filling out the form weekly starts to take significant time for each child and for you to review them. You realize that in numerous places on the form it would be more efficient to ask for information in a different way, but you’re not sure if it will have tax implications, so you decide not to make the changes… yet. You decide you need your own checklist to make sure that the forms are being filled out correctly. A new tax law means that you could claim some refunds if you have some additional information… and it can be applied retroactively, so you ask your kids to help transcribe all the old versions of the form into the latest version. It takes three days, and there is lots of guess-work. Your allowance tracking forms have become a bureaucratic nightmare.
The forms and their handling is what software developers have to deal with on a daily basis – and the business usually doesn’t give time to do that simplification step. The difference is that in software development there are tools, techniques and skills that allow your developers to maintain a system so that it doesn’t get into that nightmare state.
For a more in-deth description of this process of systems gradually becoming more and more difficult to improve, please see these two excellent articles by Kane Mar:
Ultimately, a software system can become so crufty that it costs more to add features than the business benefit of adding those features. If the business has the capacity, it is usually at this point that the business makes a hard decision: let’s re-write the system from scratch.
I used the word “decision” in that last sentence. What are the other options in that decision? Ignoring the problem might be okay for a while longer: if the company is still getting benefit from the operation of the system, then this can go on for quite a while. Throwing more bodies at the system can seem to help for a bit, but there are rapidly diminishing returns on that approach (see The Mythical Man-Month for details). At some point, however, another threshold is reached: the cost of maintaining the operation of the system grows to the point where it is more expensive than the operational value of the system. Again, the business can make a hard decision, but it is in a worse place to do so: to replace the system (either by re-writing or buying a packaged solution), but without the operating margin to fund the replacement.
In his articles, Kane Mar describes this like so:
It’s pretty clear that a company in this situation has some difficult decisions ahead. There may be some temporary solution that would allow [a company] to use the existing system while building a new product, [A company] may decide to borrow money to fund the rewrite, or [a company] may want to consider returning any remaining value to their shareholders.
In other words, refactor or die.
Refactoring and Business
In the Scrum Master and Product Owner classes that we teach, this topic comes up frequently: how does the business account for refactoring? How do we “govern” it? How do we make good decisions about refactoring?
There are a few principles that are important in helping to answer these questions. All of these principles assume that we are talking about refactoring in an Agile team using a framework like Scrum, OpenAgile, or Kanban.
Refactoring Principle One: Keep It Small
Refactoring is safest and cheapest when it is done in many small increments rather than in large batches. The worst extreme is the complete system re-write refactoring. The best refactoring activities take seconds or minutes to execute. Small refactorings create a constant modest “overhead” in the work of the team. This overhead then becomes a natural part of the pace of the team.
Not all refactoring moves can be kept so small. For example, upgrading a component or module from a third party might show that your system has many dependencies on that module. In this case, efforts should be made to allow your system to use both the old and the new versions of the component simultaneously. This allows your system to be partially refactored. In other words, to break a large refactoring into many small refactorings. This, in turn, may force you to refactor your system to be more modular in its dependencies.
Another common problem with keeping refactorings small is the re-write problem. Your own system may have a major component that needs to be re-written. Again, finding creative technical means to allow for incremental refactoring of the component is crucial. This can often mean having temporary structures in your system to allow for the old and new parts to work harmoniously. One system that I was working on had to have two separate database platforms with some shared data in order to enable this “bi-modal” operation.
Refactoring Principle Two: Business Catalysts
When is the earliest that a refactoring should be done? Not whenever the technical team wants to do it. Instead, the technical team needs to use business requests as catalysts for refactoring. If the business needs a new feature, then refactoring should only be done on those parts of the system that are required to enable that feature. In other words, don’t refactor the whole user interface, just refactor the parts that relate to the specific business request.
Again, there can be exceptions to doing this… but only in the sense that some refactorings might be delayed until a later date. This is tricky: we want to make sure that we are not accumulating technical debt or creating legacy code. So, instead, we need to allow the technical team to refactor when they detect duplication. Duplication of code, data or structure in the system. A business request might impact a particular part of the system and the team sees how it might be necessary to refactor a large swath of the system as a result. But, the cost of doing so is not yet justified: the single request is not enough of a catalyst, and the team can also choose a simple temporary solution. Later, the business makes another request that also implies the same large refactoring. Now is the time to seriously consider it. It is now a question of duplication of another simple temporary solution. The business may not be happy with the extra expense of the large refactoring so the principle of keeping it small still applies. However, the technical team must also be willing to push back to the business under the right circumstances.
Refactoring Principle Three: Team Cohesion
Teamwork in Agile requires high levels of communication and collaboration. In refactoring work, teamwork applies just as much as in any other activity. Here, it is critical that all members of the team have a unified understanding of the principles and purpose of refactoring. But that is just the first level of team cohesion around refactoring.
The next level of team cohesion comes in the tools, techniques and practices that a team uses in refactoring. Examples include the unit testing frameworks, the mocking frameworks, the automation provided by development tools, continuous integration, and perhaps most importantly, the team working agreements about standard objectives of refactoring. This last idea is best expressed by the concept of refactoring to patterns.
The highest level of team cohesion in refactoring comes from collective code ownership and trust. Usually, this is built from practices such as pair programming or mob programming. These practices create deep levels of shared understanding among team members. This shared understanding leads to self-organizing behaviour in which team members make independent decisions that they know the other team members will support. It also impacts research and learning processes so that teams can do experiments and try alternatives quickly. All of which leads to the ability to do refactoring, large and small, quickly and without fear.
Refactoring Principle Four: Transparency
In many ways, this is the simplest refactoring principle: the team needs to be completely open and honest with all stakeholders about the cost of refactoring. This can be difficult at first. Another analogy helps to see the value of this. A surgeon does not hide the fact that care is put into creating a clean operating environment: washing hands, sterilizing instruments, wearing face masks and hair covers, restricted spaces, etc. In fact, all of those things contribute to the cost of surgery. A surgeon is a professional who has solid reasons for doing all those things and is open about the need for them. Likewise, software professionals need to be open about the costs of refactoring. This comes back to the main point of the first part of this article: hidden and deferred costs will still need to be paid… but with interest. Software professionals are up-front about the costs because doing so both minimizes the costs and gives stakeholders important information to make decisions.
The challenge for business stakeholders is to accept the costs. Respecting the team and trusting their decisions can sometimes be very hard. Teams sometimes make mistakes too, which complicates trust-building. The business stakeholders (for example, the Product Owner), must allow the team freedom to do refactoring. Ideally, it is continuous, small, and low-level. But once in a while, a team will have to do a large refactoring. How do you know if the cost is legitimate? Unfortunately, as a non-technical stakeholder, you can’t know with certainty. However, there are a few factors that can help you understand the cost and it’s legitimacy, namely, the principles that are described here.
If the refactoring is small, it is more likely to be legitimate.
If the refactoring is in response to a business catalyst, it is more likely to be legitimate.
If the refactoring is reflective of team cohesion, it is more likely to be legitimate.
And, of course, if the refactoring is made transparent, it is more likely to be legitimate.
The Perfect Agile Tool doesn’t yet exist. In my training and consulting work, I often have strong words to say about electronic tools. Most of the tools out there are really bad. Unfortunately, JIRA, the most common tool, is also the worst that I know of. (Actually, the only tool worse than JIRA for an Agile team is MS Project – which is just plain evil). Some Agile tools do a bit better, but most fall far short of a good physical task board (information radiator). I am often asked to evaluate and / or partner with tool vendors to “bless” their products. Here is what I am looking for before I will consider an outright endorsement of such a tool.
Features for a Perfect Agile Tool
This list is roughly organized in order of features which do show up in some tools to those which I have never seen or heard of in tools.
1. Skeumorphism: Cards and Wall
The tool should display the current work of an Agile team in a way that is immediately recognizable as a set of note cards or PostIt’s on a physical wall. This includes colours, sizes, etc. Most people will type to enter data so fonts should be chosen to mimic hand-printed letters. Every aspect of the display should remind people of the physical analogue of the tool.
2. Live Update
As team members are using the tool, all updates that they make should be visible as immediate updates to all the other team members including typing, moving cards around, etc. There is no off-line mode for the tool. In fact, if the tool is not receiving live updates, it should be clearly disabled so that the team member knows there is a problem with the information they have displayed.
3. Simple or No Access Control
Most Agile methods strongly de-emphaisize or even disallow traditional roles and encourage self-organizing teams. This means that fine-grained access control to different features of the tool should be eschewed in favour of extremely simple access control: everyone can do anything with the tool. (It actually helps if there is no “undo” feature, just like there’s no easy way to erase Sharpie written on a note card.)
4. Infinite Zoom In/Out
When you are using cards on a wall, it is easy to see the whole wall or to get up close and see even very fine details on a single note card. Although it does not have to be literally infinite, the wide and tight zoom levels in the tool should be at least a few orders of magnitude difference. As well, the zoom feature should be extremely easy to use, similar perhaps to the way that Google Maps functions. Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.
5. Touch Device Compatible
This seems like a super-obvious feature in this day and age of tablets, smart phones and touch-screen laptops. And it would take the cards on the wall metaphor just that extra little way. But very few tools are actually easy to use on touch devices. Dragging cards around and pinch to zoom are the obvious aspects of this feature. But nice finger-drawing features would also be a big plus (see below)!
6. Size Limit on Cards
For techies, this one is extremely counter-intuitive: limit the amount of information that can be stored on a “card” by the size of the card. It shouldn’t be possible to attach documents, screen shots, and tons of meta-data to a single card. Agile methods encourage time-boxing (e.g. Sprints), work-boxing (e.g. Work-in-Process limits), and space-boxing (e.g. team rooms). This principle of putting boundaries around an environment should apply to the information stored on a card. Information-boxing forces us to be succinct and to prefer face-to-face communication over written communication. Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.
7. Minimal Meta-Data
Information-boxing also applies to meta-data. Cards should not be associated with users in the system. Cards should not have lots of numerical information. Cards should not have associations with other cards such as parent-child or container-contained. Cards should not store “state” information except in extremely limited ways. At most, the electronic tool could store a card ID, card creation and removal time-stamps, and an association with either an Agile team or a product or project.
8. Overlapping Cards
Almost every electronic tool for Agile teams puts cards in columns. Get rid of the columns, and allow cards to overlap. If there is any “modal” behaviour in the tool, it would be to allow a team member to select and view a small collection of cards by de-overlapping them temporarily. Overlapping allows the creation of visually interesting and useful relationships between cards. Cards can be used to demarcate columns or groupings without enforcing strict in/out membership in a process step.
9. Rotatable, Foldable, Rip-able Cards
Increase the fidelity of the metaphor with physical cards on a wall. Rotation, folding and ripping are all useful idioms for creating distinct visual cues in physical cards. For example, one team might rotate cards 45 degrees to indicate that work is blocked on that card. Or another team might fold a dog-ear on a card to indicate it is in-progress. Or another team might rip cards to show they are complete. The flexibility of physical cards needs to be replicated in the electronic environment to allow a team to create its own visual idioms. Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.
10. Easy Sketching on Cards… Including the Back
Cards should allow free-form drawing with colours and some basic diagramming shapes (e.g. circles, squares, lines). Don’t make it a full diagramming canvas! Instead, allow team members to easily sketch layouts, UML, or state diagrams, or even memory aides. The back side of the card is often the best place for more “complex” sketches, but don’t let the zoom feature allow for arbitrarily detailed drawing. Lines need a minimum thickness to prevent excessive information storage on the cards.
11. Handwriting Recognition
With Siri and other voice-recognition systems, isn’t it time we also built in handwriting recognition? Allowing a team member to toggle between the handwriting view and the “OCR” view would often help with understanding. Allow it to be bi-directional so that the tool can “write” in the style of each of the team members so that text entry can be keyboard or finger/stylus.
12. Sync Between Wall and Electronic Tool
This is the most interesting feature: allow a photo of cards on a wall to be intelligently mapped to cards in an electronic tool (including creating new cards) and for the electronic tool to easily print on physical note cards for placement on a wall. There is all sorts of complexity to this feature including image recognition and a possible hardware requirement for a printer that can handle very small paper sizes (not common!)
These are the features that many electronic tools implement as part of being “enterprise-ready”. I’ll be brief on these points:
No Individual Tracking – the team matters, not who does what.
No Dependency Management – teams break dependencies, tools don’t manage dependencies.
No Time Tracking – bums in seats typing doesn’t matter: “the primary measure of progress is working software” (or whatever valuable thing the team is building) – from the Agile Manifesto.
No Actuals vs. Estimates – we’re all bad at predicting the future so don’t bother with trying to get better.
No Report Generation – managers and leaders should come and see real results and interact directly with the team (also, statistics lie).
No Integration Points – this is the worst of the anti-features since it is the one that leads to the most anti-agile creeping featuritis. Remember: “Individuals and interactions [are valued] over processes and tools” – from the Agile Manifesto.
Evaluation of Common Agile Tools
I go from “Good” to “Bad” with two special categories that are discontinuous from the normal scale: “Ideal” and “Evil”. I think of tools as falling somewhere on this scale, but I acknowledge that these tools are evolving products and this diagram may not reflect current reality. The scale looks like this, with a few examples put on the scale:
Plea for the Perfect Agile Tool
I still hope that some day someone will build the perfect Agile tool. I’ve seen many of the ideal features listed above in other innovative non-Agile tools. For example, 3M made a PostIt® Plus tool for the iPhone that does some really cool stuff. There’s other tools that do handwriting recognition, etc. Putting it all together in a super-user-friendly package would really get me excited.
Let me know if you think you know of a tool that gets close to the ideal – I would be happy to check it out and provide feedback / commentary!
I’m going to be presenting a quick session on the use of a Skills Matrix to help launch a team. If you are in the Toronto area on the evening of March 16, come check it out: Agile TO Meetup.
Many organizations are attempting to use Agile methods. Banks, telecom companies, government agencies, and all manner of mid-size and small organizations. Most of these attempts are limited in that they think of Agile as a solution instead of as a culture. In this video, I explore some of the conditions for creating Real Agility.
This is the first video in a series of eleven that is oriented towards what managers need to know to create Real Agility in their organizations. The final two videos in the series are going to be content exclusively available to subscribers to our Real Agility Newsletter. Those final two videos are about “Dealing with Crisis” and “The Knowing-Doing Gap”. (Our newsletter also includes other great content including interviews – we are featuring an interview with Mary and Tom Poppendieck in just a few weeks!)
The Scrum Myths videos have been popular and I’m very happy with people’s comments about the videos. I’m going to be making an even more extensive new series for publication in just a few weeks.
Unbeknownst to me, the videographer, my brother Alexei Berteig, created a bloopers video from some of the many, many, many, MANY mistakes I made while making the videos. I hope you will watch it…. but I strongly recommend taking a look at one or two of the “good” videos first. Try these:
Now, without further ado, here is the Scrum Myths Bloopers video:
In-depth article here: Using Lego[sic] to capture raw data for Cycle Time and Process Cycle Efficiency.
From the article:
The typical way to collect baseline numbers for these metrics is to conduct a value stream mapping workshop that involves most or all team members for one day or longer. The client is worried about losing too much time in the workshops when the teams could be doing value-add work. Therefore, we needed a less intrusive way to collect baseline measurements. There is also the question of how accurate the PCE data will be when it is obtained through a workshop rather than by direct observation of team activity.
I came up with the idea of using Lego bricks to collect the raw data for Cycle Time and PCE. There is some impact on team member time, but hopefully it is not too intrusive. My observation is that people enjoy manipulating Lego bricks, and they don’t mind sticking the bricks on a plate to represent their work.
Great video shared by Robin Dymond:
Great article by Mike Griffiths: http://leadinganswers.typepad.com/leading_answers/2015/10/agile-talent-management.html
Question from Meredith:
As a product owner, what are the best ways to record technical debt and what are some approaches to prioritizing that work amid the continuous delivery of working software?
Hi Meredith! This is an interesting question. I’ll start by answering the second part of your question first. The two most common ways of handling technical debt, quality debt and legacy debt are:
- Fix as you go. The Scrum Team works on new PBIs every Sprint, but every time a PBI touches a technical, quality or legacy debt area, the team fixes “just enough” to make the PBI implementation have no debt. This means that refactoring and the creation of automated tests (usually through TDD) are done on the parts of the product/system that have the problems.
- Allocate a percentage. In this scenario, the Scrum Team reduces its velocity (sometimes significantly) to allow for time to deal with the technical, quality and legacy issues. This reduction could be adjusted every Sprint, but is usually consistent for several Sprints in a row.
In both approaches, the business is paying for the debt accumulated, and the cost includes an “interest” fee. In other words, the sooner you fix technical, quality and legacy debt, the less it costs. This approach to thinking about your product/system is essential for long-term sustainability. One organization I worked with took three years working on their system to clean it up without being able to add any new features! Don’t let your system get to that point.
Now to the first part of your question…
As a Product Owner, you shouldn’t really be making decisions about this cleanup work. Your authority is limited to the Product Backlog which should not include technical items. The only grey area here is with defects which may be hard to classify as either fully business or fully technical. But technical design, duplication of code, technical defects, and legacy code all are under the full authority of the Scrum Development Team. Practically, this means that every Sprint the team has the authority to choose however few PBIs they feel they can take while considering the technical state of the product/system. We trust and respect the team to make wise decisions.
Therefore, your main job as a Product Owner is to provide the team with as much information as possible about the business consequences of the work they are doing. With strong communication and collaboration about this aspect of their work, the technical members of your team can make good trade-off decisions, and balance the need for new features with the need to clean up previous compromises in quality.
A final note: in order for this to work well, it is critical that the team not be pushed to further sacrifice quality and that they are given the support to learn the techniques and skills to create debt-free code. (You might consider sending someone to our CSD training to learn these techniques and skills.)
Using these techniques, I have been able to help teams get very close to defect-free software deliveries (defect rates of 1 or 2 in production per year!)
Let me know in the comments if you would like any further clarification.