What is agile exactly? How do we practice it? What does it look like to be an agile product owner? What is an agile team?
One of the qualities I’ve come to admire the most about agile teams and agile ambassadors is this continuous state of learning which everyone agrees to be in.
It seems as though “being agile” gives us permission to sometimes know an answer and sometimes not to. It gives us permission to sometimes understand a situation and have a solution and sometimes not to. Agile methods have a built-in “Reality Check” which is so refreshing.
By openly communicating often in retrospectives and by making work and backlog visible the process is taken out of the abstract and into the concrete. Agile seems to put everyone on the same page ~ even if people are coming at agile from very different angles.
Recently I posted a question to the 2500+ members of the Facebook Scrum group, asking for good recommendations for meaningful resources.
Here are the TOP Four Responses:
I’m interested to read your comments about any of these four articles or sites. What do you agree with? What do you disagree with?
What has been your biggest challenge and greatest success with implementing agile methods in your work environment?
At a recent Agile Coach Camp I attended in June, a fellow participant said it best when he commented that as a software developer who had not really had an interest in people or relationships before, agile changed everything for him. He said Agile methods gave him a way to communicate with others, to trust them, and to understand how to work together to deliver excellent products.
What this gentleman was referring to, perhaps unknowingly, is one of the stages of team development. When an individual moves through stages of not trusting to trusting they are participating in an evolutionary process with a positive end result.
Daniela Moinau writes about this process in her article entitled, “High-performance Teams: Understanding Team Cohesiveness.”
She writes, “Once the storming stage is overcome the team is ready to establish open communications, stable positions and norms – the norming phase. Trust is finally gained, and “when the trust account is high, communication is easy, instant, and effective.” These are the first steps towards cohesiveness. Once cohesiveness is achieved, teams will move from norming to performing and subsequently to highly performing. ”
So while it may appear somewhat obvious that teams would trust one another, its not obvious to everyone. Each person carries their own unique history and their life experiences are of value. These experiences shaped who they are and what they have become. These experiences, whether pleasant or traumatic, shape a person’s ability to trust.
Agile methods to challenge teams to trust on a more profound level because of the nature of the values and principles which insist on it.
When a person, such as the software developer I mentioned in the first paragraph, overcomes his own internal patterns and learns to trust in a new way it leads to cohesiveness for him and his team. And this cohesiveness leads to high-performance.
As it turns out, trust is essential.
Under the right conditions Scrum can be a tremendous success story, but it often requires hard work to get there. For new Scrum teams it means learning to fundamentally work very differently than they are used to, such as relying on a lot more collaboration, making and delivering on shared commitments and building a high degree of trust. For existing Scrum teams it means constantly renewing the team commitment to each other, the cause, and to the Scrum framework. This includes the rather painful practice of revisiting the fundamentals and ensuring any deviations from accepted processes or practices were for the right reasons and had the right results.
To have a chance at achieving high performance a new-to-Scrum team will not only need to just change their processes, but fundamentally change the culture and behaviour of the team and all of the supporting roles (that includes their leadership). Meanwhile, a mature or well-established team should never assume they are high performance; they should always be checking (and rechecking) that they are still living the Agile values.
Needless to say this can become an extremely complex challenge! To be absolutely clear, I’m not proposing there is a single formula or recipe that works, but I do believe certain criteria can dramatically improve your Scrum team’s chances of success. To that end here are 10 tips (plus a bonus) that may help you focus your efforts towards building a successful Scrum team and experience.
1. Right Number of Team Members
Currently the Scrum Guide recommends that Scrum teams will work best with three to nine people (not including the Scrum Master and Product Owner). Too few people on the team and you risk not having enough technical expertise and coverage of critical skills. Too many people on the team and you may become challenged to truly collaborate effectively. Remember, this is just a guideline and you may be successful with different numbers, you just need to be aware of the impacts and make sure the gaps are covered.
2. Appropriate Balance of Skills
Scrum teams really should be balanced and cross-functional. Having all of the necessary skills on the team for delivering a complete solution (not roles, but skills) will encourage and support end-to-end thinking all the way from design to implementation. This approach will result in a better solution and a superior customer experience, but it relies on whole team collaboration. Note this does not mean individual team members need to be fully cross-functional, but what is important is that all the critical skills are represented on the team and each team member contributes their domain expertise towards the collective strength.
3. Propensity for Engineering Technical Ability
For increased chances of success, a Scrum team should leverage technology and engineering practices whenever possible. Techniques, skills and tools that facilitate Agile approaches such as Continuous Integration, Automated Testing and Test Driven Development all make technical excellence, continuous improvement and truly being “Done” every Sprint a possible reality for a Scrum team.
4. High Team Member Allocation
Scrum team members should be allocated to as few different initiatives as realistically possible. The more projects you are allocated to, the more task switching you may have to perform, the longer it will take to complete any one item, the thinner you will be spread and the less effective you will be. In other words, people (and teams) should limit their work in progress as much as possible and focus on completing those things that truly matter most. This is true for any framework, but it is particularly emphasized with Agile ones. Note there is a slight but fundamental difference between being allocated to a team and being dedicated to a team – that is a topic for a future article.
5. Empowered and Knowledgeable Product Owner
Your Product Owner needs to be informed, available, business-savvy, knowledgeable, collaborative, and empowered to make decisions about what to build and what order to do it in. They also need to be a strong negotiator and very capable at conducting business driven trade-offs. In the end, a Product Owner needs to effectively communicate, convey and deliver on a clear vision to the Team and Stakeholders to ensure a useful solution is created. Without empowerment, knowledge, and vision in a Product Owner the team will struggle.
6. Equitable Scrum Master
Having a good process is only part of the equation. A good Scrum Master will champion and enforce that process, protect the team, encourage collaboration, highlight (escalate when necessary) and encourage the removal of obstacles, facilitate discussions, provide fair and constructive feedback, cultivate a culture of continuous improvement and learning, and work to help the team live the Agile values.
Remember that the Scrum Master has authority over the process but not over the team. As the process champion the Scrum Master may sometimes even find themselves in a conflict between maintaining the Scrum rules and guiding the team as they discover the need to adapt practices to better align with their own needs and ways of working. In that regard a Scrum Master should understand and embrace the servant leader role. In the end, a Scrum Master needs to be the person that helps the team make decisions, but not the person that makes decisions for them.
7. Strong Executive Support
Leadership is the key to driving change and progress. Executives and managers of Scrum teams need to nurture the environment, let go of the “how”, allow the team to learn from mistakes, and encourage and coach the growth of the collective team knowledge and overall experience.
Understanding the dramatic impact leadership has on a transitioning team is also very critical, as a single word or direction from the executive level can single-handedly affect (either positively or negatively) the team’s future behaviours and resulting successes or failures. And without a true environment of trust built by the leadership, team members will often shy away from taking a risk to try something new or unknown.
8. Solid Partnership Commitment
There must be a consistent commitment and engagement from all parties in the organization towards adopting the Scrum framework, Agile methods, and thinking. The initiative must be an open, collaborative experience and there must be complete understanding and alignment by all parties in assuming the risks and rewards as well as sharing in the effort. This includes not only business partners and their IT counterparts, but their leadership as well as all of the people and teams supporting an Agile initiative.
9. Reduced Team Dispersion
Co-located teams are more effective communicators and can sometimes experience increased productivity by up to 60% if situated together in the same room. More simply stated, the greater the dispersion factor, the greater the challenge of collaboration. Note that time zones are often considered the largest dispersion factor and can have a greater impact than geography.
Although it is strongly recommended that teams be co-located, it is not mandatory to success. In fact, certain Agile practices have factors, tools and techniques inherent to them to help bridge some of the shortcomings of increased dispersion, such as a higher reliance on frequent collaboration and communication. But to be clear, they do not replace the value of face-to-face conversation, they are merely a crutch to not having it.
10. Consistent Education and Coaching
To ensure consistency and a shared understanding, whole teams (including the business, IT, and their leadership of executives and managers) should receive a common skills development and education experience in proper Agile Thinking, the Scrum Framework, aligned practices and mindset training. Coaching should then reinforce this new knowledge and encourage appropriate behaviours to turn these new practices into habits. Indeed, learning should be a continuous cycle and endless journey towards excellence, and Scrum leverages this through frequent retrospection and continuous improvement.
11. The Right Attitude!
Mutual respect and caring are the cornerstone to the team’s success and it needs to be integral to their culture and beliefs. Not just saying but living the belief there are no heroes or scapegoats. Everyone, including the business, executives, team members and leadership must collaborate and share in celebrating the successes as well as accepting responsibility for setbacks and failures.
Everyone must have the right attitude and commit to not only DOING as needed by attending the ceremonies or following the process and practices but truly wanting to BE part of the solution by willingly changing the way they think, work and collaborate.
At the end of the day your goal should not be to become Agile or Scrum savvy. Instead your real goals and outcomes should align with achieving the key benefits of Agility, and with what Scrum offers. These should include (but are not limited to) increased customer satisfaction, faster delivery of value, improved feedback loops, adopting a continuous improvement mindset, improved employee morale and increased employee retention. Scrum is just one of the many tools or approaches you may choose to get there, but it is certainly an important one to consider if these outcomes align with your goals.
For Scrum to be truly successful at your organization, you must dramatically transform your very culture and business approach. To be clear, this is not easy to do but the rewards are well worth the effort. By embracing such a transformation, the adopted change in behaviour, beliefs and practices should result in a more successful Scrum experience and a higher degree of satisfaction for both your customers and employees.
Can you think of other success factors that might help your Scrum team succeed? There are lots, so feel free to reach out and share them below.
Thanks to Photographer: Chris Potter for this awesome photo.
Sourced from stockmonkeys.com | Flickr Portfolio
Lately I’ve been appreciating the Top 100 Agile Websites compiled by Oikosofy.
Just out of curiosity, I thought I’d check out Number 100, just to see who was the lucky guy who wasn’t listed as number 101.
What I found was a delightfully surprising and pleasantly entertaining blog by a coach named Yves Hanoulle. One article which particularly caught my attention is called “Getting out of your comfort zone.”
8 key points for coaches creating safety for their teams
- Getting out of your comfort zone is important for personal improvement
- When you do experiements as a coach to learn people about this, people might see things differently, because of their earlier experiences.
- Give people a safe environment so they can learn to push their boundaries.
- People need to feel safe to move out of their comfort zone.
- The Safety Zone is bigger then Comfort Zone.
- Stepping out of your Comfort zone increases the size of Safety Zone.
- Staying to long in your Comfort Zone decreases your safe zone.
- Safety zone is perceived.
His reflections after a training seminar on the topic really made a lot of sense to me.
Basically, an agile team is striving to create a Safe Zone for themselves and their team-mates so people will take risks and move out of their comfort zone. In order to do this, they are or become really comfortable with that uneasy state of being uncomfortable.
It’s as though it is no longer uncomfortable to be uncomfortable.
When an agile team moves out of its Comfort Zone together and everyone feels safe and supported, the end result is the type of team described in the Agile Manifesto. It’s the type of team companies really get excited about. It’s the type of team people love to work with and in doing so they may find they love their work more than ever.
Agile transformation coaches promise their clients the positive outcome of “high-performance teams.”
According to the well-cited Psychologist B.W Tuchman, teams go through four stages on their way to high-performance. The end result seems to be a self-organizing team which effectively delivers to clients or customers with increasing satisfaction and continuous development and growth.
However, agile teams are different than regular teams. Aren’t they?
What I mean is, right from the outset individuals in an agile culture expect to confront change with positive stride. They are expected to be able to adapt to quickly even in uncertain environments. Therefore, their experience of team development is different, right from the outset.
Consider what Debbie Madden has to say in her article The Increasing Fluidity of Agile Practices Across Teams. She writes that, “most companies either claim they are Agile, are trying to become Agile, or have tried Agile. In truth, what I see today is a lot of customized Agile. In fact, the term “Traditional Agile” has come to mean the pure, original implementation of Agile. And, most companies are not following “Traditional Agile”. Instead, teams are customizing Agile to fit their needs, making the fluidity of Agile more prominent now than ever before.”
What this says to me is that since “Traditional Agile” has been around long enough now, teams have internalized the principles and values enough to understand change is to be expected and they have strategies in place to adapt well.
It says to me that teams are now taking Agile to a whole new level. They are making it their own. Adapting. Shaping. Moulding. Sculpting. The fluid nature of Agile gives teams permission to do this.
If we take Tuchman’s four-stage model and insert some agile thinking what we might come out with is an awareness that agile teams do what Debbie said they do. They make things up as they go along and they get the job done.
In this way, what might have been called “storming” by the old standards and definitions of team development can really also be called “high-performance” when the team is agile.
Perhaps some agile teams can create their own team development model and one of the stages is “high-performing storming” and maybe that is not even the final outcome but maybe it is the starting point on Day One!
Wouldn’t that be something?
Often times, as I’ve been researching about agile methods and how to apply these to create real and sustainable change in an organization, I come across reference to the Agile Manifesto. I list it here today for those who are new to the field or who are getting back to the roots after trying a few things with different-than-expected results. It is an instrumental document. The values and principles listed here truly do shape the way agilists think and operate and to some degree or another the results appear to be better than before this founding document was introduced. So here is my “hats off” to this remarkable item which plays a pivotal role in cultural transformation.
The four key values are:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Personally, I find the first one the most meaningful of all. When we value individuals and interactions over process and tools we are truly improving in leaps and bounds in creating collaborative environments which are continuously improving.
The Hunt for Better Retrospectives
The rumours had started to spread, retrospectives at our organization were flat, stale and stuck in a rut. The prevailing thought was that this was stalling the pace of continuous improvement across our teams. In truth, I wasn’t sure if this was at all true, it’s a complex problem that has many possible contributing factors. Here are just some possible alternative or co-contributing causes: how the teams are organized, the level of safety, mechanisms to deal with impediments across the organization, cultural issues, levels of autonomy and engagement, competence & ability and so on…
Despite this, it didn’t hurt to have a look for some inspiration on good retrospectives. I really liked Gitte Klitgaard’s talk called Retrospectives are Boring and Useless – Or are They? In particular the parts around preparing and establishing safety.
On the theme of safety, I thought we could try to go as far as having fun; we’d already had lots of success with the getKanban game (oh Carlos you devil!). Where it all tied together for me, was being inspired by the great question-based approach from cultureqs.com that I’d had a chance to preview at Spark.
If I could create a game with the right prepared questions, we could establish safety, the right dialogue and maybe even have some fun.
The Retro Game
This is a question-based game that I created that you could use to conduct your next retro for teams of up to 10 people. The rules of the game are fairly simple and you could play through a round or two in about 1 to 2 hours depending on team size and sprint duration. Prep time for the facilitator is about 2-4 hours.
Prepping to play the game
You, as facilitator, will need to prepare for 3 types of questions that are thought of ahead of time and printed (or written) on the back of card-stock paper cards.
One question per card. Each question type has its unique colour card. About 8 questions per category is more than enough to play this game.
The 3 types of questions are:
In the Moment – These are questions that are currently on the mind of the team. These could be generated by simply connecting with each team member ahead of time and asking, “if you could only talk about one or two things this retro, what would it be?” If for example they responded “I want to talk about keeping our momentum”, you could create a question like “what would it take to keep our momentum going?”
Pulse Check – These are questions that are focused on people and engagement. Sometimes you would see similar questions on employee satisfaction surveys. An example question in this category could be “What tools and resources do we need to continue to be successful?”
Dreams and Worries – This is a longer-term view of the goals of the team. If the team has had any type of Lift Off or chartering exercise in the past, these would be questions connected to any goals and potential risks that have been previously identified. For example if one of a team’s goal is to ship product updates every 2 weeks, a question could be “What should we do next to get closer to shipping every 2 weeks?”
On the face-up side of the card it should indicate the question type as well as have room to write down any insights and actions.
You will also need:
- To print out the game board
- To print out the rule card
- Bring a 6-sided dice
Playing the Game
Players sit on the floor or at a table around the game board. The cards are in 3 piles, grouped by type, with the questions face down.
- The person with the furthest birthday goes first.
- It is their turn and they get to roll the dice.
- They then choose a card from the pile based on the dice roll. A roll of 1 through 3 is an “In the Moment” card, 4 is a “Pulse Check” and 5 to 6 “Dreams & Worries”
- They then read the card question on the card out loud and then pass the card to the person on the right.
- The person on your right is the scribe, they will capture notes in the Insight and Actions boxes of the card for this round.
- Once they have read the question, they have a chance to think and then answer the question out loud to the group. Nobody else gets to talk.
- Once they’ve answered the question, others can provide their thoughts on the subject.
- After 3 minutes, you may wish to move on to the next round.
- At the end of each round the person whose turn it was chooses the person who listened and contributed to the discussion best. That person is given the card to keep.
- The person to the left is given the dice and goes next.
Winning the Game
- The game ends at 10 minutes prior to the end of the meeting.
- At the end of the game, the person with the most cards wins!
- The winner gets the bragging rights (and certificate) indicating they are the retrospective champion!
- You should spend the last 10 minutes reflecting on the experience and organizing on the action items identified.
Concepts at Play
Context & Reflection – Preparation is key, particularly for the “In the Moment” section. The topics will be relevant and connect with what the team wants to talk about. Also when presented in the form of a question they will likely trigger reflection for all those present.
Sharing the Voice – Everyone gets a chance to speak and be heard without interruptions. The game element also incentivises quality participation.
Coverage of topic areas – The 3 question categories spread the coverage across multiple areas, not just the items in the moment. The probabilities are not however equal, for example there is a 50% chance of “In the Moment” being chosen in each turn.
Fun & Safety – The game element encourages play and friendlier exchanges. You are likely to have dialogue over debate.
Want to play the game?
I’d love to hear how this game worked out for you. I’ve included everything you need here to setup your own game. Let me know how it went and how it could be improved!
This post is a follow-up to an earlier article: There Are No Breaks Between Sprints.
Breaks between Sprints indicate a problem. Usually such breaks are filled with planning activities including research, requirements gathering, design & preparation, negotiations & approvals and the problem is threefold:
- Such plans are based on conjecture (risky and not compatible with Scrum) rather than empiricism (less risky and compatible with Scrum). Those activities are most beneficial when diligently performed by skilled inspectors at the point of the work. The four formal events within each Sprint provide the team and stakeholders adequate opportunity for inspection and ensure that decisions are being made in light of the up-to-date product increment and with respect to current user needs and market conditions.
- Breaks between Sprints often include activities which do not add value to the product or are entirely unrelated.
- Breaks between Sprints defer the delivery of value because the work performed does not result in potentially-releasable increment of “Done” product.
To correct this problem it is important to identify whether any of the effort spent between Sprints is adding value to the product — that is, which activities effect the form, fit, or function of the actual product. If determined to not be value adding, stop the activity entirely — it is waste. If determined to be value adding then the work ought to be part of their Sprints and the Scrum Team may decide that either the activity should be represented and ordered in the Product Backlog, or should be represented in the team’s Definition of Done.
I recently said goodbye to one of my organization’s Agile Coaches and I felt that I needed to take a pause and reflect to consider my next move. The engagement had gone well, in fact one of the best we’ve had, but not without its share of successes and failures. But the successes had clearly outpaced any failures, and so there was a lot of good I wanted to build on.
The departing coach was part of a 3rd generation of Agile Coaches that I had worked with in the 3 years since we had begun our company’s transformation to Agile. And while he was a great coach, so were his predecessors and yet they had had fewer successes.
On reflection, what had really happened is that we had changed as a company; we had learned how to better execute our engagements with an Agile Coach.
Deciding to hire an Agile Coach.
Deciding to hire an Agile Coach can be a big step. A couple of things need to have happened, you’ve recognized that you need some help or at least another perspective. And given that Agile Coaches are typically not very cheap, you have decided to invest in your Agile transformation, however big or small. You’re clearly taking it seriously.
However, through my experiences I noticed that things can get a little tricky once that decision has been made. Many organizations can fall into a trap of externalizing transformation responsibilities to the Agile Coach.
In essence thinking along the lines of “as long as I hire a good coach, they should be able to make our teams Agile” can take you into an engagement model that is not very Agile in the first place.
Much like how Scrum and other Agile Practices connect customers with teams and establishes shared risk, an organization’s relationship with their Agile Coaches need to be a working partnership.
Positive Patterns for Coaching Engagements
So it’s important for you to setup the right engagement approach to get value out of your Agile Coach and this goes beyond the hard costs of their services, but also the high cost of failure with not having the right coaching in the right areas.
Here are 5 positive patterns for coaching engagements that I’ve observed:
1. Identify the Customer
Usually it is management who will hire a coach, and they may do so to help one or more teams with their Agile adoption needs. So in this scenario who is the customer? Is it the person that hired the coach or the teams (the coachees) who will be receiving the services? In some cases, the coachees aren’t clear why the coach is there, they haven’t asked for their services and in some cases may even feel threatened by their presence.
For this reason, if management is hiring coaches you need to recognize that there is a 3-pronged relationship that needs to be clearly established and maintained.
With the customer in this case being someone in management, i.e. the person who hired the coach in the first place. The customer’s responsibility will be to not only identify the coachee but then work with the coach to establish and support that relationship.
2. Set the Mandate
Agile Coaches typically tend to be more effective when they have one or two specific mandates tied to an organization’s goals. Not only is the mandate important to establish why the coach is there, too many goals can significantly dilute the coach’s effectiveness. Put another way, Agile Coaches are not immune to exceeding their own Work in Progress limits.
The mandate establishes why the coach is there, and should be tied to some sort of organizational need. A good way of developing this is to articulate what is currently happening and the desired future state you want the coach to help with.
The teams on our new program are great at consistently delivering their features at the end of each sprint. However, we still experience significant delays merging and testing between teams in order for the program to ship a new release. We’d like to reduce that time significantly, hopefully by at least half.
Once the engagement is well underway you may find that the coach, through serendipity alone, is exposed to and gets involved with a wide variety of other areas. This is fine, but it’s best to just consider this to be a side show and not the main event. If other activities start to take on a life of their own, it’s probably a good time to go back to inspect and potentially adjust the mandate.
If you’re not sure how to establish or identify your Agile goals, this could be the first goal of any Agile coach you hire. In this scenario, the customer is also the coachee and the mandate is to get help establishing a mandate.
3. Hire the Coach that fits the need
Agile coaches are not a homogeneous group, with many degrees of specialty, perspective and experiences. Resist the desire to find a jack-of-all-trades, you’re as likely to find them as a unicorn.
Your now established mandate will be your biggest guide to what kind of coach you should be looking for. Is the need tied to technical practices, process engineering, team collaboration, executive buy-in, transforming your culture, etc?
The other part is connected with the identified coachee. Are the coachees team members, middle management or someone with a “C” in at the start of their title? Will mentoring be required or are you just here to teach something specific?
Using something like ACI’s Agile Coaching Competency Framework, would be a good model to match the competencies required of the perspective coach.
In my example earlier, in order for your team to get help with their merging & testing needs, you may have to look for a coach with the right skills within the Technical Mastery competence. And if you have technical leaders who are championing the change, potentially the ability to Mentor.
4. Establish Feedback Loops
With the coach, customer and mandate clearly identified, you now need to be ready to devote your time to regularly connect and work with the coach. Formalizing some sort of cadence is necessary, if you leave it to ad hoc meetings you will typically not meet regularly enough and usually after some sort of failure has occurred.
The objective of these feedback loops is to tie together the communication lines between the 3 prongs established: the customer, the coach and the coachees. They should be framed in terms of reviewing progress against the goals established with the mandate. If the coachees ran any experiments or made any changes that were intended to get closer to the goals, this would be the time to reflect on them. If the coachees need something from the customer, this would be a good forum to review that need.
Along with maintaining a cadence of communication, feedback loops if done regularly and consistently, could be used to replace deadlines, which in many cases are set simply a pressure mechanism to maintain urgency. So statements like “Merge & test time is to be reduced by half by Q2” now become “We need to reduce merge and test time by half and we will review our progress and adjust every 2 weeks.”
5. It doesn’t need to be Full Time
Resist the temptation to set the coach’s hours as a full-time embedded part of the organization or team. While you may want to have the coach spend a significant amount of time with you and your coachees when the engagement is starting, after this period you will likely get a lot more value from regular check-ins.
This could look like establishing some sort of rhythm with a coachee: reviewing challenges, then agreeing on changes and then coming back to review the results after sufficient time has passed.
This approach is more likely to keep the coach as a coach, and prevents the coach from becoming entangled in the delivery chain of the organization. The coach is there to help the coachees solve the problems, and not to become an active participant in their delivery.
Time to get to work
Bringing in an Agile Coach is an excellent and likely necessary part of unlocking your Agile transformation. However, a successful engagement with a coach will have you more connected and active with your transformation, not less. So consider these 5 positive coaching engagement patterns as I consider them moving into my 4th generation of Agile coaches. I expect it will be a lot of work, along with a steady stream of great results.
The best architectures, requirements and designs emerge from self-organizing teams.
The quality of our software systems depends on refactoring. In fact, I believe that the only way that an organization can avoid refactoring is by going out of business. Maybe I should explain that.
Refactor or Die
Every software system that we build is inside a dynamic environment. The organization(s) using the software are all in a state of constant change. The people using the software are also constantly changing. Due to this constant change, every software system needs to be adapted to the environment in which it is used. Most of the time, businesses think of this constant change in terms of new features and enhancements – the scope of functionality that a system can handle. Less commonly, businesses think of this change in terms of the obvious external qualities and attributes of the system such as performance or security. But almost never does an organization, from a business perspective, think of the invisible qualities of the software system such as simplicity and technical excellence.
What happens when the business does not recognize those invisible qualities? I’m sure almost every software developer reading this can answer this question easily: the system becomes “crufty”, hard to maintain, bug-prone, costly to change, maze-like, complex. Some people refer to this as legacy code or technical debt.
The longer this state is allowed to continue, the more it costs to add new features – the stuff that the business really cares about. It is pretty easy to see how this works – for someone who has a technical background. But for those without a technical background it can be hard to understand. Here is a little analogy to help out.
Imagine that you set up a system for giving allowance to your kids. In this system, every week your kids have to fill out a simple form that has their name, the amount that they are requesting, and their signature. After a few weeks of doing this, you realize that it would be helpful to have the date on the form. You do this so that you can enter their allowance payments in your personal bookkeeping records. Then you decide that you need to add a spot for you to counter-sign so that the paper becomes a legal record of the allowance payment. Then your kids want extra allowance for a special outing. So you add some things on the form to allow them to make these special requests. Your accountant tells you that some portions of your kids allowance might be good to track for tax purposes. So, the form gets expanded to have fields for the several different possible uses that are beneficial to your taxes. Your form is getting quite complex by this point. Your kids start making other requests like to be paid by cheque or direct-deposit instead of in cash or to be paid advances against future allowances. Every new situation adds complexity to the form. The form expands over multiple pages. Filling out the form weekly starts to take significant time for each child and for you to review them. You realize that in numerous places on the form it would be more efficient to ask for information in a different way, but you’re not sure if it will have tax implications, so you decide not to make the changes… yet. You decide you need your own checklist to make sure that the forms are being filled out correctly. A new tax law means that you could claim some refunds if you have some additional information… and it can be applied retroactively, so you ask your kids to help transcribe all the old versions of the form into the latest version. It takes three days, and there is lots of guess-work. Your allowance tracking forms have become a bureaucratic nightmare.
The forms and their handling is what software developers have to deal with on a daily basis – and the business usually doesn’t give time to do that simplification step. The difference is that in software development there are tools, techniques and skills that allow your developers to maintain a system so that it doesn’t get into that nightmare state.
For a more in-deth description of this process of systems gradually becoming more and more difficult to improve, please see these two excellent articles by Kane Mar:
Ultimately, a software system can become so crufty that it costs more to add features than the business benefit of adding those features. If the business has the capacity, it is usually at this point that the business makes a hard decision: let’s re-write the system from scratch.
I used the word “decision” in that last sentence. What are the other options in that decision? Ignoring the problem might be okay for a while longer: if the company is still getting benefit from the operation of the system, then this can go on for quite a while. Throwing more bodies at the system can seem to help for a bit, but there are rapidly diminishing returns on that approach (see The Mythical Man-Month for details). At some point, however, another threshold is reached: the cost of maintaining the operation of the system grows to the point where it is more expensive than the operational value of the system. Again, the business can make a hard decision, but it is in a worse place to do so: to replace the system (either by re-writing or buying a packaged solution), but without the operating margin to fund the replacement.
In his articles, Kane Mar describes this like so:
It’s pretty clear that a company in this situation has some difficult decisions ahead. There may be some temporary solution that would allow [a company] to use the existing system while building a new product, [A company] may decide to borrow money to fund the rewrite, or [a company] may want to consider returning any remaining value to their shareholders.
In other words, refactor or die.
Refactoring and Business
In the Scrum Master and Product Owner classes that we teach, this topic comes up frequently: how does the business account for refactoring? How do we “govern” it? How do we make good decisions about refactoring?
There are a few principles that are important in helping to answer these questions. All of these principles assume that we are talking about refactoring in an Agile team using a framework like Scrum, OpenAgile, or Kanban.
Refactoring Principle One: Keep It Small
Refactoring is safest and cheapest when it is done in many small increments rather than in large batches. The worst extreme is the complete system re-write refactoring. The best refactoring activities take seconds or minutes to execute. Small refactorings create a constant modest “overhead” in the work of the team. This overhead then becomes a natural part of the pace of the team.
Not all refactoring moves can be kept so small. For example, upgrading a component or module from a third party might show that your system has many dependencies on that module. In this case, efforts should be made to allow your system to use both the old and the new versions of the component simultaneously. This allows your system to be partially refactored. In other words, to break a large refactoring into many small refactorings. This, in turn, may force you to refactor your system to be more modular in its dependencies.
Another common problem with keeping refactorings small is the re-write problem. Your own system may have a major component that needs to be re-written. Again, finding creative technical means to allow for incremental refactoring of the component is crucial. This can often mean having temporary structures in your system to allow for the old and new parts to work harmoniously. One system that I was working on had to have two separate database platforms with some shared data in order to enable this “bi-modal” operation.
Refactoring Principle Two: Business Catalysts
When is the earliest that a refactoring should be done? Not whenever the technical team wants to do it. Instead, the technical team needs to use business requests as catalysts for refactoring. If the business needs a new feature, then refactoring should only be done on those parts of the system that are required to enable that feature. In other words, don’t refactor the whole user interface, just refactor the parts that relate to the specific business request.
Again, there can be exceptions to doing this… but only in the sense that some refactorings might be delayed until a later date. This is tricky: we want to make sure that we are not accumulating technical debt or creating legacy code. So, instead, we need to allow the technical team to refactor when they detect duplication. Duplication of code, data or structure in the system. A business request might impact a particular part of the system and the team sees how it might be necessary to refactor a large swath of the system as a result. But, the cost of doing so is not yet justified: the single request is not enough of a catalyst, and the team can also choose a simple temporary solution. Later, the business makes another request that also implies the same large refactoring. Now is the time to seriously consider it. It is now a question of duplication of another simple temporary solution. The business may not be happy with the extra expense of the large refactoring so the principle of keeping it small still applies. However, the technical team must also be willing to push back to the business under the right circumstances.
Refactoring Principle Three: Team Cohesion
Teamwork in Agile requires high levels of communication and collaboration. In refactoring work, teamwork applies just as much as in any other activity. Here, it is critical that all members of the team have a unified understanding of the principles and purpose of refactoring. But that is just the first level of team cohesion around refactoring.
The next level of team cohesion comes in the tools, techniques and practices that a team uses in refactoring. Examples include the unit testing frameworks, the mocking frameworks, the automation provided by development tools, continuous integration, and perhaps most importantly, the team working agreements about standard objectives of refactoring. This last idea is best expressed by the concept of refactoring to patterns.
The highest level of team cohesion in refactoring comes from collective code ownership and trust. Usually, this is built from practices such as pair programming or mob programming. These practices create deep levels of shared understanding among team members. This shared understanding leads to self-organizing behaviour in which team members make independent decisions that they know the other team members will support. It also impacts research and learning processes so that teams can do experiments and try alternatives quickly. All of which leads to the ability to do refactoring, large and small, quickly and without fear.
Refactoring Principle Four: Transparency
In many ways, this is the simplest refactoring principle: the team needs to be completely open and honest with all stakeholders about the cost of refactoring. This can be difficult at first. Another analogy helps to see the value of this. A surgeon does not hide the fact that care is put into creating a clean operating environment: washing hands, sterilizing instruments, wearing face masks and hair covers, restricted spaces, etc. In fact, all of those things contribute to the cost of surgery. A surgeon is a professional who has solid reasons for doing all those things and is open about the need for them. Likewise, software professionals need to be open about the costs of refactoring. This comes back to the main point of the first part of this article: hidden and deferred costs will still need to be paid… but with interest. Software professionals are up-front about the costs because doing so both minimizes the costs and gives stakeholders important information to make decisions.
The challenge for business stakeholders is to accept the costs. Respecting the team and trusting their decisions can sometimes be very hard. Teams sometimes make mistakes too, which complicates trust-building. The business stakeholders (for example, the Product Owner), must allow the team freedom to do refactoring. Ideally, it is continuous, small, and low-level. But once in a while, a team will have to do a large refactoring. How do you know if the cost is legitimate? Unfortunately, as a non-technical stakeholder, you can’t know with certainty. However, there are a few factors that can help you understand the cost and it’s legitimacy, namely, the principles that are described here.
If the refactoring is small, it is more likely to be legitimate.
If the refactoring is in response to a business catalyst, it is more likely to be legitimate.
If the refactoring is reflective of team cohesion, it is more likely to be legitimate.
And, of course, if the refactoring is made transparent, it is more likely to be legitimate.
Hi, David here.
The Perfect Agile Tool doesn’t yet exist. In my training and consulting work, I often have strong words to say about electronic tools. Most of the tools out there are really bad. Unfortunately, JIRA, the most common tool, is also the worst that I know of. (Actually, the only tool worse than JIRA for an Agile team is MS Project – which is just plain evil). Some Agile tools do a bit better, but most fall far short of a good physical task board (information radiator). I am often asked to evaluate and / or partner with tool vendors to “bless” their products. Here is what I am looking for before I will consider an outright endorsement of such a tool.
Features for a Perfect Agile Tool
This list is roughly organized in order of features which do show up in some tools to those which I have never seen or heard of in tools.
1. Skeumorphism: Cards and Wall
The tool should display the current work of an Agile team in a way that is immediately recognizable as a set of note cards or PostIt’s on a physical wall. This includes colours, sizes, etc. Most people will type to enter data so fonts should be chosen to mimic hand-printed letters. Every aspect of the display should remind people of the physical analogue of the tool.
2. Live Update
As team members are using the tool, all updates that they make should be visible as immediate updates to all the other team members including typing, moving cards around, etc. There is no off-line mode for the tool. In fact, if the tool is not receiving live updates, it should be clearly disabled so that the team member knows there is a problem with the information they have displayed.
3. Simple or No Access Control
Most Agile methods strongly de-emphaisize or even disallow traditional roles and encourage self-organizing teams. This means that fine-grained access control to different features of the tool should be eschewed in favour of extremely simple access control: everyone can do anything with the tool. (It actually helps if there is no “undo” feature, just like there’s no easy way to erase Sharpie written on a note card.)
4. Infinite Zoom In/Out
When you are using cards on a wall, it is easy to see the whole wall or to get up close and see even very fine details on a single note card. Although it does not have to be literally infinite, the wide and tight zoom levels in the tool should be at least a few orders of magnitude difference. As well, the zoom feature should be extremely easy to use, similar perhaps to the way that Google Maps functions. Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.
5. Touch Device Compatible
This seems like a super-obvious feature in this day and age of tablets, smart phones and touch-screen laptops. And it would take the cards on the wall metaphor just that extra little way. But very few tools are actually easy to use on touch devices. Dragging cards around and pinch to zoom are the obvious aspects of this feature. But nice finger-drawing features would also be a big plus (see below)!
6. Size Limit on Cards
For techies, this one is extremely counter-intuitive: limit the amount of information that can be stored on a “card” by the size of the card. It shouldn’t be possible to attach documents, screen shots, and tons of meta-data to a single card. Agile methods encourage time-boxing (e.g. Sprints), work-boxing (e.g. Work-in-Process limits), and space-boxing (e.g. team rooms). This principle of putting boundaries around an environment should apply to the information stored on a card. Information-boxing forces us to be succinct and to prefer face-to-face communication over written communication. Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.
7. Minimal Meta-Data
Information-boxing also applies to meta-data. Cards should not be associated with users in the system. Cards should not have lots of numerical information. Cards should not have associations with other cards such as parent-child or container-contained. Cards should not store “state” information except in extremely limited ways. At most, the electronic tool could store a card ID, card creation and removal time-stamps, and an association with either an Agile team or a product or project.
8. Overlapping Cards
Almost every electronic tool for Agile teams puts cards in columns. Get rid of the columns, and allow cards to overlap. If there is any “modal” behaviour in the tool, it would be to allow a team member to select and view a small collection of cards by de-overlapping them temporarily. Overlapping allows the creation of visually interesting and useful relationships between cards. Cards can be used to demarcate columns or groupings without enforcing strict in/out membership in a process step.
9. Rotatable, Foldable, Rip-able Cards
Increase the fidelity of the metaphor with physical cards on a wall. Rotation, folding and ripping are all useful idioms for creating distinct visual cues in physical cards. For example, one team might rotate cards 45 degrees to indicate that work is blocked on that card. Or another team might fold a dog-ear on a card to indicate it is in-progress. Or another team might rip cards to show they are complete. The flexibility of physical cards needs to be replicated in the electronic environment to allow a team to create its own visual idioms. Among all the other features I mention, this is one of the top three in importance for the perfect Agile tool.
10. Easy Sketching on Cards… Including the Back
Cards should allow free-form drawing with colours and some basic diagramming shapes (e.g. circles, squares, lines). Don’t make it a full diagramming canvas! Instead, allow team members to easily sketch layouts, UML, or state diagrams, or even memory aides. The back side of the card is often the best place for more “complex” sketches, but don’t let the zoom feature allow for arbitrarily detailed drawing. Lines need a minimum thickness to prevent excessive information storage on the cards.
11. Handwriting Recognition
With Siri and other voice-recognition systems, isn’t it time we also built in handwriting recognition? Allowing a team member to toggle between the handwriting view and the “OCR” view would often help with understanding. Allow it to be bi-directional so that the tool can “write” in the style of each of the team members so that text entry can be keyboard or finger/stylus.
12. Sync Between Wall and Electronic Tool
This is the most interesting feature: allow a photo of cards on a wall to be intelligently mapped to cards in an electronic tool (including creating new cards) and for the electronic tool to easily print on physical note cards for placement on a wall. There is all sorts of complexity to this feature including image recognition and a possible hardware requirement for a printer that can handle very small paper sizes (not common!)
These are the features that many electronic tools implement as part of being “enterprise-ready”. I’ll be brief on these points:
No Individual Tracking – the team matters, not who does what.
No Dependency Management – teams break dependencies, tools don’t manage dependencies.
No Time Tracking – bums in seats typing doesn’t matter: “the primary measure of progress is working software” (or whatever valuable thing the team is building) – from the Agile Manifesto.
No Actuals vs. Estimates – we’re all bad at predicting the future so don’t bother with trying to get better.
No Report Generation – managers and leaders should come and see real results and interact directly with the team (also, statistics lie).
No Integration Points – this is the worst of the anti-features since it is the one that leads to the most anti-agile creeping featuritis. Remember: “Individuals and interactions [are valued] over processes and tools” – from the Agile Manifesto.
Evaluation of Common Agile Tools
I go from “Good” to “Bad” with two special categories that are discontinuous from the normal scale: “Ideal” and “Evil”. I think of tools as falling somewhere on this scale, but I acknowledge that these tools are evolving products and this diagram may not reflect current reality. The scale looks like this, with a few examples put on the scale:
Plea for the Perfect Agile Tool
I still hope that some day someone will build the perfect Agile tool. I’ve seen many of the ideal features listed above in other innovative non-Agile tools. For example, 3M made a PostIt® Plus tool for the iPhone that does some really cool stuff. There’s other tools that do handwriting recognition, etc. Putting it all together in a super-user-friendly package would really get me excited.
Let me know if you think you know of a tool that gets close to the ideal – I would be happy to check it out and provide feedback / commentary!
I’m going to be presenting a quick session on the use of a Skills Matrix to help launch a team. If you are in the Toronto area on the evening of March 16, come check it out: Agile TO Meetup.