Agile Advice - Working With Agile Methods (Scrum, XP, Lean)

This post is a little geeky and technical and product-related for AgileAdvice, and is a shameless self-promotion. Nevertheless, since testability, test-driven-development, and incremental design are non-exclusive sub-topics of Agile, I though I’d report this here anyway.

Many developers use the Dependency Injection and Inversion of Control (IoC) patterns through such IoC containers as Spring, Hivemind, Picocontainer, and others. They have all sorts of benefits to testability, flexibility, etc. that I won’t repeat here, but can be read about here, here, and here. A great summary of the history of “IoC” can be found here. J2ME developers, however, especially those on limited devices that use the CLDC configuration of J2ME, cannot use the substantial number of IoC/DI containers out there, because they nearly all rely on reflection. These also often make use of APIs not present in the CLDC - APIs which could not easily be added. Lastly there’s a tendency among developers of “embedded software” to be very suspicious of complexity.

In working out some examples of DI as part of a testability workshop at one of my clients, I whipped up a quick DI container, and being the freak that I am, hardened it until it was suitable for production, because I hate half-finished products. So allow me to introduce the Israfil Micro Container. (That is, the Container from the Israfil Micro project). As I mention in the docs, “FemtoContainer” just was too ridiculous, and this container is smaller than pico-container. The project is BSD licensed, and hosted on googlecode, so source is freely available and there’s an issue/feature tracker, yadda yadda.

Essentially I believe that people working on cellphones and set-top boxes shouldn’t be constrained out of some basic software design approaches - you just have to bend the design approach to fit the environment. So hopefully this is of use to more than one of my clients. It currently supports an auto-wiring registration, delayed object creation (until first need), and forthcoming are some basic lifecycle support, and a few other nicities. It does not use reflection (you use a little adapter for object creation instead), and performs quicker than pico-container. Low, low overhead. It’s also less than 10 classes and interfaces (including the two classes in the util project). It’s built with Maven2, so you can use it in any Maven2-built project with ease, but of course you can always also just download the jar (and the required util jar too). Enjoy…

P.S. There are a few other bits on googlecode that I’m working on in the micro-zone. Some minimalist backports of some of java.lang.concurrency (just the locks), as well as some of the java.util.Collections stuff. Not finished, but also part of the googlecode project.

This was actually cribbed from a Bruce Schneier blog post about security…

Security engineers see the world differently than other engineers. Instead of focusing on how systems work, they focus on how systems fail, how they can be made to fail, and how to prevent–or protect against–those failures. Most software vulnerabilities don’t ever appear in normal operations, only when an attacker deliberately exploits them. So security engineers need to think like attackers.People without the mindset sometimes think they can design security products, but they can’t. And you see the results all over society–in snake-oil cryptography, software, Internet protocols, voting machines, and fare card and other payment systems. Many of these systems had someone in charge of “security” on their teams, but it wasn’t someone who thought like an attacker.  

There’s an interesting parallel between this statement and how most software quality is handled. Quality and Security are similar. In fact, I see security as a very specific subset of quality-mindedness. Certainly both require the same mindset to ensure - rather than thinking merely “how will this work”, a quality-focused person will also, or perhaps alternately think: “how might this be breakable”. From this simple change in thinking flows several important approaches

• Constraint-based thinking (as opposed to solution based thinking): allows an architect/developer to conceive of the set of possible solutions, rather than an enumeration of solutions. By looking at constraints, a developer implements the lean principle of deciding as late as possible, with as full information as possible.
• Test-First: As one thinks of how it might break, scenarios emerge that can form the basis of test cases. These cases form a sort of executable acceptance criteria
• Lateral Thinking: The constraint+test approach starts to get people into a very different mode, where vastly different kinds of solutions show up. The creative exercise of trying to break something provides insights that can change the whole approach of the system.

 Schneier goes on to ponder 

This mindset is difficult to teach, and may be something you’re born with or not. But in order to train people possessing the mindset, they need to search for and find security vulnerabilities–again and again and again. And this is true regardless of the domain. Good cryptographers discover vulnerabilities in others’ algorithms and protocols. Good software security experts find vulnerabilities in others’ code. Good airport security designers figure out new ways to subvert airport security. And so on.  

 Here again - I think it’s possible to help people get a mind-set about quality, but some do seem to have a knack. It’s important to have some of these people on your teams, as they’ll disturb the waters and identify potential failure modes. These are going to be the ones who want to “mistake proof” (to borrow Toyota’s phrase) the system by writing more unit tests and other executable proofs of the system. But most importantly (and I can personally testify to this) it is critical that people just write more tests. It is a learned skill to start to think of “how might this fail” until it becomes a background mental thread, always popping up risk models.A related concept is Demmings’ “systems-thinking”, which, applied to software quality, causes one to start looking at whole ecosystems of error states. This is when fearless re-factoring starts to pay off, because the elimination of duplication allows one to catch classes of error in fewer and fewer locations, where they’re easier to fix. There are many and multifarious spin-off effects of this inverted questioning and the mindset it generates. Try it yourself. When you’re writing code, ask yourself how you might break it? What inputs, external state, etc. might cause it to fail, crash, or behave in odd ways. This starts to show you where you might have state leaking into the wild, or side-effects from excessively complex interactions in your code. So quality focus can start to improve not only the external perception of your product, but also its fitness to new requirements by making it more resilient and less brittle. Cleaner interactions and less duplication allow for much faster implementation of new features.I could go on, but I just wanted to convey this sense of “attitude” or “mindset,” over mere technique. Technique can help you get to a certain level, but you have to let it “click”, and the powerful questions can sometimes help.

Everywhere I go, there are three practices that make the biggest difference in overall productivity for teams and organizations. All three practices are part of agile methods such as Scrum and Extreme Programming, but you don’t need to be doing these methods to take advantage of these practices. All of them are relatively inexpensive, and the return on investment for these practices is HUGE!!! Without further ado…

1. A Proper Team Room

This is astonishing: you can expect a 60% boost in team productivity from this single practice! The cost of re-stacking your cubes or office spaces is trivial compared to the benefits. If you are going to do this, do it right! Do the research, hire an agile coach or consultant, but make sure it is done well. One organization I worked with was very excited about their new team room setup. They had a nice open-concept layout with lots of windows etc. But they had also made some big mistakes including that all the developers on a single team would have a low wall separating them from each other. Because of poor layout that would block communication paths, their new setup would actually be worse than their old setup! Some research has shown that you can expect as much as a doubling of productivity (reference). This is one practice you don’t want to let your competitors pick up before you do! Here are some tips on agile team room setup.

2. Short Iterations

Once you have set up your team room, it is critical for your team to have something to do! The fastest way to get your team doing something is to start using short cycles of work (iterations, sprints) to deliver valuable results such as working software. Many software development projects use iterations that are two weeks long or even a month long. I strongly recommend iterations that are only one week long. Again, the benefits are incredible: your team will move through the stages of team development (forming, storming, norming and performing - reference) much more quickly than with longer iterations or no iterations… thus leading to high productivity much sooner. The value here is in the time gained. This chart demonstrates how this works:

The short iterations provide a certain type of pressure that forces team and project crisis to happen quickly. However, because iterations deliver working, valuable results, the pressure is not demoralizing, instead it motivates teams to get through the crisis and reach the norming and performing stages of development quickly. Again, to make this work, there are some critical success factors including methods of allowing team commitment, self-organizing and obstacle removal.

3. Test Driven Development

There is a myth that speed and quality are mutually exclusive. This comes from the idea that you need to slow down to make stuff high quality or that you need to sacrifice quality in order to go fast. It is true that initially you might get gains through these approaches. The really amazing thing happens when you try, deliberately and systematically, to do both high speed and high quality work. In software development this is best done through test driven development. In informal polling I’ve done with teams I’ve worked with, test driven development produces a noticeable long-term productivity gain as well as a simultaneous increase in developer and end user satisfaction due to a substantial reduction in defects discovered after the code leaves the developers. I have seen teams doing this that reduce defect rates to 5% (or less!) of what they once were prior to test driven development… while at the same time delivering projects faster than expected. Since substantial expense is squandered on defect management (tools, support teams, user training, lost productivity, etc.) and since staff turnover is also high in IT and high-tech, the results of test driven development on the bottom line are substantial.

Benefit of All Three Practices

If a team and an organization adopt these practices, get through the initial cost of learning them, then I would like to suggest that your teams can easily double their productivity if not more. For a team of 5 people working on a 100 day project this amounts to shortening the project to 50 days (save $200,000) or get twice as much work done. Clearly, an organization that adopted these practices and perfected them would save huge amounts of money and would be able to crush any competitors not doing this.

Previously I wrote a more general treatment of the benefits of agile and an article that lists other resources discussing the benefits of agile.

Any discussion of benefits should at least say a few words about how exactly to measure those benefits! However, I’m out of time. How do you measure the benefits of agile?

Hey folks, if you found this useful, could you please “Digg” and “Reddit” this article?  Thanks!

Not getting the benefits of agile? Consider the Agile Clinic!