Notes from the talk by Dr. Mark Paulk

PLEASE NOTE: These are raw notes based on both the talk and the slides for the talk. Any errors or omissions are mine. – Mishkin

Research
Practices that characterize the Scrum agile method, along with common variants and tailorings
Which Scrum practices, or variants thereof, have been implemented and the perceived value of the method
Factors affecting Scrum adoption

Scrum

a process for incrementally building software in complex environments
(really likes the Nokia test)
Backlog – all outstanding work for a product area
Sprints – 30-day increments of work that produce a deliverable
Scrums – daily status check meetings

Scrum Practices

practices: 30-day Sprint, Product Backlog, Daily Scrum, …
roles: ScrumMaster, Development Team, Product Owner, …

Variations on the Scrum methodologies

different lengths of Sprints
ScrumMaster = Project manager

People do stupid things because they interpret e.g. CMM to be waterfall
Waterfall is a stupid idea – CMM introduces 7 different lifecycles
Loves iterative, evolutionary lifecycles
What are the variations of Scrum that are legitimate?
From an impericist perspective, Dr. Paulk doesn’t care
Where is the dividing line on Sprint length?
2 weeks? 4 weeks? 6 weeks? 6 months?
Probably universal agreement that 6 months is counter to the Scrum philosophy
But 6 weeks vs. 6 weeks + 1 day?
As empiricist, when hears role of ScrumMaster, says it’s a Project Manager
what about teams that have both a ScrumMaster and a Project Manager?
There are going to be terminology issues
when doing survey, it is important to make sure that people know what you mean when you use certain terms e.g. Product Owner
E.g. What is a “project” vs. “task” (8 hour projects in a CR environment)
Not officially part of the methodology:

Defining “done”
Quitting/stopping and being “done” are not the same thing wrt End of Sprint
Design reviews
Code reviews
Unit tested

Practices commonly associated with other agile methods

if every successful Scrum project did TDD, then we might consider bring TDD into the method definition (relationship to “done”)
Test-driven development
Pair programming
Good engineering and management practices in general
Top-10 risks list
Risk management is done differently in agile approaches (note: the spiral model is explicitly targeted at dealing with risk)
Customer-supplier relationship

Empirical research always leads to surprises – we go in with pre-conceptions
Practice Effectiveness

The Scrum method in principle should be used without much variation – significant variation probably indicates a “ScrumBut” implementation
early termination of a project is not failure – business decision
Practices that are perceived as working will continue to be used…
Perception is important! People will continue to use what they perceive to be effective
“Superstitious” learning
Were any Scrum practices perceived as not being feasible?
Did any Scrum practices not work well when tried?
Can we measure this in any meaningful sense?
Did it increase market share? Etc.?

What is the definition of a successful project?

Dept. Of Def. – meeting schedule is the most important factor
System, culture and environment play a huge part!
cultural clash between DoD and agile approaches do things
overcoming this clash is the only way that the DoD can get benefit from agile

Why Adopt Agile Methods?

pilot the agile practices and see how they work?
Use agile methods on my projects (because I like it)?
Adopt agile methods in a particular domain – where appropriate?
– Scrum is not as software-specific as other agile methods
– used a Scrum-like approach for the development of CMM!
– Some areas where these methods just might not be appropriate e.g. DoD
– Architecture breaker – a problem that requires a project be re-started (Barry Boehm)
– reliability on life-critical systems
Adopt agile methods for the organization?

CMM and agile – complementary
CMM eyes-wide-open, tailor when needed, standardize when possible
what class of projects would agile methods be applicable?
Process community has learned things about the factors that affect adoptions
Assessment readiness survey – even this can be difficult

Assessment failed (1/3)
Assessment completed, but nothing done with it (1/3)
Assessment completed and followed up with changes (1/3)

… fired “customers” who were dead in the water
Measuring Business Success

Does the project have a “product vision” that characterizes success?

How does the project measure success?

financial/market (profit, market share)
quickly responding to changing customer needs
cost and schedule drivers
quality
customer satisfaction / delight
innovation (building for the future)

Where does Scrum/agile fit in terms of business drivers?
Factors Affecting Adoption
some general adoption issues

sponsorship – addressing business problems
resistance to change
we need a requirements specifications
shelfware (facades)
training, user groups, conferences
scope of piloting / adoption
politics, responsibility, accountability

some cultural issues

power and control
uncertainty (when will we be done?)
confrontation vs. Compromise
risk aversion
National cultures are a big factor in methodology adoption

The Agile Alliance (quote the Agile Manifesto)
values need to inform research
e.g. Need to provide human feedback not just documented feedback
Cultural Misfits (Using the DoD as an example)

regulatory requirements for a level playing field raise challenges for evolutionary and incremental development
level playing field for vendors
The need by the contracts officer for a requirements specification
one project the proposal took more effort than the project itself
Progress payments defined from a waterfall mentality
Barriers – regulatory and cultural – to a collaborative customer relationship
Protests from competitors

Research Realities

We’d like more information than most people are willing to take the time to provide
participation by people doing the work is crucial to insight
Surveys inspire more questions
Follow-up interviews provide deeper, but less comparable information
e.g. To explore causality vs. correlation

Research Plans

survey of Scrum projects
Interviews and case studies of selected projects
hoping to chat with people here at the conference!
Publications confirming / testing “what everyone knows”
Scrum practices, variations, and associated practices
business value of Scrum
factors affecting Scrum adoption

Questions and Answers
haven’t seen much evidence that research changes anything – is spending this money worth it?

We don’t use empirical research very much!
Having evidence in toolkit can help discussions with people who have limited time/information to investigate independently e.g. executives
Research can help us understand the barriers to Scrum adoption

Starting with a theory of Scrum to test? If so, what is it?

Starting actually with pure exploration of factors that affect adoption, and practices actually being used – surfacing issues

Couldn’t hear question – seemed to be about security
Would you use the Scrum approach for the research itself?

Collaborative approach with the Scrum Alliance, Ken Schwaber, Jeff Sutherland, etc.
Getting feedback from public at large
Not going to use all the practices e.g. Daily Scrum because working part-time as an individual, not with a team

Dr. Mark C. Paulk
mcp@sc.cmu.edu
http://www.cs.cmu.edu/~mcp/
Carnegie Mellon University
Institute for Software Research
407SCR 115
5000 Forbes Avenue
Pittsburgh, PA 15213 USA