Summary of the First eWorkshop on Agile Methods
April 8, 2002
Participants:
§
Alistair Cockburn
§
Atif Memon, University of Maryland and Fraunhofer Center
for Experimental Software Engineering
§
Barry Boehm, University of Southern California
§
Bil Kleb, NASA Langley Research Center
§
Don Wells, ExtremeProgramming.org and co-chair of XP/Agile
Universe conference
§
Forrest Shull, Fraunhofer Center for Experimental Software
Engineering
§
Frank Maurer, University of Calgary
§
Gary Pollice, Rational Software
§
Granville (Randy) Miller, TogetherSoft
§
Hakan Erdogmus and Joel Martin, National Research Council
of Canada
§
Ken Auer, RoleModel Software, Inc
§
Ken Schwaber, Advanced Development Methods, Inc. and one of
the developers of Scrum
§
Kent Beck, founder and director of the Three Rivers
Institute
§
Laurie Williams, North Carolina State University
§
Marv Zelkowitz, University of Maryland and Fraunhofer
Center for Experimental Software Engineering
§
Peter Hantos, Xerox
§
Philip Johnson, University of Hawaii
§
Scott Ambler, Ronin International, Inc. and Thought Leader
of the Agile Modeling methodology
§
Tim Mackinnon, Connextra Ltd.
§
Vic Basili, University of Maryland and Fraunhofer Center
for Experimental Software Engineering
§
William Wood, NASA Langley Research Center
§
Winsor Brown, University of Southern California
Definition
Early in the meeting, there was some discussion about the
definition of agile software development.
§
Ken_Schwaber suggested the following definition: An agile
method is iterative, incremental, self-organizing, and emergence. It
must include all of these attributes; otherwise it is a “lightened defined
process”.
§
Barry Boehm suggested using the 4 values and 12 principles
described in the Agile Manifesto (http://www.agilemanifesto.org/) for the
eWorkshop. Almost everyone – 15 participants – agreed to go along with this
definition for the eWorkshop.
Despite this agreement, later in the eWorkshop there was
more discussion about the definition. Marv Zelkowitz questioned whether there is
any common definition or is it’s just a new buzzword. In response to his
question, Ken Schwaber said: “Yes
... it is based on empirical methods rather than defined methods. Defined
methods are when you plan what you want and then enforce it. Agile, you lightly
plan what you want and then adapt to what you get. Self-organization,
emergence.”
Size of projects using Agile Methods
The first topic of the eWorkshop was the size of projects
that are better suited for agile methods.
What size teams people have seen?
In answering the question: “What size projects have you
successfully worked with? References?” The participants responded with the
following examples:
§
Alistair Cockburn: 6, 18, 45 and 90 people. These are
referenced in his book (“Agile Software Development”)
§
Tim Mackinnon: 3, 6, 12 people using XP and 8 and 25 (with
sub-teams) using Agile.
§
Ken Auer: 12 using pure XP and several smaller, 150+ with a
12 person subgroup using XP.
§
Ken Schwaber: OOPSLA workshop: unable to rule out any types
of projects (small to large and all types of projects) and 800 person team at
IDX (teams coordinated around product lines all using agile - scrum). There was
a project plan used to guide all teams. Teams are decomposed into small-sized teams.
They coordinate using scrums of scrums or have teams with individuals from
multiple product lines. Teams are small for self-organizing purposes, but teams
of teams can be formed to scale up.
§
Don Wells: 150 (mixed methods)
§
Scott Ambler: 4-8, 25-30, several hundred people (but only
one subgroup was “reasonably agile”)
§
Frank Maurer: In a university setting, small teams work
effectively.
Strategies for scaling up to use these methods with larger
teams
Randy Miller raised the point that size is not an issue
specific agile development. It’s a best practice for any type of development to
break into smaller teams.
Alistair Cockburn said size is an issue. As size grows,
coordinating interfaces becomes a dominant issue. Agile with face-to-face breaks
down (becomes more difficult/complex) past 20-40 people (interface becomes an
issue). Most people agreed (12 participants), but think that this statement is
true for any development process.
Following this discussion on size of projects, the
participants discussed the interface issues of using teams of teams.
§
In Ken Schwaber’s 800-person team they handled the
interfaces using “scrums of scrums” where the senior people on the teams met or
teams with members from multiple product lines. Part of this project is
documented in Jim Highsmith’s book (Agile Software Development Ecosystems: Problems, Practices,
and Principles, Addison-Wesley, 2002)
§
Scott Ambler mentioned that a core team
responsible for the glue is necessary. These people work actively with the
subteams, but also work on core architecture/standards.
§
Tim Mackinnon said that at OTI
they relied on yearly conferences to align interfaces, 3-month internships
(rotation of teams) – more to build trust in each other; Interfaced with emails,
conference calls, and test case results.
§
Alistair Cockburn said that Ron
Crocker's "Grizzly" method from two large Motorola projects is an example –
large teams have to do agile differently to make it work. The only example of a
"large" international project run with agile that he has is Ron Crocker's. The
mechanisms he (Ron’s) used were fundamentally different than his (Alistair)
face-to-face ones, but otherwise fit all the principles. It was as agile as he
(Alistair) could imagine for that group (more, in fact). Crocker’s large project used simulators and stubs, three cross-project
subteams with international representation – these teams met to keep in sync.
The book is still in progress, but XP
2001 in Sardinia and JAOO 2001 in Denmark have papers and slides.
§
Gary Pollice concluded that projects of projects are treated like modules that have
interfaces to the others.
Ken Auer mentioned that having subgroups of larger projects doing “agile” improves coordination effort for others. He was part of a 150+ person project which was chaos except the 12 person sub-group he led which was certainly "agile" and made real progress and started making coordination efforts with others happen.
Suggested reference: Scott Ambler suggested his Agile Architecture essay as good reference for this topic.
Managing Agile Development
Success Factors
The variables that affect agile development and what are
the success factors for agile development were another topic of discussion.
These are some of the comments from the participants:
Culture
§
Scott Ambler: The factors that affect agile development are
different depending upon the agile method chosen. The corporate culture, working
together and trust play an important role.
§
Ken Auer: Agile
is about taking control of your own destiny to the point that you can. Saying,
"If you work in a bog bureaucratic organization, agile is inappropriate" is a
cop-out. If someone is more interested in keeping their job than doing the right
thing, methods/approach is irrelevant.
People
§
Alistair Cockburn: Good people are key to success with big
teams
Communication
§
Scott Ambler: Process and organization – need to have a
communication vehicle (such as co-located teams – Ken Schwaber) in place.
In general, the group agreed that the following are the
success factors for agile development:
§
Negotiation: If people are willing to keep talking and
learning, you are, almost by definition, agile.
§
Ability to continuously learn
§
Responsibly responding to change.
§
Environment where rapid communication is enabled.
§
Team reasonably in charge of its own destiny (being allowed
to succeed).
§
People are valued over other factors
§
The team
needs to be more than just able, it needs to actually succeed. There needs to be
a support network for each individual in the team to make a contribution however
small.
Alistair Cockburn made the following comment regarding the
success factors: There's a cost element to this. I used agile techniques on my
last book - the book came out sooner, but production cost more. Predictable
situation + low cost as driver -> less appropriate for agile. That's the only
one I can think of.
When is Agile appropriate or not appropriate?
There was also some discussion about when agile is and is
not appropriate. The comments of the participants were:
§
Scott Ambler: When the culture is not there. When the
development team is not in control of its own destiny.
§
Bil Kleb: In life critical systems (but found testing in XP
to be helpful for these types of systems).
§ Hakan Erdogmus and Joel Martin: We
believe that some agile methods are always appropriate. If safety is critical
and documentation is paramount (customer wants it), they just become added
tasks. The more emergent and rapidly changing the requirements, the more
appropriate agile methods are. However, that doesn't imply that agile methods
are not appropriate for more stable environments. Plus, there is always
technical uncertainty that you cannot eliminate.
Marv Zelkowitz observed that project time and schedule
don’t seem to be defining attributes for
when agile methods are appropriate and asked what are the defining attributes.
Scott Ambler and Gary Pollice agreed with him. Gary Pollice added that he believes that there
are many factors and the dependencies between the different
factors will be very complicated. In response to this discussion:
§
Randy Miller suggested minimizing pain/risk.
§
Frank Maurer: the ability to get fast feedback from the
customer (which may be tricky for embedded systems)
§
Kent Beck said: Negotiation. If people are willing to keep
talking and learning, you are, almost by definition, agile. Scott Ambler and Ken
Auer agreed.
§
Randy Miller raised the issue that a case could be made
that BDUF is talking and learning.
§ Bil
Kleb added that "continued learning" is one of the fundamental differences
between "agile" and "others".
What are indicators of problems with Agile projects
The group also discussed indicators of problems with agile
projects so that management can make decisions on time in order to
minimize/mitigate risks. The group agreed upon the following indicators:
§
Ken Schwaber: During the daily meetings
§
Alistair Cockburn: When software is not being produced,
morale is down
§
Randy Miller: When we stop focusing on what is supposed to
be delivered
§
Gary Pollice, Scott Ambler: Activities that don't provide positive value
to the overall effort
§
Gary Pollice, Scott Ambler: when useless documentation is
being produced
§
Ken Schwaber: When the customer refuses to fund another
iteration
§
Peter Hantos: When you are getting behind on planned
iterations
People
Type of personnel needed for Agile projects
The type of personnel needed for Agile projects and their
role in the success of the agile development was another topic discussed during
the meeting. The comments were:
§
Ken Auer: We need high quality people. Doesn’t mean
necessarily experienced skilled people, but honest, collaborative people.
§
Scott Ambler: People must be responsible, ready to learn
and work with others.
§
Randy Miller: Experience helps too!
§
Gary Pollice and Randy Miller: Mentors and coaches come in
to help deliver the project (with inexperienced people)
§
Barry Boehm thinks that
the criterion is that for any success-critical issue, at least one person who is
listened to that has the time and expertise to resolve the issue.
§
Peter Hantos: team lead needs to be trained (at least),
external experts are useless in highly agile, tactical environments
Percentage of experienced people needed to bring the
project along
Participants agreed that a certain percentage of
experienced people are needed to bring the project along, however they were
uncomfortable with putting an exact number on the total number of competent and
experienced people needed for effective development. 25% was suggested as first,
but several of the participants felt that a range would be better.
§
Hakan Erdogmus and Joel Martin agreed with 25%, but would
reduce to 10% if rotating pairs, needs to be one person that everybody listens
to
§
Tim Mackinnon believed 33% expert developers are needed
§
Hakan Erdogmus and Joel Martin, Tim Mackinnon and Scott
Ambler agreed there must be 10 - 30% competent and experienced people on the
project for effective development
Definition of Competent
Scott Ambler raised the point that we need to define
competent. In response to his question:
§
Randy Miller: I
think if you know the most about something that makes you the expert on the
project
§
Ken Auer: On most projects, the most “expert” depends on
the task at hand.
§
Scott Ambler: Competent to me means that they have real-world experience
in the technologies, have built similar systems in the past, have good people
skills, good communication skills, ...
Training
The discussion on competence led to a discussion on
training. How much training is necessary?
Vic Basili raised the question whether agile methods
require less formal training than traditional methods. Most participants (13)
agreed that agile methods do require less formal training.
Hakan Erdogmus and Joel Martin and Bil Kleb think that
agile require less formal training for XP because of pair programming. Peter
Hantos mentioned that it is not training that is needed but a , a
fair and open, but "professionally" guided discussion. Gary Pollice thinks it's
probably less formal training, but more mentoring type training ... exploiting
tacit knowledge transfer that is needed. Alistair Cockburn
added that we should think of skills development instead of training. The mentor
helps individuals grow professionally, in techniques, insights, and method. Don
Wells pointed out that perhaps there is a difference between training and
re-training. The entire team can train it self by it self. But train it must.
Ken Auer observed that if you have honest/collaborative people, training is
relatively simple. Tim Mackinnon pointed out that training people to be
honest and collaborative is not so simple.
Hakan Erdogmus and Joel Martin mentioned that a participant
in one of their XP course that had taken RUP training before found the XP one to
be easier. Randy Miller mentioned that he has seen groups that have trained
themselves on XP, but he has also seen it with non-agile methods like RUP. He
believes there are enough materials for groups to train themselves on Crystal,
SCRUM and FDD too. Scott Ambler has also seen groups being able to train
themselves.
There was a discussion regarding the statement in Barry’s paper: “Agile methods fit applications that can be built quickly and don't require extensive quality assurance. Agile methods work less well for critical, reliable, and safe systems."
The moderator proposed this discussion but presented the statement as "Agile methods fit applications that can be built quickly". Alistair Cockburn disagreed with the statement proposed by the moderator that only considered part of the original statement. He (Alistair) mentioned a $15M, 18-month fixed-price fixed time contract where Agile worked very nicely. Later, Barry Boehm clarified what statement 3 was and the discussion proceeded regarding the complete statement from Barry's paper.
For some participants the issue of applying the agile methodology for critical systems is just a matter on how you treat the criticality, reliability and safety requirements. Peter Hantos said that as long as the issues that make the mission "critical" are explicitly expressed in the requirements, agile should work. Ken Auer said that if embedded, safety-critical... stuff is high priority, it should be stated as such at the beginning of a project and "Agile" will handle it by putting in the proper level of testing and emphasis on the critical issues. Hakan Erdogmus and Joel Martin, Tim Mackinnon and Bilkleb agreed with them.Randy Miller believes that agile process like XP are acceptable for critical mission. Scott Ambler agreed and added that FDD can be used for critical mission.
Winsor Brown mentioned that despite the fact that agile
approaches include planning, the manifesto seems to counter the amount of
planning that has proved necessary for successfully achieving high levels of
security. Scott Ambler replied saying that if security is an issue, then we
should identify it as a requirement and act accordingly. Non agile methods will
fall down on this issue if they don't have it as a requirement as well. Bill
Wood added that if security is important, an agile method can put it in early
and demonstrate it early. The alternative is to wait until late in the process
to implement security and risk finding out then that the plan didn't exactly
work as planned.
Barry Boehm agreed with the participants but mentioned that
what worries him about agile is that the manifesto emphasizes responding to
change over following a plan. Many crashes of satellites, telephone systems,
etc. resulted from departing from the plan to follow an impulse. Kent
Beck argued saying he stole the feature-driven fixed-length iterations from a
pacemaker project Jon Hopkins worked on. Alistair Cockburn’s opinion is: "How
agile can we be, given that this is going to be critical, reliable, safe?" The
answer will give strategies that will involve reviews, tests, interface
simulators, etc., AND ALSO drive up the collaboration & morale components,
since they will be equally crucial on such a project. Scott Ambler
agreed. Alistair Cockburn added: I
believe there are also examples of following the plan, not noting current
conditions, and obtaining catastrophe. I would put the Challenger space shuttle
explosion in this category. But it's "responding to change" with a new plan
based on learning, not just "responding to change".
Some participants believe that tests are the means to
respond to change safely. Tim Mackinnon emphasized that responding to change should
be balanced with the "working software" value - if you change
on a whim and your tests don't pass then its not working. Bill Wood stated: I think agile can have a very strong
emphasis on safety and reliability through testing. Bil Kleb agreed.
Hakan Erdogmus and Joel Martin believe that for safety-critical stories, customers can write acceptance
tests that measure performance, reliability, etc. They are just more difficult
to write, and may require environments more complicated than JUnit.
Based on these discussions Barry Boehm suggested the use of
the term “Responsibly responding to change”. Scott Ambler agreed but
Tim Mackinnon believes responsibly responding is too
weak. “Sure I thought I did the right thing and we did it as a pair but it still
broke because I decided not to write a test. That’s not good is it?”
Alistair Cockburn answered Tim Mackinnon’s point stating
that: Working in a
agile way and goofing / failing is not the same as not being agile, or not
supposed to be agile. Mistakes happen in all scenarios. He also raised the point the agile manifesto phrase is worded in an odd way, that
generates this worry. The responsible way to "respond to change" is to
investigate the quality of the source, not simply to shrug and change. An error
in agile interpretation will be to shrug and change, not check sources and
respond with dialog. Tim Mackinnon replied: I still stand by, simply being
responsible is not good enough. We all need an extra level of discipline that is
more concrete. Test First does that - I have been responsible if I wrote the
tests (period) - actually the same applies to all XP practices. I/we get a good
benchmark to rate ourselves on. Scott Ambler added: But if you're responsible
then you're willing to learn more and are likely to run into this concept of
test first development. You'll consider it and adopt it if it makes sense for
your current situation, and if it doesn't then you might lobby to change things
so that it does.
Refactoring
Refactoring and architectural evolution were other topics
that arose during the discussion. Some participants were uncomfortable with the
idea of refactoring the architecture while others thought that refactoring the
architecture is not a problem and that it is often necessary because of changing
requirements.
How much refactoring is reasonable?
Peter Hantos had a concern regarding refactoring at the
architecture level and stated: “If any requirement pops up down the line that
questions any aspects of the architecture, then refactoring will not work, or
will take forever. I am for frequent refactoring of a reasonable size code, but
not architecture, which affects all the internal/external players. It seems that
I was brainwashed into respecting architecture. I conducted risk assessments on
many major programs, and most of them failed because even though they knew about
performance, exception handling, etc. requirements, they choose to deal with
them at a very late stage if the game.
Scott Ambler said that when you do BDUF, you are more
locked into the current architecture. He added that the BDUF proponents are
often lying to themselves, often because they don't actually get in and roll up
their sleeves to do some coding. They don't see how far off their designs
actually are, and don't feel the impact of getting things wrong.
Barry
mentioned that if you do BDUF to a snapshot set of requirements, you'll usually
get a hard-to-evolve point solution. It you get people to do the best they can
to define their evolution requirements, and these are stable and reasonably
accurate, you can get an architecture that accommodates future evolution.
Bil Kleb brought up test: refactoring with a set of
automated tests as a safety net is an awesome experience. big architecture
changes are no longer scary nor require much effort -- the minimal effort is
also due to other factors such as the KISS principal (keep things simple,
stupid).
Participants agreed that requirements do change over time
and your architecture needs to evolve. These are some comments from the
participants regarding this issue:
Experimentation
People seem to believe that it is obvious that agile
methods work. Marv Zelkowitz asked if there was a need for validation.
Despite the perception that agile methods are obviously
beneficial, some participants (Philip Johnson and Alistair Cockburn) felt that
there is a need for proper validation (especially for late adopters) and
suggested possible studies.
Is experimentation important?
Some participants (6) agreed that experimentation is
important. Laurie Williams pointed out that her experiments results
have helped to overcome resistance to pair programming. Don Wells said he
experienced the same thing; “We
had to get PP going by mixing people up till something mysterious and magical
happened.” Ken Auer mentioned that experimentation is extremely useful to help
people get over their fears... it's OK to stumble in an experiment!
Scott Ambler believes that experimentation is important for many in the "late majority" and "laggards" on Moore's adoption curve. "If it ain't proved I ain't interested" is a common refrain among these folks. People at this workshop are early adopters or innovators and are willing to try new things. But, there are many organizations that are still struggling with adopting OO, let alone agile stuff. Many participants (12) agreed that the late adopters would need to see data before they would use agile methods. Peter Hantos believes tha since the application of these methods is so politicized, controlled experiments, and particularly if they are conducted in a university setting will miss those cruicial political and contextual points. Alistair Cockburn added that they may ignore the data from experiments, but if you don’t have data, they will ignore you.
Alistair Cockburn asks for proper validation. At the USC workshop, his suggestions were turned down, because they were "too obviously true". But we still need the data to show people, otherwise they won't believe.
Philip Johnson agreed but believes it will be difficult to
experiment with agile development because the method is under specified.
What kind of studies you would like to see?
The following studies were suggested:
§
Scott Ambler: Exploring the effectiveness of manual
modeling techniques (CRC cards, white board sketches, low fidelity prototypes)
The idea of discarding temporary models and traveling light should be
addressed
§
Hakan Erdogmus and Joel Martin: Experiments to measure how
information about the system, the process, and technical skills are propagated
through the team using agile practices
§
Don Wells: some sort of experiment in cognition versus
strict plans or looser collaboration styles.
§
Alistair Cockburn: I want to see quantitative studies
comparing team morale / citizenship / communication paths used, to project
outcome. My prediction is that they link better than most others (other than the
standard project killers). Without studying those factors, we only see the
"Process", and the important factors slip through the survey's net. (Several
participants were interested in this experiment – Scott Ambler, Winsor Brown,
Tim Mackinnon, Ken Auer)
§
Bil Kleb: BDUF vs Test Driven Development designs
§
Alistair Cockburn: Study of agile and architecture
evolution
Some of the participants felt that it might be difficult to
experiment with agile methods using controlled studies and suggested case
studies or less formal experimentation:
§
Forrest Shull said: If
controlled experiments are hard/unfeasible to do right now, we could get a lot
of value out of case studies. It would be great to say that we followed an agile
methodology group and document that it achieved a highly dependable real-time
system (or whatever it is that objectors argue against agile ever being able to
reach. Peter Hantos and Barry Boehm agreed.
§
Hakan Erdogmus and Joel Martin stated that not
all empirical studies must be "controlled". A great deal can be learned by
careful, qualitative observational studies by people who know what they are
doing. The "controlled" bit is necessary for making generalizations about
causation. That is not always possible.
§
Scott Ambler agreed and added: We need to keep things
honest. However, someone who knows what they're looking at needs to write up the
failure. In the mid-90s there was a lot of articles about how you couldn't store
objects in relational databases, but they often blamed lack of experience with
C++ on mapping objects to RDBs. I'd hate to see teams blaming a process for
their own mistakes. Software development is fragile, one bad decision can
kill your project. A common bad decision is to go against the iron triangle and
try to set the level of quality, the level of resources, and the scope of your
effort -- at least one of these factors has to vary.
Documentation
Is any documentation necessary at all? If so, how do you
know how much?
There was also some discussion about the use of
documentation with agile projects. Scott Ambler commented that documentation
becomes out of date and should be updated only “when it hurts”. Documentation is
a poor form of communication, but sometimes you need it to retain critical
information over time. Many organizations demand more than is needed. The goal
should be to communicate effectively and documentation should be one of the last
options to fulfill that goal. Barry Boehm mentioned that a documented project
makes it easier for an outside expert to diagnose problems. Kent Beck disagreed
saying that as an outside expert who spends a large percentage of his time
diagnosing projects, he is looking for people “stuff” (like quiet asides) and
not technical details.
Bil Kleb said that with agile, documentation is assigned a
cost and its extent is determined by the customer (excepting internal
documentation).
Suggested reference: Scott Ambler suggested his Agile Documentation essay as good reference for this topic.
Next
eWorkshop
In the end of the meeting, the participants were asked
whether they were interested in another workshop sometime in June. Most of them
had a positive response.