Summary of the First eWorkshop on
Agile Methods
April 8, 2002
Part 3
There was a discussion regarding the statement in Barry’s paper: “Agile methods fit applications that can be built quickly and don't require extensive quality assurance. Agile methods work less well for critical, reliable, and safe systems."
The moderator proposed this discussion but presented the statement as "Agile methods fit applications that can be built quickly". Alistair Cockburn disagreed with the statement proposed by the moderator that only considered part of the original statement. He (Alistair) mentioned a $15M, 18-month fixed-price fixed time contract where Agile worked very nicely. Later, Barry Boehm clarified what statement 3 was and the discussion proceeded regarding the complete statement from Barry's paper.
For some participants the issue of applying the agile methodology for critical systems is just a matter on how you treat the criticality, reliability and safety requirements. Peter Hantos said that as long as the issues that make the mission "critical" are explicitly expressed in the requirements, agile should work. Ken Auer said that if embedded, safety-critical... stuff is high priority, it should be stated as such at the beginning of a project and "Agile" will handle it by putting in the proper level of testing and emphasis on the critical issues. Hakan Erdogmus and Joel Martin, Tim Mackinnon and Bilkleb agreed with them.Randy Miller believes that agile process like XP are acceptable for critical mission. Scott Ambler agreed and added that FDD can be used for critical mission.
Winsor Brown mentioned that despite the fact that agile
approaches include planning, the manifesto seems to counter the amount of
planning that has proved necessary for successfully achieving high levels of
security. Scott Ambler replied saying that if security is an issue, then we
should identify it as a requirement and act accordingly. Non agile methods will
fall down on this issue if they don't have it as a requirement as well. Bill
Wood added that if security is important, an agile method can put it in early
and demonstrate it early. The alternative is to wait until late in the process
to implement security and risk finding out then that the plan didn't exactly
work as planned.
Barry Boehm agreed with the participants but mentioned that
what worries him about agile is that the manifesto emphasizes responding to
change over following a plan. Many crashes of satellites, telephone systems,
etc. resulted from departing from the plan to follow an impulse. Kent
Beck argued saying he stole the feature-driven fixed-length iterations from a
pacemaker project Jon Hopkins worked on. Alistair Cockburn’s opinion is: "How
agile can we be, given that this is going to be critical, reliable, safe?" The
answer will give strategies that will involve reviews, tests, interface
simulators, etc., AND ALSO drive up the collaboration & morale components,
since they will be equally crucial on such a project. Scott Ambler
agreed. Alistair Cockburn added: I
believe there are also examples of following the plan, not noting current
conditions, and obtaining catastrophe. I would put the Challenger space shuttle
explosion in this category. But it's "responding to change" with a new plan
based on learning, not just "responding to change".
Some participants believe that tests are the means to
respond to change safely. Tim Mackinnon emphasized that responding to change should
be balanced with the "working software" value - if you change
on a whim and your tests don't pass then its not working. Bill Wood stated: I think agile can have a very strong
emphasis on safety and reliability through testing. Bil Kleb agreed.
Hakan Erdogmus and Joel Martin believe that for safety-critical stories, customers can write acceptance
tests that measure performance, reliability, etc. They are just more difficult
to write, and may require environments more complicated than JUnit.
Based on these discussions Barry Boehm suggested the use of
the term “Responsibly responding to change”. Scott Ambler agreed but
Tim Mackinnon believes responsibly responding is too
weak. “Sure I thought I did the right thing and we did it as a pair but it still
broke because I decided not to write a test. That’s not good is it?”
Alistair Cockburn answered Tim Mackinnon’s point stating
that: Working in a
agile way and goofing / failing is not the same as not being agile, or not
supposed to be agile. Mistakes happen in all scenarios. He also raised the point the agile manifesto phrase is worded in an odd way, that
generates this worry. The responsible way to "respond to change" is to
investigate the quality of the source, not simply to shrug and change. An error
in agile interpretation will be to shrug and change, not check sources and
respond with dialog. Tim Mackinnon replied: I still stand by, simply being
responsible is not good enough. We all need an extra level of discipline that is
more concrete. Test First does that - I have been responsible if I wrote the
tests (period) - actually the same applies to all XP practices. I/we get a good
benchmark to rate ourselves on. Scott Ambler added: But if you're responsible
then you're willing to learn more and are likely to run into this concept of
test first development. You'll consider it and adopt it if it makes sense for
your current situation, and if it doesn't then you might lobby to change things
so that it does.
Refactoring
Refactoring and architectural evolution were other topics
that arose during the discussion. Some participants were uncomfortable with the
idea of refactoring the architecture while others thought that refactoring the
architecture is not a problem and that it is often necessary because of changing
requirements.
How much refactoring is reasonable?
Peter Hantos had a concern regarding refactoring at the
architecture level and stated: “If any requirement pops up down the line that
questions any aspects of the architecture, then refactoring will not work, or
will take forever. I am for frequent refactoring of a reasonable size code, but
not architecture, which affects all the internal/external players. It seems that
I was brainwashed into respecting architecture. I conducted risk assessments on
many major programs, and most of them failed because even though they knew about
performance, exception handling, etc. requirements, they choose to deal with
them at a very late stage if the game.
Scott Ambler said that when you do BDUF, you are more
locked into the current architecture. He added that the BDUF proponents are
often lying to themselves, often because they don't actually get in and roll up
their sleeves to do some coding. They don't see how far off their designs
actually are, and don't feel the impact of getting things wrong.
Barry
mentioned that if you do BDUF to a snapshot set of requirements, you'll usually
get a hard-to-evolve point solution. It you get people to do the best they can
to define their evolution requirements, and these are stable and reasonably
accurate, you can get an architecture that accommodates future evolution.
Bil Kleb brought up test: refactoring with a set of
automated tests as a safety net is an awesome experience. big architecture
changes are no longer scary nor require much effort -- the minimal effort is
also due to other factors such as the KISS principal (keep things simple,
stupid).
Participants agreed that requirements do change over time
and your architecture needs to evolve. These are some comments from the
participants regarding this issue:
[Main Page] [Part 1] [Part 2] [Part 4]