Post your comments

 

 

Empirical Studies of Agile Software Development

The push for "agile" software development practices is a relatively new phenomenon. Although agile practices have many proponents and are gaining in popularity, there is as yet little empirical data that can be used to discuss the merits and costs of this approach, and the contexts in which it may be relevant. The question of how this type of development practice can be best evaluated by empirical methods has been of interest to some members of the ISERN community, and some empirical studies are beginning to be run by ISERN members.

 

ISERN Presentations

Forrest Shull moderated a session on this topic at the 2002 ISERN meeting in Nara, Japan. Four short presentations were given to provide an overview of the available information.

  • Forrest Shull (Fraunhofer Center - Maryland) provided a description of expert workshops that were run to develop testable hypotheses in a "green field" with little available empirical evidence.
    • Summaries of the eWorkshop discussions , containing an overview of the experiences and other resources described, are available.
  • Marcus Ciolkowski (University of Kaiserslautern) provided an example of a classroom study that tests a key agile practice, pair programming.
  • Erik Arisholm (Simula Research Labs) provided a design of a study for a controlled experiment on paired programming, allowing the use of previous experimental results as a baseline for measuring improvement.
  • Jyrki Kontio (Helsinki University of Technology) gave a brief discussion of industrial perspectives and a survey conducted at Nokia showing how agile practices were adopted.

 

Other Resources

ISERN member VTT Electronics has published a literature review , surveying several different agile development methodologies.

 

Agile Methods Data

To facilitate the drawing of empirically-based conclusions about agile practices, we would like to begin creating a list of what empirical studies and data do exist on this topic. If you know of a published and/or web-accessible source of data, please use the form below to share it with the ISERN group. Please submit ONLY sources with some empirical component - there's no shortage of publications on agile, but the aim here is to help ISERN researchers find the subset they are most likely to find useful. For each submission, please:

  • Give a reference or preferably a web link so that readers can easily find the original source.
  • Feel free to provide some annotation describing your feelings on the validity of the study and/or summarizing the overall contribution.

Name:
Email:
Topic:
Message:


 

 

 

 

 

Posted Comment

Williams' work on XP practices
  Forrest Shull (fshull@fc-md.umd.edu) Date:12/13/2002
  Williams, Laurie, Kessler, Robert R., Cunningham, Ward, and Jeffries, Ron, Strengthening the Case for Pair-Programming, IEEE Software July/Aug 2000
http://collaboration.csc.ncsu.edu/laurie/Papers/ieeeSoftware.PDF
One of the earliest and still most often cited empirical studies of Pair Programming, a key practice of eXtreme Programming. Classroom study.

George, B. and Williams, L., An Initial Investigation of Test-Driven Development in Industry, ACM Symposium on Applied Computing, March 2002.
http://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf
Empirical study of Test-Driven Development, another key XP practice.

Classroom study on test-first programming
  Forrest Shull (fshull@fc-md.umd.edu) Date:12/13/2002
  Matthias M. Müller, Oliver Hagner. Experiment about Test-first programming. In Conference on Empirical Assessment In Software Engineering (EASE), Keele, April 2002. http://www.ipd.uka.de/~muellerm/publications/ease02.pdf

Industrial experiment on pair programming
  Forrest Shull (fshull@fc-md.umd.edu) Date:12/13/2002
  Matevz Rostaher, Marjan Hericko: Tracking Test First Pair Programming - An Experiment. XP/Agile Universe 2002: 174-184
http://link.springer-ny.com/link/service/series/0558/papers/2418/24180174.pdf

Unpublished study of XP effort
  Forrest Shull (fshull@fc-md.umd.edu) Date:12/13/2002
  Wei Li , Mohammed Alshayeb, U. Ala-Huntsville, An Empirical Study of XP Effort
http://sunset.usc.edu/events/2002/cocomo17/Wei%20Li--Some%20Observations%20in%20Extreme%20Programming.zip
This study has not yet been published (to my knowledge) but I found this an interesting talk containing some data. Although there are methodological problems with the study, I haven't seen another treatment yet of similar research questions (concerning the types of activities on which XP programmers actually spend their time).

XP Case Study & XP Introduction Decicion Support
  Daniel Karlström (daniel.karlstrom@telecom.lth.se) Date:12/16/2002
  Karlström, D., "Introducing Extreme Programming - An Experience Report", Proceedings Third International Conference on eXtreme Programming and Agile Processes in Software Engineering (XP'2002), Sardinia, Italy, 2002. (http://www.xp2003.org/xp2002/atti/DanielKarlstrom--IntroducingExtremeProgramming.pdf) Karlström, D., Runeson, P., "Decision Support for Extreme Programming Introduction and Practice Selection", Proceedings The Fourteenth International Conference on Software Engineering and Knowledge Engineering (SEKE'02), Ischia, Italy, 2002. (http://www.lucas.lth.se/lucas-dagar/publications/SEDEC_2002_XP_1_4.pdf)

Pair Learning in a Beginning Computer Science course
  Laurie Williams (williams@csc.ncsu.edu) Date:12/19/2002
  Williams, L., Wiebe, E., Yang, K., Ferzli, M., Miller, C., In Support of Pair Programming in the Introductory Computer Science Course, Computer Science Education, September 2002. http://collaboration.csc.ncsu.edu/laurie/Papers/PP%20in%20Introductory_CSED.pdf
The results of using pair programming in the first computer science course at North Carolina State University
The Impact of Pair Programming on Student Performance and Pursuit of Computer Science Related Majors H. Bullock, J. Fernald, C. McDowell, and L. Werner http://www.cse.ucsc.edu/~charlie/pubs/retention2003.pdf
A study of pair programming in an introductory computer science class at the University of California - Santa Cruz. Also examines how students go on to succeed in future classes and how pair programming seems to impact the participation and retention of women in computer science.