9:00 -9:15

Welcome and Introduction

Dr. Forrest Shull, FC-MD

9:15-10:15

Before diving into specific approaches, a high-level, thought-provoking presentation will focus on larger issues of matching study design to researcher goals.

Technology Evaluation

Evaluating a technology or method often invokes the image of the controlled experiment - many individuals performing the same task under differing conditions - and then using various statistical tests to determine if the proposed method improves the process. However, there are many ways to evaluate a new technology; the controlled experiment being only one. In this talk we will discuss various ways in which a researcher can approach the technology validation problem and we will look at historical data showing how prevalent was each method in validating various computer science technologies.

Dr. Marvin Zelkowitz, University of Maryland and FC-MD

10:15-10:45

-- Break --

10:45-12:15

Mini-tutorial 1: How to run empirical studies using project repositories (and avoid common pitfalls)

Over the last few years software project support tool repositories are increasingly being used for empirical studies of software. This has been primarily driven by the interest in open source projects and the wide availability of their repositories. Unfortunately, the project support tools are not designed as data sources for empirical studies
and, therefore, many pitfalls and problems await. This mini-tutorial will discuss and illustrate some of the common things to avoid when analyzing software repositories. The topics range from the high level objectives of empirical study having an irrelevant topic or a too narrow audience to common mistakes related to the impact of systematically missing, default, or tool-generated data.

Dr. Audris Mockus,
Avaya Labs Research

12:15-13:30

-- Lunch --

13:30-

15:00

Mini-tutorial 2: The role of replication in Software Engineering

Recently, software engineering has witnessed a great increase in the amount of work with an empirical component; however this work has often little or no established empirical framework within the topic to draw upon. Frequently, researchers use frameworks from other disciplines in an attempt to alleviate this deficiency. A common underpinning in these frameworks is that experimental replication is available as the cornerstone of knowledge discovery within the discipline. This tutorial investigates the issues involved in accepting this premise as a fundamental building block with empirical software engineering and recommends extending the traditional view of replication to improve the effectiveness of this essential process within our domain.

Dr. James Miller,
University of Alberta, Canada

15:00-15:30

-- Break --

15:30-17:00

Mini-tutorial 3: Software engineering process simulation

One way of analyzing the effectiveness of software development techniques and tools is to build simulation models that represent the management and engineering processes applied in software development organizations. In combination with empirical experimentation (i.e., case studies and controlled experiments) process simulation models can support decision makers in the following areas:

• analysis of recurring problems that have been observed in software development projects,
• assessment of risks and identification of improvement opportunities,
• investigation of the effects of suggested improvement actions (before their implementation), and
• planning and control of projects and processes.

This mini-tutorial gives an introduction into process simulation techniques applied in software engineering. Example of process simulation models applied in software industry will be presented.

Dr. Dietmar Pfahl,
University of Calgary, Canada

19:00

Attendance at IDoESE/IASESE/ISESE welcome reception