New search:
Moderator | has entered the room.
105 Moderator: /clear
Moderator | has left the room.
Moderator | has entered the room.
Joerg | has entered the room.
Joerg1 | has entered the room.
Joerg1 | has left the room.
Rose_Pajerski | has entered the room.
sandro_morasca | has entered the room.
muench | has entered the room.
Moderator | has left the room.
Moderator | has entered the room.
Vic_Basili | has entered the room.
MarvZelkowitz | has entered the room.
Carolyn_Seaman | has entered the room.
Kurt | has entered the room.
Tricia_Larsen | has entered the room.
Kurt | has left the room.
1 Moderator: Hello Kurt!
Kurt_Schneider | has entered the room.
2 Kurt_Schneider: Hello!
3 Vic_Basili: Welcome to the first measurement workshop.
4 Vic_Basili: Should we begin or wait for 3 minutes?
5 Kurt_Schneider: Wait
6 muench: Welcome to all!
Oliver_Laitenberger | has entered the room.
Frank_Sazama | has entered the room.
7 Vic_Basili: We have two topics, depending on time we will get to both.
8 Moderator: Let's start!
9 Vic_Basili: The first topic is calculating ROI, i.e., gathering stories, types of issues encountered, and what needs to collected, and do we convince an organization that the empirical evidence of ROI will be true for them.
10 Vic_Basili: The first question is: o What experiences have you had in calculating ROI?
11 Kurt_Schneider: Both in my Industrial job and in conversations with other companies, many people say they need ROI to convince THEIR own management (QS people, for example, say that).
12 Vic_Basili: o What experiences have you had in calculating ROI?
13 Rose_Pajerski: I guess one would be in the SEL - where it took 10 years to show improvement and then we never knew why things improved (productivity)
14 MarvZelkowitz: 11 Has management ever been convinced?
15 Kurt_Schneider: 12 A little on reviews
16 Vic_Basili: o I would say we had this experience at the NAS SEL. We were able to show that there were improvements in our approach, i.e., the experience factory and the various technologies we used with data in 87, 91, and 95 to show improvements in cost and defect rates.
17 muench: I used ROI models mainly in the context of process management, I applied similar metrics such as efficiency and productivity in areas such as defect reduction or technology change.
18 Kurt_Schneider: 12 Also some on an experience gathering technique of which we estimated the "ROI"
19 muench: 12 ROI models are one of several arguments to convince companies to change a process. Addressing a decision maker on the engineering level requires in my experience that the change is also one of the priorities of the engineer. If decision makers on higher levels are addressed, high ROI numbers are expected. As Oliver said in his position statement I think that it is important to find out other factors relevant for process change.
20 Oliver_Laitenberger: I think management believes in ROI-studies created elsewhere. So do we really need more of them and if yes, which ones?
21 MarvZelkowitz: 16 But the problem with the SEL is do we really kknow why? We have a cultural change that showwed improvement, but I don't think we know exactly what individually contributed.
22 Vic_Basili: 20 I am not so sure I believe this. I think people want to the the RIOI in their own environment.
23 Carolyn_Seaman: I think that management is more likely to believe, or even pay attention to, a report of a high ROI elsewhere if they can look at the costs and benefits that went into that calculation and can see that those same costs and benefits are applicable in their organization.
24 muench: 20 This is also my impression. A successful application of a ROI model in a similar context seems to help convincing management. It is far better than just qualitative experience.
25 Kurt_Schneider: 22 I am not even sure they will believe that: Maybe they just like to think they would
Sally_Godfrey | has entered the room.
Frank_Sazama | has entered the room.
26 Kurt_Schneider: 24 +
27 MarvZelkowitz: Cuemudgeon here! Problem with most of this is that better software development means makiing fewer errors. How do you convince management that you would have made an error you didn't make because of some new technology.
28 Moderator: Hi Frank!
29 Moderator: Is everything alright now?
30 Oliver_Laitenberger: We have sufficient evidence about basic technologies such as inspections, requirements engineering, testing and a few others. Still these are still not used widely.
31 sandro_morasca: I think that one of the problems is that ROI may be conputed over a number of years, and it's not just a short-term index.
32 MarvZelkowitz: 30 SO we haved 25 years of such positive experiences. Why not used?
33 Kurt_Schneider: 30 ok: So do you think we should do ROI on non-standard things only - or not bother at all?
34 Rose_Pajerski: 30 Is it because it takes too much time to gather real data?
35 Vic_Basili:  27 It was always a problem to explain to people that we had no control group and therefore no basis to decide if we would have gotten that improvement if nothing happened.
36 Kurt_Schneider: 35 + very good point!
37 Oliver_Laitenberger: Top-Management believes you if you can link the ROI studies to the business-numbers! And if business numbers improve management will be happy.
Frank_Sazama | has entered the room.
38 Vic_Basili: 37 This link is not so easym because it is hard to show cause and effect.
39 Moderator: hi frank...
40 sandro_morasca: 37 I agree with that, but some of the effects may be long-lasting, so the improvement in your ROI may show up several years down the road
41 muench: 30 I think that it depends on the priorities of the companies. SMEs, for instance, depend on customer relations and reliability. Therefore, innovations in the areas of customer mangement and testing are easier to be accepted than others.
42 Frank_Sazama: hi to all, I changed to netscape and it runns
43 Carolyn_Seaman: 35 There's a double-edged sword here. THe most reliable ROI calculations happen over many years (like in the SEL), but that passage of time also makes it harder to say that the improvement was due only or even mainly to the technology introduced, and not to the natural evolution of people and processes.
44 Oliver_Laitenberger: 38: That's the point. Do we therefore need more ROI studies of the same type?
45 Kurt_Schneider: I would like to come back to my observation: many middle managers pretend they would react if they only had quantitative data. But this is not always true. Why?
Rose_Pajerski | has entered the room.
46 Carolyn_Seaman: 45 My suspicion (not based on empirical evidence) is that the evidence that really matters to managers is the opinions of people they respect and trust. However, they need quantitative evidence to make the case to the rest of the organization.
47 Carolyn_Seaman: 44+
48 Kurt_Schneider: 46 + I agree in principle. But there is a whole hierarchie within the organization, and in the end, every one will need a respected person more than the data from below. Would you agree, Carolyn?
49 MarvZelkowitz: 48 I would agree (even though I am not Caraolyn)
50 Vic_Basili: 44, 45 I think we need ROI data from other places, as closely related to the current situation as possible, a plan to collect ROI data in the orgaization, e.g., hypotheses of what will happen if the technolgy is used, and a link to the business. Woud these three things be enough?
51 Kurt_Schneider: 49 thanks!
52 Oliver_Laitenberger: 48: At the end all that matters is that the person at the top is convinced.
53 Carolyn_Seaman: 44 However, all ROI studies have to be very clear about what the variables are that went into the ROI calculation, because that is the most valuable part of the process. If a manager can see that, given a set of costs and benefits that are relevant for him, that there was a positive ROI in another organization, they are more likely to believe it.
54 Kurt_Schneider: 52 ++ exactly! But how can ROI or any other data help with this?
55 muench: 46+ I agree. It is a difference in convincing people that trust you and people you do not know. However, ROI models can focus on a more objective
56 sandro_morasca: 44 I would say that we need mor ROI studies. You can never be sure of a cause-effect relationship, but if you provide more and more--even diverse--evidence, people will be inclined to accept
57 muench: 46 ... level of the decision.
58 Carolyn_Seaman: 48 I agree, but often the trusted expert is someone in the same organization, i.e. the idea of a champion. If the champion is convinced, that goes a long way in convincing other people in the organization who respect the champion, but not everyone will rely only on the champion's word, and empirical evidence is needed in those cases (especially for those who have to spend the money).
59 MarvZelkowitz: I'd like to briing up another related point. On a study I did, academics prefer studies that are more reproducible - such as within universities. These are not "real world." On the other hand, industrial managers perfer real world validation such as large case studies. So a disconnect between what evidence there is and evidence that will convinice managers. How to address this?
60 Vic_Basili: 56 +
61 Kurt_Schneider: 58 yes, and some people may need the data to point to if anything goes wrong. You cannot just point to a friend who told you it was ok.
62 muench: 59 I think that a combination of both, controlles
63 Frank_Sazama: my experience is from one example as a preparation for a bussines plan, it was a very clear goal and close related to the goals of the management
64 Carolyn_Seaman: 61 + VEry good point. We call that CYA.
65 Rose_Pajerski: 50, 56 - need a model that addresses the business case for the top manager
66 Frank_Sazama: 65 have a brief look on my slides from yesterday, sorry to all that this came late
67 Kurt_Schneider: 65 I heard a talk saying: you do not need so much data, but you need a convincing theory that is supported by the little data you have
68 muench: 59 I think that a combination of both, controlled experiments (repeatable, statistically significant results) and case studies in realistic contexts are an appropriate means to come up with convincing results (if the results are positive in both cases).
69 Vic_Basili: Can I come back to these points:
70 Vic_Basili: I think we need ROI data from other places, as closely related to the current situation as possible, a plan to collect ROI data in the orgaization, e.g., hypotheses of what will happen if the technolgy is used, and a link to the business. Woud these three things be enough?
71 Kurt_Schneider: 70 no
72 Carolyn_Seaman: 67 + This is a good point, I agree. I think practitioners and managers need some way to look at the various bits of evidence out there. A good, convincing theory can go along to doing that, to helping managers "make sense" of the evidence they do see. That way, it is more convincing.
73 Vic_Basili: 71 so what else is needed?
74 Frank_Sazama: 70 -
75 Kurt_Schneider: 71 the point is: any data is only a piece in the puzzle. If you really want to convince anybody, unsolicited data won´t do. NO unsolicited data will.
76 MarvZelkowitz: 70 I agree with Kurt. More data of the same type won't work.
77 Tricia_Larsen: 59 Being at the NDIA conference and listening to the talks on ROI, companies want more of real life examples, and they want to understand how they can relate it to their company.
78 MarvZelkowitz: 77 That is exactly the point I was trying to make
79 Vic_Basili: 75 who said unsolicicited data?
80 Frank_Sazama: I agree also with kurt, and I think only data from the own company wil convince the managers
81 Oliver_Laitenberger: 77: +
82 muench: 70 I agree. It is necessary to look at different (similar) contexts to draw more general conclusions for these contexts.
83 Oliver_Laitenberger: 80: ... but it must be the business data. Not technical data!
84 Vic_Basili: 80 But how do you get the compnay to collect data? Yhat is, where do you start?
85 Carolyn_Seaman: 77 + Yes, and understanding how they can relate it to their company means presenting the cost and benefit variables very precisely. This is real contribution of a ROI analysis, not the end number.
86 Kurt_Schneider: 79 I assume if you present ROI from somewhere else, it is not exactly representing the company you are talking to. So it may look "foreign", or unsolicited to them. It does not come from their own environment, and it was collected before they even asked a question.
87 Frank_Sazama: 83 yes it must be related to the business
88 Kurt_Schneider: 77 +
89 MarvZelkowitz: So after discussing this for 25 minutes, where is all this leading us?
90 Tricia_Larsen: 85 +
91 Frank_Sazama: 83 ROI is business data and not technical
92 muench: 77 My experience is: Presenting reference projects that were successful is very convincing to industry people. However, from a scientific perspective it is not sufficient. Relying on only one successful case can lead to the introduction of great risks in a company.
93 sandro_morasca: 92+
94 Kurt_Schneider: 92 good point. I agree we need different kinds of data or experiences to convince researchers vs. managers
95 Vic_Basili: 86 So the first step is to have data, the second step is to say exactly what hypothese you expect to be true in the organization and what data they shoudl collect. Theh you have to show it realtes to the business issue. And then maybe you can collect the deata to show ROI in the compnay itself.
96 Frank_Sazama: 72 what kind of reference to what kind of person ?
97 Frank_Sazama: sorry I mean 92
98 Kurt_Schneider: 95 ok, that is the clean process.
99 Frank_Sazama: 98 but this can't be the process at my customer in the german automotive
100 Moderator: Please review the whiteboard and see if you agree with it...
101 Frank_Sazama: I takes too long
102 muench: 96 As a reference typically it is sufficient to explain what has been done in another company and what were the effects. Person: Usually department managers of managers of SPI initiatives (that is my experience).
103 Vic_Basili: 99 So what is the process at the german automotive?
104 Kurt_Schneider: 95 I think Vic had a good point putting the usage of data in perspective: we need it and we get it not as the only thing; and I would add: we cannot convince anybody by data, but a convinced person can be reassured
105 Vic_Basili: 104 So we need to add the part about adding a convincing argument form a respected person.
106 Frank_Sazama: 103 they will not trust external data and results, they will see it with their own eyes and teir own data related to teir goals and environment
107 Kurt_Schneider: 105 + yes, and we might have a closer look at where ROI data really comes into the picture.
108 Carolyn_Seaman: 104+ Well put, that a convinced person can be reassured (and well armed) by data, but the data by itself cannot do the convincing. Summarizing from previous points, it takes a convincing theory and a reputable expert.
109 Vic_Basili: 106 So what will work?
110 Frank_Sazama: 106 this is my experience starting with GQM and the results from the SoftQuali project
111 Tricia_Larsen: David Raffo is doing a lot of work at Portland State in the area of process modeling and simulation to understand the ROI of process improvement activities. Is this an area that could help to solve the issue of ‘real’ data and the context of the company? He is presenting a number of times in this area.
112 Frank_Sazama: 109 start collecting internal data, start like GQM
113 Kurt_Schneider: 111 sounds interesting! What kind of simulations? Interactive? System dynamics again?
114 Vic_Basili: 111 Simulation models are an opportunity but it is hard to get the data to use them
115 muench: 111 Simulation is very sucessfully applied in non-software domains (e.g., production domain). My impression is that software-related companies are not willing to invest in model building at the moment.
116 Carolyn_Seaman: 111 + David's work is very good, and he is well motivated, I think, by industrial partners. His approach might be a way to take ROI data from various contexts and allow a manager to manipulate it in terms of the variables relevant to his/her company in order to get a better idea of what the ROI would be in the new context.
117 Kurt_Schneider: 115 one of the problems might be: software is a people discipline, not one that is easily modeled or simulated
118 Oliver_Laitenberger: System Dynamics is too complicated for managers. They rather buy in a case study than a simulation.
119 MarvZelkowitz: We did some work with a NASA center. They were very conservative and I don't think ANY evidence would have convinced them to change their software process. But they are using new hardware on the next space mission. Why ok to change hardware but not software development?
120 sandro_morasca: 115 Simulation models may be useful, but simulation and data don't tell everything. There are many qualitative elemnts that need to be taken into account, so you need to tell the entire story of why you got that ROI
121 Kurt_Schneider: 118 + but for researchers it is an intersting tool.
122 Frank_Sazama: 117+
123 Vic_Basili: 118 We did some simulation modelling for Motorla a few years ago and I belive they are using it successfully. But it requires a long time, a lot of data, and an internal champion.
124 Oliver_Laitenberger: 121: I think we need to distiguish ROI-Studies for researchers and ROI-studies for managers as an argument to use a technology in real life.
125 muench: 115+
126 Kurt_Schneider: 124 + good! So where can you convince researchers by ROI?
127 Kurt_Schneider: 126 I mean: do you have any example in mind in which one can do that?
128 Kurt_Schneider: 127 or is ROI always and only for managers - and why?
129 Oliver_Laitenberger: 126: It may not be limited to ROI but studying cause-effect relationship in general (under semi-controlled conditions)
130 Vic_Basili: Are there any other issues (model steps) that we need?
131 muench: 118 On the level of decision making about engineering process companies often do not fulfill the prerequisited for model building (e.g., a defined process).
132 Vic_Basili: What we can do now is build a set of steps that people should try to follow where possible. We will send this out to all particpant and ask them to make comments and suggestions.
133 MarvZelkowitz: 132 WHich people?
134 Frank_Sazama: 131 + yes if you are starting an SPI program than you are on level 1 and there is no measurement in place
135 Vic_Basili: 133 People refers to those trying to do technolgoy transfer
136 Frank_Sazama: 135 I didn't get it
137 muench: 134 In addition, not only a defined process is necessary but also a good process adherence to the defined process.
138 Vic_Basili: One last thought, we need to include in the ROI model for an organization an estimate of the costs of various steps, such as training, measurement, etc. to calculate ROI
139 MarvZelkowitz: 136 The technical staff within a company charged with improving the development activities - I think
140 Kurt_Schneider: 111, 118 if System dynamics is too complicated - what else could he use to simulate and get reasonable data?
141 Oliver_Laitenberger: 138: + at least you have to take these costs into account (as well as the costs for reducing the development staff due to the improvemenet!)
142 Moderator: /vote Do you agree with the first three lines on the whiteboard?
143 Moderator: I meant three sentences!
144 Carolyn_Seaman: 140 I'm not sure what David Raffo uses in his work, but there are numerous simulation modeling techniques, each differing in the level of sophistication, level of understandability, etc.
145 MarvZelkowitz: [voteyes]
145 Oliver_Laitenberger: [voteyes]
145 Frank_Sazama: [voteyes]
145 Carolyn_Seaman: [voteyes]
145 Kurt_Schneider: [voteyes]
145 muench: [votenotsure]
145 Rose_Pajerski: [voteyes]
145 Vic_Basili: [votenotsure]
145 sandro_morasca: [voteyes]
145 Tricia_Larsen: 144 I probably can't give you all the specifics, but his talk was on case studies related to unit testing and inspection processes. He has developed a tool and I can provide his talks if you'd like them.
146 Oliver_Laitenberger: Has anybody observed a linkage between balanced scorecard and GQM?
147 Moderator: /endvote
147 147 | 11/17/2004 11:46:48 AM | Vote question: Do you agree with the first three lines on the whiteboard?
Vote results:
       Yes  (7/9) 78%
        No  (0/9) 0%
  Not Sure  (2/9) 22%
  Total: 9 votes

148 Tricia_Larsen: [voteyes]
148 Moderator: The ones that said "not sure" would like to comment on that?
149 Vic_Basili: 146 Yes, we are currently workin on that link.
150 Frank_Sazama: 146 I also think that this is the right way to go. I only think about that, but have no practical experience
151 Oliver_Laitenberger: 149: I think that would help. Many companies we work with use a kind of balanced scorecard approach.
152 muench: 146 In the context of monitoring systems (such as dashboards) I recognized very often that the areas Balanced Scorecard (Business Goals) and Software Goals (e.g., defined with GQM) are completely seperated. The input for balances scorecards, for instance, is typically not derived from a software measurement activity. I have seen several systems in the non-software production domain, where measures of the production process and products are fed in business systems (such as ERP systems) but not many in the software domain.
153 Frank_Sazama: 151 +
154 Frank_Sazama: 152 GQM is not only for software, or ?
155 Vic_Basili: I said not sure because I sort of agree but do not know if that is the full answer. I still think the model i suggested is correct, there are lots of things we need to do and we cannot just asssume that the manager will ignore data.
156 Moderator: We need to wrap-up and move on to the second topic.
157 Frank_Sazama: If you have goals and you would like to measure than use GQM
158 Vic_Basili: So no we can rap up this first topic and go to topic 2
159 muench: 154 Yes, I agree. GQM could be a good tool to connect business goals and software goals.
160 Kurt_Schneider: 155 correct: we cannot ignore data. But we need to embed it into a convincing longer story (with case studies to start with)
161 Oliver_Laitenberger: 155: Managers are interested in data but it must be the data they understand. And if a CEO is for example a mechanical engineer, he has no clue about software data. This requires translation.
162 Frank_Sazama: 160 +
163 Vic_Basili: Goal: Gather a set of measurement goals relating to CMM,CMMI, and Profes; put these goals into context on project and/or organizational levels; and define relationships between CMM, CMMI, Profes measures and “management through measurement” program measures. o What are the goals of measurement in these models (CMM,CMMI, and Profes)?
164 Frank_Sazama: 163 please see my slides. there are too much to explain all now
165 Moderator: Please review the 'Conclusions on ROI' on the whiteboeard.
166 Vic_Basili: o I think CMM traditionally have been process conformance, or even process definition, rather than process effect. I have seen little in the product measurement area. I agree with Juergen that QIP approaches are different but CMMI and SPICE are more like CMM.
167 muench: CMMI seems to focus on establishing measurement competence rather that on measurement process-product relations for key issues. Profes focuses more on concrete problems and measuring the effects of solutions for these problems.
168 Frank_Sazama: 164 the goals are to support different measurement customers like project leader, quality people, SEPG, line managers
169 Vic_Basili: 163 Frank, Can you swummarize some of thekey ints on your slides?
Dave_Weiss | has entered the room.
170 Frank_Sazama: 169 we are starting with GAM (like GQM), we are thinking about measurement customers, we prepared some szenarios an we had a look on the CMM/CMMI and SPICE goals
171 Moderator: 168: Frank, does this apply to all CMM/CMMI/Profes
172 Moderator: Welcome, Dave!
173 Frank_Sazama: 171 what ist profes in you eyes ?
174 Oliver_Laitenberger: I'm somewhat lost: What is really the key question here?
175 MarvZelkowitz: 174+
176 Frank_Sazama: from my point of view it is the question what kind of measures are in the CMM(I)
177 Vic_Basili: 174 Are we really talking about two different models? the CMM like models and the QIP/EF/Profes model?
178 Kurt_Schneider: 174 +
179 MarvZelkowitz: 177 I think issue is do we measure process quality, do we measure product quality, and is there a relation between the 2?
180 Tricia_Larsen: I think with the M&A PA of the CMMI, there has been an emphasis on linking the organization business goals to the projects measurements. I think there is a flavor of creating a measurement program, although not explicitly defined as such.
181 Frank_Sazama: 180 ++
182 Frank_Sazama: 180 this is the way I went the last 4 years with Bosch
183 muench: My impression is that CMM/I gives a lot of guidance about best practices and the current state of the usage of best practices in a company. If you have identified concrete problems, QIP/Profes provide guidance on how to improve in these problem areas.
184 Kurt_Schneider: 183 +
185 Rose_Pajerski: 183 so the measures for each are/need to be different?
186 MarvZelkowitz: 183 So what is the link between best practices and good products?
187 Vic_Basili: 183 Does this mean you believe that CMMI is abotu measureing the product?
188 Frank_Sazama: 187 there are also product measures included, yes
189 muench: 187 No. I mean, models like CMM/SPICE and continuous models like QIP do not contradict. CMM could give orientation (what are improvement areas) and QIP helps to identify key issues in these areas and helps to improve.
190 Oliver_Laitenberger: 188: Are these product measures concretly defined?
191 Vic_Basili: It has been my belief that the CMM stuff is about measuring process, not product, while QIP is about measureing both. So what project meausres does CMM/I support?
192 Frank_Sazama: 190 this is my or our interpretation and implementation
193 Oliver_Laitenberger: 189: Does that mean we use CMM to identify what should be done and QIP to describe how to do it?
194 Dave_Weiss: 191 In companies where I have worked, CMM is viewed as measuring the orgnaization, not the process or the product.
195 Oliver_Laitenberger: 194: +
196 Vic_Basili: 194 +
197 Dave_Weiss: This often turns out to be very destructive to the organization.
198 Frank_Sazama: 193 QIP is for me the continuous Improvement, and a way how to do it, BUT not the eyplanation what to do
199 Tricia_Larsen: 191 I don't think the CMMI limits you to process measures, although they are emphasized. But you define your process which includes the development of products. You in turn base your measurements accordingly.
200 Oliver_Laitenberger: 197: and the levels are also something that management understands.
201 Frank_Sazama: 194 if you see the assessment result as a measure than yes
202 Vic_Basili: 197 What do you mean by destructuve?
203 Rose_Pajerski: 197 destructive in what sense?
204 muench: 193 I would not agree in general. We use, for example, often SPICE as a starting point for identifying improvement aras. It gives companies kind of a benchmark.
205 Frank_Sazama: 204 +
206 Dave_Weiss: 202 The organization becomes conerend with achieving a certain number, rather than becoming a more effective development organization.
207 sandro_morasca: 206+
208 Frank_Sazama: 200 +
209 Vic_Basili: 204 But are those imporvement areas associated with where the process or managment is weak?
210 Tricia_Larsen: 206+
211 Dave_Weiss: When marketing gets into the act, the marketers are concerned that the organization gets the higest possible rating for the sake of appearance to customers.
212 Vic_Basili: So are we saying that providing number is interesting tot he management but distructive to the organization?
213 Frank_Sazama: 206 + yes in the beginning, in the background there will start a learnung process on the management level and thei understand their organization better and better
214 Dave_Weiss: 212+
215 sandro_morasca: 212 + It depends on what the number really means. If it's only good for marketing purposes ...
216 Tricia_Larsen: 212 I think it is an education process for the management to understand the true benefit
217 Dave_Weiss: That's why a goal-oriented approach is needed. Goals should be to improve the organization. Start from there and not from achieving a certain number or level.
218 Tricia_Larsen: of actually 'doing' process improvement
219 muench: 209 No. I think that it is not so much helpful for companies with a very high maturity. But it is a good starting point for others because it gives hints on missing practices that others sucessfully apply.
220 Frank_Sazama: 212 + it depends on the analysis and use of data
221 Oliver_Laitenberger: 212: At least the number motivates for some action instead of maintaining the status quo
222 MarvZelkowitz: 212 But the "true benefit" is often certification to bid on a proposal. A better process is possibly an unintended side effect.
223 Frank_Sazama: 219 ++
224 Vic_Basili: 219 I agree that CMM.. is a good consciousness raising approach, but if taken lterally, it becomes destructive
225 Frank_Sazama: 224 do you mean that we only should go to level 2 or 3 and than start with QIP and our own goals
226 Moderator: /vote Do you agree with statement 224?
227 MarvZelkowitz: 225+ Some companies often think level 3 is enough and 4 and 5 is overkill with no ROI (to go back to the earlier discussion)
228 Vic_Basili: 225 No tink going to a 3 is the most dangerous as it brings in a lot of process and then at level 4 you start to pare it away.
229 sandro_morasca: [voteyes]
229 MarvZelkowitz: [voteyes]
229 Oliver_Laitenberger: [votenotsure]
229 Tricia_Larsen: [voteno]
229 Vic_Basili: [voteyes]
229 Carolyn_Seaman: [voteno]
229 Dave_Weiss: Yes.
230 Frank_Sazama: 224 or do you mean that the process focus will be to much and the product and business focus to less
231 Dave_Weiss: [voteyes]
231 muench: 225 CMMI is quite comprehensive on a higher level. A goal-oriented continuous approach could be part of CMMI nowadays. But CMMI does not really provide guidance for continuous improvement compared to Profes.
232 Frank_Sazama: [voteno]
232 muench: [voteyes]
232 Kurt_Schneider: [voteno]
232 Moderator: /endvote
232 232 | 11/17/2004 12:16:04 PM | Vote question: Do you agree with statement 224?
Vote results:
       Yes  (5/10) 50%
        No  (4/10) 40%
  Not Sure  (1/10) 10%
  Total: 10 votes

233 Rose_Pajerski: [voteno]
233 Vic_Basili: So can the no's say why they voted no
234 MarvZelkowitz: Ditto for not sure and those who didn't vote
235 Moderator: 232: could you please comment on what on don't agree?
236 Oliver_Laitenberger: I think destructive needs to be explained. On the one hand we argue that we need some processes but at the same time it appears that these are destructive?
237 Kurt_Schneider: 233 I think 224 is right in conscious is raised. However, on the lower levels, CMM/SPICE etc is more: it gives guidance to a number of very concrete areas to work on. This does a lot of manager-convincing, and is not destructive. Again: it is a piece in the puzzle.
238 Carolyn_Seaman: 233 My problem was with wording. I think I agree with what you meant to say I think that if an organization takes the CMM/I "literally" then they are paying attention to all the guidance and rationale that it provides. The problem comes in when the focus is on the end number and nothing else. So the problem is not taking it "literally" but taking it too much on the surface level.
239 Rose_Pajerski: destructive at higher levels but not at lower when companies are just getting started
240 Dave_Weiss: 238+ 239-
241 Tricia_Larsen: I think it is all about how it it implemented. I also feel it is about education. And lately, it is about the education of the customer (in many cases the government) - that is the group currently driving the level numbers.
242 Vic_Basili: 238 I disagree. I tink putting all tha tproces in place, without worrying a priori about what is already working, is not the right thing to do
243 Oliver_Laitenberger: 241: ... and companies in industry follow this path.
244 Frank_Sazama: my experience is that the maturity and understanding for ROI and all the relations between processes is coming on level 4, so you have to go also for level 2 and 3
245 Dave_Weiss: I work for a company whoe major markets are outside the government sector. There is almost no discussion of CMM levels in the software development organizations.
246 Dave_Weiss: If your pdocuts are getting to the marekt on time, your customers are happy, and your revenues are growing, you don't care about what your level is.
247 Oliver_Laitenberger: 242: sometimes companies have not implemented basic processes such as a systematic test process. Is this also true for those?
248 Vic_Basili: 244 Why go through the levels. Why not start wil a evel 5 way of thinking and then put in place what is needeed to get there. This is QIP/EF.
249 Dave_Weiss: If your products are late, your customers are unhappy, and revenues are decreasing, you don't tyr to increase your CMM level to solve the problems.
250 Frank_Sazama: 242 you should not follow the model blind, open your eyes and think what you would like to do, think about business goals and concentrate
251 MarvZelkowitz: 248- See 246
252 Kurt_Schneider: 248 Because it takes such a lot of convincing
253 Dave_Weiss: 250+
254 Oliver_Laitenberger: 250: This requires skilled managers
255 Frank_Sazama: 248 because there is no maturity and no experience and no goal orientation in the company
256 Oliver_Laitenberger: 255 +
257 muench: 248 + I think it is better to start focusing on the main problems. However, a lot of companies do not have, for instance, change management or configuration management processes in place. To identify such basic weaknesses SPICE is very helpful.
258 Frank_Sazama: 248 you have to start simple and you have to show the way before you start. that is the reason to use CMM(I) staged
259 Kurt_Schneider: 248 CMM et al. provide guidance AND numbers (1..5) to reassure managers AND are well-respected in many companies. However, something else needs to be added once it is running. For example: QIP, EF
260 Oliver_Laitenberger: 257: SPICE is helpful for Step 1 of QIP: Characterize
261 Frank_Sazama: 259 ++
262 Vic_Basili: I still find this it problem to go from level to level. We add processes without understanding the problems and then learn to select and evolve processes over time - this is a hange in culture
263 muench: 257+
264 Frank_Sazama: 259 this is my experince coming from EF and going to CMM(I)
265 Vic_Basili: 260 +
266 Kurt_Schneider: 263 do you agree with yourself? Or what did I miss?
267 Dave_Weiss: 257 A very good example. If I can show a product manager that I can reduce the build time for his proudcts from 2 weeks to 2 hours, thereby removing a major bottleneck in getting products developed, then he will be enthusiastic in his support.
268 muench: 260+
269 Frank_Sazama: 267 ++
270 Oliver_Laitenberger: We have used a kind of SPICE-Approach to identify the problems. And it was very helpful for this purpose and motivate changes.
271 Kurt_Schneider: 267 right, and you do not need ROI nor QIP nor CMM. But what if you cannot do such a silver-bullety thing?
272 Vic_Basili: The time is up so are we ready to close up?
273 Frank_Sazama: 272 I have an open end
274 Vic_Basili: Any further comments? We did not get to the measure here but ...
275 Dave_Weiss: 271 Unless the company is very well-disciplined and everything is running smoothly, my experience is that it is easy to identify the big problems. Everyone knows them. It's jsut that they are often afraid to admit it.
276 Moderator: I will ask you a couple of questions regarding this meeting...
277 Kurt_Schneider: 275 IMPORTANT point! who agrees?
278 Frank_Sazama: 275 +
279 Oliver_Laitenberger: 275 +
280 MarvZelkowitz: 275+
281 Vic_Basili: 275 Does that mean they don't need CMMI?
282 Oliver_Laitenberger: Perhaps CMMI is the way to communicate the problems to the managers?
283 Frank_Sazama: they can used it before to reach the maturity
284 Tricia_Larsen: 282+
285 Dave_Weiss: 281 Didn't say that. They need continuous improvement; it's just got to start with the critical problems.
286 Moderator: /vote Was this eWorkshop of value to you?
287 Kurt_Schneider: 285 + ... and those are easy to see
288 MarvZelkowitz: [voteyes]
288 Dave_Weiss: [votenotsure]
288 Carolyn_Seaman: [voteyes]
288 Kurt_Schneider: [votenotsure]
288 Vic_Basili: [voteyes]
288 Frank_Sazama: [votenotsure]
288 sandro_morasca: [voteyes]
288 muench: [voteyes]
288 Rose_Pajerski: [voteyes]
288 Oliver_Laitenberger: [voteyes]
288 Tricia_Larsen: [voteyes]
288 Moderator: /endvote
288 288 | 11/17/2004 12:32:18 PM | Vote question: Was this eWorkshop of value to you?
Vote results:
       Yes  (8/11) 73%
        No  (0/11) 0%
  Not Sure  (3/11) 27%
  Total: 11 votes

289 Moderator: /vote Would you like to participate in another eWorkshop on measurement?
290 MarvZelkowitz: [voteyes]
290 Tricia_Larsen: [voteyes]
MarvZelkowitz | has left the room.
290 Kurt_Schneider: [votenotsure]
290 sandro_morasca: [voteyes]
290 Vic_Basili: [voteyes]
290 Carolyn_Seaman: [votenotsure]
290 Dave_Weiss: [voteno]
290 muench: [voteyes]
290 Rose_Pajerski: [voteyes]
290 Frank_Sazama: NOT SURE: it was a good experience, but I have to read all the stuff again
291 Oliver_Laitenberger: [voteyes]
291 Frank_Sazama: [voteyes]
291 291 | 11/17/2004 12:33:27 PM | Vote question: Would you like to participate in another eWorkshop on measurement?
Vote results:
       Yes  (8/11) 73%
        No  (1/11) 9%
  Not Sure  (2/11) 18%
  Total: 11 votes

292 Moderator: Thank you all for your time! We will send out a summary of the meeting for you to comment on.
293 Vic_Basili: Thanks everyone, we will send out a summary for comment
294 Kurt_Schneider: Thank you for organizing this!
295 muench: Thank you for the interesting discussion!
296 Oliver_Laitenberger: Thank you!
297 Frank_Sazama: thank you all and chat again
Rose_Pajerski | has left the room.
Tricia_Larsen | has left the room.
298 Vic_Basili: Bye
299 Frank_Sazama: kurt bis du zuhause oder im office, wuerde gern kurz anrufen
Vic_Basili | has left the room.
300 Moderator: bye
301 muench: Bye
302 Dave_Weiss: Thanks. Bye.
303 Kurt_Schneider: bin im büro
Dave_Weiss | has left the room.
sandro_morasca | has left the room.
Moderator | has left the room.
304 Frank_Sazama: bye
Oliver_Laitenberger | has left the room.
Frank_Sazama | has left the room.
muench | has entered the room.
Moderator | has entered the room.
305 Moderator: ewrer