| COTS Lessons Learned |
| Lesson Learned Statement |
Recommended action |
Date Added |
Details |
| COTS products pose new challenges for testing at both the product and system levels. |
Plan your system testing carefully. Make a realistic determination of what kind/levels of testing are required for the COTS and non-COTS parts of a system. Prototype your testing strategy, process, and tools early in the life cycle |
12/10/2003 |
Click here for details |
| COTS products pose new challenges for system integration. |
Get the products in early and do integration early in the development schedule. |
12/10/2003 |
Click here for details |
| It is important to establish and use robust testbeds that include interfaces to any external systems and capacity simulation and/or testing. |
unknown |
12/10/2003 |
Click here for details |
| Changes and upgrades to COTS products pose a new challenge for your system. |
Take into account different vendor release cycles, the cost of postponing upgrade vs. doing more frequent upgrades, etc. |
12/10/2003 |
Click here for details |
| COTS product modification and customization make system evolution more difficult. |
Avoid COTS modification. |
12/10/2003 |
Click here for details |
| Find a new engineering process for your CBS effort. |
Develop a new COTS development process for your organization. |
12/10/2003 |
Click here for details |
| Product modification complicates system and software engineering for COTS-based systems. |
Avoid COTS modification. |
12/10/2003 |
Click here for details |
| Plan ahead for technology insertion |
Build a good testbed for experimenting with new products. |
12/10/2003 |
Click here for details |
| It is important to establish relationships with COTS vendors and to make sure they are good relationships. |
Take the time and effort necessary to build good vendor relationships. |
12/10/2003 |
Click here for details |
| Vendor relationships are influenced by many factors. It is important to find out what motivates your vendors. |
Understand the presence your vendor has in the market, the maturity of the vendor’s product, how large a part you are of their customer base, and the importance of their product in your system. |
12/10/2003 |
Click here for details |
| It may be a challenge to establish and maintain good COTS vendor relationships. |
unknown |
12/10/2003 |
Click here for details |
| Out-sourced CBS applications that don’t require refreshed COTS components in their contracts for delivered applications often have to live with obsolete COTS products |
When contracting for CBS maintenance, make sure to plan for and specify the refresh of COTS components prior to acceptance |
6/5/2003 |
Click here for details |
| Maintenance complexity (and costs) will increase exponentially as the number of independent COTS packages integrated into a system increases. |
Projects should understand the maintenance implications of integrating large numbers of COTS products into a system. In addition to the effort involved in the initial integration, they need to consider that each product will evolve in its own way, according to different timetables, at the discretion of the vendors. You will have to expend considerable effort to handle the continuing evolution of these products (e.g., understanding the impact of an upgrade on the rest of the system, making changes to glue code). |
6/5/2003 |
Click here for details |
| Significant time and effort must be spent up-front analyzing the impact of version updates (even when the decision is made not to incorporate the updates). |
Maintenance modeling needs to assume that there are fixed and variable costs for CBS. Fixed costs are those associated with market watch and continued product evaluation. Variable costs are a function of the work performed to incorporate updates and fixes, changes and optimizations into the impending release. Workload balancing is needed to optimize the workload performed by the fixed staff as part of this process. |
6/5/2003 |
Click here for details |
| Flexible CBS software licensing practices lead to improved performance, reliability and expandability |
Partnering instead of conflict management was identified as the preferred approach to licensing. Shared goals lead to products with improved “goodness of fit” and “functionality” for the buyer. Leveraging relationships to achieve shared goals is strongly desired. Innovative contracting under such arrangements lead to deep volume discounts and priority service/bug fixes. Traditional approaches to licensing where contracts govern instead of relationships lead to distrust and poor results. |
6/5/2003 |
Click here for details |
| Wrappers can be effectively used to protect a CBS from unintended negative impacts of version upgrades |
CBS architectures should accommodate COTS changes throughout the system life cycle. |
6/5/2003 |
Click here for details |
| COTS software capability and quality evaluation needs to be managed as a continuing task during the maintenance phase |
Most COTS software studies recommend that firms (1) establish a market watch function to keep track of where their packages are heading and (2) an evaluation activity to continuously assess options. A market watch looks at the marketplace as a whole, monitoring the health and viability of a specific vendor as well as what the competitors are coming out with. The COTS evaluation activity provides you with a detailed assessment of package capabilities, quality issues and future options. It typically involves conducting some form of operational demonstration. |
6/5/2003 |
Click here for details |
| The cost to maintain COTS-based systems often equals or exceeds that of developing custom software. Maintenance in this context involves updating CBS with new releases, modifying wrappers and glue code and incorporating fixes/repairs into the system.
Costs average 10 percent of the development cost per year over a projected ten year life for the system. Although releases occur every year, technology refreshes for COTS occur every two years. Defect rates per release for CBS are poorer than for custom-built software, averaging 10 to 40 percent higher. |
Even though firms can save time and effort during development using CBS, they should evaluate the total life cycle cost of options prior to making commitments. Such analysis could identify risks that negate many of the advantages that CBS brings to the table. For example, firms must coordinate glue code updates along with package improvements. As a line of glue code costs, on average, three times the cost of a line of custom code to develop and maintain, the effort during maintenance can get quite expensive. In situations where COTS functionality is relatively small and the CBS has a long life, custom solutions may work out to be cheaper than COTS alternatives. COTS-based systems need a continual stream of funding throughout their life cycle. Such funding is required to keep up with the market place which is dynamic and recognizes the fact that vendors are continually releasing new versions. Funding is an issue because maintenance budgets get cut often. Maintenance budget cuts hurt a CBS more than a custom system because they are more able to delay maintenance in the latter if necessary. |
6/5/2003 |
Click here for details |
| The most significant variables that influence the cost of COTS-based systems maintenance include the following (in priority order): Number of COTS packages that need to be synchronized within a release, Technology refresh and renewal cycle times, Maintenance workload for glue code and wrapper updates, Maintenance workload to reconfigure packages, Market watch and product evaluation workload during maintenance, Maintenance workload to update databases, Maintenance workload to migrate to new standards |
Cost models like USC’s COCOTS (see http://sunset.usc.edu for information) need to be updated to encompass the full CBS life cycle. Currently, they focus on estimating the costs associated with evaluating, adapting and deploying COTS software packages during development and maintenance. In the future, such models need to incorporate additional variables like the last three bullets on our list to permit those assessing life cycle costs to estimate the full cost of the maintenance portion of the CBS life cycle. |
6/5/2003 |
Click here for details |
| The refresh and renewal process for COTS-based systems (CBS) needs to be defined a priori and managed so COTS package updates can be synchronized with each other and the organization’s release and business cycles. |
Guidance is needed to define the activities that take place during the refresh and renewal process. Criteria for making decisions relative to when to incorporate updates within releases also need to be defined along with their associated risks and business implications. |
6/5/2003 |
Click here for details |
|
Submit new lessons learned
First
Previous
[ 2 3 4 of 7 ]
Next
Last
|