Proceedings

Focused Workshop #7

 

System Integration

with

Commercial Off the Shelf

(COTS) Software

 

November 6 - 8, 1996

 

Center for Software Engineering

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Editors:

Christopher Abts

Barry Boehm

 

Proceedings

USC Center for Software Engineering

Focused Workshop #7:

System Integration with COTS Software

 

Table of Contents

 

Workshop Overview.............................................................................................................. p.1

(Dr. Barry Boehm & Dr. Walt Scacchi)................................................................ p.19

 

Workshop Issue Papers......................................................................................................... p.53

(Noll & Scacchi)……........………………………………………………….......... p.55

Safety Systems" (Scott, Preckshot & Gallagher)…...…………….......................... p.93

(Garlan, Allen & Ockerbloom)……………………………………….................... p.117

 

Technical Presentations........................................................................................................ p.127

USC Faculty and Graduate Students

New Tools Over Wide Area Networks)...………………………………………..... p.213

 

 

(continued)

iii

 

 

Table of Contents

(continued)

 

Technical Presentations (continued)

Affiliate Members

and Goals).........……………………………………………………………............ p.257

and Predictions).....……………………………………………………................... p.277

 

Breakout Groups................................................................................................................... p.329

 

Conclusion............................................................................................................................. p.405

Relations to WinWin and Architecture Projects (Dr. Barry Boehm)...................... p.407

 

Appendices............................................................................................................................. p.431

 

iv

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Workshop Overview

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Workshop Highlights

Barry Boehm

 

Introduction

 

Although the workshop concentrated on COTS software integration risks and ways to improve COTS integration, this was done in a context that COTS usage is generally a good thing. For most applications, deciding not to use COTS is simply unrealistic. But the road to successful COTS integration has many risks and pitfalls. In this context, I’d like to summarize the workshop results in terms of:

 

    1. Four characteristics of COTS integration which I found helpful in explaining the various pitfalls and recommendations highlighted by the workshop participants (several of the points are adapted from a particularly good Lockheed Martin briefing included in appendix A of these proceedings, "COTS Integration: Application Lessons Learned," which is well worth your study).
    2.  

    3. The primary research priorities identified by the participants, and USC-CSE’s plans for addressing them.
    4.  

    5. A set of corporate-level COTS integration issues emerging from the workshop which we plan to address at the upcoming USC-CSE Executive Workshop for Affiliates on March 12, 1997.

 

 

 

I. Four Key COTS Integration Characteristics and Their Implications

 

The four key COTS integration characteristics below make COTS integration significantly different from other forms of software development (including maintenance). They require traditional approaches to software development to be significantly revised. They are:

 

    1. You have no control over a COTS product’s functionality or performance.
    2. Most COTS products are not designed to interoperate with each other.
    3. You have no control over a COTS product’s evolution.

4. COTS vendor behavior varies widely.

 

 

1. You have no control over a COTS product’s functionality or performance.

 

If you can modify the source code, it’s not really COTS--and its future becomes your responsibility. Even as black boxes, big COTS products have formidable complexity: Microsoft people have indicated that Windows 95 has 25,000 entry points.

 

 

Resulting Pitfalls

 

 

Resulting Recommendations

 

 

 

2. Most COTS products are not designed to interoperate with each other.

 

The [Garlan et al, 1995] article included in these proceedings provides a good case study and explanation for why interoperability problems can cause COTS integration cost and schedule overruns by factors of four to five.

 

Resulting Pitfalls

 

Lack of COTS interoperability exacerbates each of the previously cited pitfalls. Some additional direct pitfalls are:

 

 

Resulting Recommendations

 

The previously cited recommendations on risk-driven processes and co-evolving your requirements and architecture are also appropriate here. In addition:

 

 

 

3. You have no control over a COTS product’s evolution.

 

Again, COTS vendors respond to the overall marketplace and not to individual users. Upgrades are frequently not upward compatible. And old releases become obsolete and unsupported by the vendor. If COTS architectural mismatch doesn’t get you initially, COTS architectural drift can easily get you later. Figure 1, from the Lockheed Martin briefing, indicates that current COTS-intensive systems often have higher software maintenance costs than traditional systems, but that good practices can make them lower. (Don't over-interpret the chart; it's not meant to imply that its cheaper for you to maintain you own home-brew relational DBMS than to pay the COTS maintenance fees for a clean SQL-compliant product.)

 

 

 

Resulting Pitfalls

 

Lack of evolution controllability exacerbates each of the previously cited pitfalls. Some additional direct pitfalls are:

 

 

Resulting Recommendations

 

The previously-cited risk-driven and architecture-driven recommendations are also appropriate here. In addition:

 

 

 

  1. COTS vendor behavior varies widely.

 

Vendor behavior varies widely with respect to support, cooperation, and predictability. Sometimes a COTS vendor is not even the developer, just a value-added reseller. Given the three major sources of COTS integration difficulty above, an accurate assessment of a COTS vendor’s ability and willingness to help out with the difficulties is tremendously important. The workshop identified a few assessment heuristics, such as the

experience that the value of a COTS vendor’s support follows a convex curve with respect to the vendor’s size and maturity (see Figure 2). Small vendors often lack the capability to support you; very large vendors have the capability, but not the motivation (the classic example is Microsoft).

 

 

 

 

High

 

 

Value of COTS

vendor support

 

 

 

Low

Low Medium High

 

COTS vendor size and maturity

 

 

Figure 2 - Variation of COTS Vendor Support with Size

 

 

 

 

Resulting Pitfalls

 

Poor COTS vendor support exacerbates each of the previously-cited

pitfalls. Some additional direct pitfalls are:

 

 

Resulting Recommendations

 

The previously cited recommendations are also appropriate here. In addition:

 

 

 

Summary of Four Key COTS Integration Characteristics and Their Implications

 

The main results of the general workshop presentations and the Risk Management breakout group can be summarized in terms of the four key COTS integration characteristics and their associated pitfalls to avoid and recommended practices to adopt. For easy reference, these are highlighted in Table 1.

 

 

Pitfalls to Avoid

Recommended Practices to Adopt

1. You have no control over a COTS product’s functionality or performance.

 

 

  • Using the waterfall model on a COTS integration project.
  • Using evolutionary development with the assumption that every undesired feature can be changed to fit your needs.
  • Believing that advertised COTS capabilities are real.

 

  • Use risk management and risk-driven spiral-type process models.
  • Perform the equivalent of a "receiving inspection" upon initial COTS receipt.
  • Keep requirements negotiable until the system’s architecture and COTS choices stabilize.
  • Involve all key stakeholders in critical COTS decisions.

 

2. Most COTS products are not assigned to interoperate with each other.

 

 

  • Premature commitment to incompatible combinations of COTS products.
  • Trying to integrate too many incompatible COTS products.
  • Deferring COTS integration till the end of the development cycle.
  • Committing to a tightly-coupled subset of COTS products with closed, proprietary interfaces.

 

 

  • Use the Life Cycle Architecture milestone as a

process anchor point.

  • Use the Architecture Review Board (ARB) best commercial practice at the Life Cycle Architecture milestone.
  • Go for open architectures and COTS substitutability.

 

 

3. You have no control over a COTS product’s evolution.

 

 

  • "Snapshot" requirements specs and corresponding point-solution architectures.
  • Understaffing for software maintenance,
  • Tightly coupled, independently evolving COTS products.
  • Assuming that uncontrollable COTS evolution is just a maintenance problem.

 

 

  • Stick with dominant commercial standards.
  • Use likely future system and product line needs as well as current needs as COTS selection criteria.
  • Use flexible architectures facilitating adaptation to change.
  • Carefully evaluate COTS vendors’ track records with respect to predictability of product evolution.
  • Establish a pro-active system release strategy, synchronizing COTS upgrades with system releases.

 

4. COTS vendor behavior varies widely.

 

 

  • Uncritically accepting COTS vendors’ statements about product capabilities and support.
  • Lack of fallbacks or contingency plans.
  • Assuming that an initial vendor support honeymoon will last forever.

 

 

 

  • Perform extensive evaluation and reference-checking of a COTS vendor’s advertised capabilities and support track record.
  • Establish strategic partnerships or other incentives for COTS vendors to provide support.
  • Negotiate and document critical vendor support agreements.

 

Table 1 - Key COTS Integration Characteristics and Their Implication

 

 

II. Affiliate Research Priorities and USC-CSE Response

 

Figure 3 summarizes the Affiliate research priorities from the Architecture breakout group. It assesses the candidate research activities considered by the group in terms of Relative Payoff and Relative Difficulty on (High, Medium, Low) scales. The most attractive research areas were those assessed as High in relative payoff and Medium in relative difficulty:

 

1. Architecture specification and modeling techniques.

6. Interface specification techniques.

11. Extend Ahmed Abd-Allah’s work to look at architectures and styles

from multiple views.

    1. How to map a COTS component into the system architecture.
    2. System and software engineering processes to deal with COTS integration.

 

The group also identified the areas they believed would be most fruitful for the USC-CSE architecture research activity to focus on:

 

 

 

USC-CSE will respond to these priorities by reinforcing those of our existing efforts which are aligned with the priorities, and by establishing a new effort to cover the major high-priority topic we are not addressing: integration of multiple architectural views.

 

 

In the area of heterogeneous architectural style composition, Cristina Gacek is extending Ahmed Abd-Allah’s AAA tool to cover additional architectural styles, such as repository, blackboard, real-time, and closed-loop feedback control. In the area of architecture-based tradeoff analysis, we are pursuing architectural style-sensitive analysis tools.

Ahmed Abd-Allah has identified approaches for some needed extensions to classical reliability models to cover style-sensitive phenomena (e.g., spawning of multiple process threads).

In the view integration area, we plan to build on our recent experiences in specifying multiple-view guidelines for fifteen student teams developing multimedia applications for the USC Library. The results indicated that a tighter integration of the views was both feasible and desirable. Alexander Egyed, who was Teaching Assistant for the fifteen

projects, will be investigating this for his Ph.D. research.

The cost modeling breakout group focused on creating a baseline functional form for the USC-CSE COTS Integration Cost Estimation Model. CSE had developed a straw man functional form based on a literature review, analysis of Affiliate questionnaire responses, and analysis of a number of candidate cost driver factors’ influence on several sources of COTS integration costs. The baseline functional form uses the External Interface portion of Function Points for sizing, with effort multipliers to be derived from the "Major Significance" and "Intermediate Significance" variables identified in Tables

2 and 3, which show the relative effects of the cost drivers on the five main sources of COTS integration costs.

 

 

 

 

III. Corporate-Level COTS Integration Issues

 

The workshop also identified a number of significant COTS integration issues that are important to understand and address at the executive level. We will be addressing these at the USE-CSE Executive Workshop on COTS Integration at USC on March 12, 1997. The major issues are:

 

 

 

 

 

 

The preliminary agenda for addressing these issues at the Executive Workshop begins with an overview session, followed by three Affiliate executive panel sessions on COTS considerations in commercial organizations, in government contracting, and in emerging technology. A subsequent proceedings will summarize the results of the Executive Workshop.

 

 

Final Agenda

 

USC Center for Software Engineering

Focused Workshop #7

System Integration with COTS Software

Nov. 6-8, 1996

Leavey Library Auditorium, USC Campus

 

 

Wednesday, Nov. 6

07:30 - 08:15 Continental Breakfast

08:15 - 09:00 Barry Boehm, USC COTS Integration Cost Modeling

09:00 - 09:25 Chris Abts, USC COTS Integration Survey Status Report

09:25 - 09:50 Ahmed Abd-Allah, USC COTS Integration Interoperability Analysis

09:50 - 10:20 Break

10:20 - 11:00 Dave Wile, USC Software Architecture Technology for COTS Integration

11:00 - 11:30 Walt Scacchi, USC Integrating COTS, Legacy and New Software Applications

11:30 - 12:15 Lunch

12:15 - 12:45 Gail Cochrane, TRW COTS for Safety Critical Systems

12:45 - 01:15 Agi Sardi, Raytheon COTS Integration Risk Management

01:15 - 01:45 Patricia Oberndorf, SEI SEI Integrated CASE Environments: Lessons and Predictions

01:45 - 02:15 Break

02:15 - 02:45 Tony Wasserman A Tool Builder’s View of COTS Integration

02:45 - 03:15 Marvin Carr, SEI COTS and SEI Risk Taxonomy

03:15 - 04:00 Other Affiliate Presentations

04:00 - 04:30 Determine Breakout Groups

04:30 - 05:30 Breakout Group Sessions

05:30 - 07:00 Reception (Faculty Center)

 

Thursday, Nov. 7

07:30 - 08:15 Continental Breakfast

08:15 - 12:00 Breakout Group Sessions

12:00 - 01:30 Lunch & Plenary Breakout Group Progress Reports

01:30 - TBD Breakout Group Sessions

 

Friday, Nov. 8

07:30 - 08:00 Continental Breakfast

08:00 - 10:30 Breakout Group Presentations & USC Responses

10:00 - 11:30 General Discussion & Wrap-up

 

 

 

 

Breakout Groups

 

Three breakout group topic areas are discussed in the USC-CSE Issue paper:

 

    1. Architecture Technology Solutions: Judy Kerner, Aerospace Corp., chair
    2. Risk Management Approaches: Walt Scacchi, USC, chair
    3. Cost Estimation Approaches: Barry Boehm, USC, chair

 

 

 

 

 

 

 

 

 

 

 

 

Workshop Issue Papers

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Technical Presentations

 

 

 

 

 

 

 

COTS Integration cost Modeling

Dr. Barry Boehm

USC

11/06/96

 

 

 

 

 

In this talk Dr. Boehm presented an overview of COTS modeling issues as examined by research being done at USC. He identified five sources of COTS integration effort (assessment, tailoring & tuning, glue code, application volatility, and added application validation & verification effort). Then he discussed the pros and cons of four COTS modeling alternatives within the context of COCOMO (COCOMO 2.0 reuse model, COTS-integration effort multiplier or exponent factor, COTS-integration effort added to other development effort, Stand-alone COTS-driven estimation model). This was followed by a review of the Loral CotsCost model, including cost drivers and calibration results. He concluded with a discussion of the proposed USC COTS integration cost model, including a proposed estimation procedure and candidate cost drivers, and areas for further discussion.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

COTS Software Research

Effort Status Briefing

Christopher Abts

USC

11/06/96

 

 

 

 

 

Mr. Abts presented an overview of the COTS integration cost model research being done at USC, followed by a discussion of the results of a first pass COTS data collection survey. The overview described near-term goals (a prototype cost estimation tool, estimation guidelines), funding sponsorship (Air force Electronics System Center and USC-CSE Affiliates), and the data collection approach (two data survey passes) forming the basis for model development. The second part of the discussion focused on preliminary results of the first round survey, including a prioritization of candidate cost drivers.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

COTS Integration Interoperability Analysis

Dr. Ahmed Abd-Allah

USC

11/06/96

 

 

 

 

 

Dr. Abd-Allah began with an example of the architecture of a COTS based tool (Aesop) taken from Garlan. Then he presented the architectural styles of representative COTS packages (UNAS, Prolog, SoftBench, OBST, BaseWorX, Delphi). This was followed by an approach to identifying architectural mismatches among COTS products (implicating potential COTS integration difficulties) by modeling architectural styles. This included discussion of efforts focused at a uniform representation of architectures in terms of base entities and style spaces, using those representations to indicate the architectural composition of COTS products, which in turn forms a basis for identifying possible architectural mismatches between COTS products. He concluded by discussing the prototype Architect’s Automated Assistant (AAA) as a tool for analyzing architectural constraint mismatches, and pointing to future research directions.

 

 

 

 

 

 

 

 

 

 

 

 

 

Software Architecture Technology for COTS Integration

Dr. David Wile

USC ISI

11/06/96

 

 

 

 

 

Dr. Wile began by discussing the importance of "domain specific" architectures. He talked about three levels of architectural environments (domain development, specific application development, application execution), and two architectural levels (reference architecture, application architecture). This was followed by a discussion of various forms of architecture "connectors," which form the basis for all inter-module interactions. Then he discussed architectural integration paradigms (old style: co-resident—new style: distributed & autonomous). He finally concluded with a discussion of example architectural tools.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Process Integration of COTS, Legacy and

New Tools Over Wide Area Networks

Dr. Walt Scacchi

USC

11/06/96

 

 

 

 

 

Dr. Scacchi began with a discussion of conventional software environment architecture and the difficulties associated with extrapolating this architecture to the modern distributed "virtual" software enterprise. This was followed by a discussion of the goal of process integration (application of tools to data by an explicit process), the potential problems interfering with attainment of that goal (data access, multiple process models and instances, legacy and COTS tools, process guidance), and the inadequacies of existing standards (CORBA, OLE) for mitigating these problems. This was followed by discussion of the DHT model as a way of overcoming these difficulties. The solution to process integration is found in a Process Modeling Language (PML) and a PML "engine," an environment for enacting processes.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

COTS/NDI Use in Safety Critical Systems

Gail Cochrane

TRW

11/06/96

 

 

 

 

 

Ms. Cochrane began with the goal of this study done for the FAA, which was to assess the potential need for changes in the FAA safety certification process due to the impact of COTS/NDI usage in safety critical systems. This was followed by an overview of the approach the study followed (evaluate direction of software engineering, identify motives for using COTS/NDI software, determine how COTS/NDI issues are addressed by other organizations, determine recommendations). Next came a discussion of the focus of the study, particularly with respect to software as the provider of functionality. After an in-depth review of the findings under each area of interest (evaluate direction, identify motives, determine other organization practices), she concluded with—among other recommendations—the following process recommendations:

 

 

 

 

 

 

 

 

 

 

 

COTS Risk Mitigation: Strategies and Goals

Agi Sardi

Raytheon

11/06/96

 

 

 

 

 

Ms. Sardi began with a brief "history" of the recent practice of COTS integration at Raytheon, moving from the ESC PRISM/CARDS program to the RAPID lab, and a focus on COTS integration issues to the RAPID technology deployment to COTS technology performance. This was followed by an in-depth description of the RAPID process, PRISM and RAPID Domain architecture, and change management model. Then an overview of the RAPID assessment process, which is focused on choosing the "right" COTS. In summary, RAPID is

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Integrated CASE Environments: Lessons and Predictions

Patricia Oberndorf

SEI

11/06/96

 

 

 

 

 

Ms. Oberndorf began with a background overview of the history and role of CASE environments, making the point that there isn’t much evidence of extensive use of such tools and environments in actual production of code. This was followed by historical examples from three broad categories: language-centered environments, framework-based environments, and tool-kit environments. This lead to a discussion of common observations found to hold true across large organizations using CASE environments:

 

 

This was followed by an overview of the current setting for CASE environments (open systems, COTS software, better understanding of software architectures, object oriented technology, and the World Wide Web). She concluded with some predictions regarding new attempts to produce CASE environments:

 

 

 

 

 

 

 

 

 

 

 

 

 

COTS Use: Theory and Practice

Dr. Rami Razouk

The Aerospace Company

11/06/96

 

 

 

 

 

This was an informal talk given as background. As such slides are not available for distribution. Salient points of the briefing are presented below:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

A Tool builder’s View of COTS Integration

Dr. Anthony Wasserman

11/06/96

 

 

 

 

 

Dr. Wasserman discussed his experiences at IDE with the Software Through Pictures environment. He began with an overview of environments and tools falling under the following categories: programming environments, documentation tools, requirements tools, configuration management, schema generations and version control. From these examples he derived several observations: all involve point-to-point integrations, no standards used, all need commitment to updating, large software engineering effort required, and customers rarely pay for integrations. Then he discussed why tool integration is difficult (the need for interaction with a vendor; tool rarely designed for integration; incompatibility between tool releases; individual and unique data storage, environment, and resource requirements). He also made the point that tool builders are the users of other tools, which can impose their own constraints on tool development. Then he discussed tool interface decision issues, ranging from the minimum effort (entire tool is invoked from another program) to the maximum effort (everything within the tool can be invoked individually). This lead to a discussion of the types of tool integration (data, control, presentation, platform and process integration). He concluded with a discussion of the future of tool integration mechanisms (tempered by the caveat that technical solutions may not always solve business problems):

 

 

 

 

 

 

 

 

 

COTS Integration Risks

Marvin Carr

SEI

11/06/96

 

 

 

 

 

Mr. Carr gave a presentation relating COTS integration risks to the SEI Risk Taxonomy and Risk Repository. The SEI study collected empirical data on COTS integration risks from thirty-six risk assessments conducted by SEI via interviews with COTS integration practitioners. From analysis of the data collected, SEI identified seven areas of concern with respect to COTS integration activities (documentation, performance, maturity, interface, verification, vendor support/upgrades, and data rights). He concluded with the admonition that COTS software is not often integrated truly "right off the shelf." Considerable modified code may be needed to make it work, and upgrades are frequently significantly more work and error-prone than anticipated.

 

 

 

 

 

 

 

 

 

 

 

 

 

COTS Integration Process Model

Karen Kelley

Lockheed Martin

11/06/96

 

 

 

 

 

This was an informal talk given as background. As such slides are not available for distribution. Salient points of the briefing are presented below:

 

 

 

 

 

 

 

 

 

 

 

Breakout Groups

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Architecture Technology

Solutions

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Risk Management

Approaches

 

 

 

 

 

 

Cost Estimation

Approaches

 

COTS Integration Cost Estimation Approaches

Group Discussion Summary

 

Moderator: Barry Boehm (USC)

 

Participants: Marvin Carr (SEI)

Sunita Devnani (USC)

Roger Dziegiel (Rome Laboratories)

Gary Thomas (Raytheon E-Systems)

Peggy Wells (USAF/ESC)

Scribe: Christopher Abts (USC)

 

 

Initial evening session 11/06/96:

 

First order of business for the group was to agree upon a discussion agenda (though each item was not necessarily approached in the given order):

 

 

Agenda

 

 

 

 

 

This was followed by some random exchanges to further stimulate ideas:

 

Issues to consider for the second round CSE COTS integration data collection questionnaire:

 

 

 

 

 

 

 

 

 

Morning session 11/07/96:

 

Began by characterizing sources of effort, arriving at the following assessments:

 

 

COTS Integration Sources of Effort

 

 

 

General comments evolving out of the above:

 

integration activities have different cost factors?

Answer: cost factors likely the same, but the ratings of those factors for the

various activities may be different.

 

 

 

Next followed a closer study of the sources of COTS integration effort delineated previously:

 

COTS Assessment

 

Recognizing that this effort is already captured somewhat in the COCOMO 2.0 cost factor RESL (Architecture/Risk Resolution), the group looked at Table 8: RESL Rating Components from the COCOMO 2.0 Model Definition Manual (reproduced on the following page). Conclusion was that this table will be useful in defining ratings for a similar factor in a COTS integration cost model, but needs to be tailored more specifically towards COTS integration issues. In particular, another row to this table could be added to address architectural mismatch.

 

COTS Tailoring, Tuning and Installation

 

Referring to tables showing both the COCOMO 2.0 and Loral Model cost driver contributions across the five COTS integration sources of effort (reproduced here as tables A and B), the point was made that good documentation makes tailoring, configuring, etc., much simpler. The rating impact of these five sources of effort really requires a 3-D solution space; e.g., the more information you have about a product and/or vendor, the less important some of these other drivers may become.

A lot of COTS products can’t be tailored at all, but problems with tailoring are often compensated by adding more functionality in the glue code.

COTS Glue Code Development

 

This effort can be small or large, depending upon the cleanliness and openness of the COTS product external interface elements, and how much additionally functionality must be added to the glue code for the reasons noted above under COTS tailoring. As indicated previously, however, this effort is usually large, and in fact is almost invariably the source of greatest required effort during the integration task.

 

Application Volatility due to COTS Products

 

This can be a very large effort to manage and/or contain, because the developer rarely (read: almost never!) has much control over when and how often the COTS vendor releases new versions of their product. This can become particularly acute if the overall software development project encompasses a large system whose development is spread out over a significant period of time. During that same period, the COTS products vendors, in order to stay competitive within their markets, have likely released multiple updates of their own products, which by default can lead to significant volatility in the main application software as the developer struggles to keep in step with the COTS vendors.

 

 

 

 

 

Table 8: RESL Rating Components

 

 

 

 

(excerpted from COCOMO 2.0 Model Definition Manual, USC 1996)

 

 

 

 

 

 

 

 

Added Application IV&V Effort

 

COTS products usually come with more functionality than is needed, but the developer dare not forego doing IV&V on the entirety of functionality offered by the COTS product to avoid unexpected interactions and problems.

 

 

More general comments:

 

Next topic approached was a COTS Integration Cost Estimation Procedure, with discussion converging on the following:

 

COTS Integration Cost Estimation Procedure

  1. Sizing
    1. determine Unadjusted Function Points (UFP) from External Interface Files (EIFs)
    2. includes COTS-COTS, COTS-application, COTS-platform (the operating system, not the database, though not always)
  2. Breakage
    1. estimate percentage of UFP breakage (BRAK) during the development period
    2. function of: number of COTS packages, average new releases per package, average interface breakage per release

 

(continued)

 

 

COTS Integration Cost Estimation Procedure

(continued)

 

 

3) Effective Size

    1. ESIZE = UFP (1.0 + BRAK/100)
    2. (an additive term to account for installation, assessment effort independent of size might be warranted)

4) Scaling

    1. initially assumed linear for simplicity
    2. B = 1.0

5) Effort Multipliers

    1. approach same as in COCOMO 2.0
    2. EMi

6) Estimated Effort

    1. Person Months (PM) = A * (ESIZE)B * P (EMi)
    2. (A is a constant used to capture the multiplicative effects on effort with projects of increasing size)

7) Effort Distribution by Source-of-Effort

    1. downstream possibility, not included in current model design

 

 

General comments on this topic:

 

Sizing

 

Breakage

 

 

Effective Size

 

 

 

Scaling

 

 

Effort Multipliers

 

 

Estimated Effort

 

 

 

 

 

The next activity pursued by the group was to compare all the cost drivers listed independently in tables A, B and C to try to reduce them into a single set of prioritized candidate cost drivers for the CSE COTS integration cost model. The result can be found in tables D and E.

 

General comments related to this activity are as follows, focusing on Loral factors:

 

Support Offered

 

 

 

 

 

Young Vendor Behemoth (e.g., Microsoft)

 

 

 

 

 

 

 

 

Afternoon session 11/07/96:

 

Discussion of candidate cost drivers continued, this time focusing on the COCOMO 2.0 drivers:

 

 

 

 

 

 

 

 

 

 

Table 20: Module Complexity Ratings versus Type of Module

 

 

 

(excerpted from COCOMO 2.0 Model Definition Manual, USC 1996)

 

 

 

 

 

 

 

 

 

 

Future plans for the CSE COTS integration cost model were then discussed:

 

 

 

 

 

 

 

 

 

The final activity broached by the group was to create an incremental strategy for the further development of the CSE COTS integration cost model:

 

Incremental COTS Model Development Strategy

 

 

 

 

 

 

 

 

 

 

-- finis --

 

 

 

 

 

 

 

 

 

 

 

 

 

Conclusion

 

 

 

 

 

 

 

 

 

 

 

USC-CSE Response to Workshop Ideas:

COTS Integration Process Approaches

Dr. Barry Boehm

USC

11/08/96

 

 

 

 

 

Dr. Boehm’s USC-CSE response briefing related a number of the COTS integration process issues (early risk identification and resolution; anticipating and architecting for users’ and maintainers’ downstream evolutionary needs; early reconciliation of stakeholders’ COTS-related win conditions) to USC-CSE’s research on the WinWin Spiral Model and the WinWin requirements/architecture negotiation tool. The Workshop Highlights discussion found at the front of these proceedings elaborates on the process issues. The resulting discussion identified further Affiliate suggestions for USC-CSE research priorities in the architecture and cost modeling areas—again, these are summarized in the Workshop Highlights.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Wrap-up

Dr. Barry Boehm

USC

11/08/96

 

 

 

 

 

The Wrap-up discussion focused on suggestions for future workshops. Its primary outcome was to establish COTS Integration as the topic for CSE’s next Executive Workshop to be held at USC in Los Angeles on March 12, 1997.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Appendices

 

 

 

 

 

 

 

 

 

Appendix A

 

"COTS Integration: Application

of Lessons Learned"

Lockheed Martin

 

 

 

 

 

 

 

 

 

 

 

Appendix B

 

"Anchoring the Software Process"

Barry Boehm

 

 

 

 

 

 

 

 

 

 

 

Appendix C

 

"Best Current Practices: Software

Architecture Validation"

AT&T/Lucent.

 

 

 

 

 

 

 

 

 

 

 

Appendix D

 

CSE Preliminary COTS Survey

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Appendix E

 

Conference Attendees