University of Southern California
Home   Contact Us    
Center for Systems and Software Engineering

About us
Tech. Report
TR by Author
List of Affiliates
Private Area
Other Resources

PhD Dissertations

Ahmed Abd-Allah, "Composing Heterogeneous Software Architectures," PhD Dissertation, Department of Computer Science, University of Southern California, August 1996 (pdf)

A persistent problem in software engineering is how to put complex software systems together out of smaller subsystems, the problem of software composition. The emergence of software architectures and architectural styles has introduced a higher level of abstraction at which we can create and compose software systems. We examine the problem of providing formal semantics to the composition of different architectural styles within software systems, i.e. the problem of composing heterogeneous architectures. We describe a model of pure styles, and a model of their composition.

Our model of pure styles is highlighted by a uniform representation for describing many different styles. An architectural style space of major conceptual features is introduced which allows new styles to be rapidly incorporated into the model, including commercial-off-the-shelf packages which embody a specific style(s). We show a disciplined approach to the process of architectural composition, and show how architecture mismatches can be generated during composition. Finally, we describe a prototype tool which is built on top of the models.

Chris Abts , "Extending the COCOMO II Software Cost Model to Estimate Effort and Schedule for Software Systems Using Commercial-off-the-Shelf (COTS) Software Components: The COCOTS Model," PhD Dissertation, Department of Industrial and Systems Engineering, University of Southern California, May 2004 (pdf)

This dissertation discusses changes in software development practice that have occurred since the 1970s. In particular, the focus is on the rise of the "off-the-shelf"approach whereby large software systems are constructed out of a mix of original code and adapted code as well as pre-existing "black box" components for which the developer has no access to the source code. While certainly used prior to 1980, this technique of using black box elements has taken on greatly increased importance in recent years as one way to manage development risks as the overall complexity of software has grown.

Software estimation models, however, have generally not kept apace with off-the-shelf techniques. This includes the widely-used COCOMO software estimation model. While recently updated as the COCOMO II estimation model to reflect other changes in software development since its original publication in 1981, it still is unable to accommodate the use of black box off-the-shelf components in its estimation framework.

The focus of this dissertation then is the investigation of a potential extension of COCOMO II that is intended to provide an estimation capability for any off-the-shelf elements being designed into a software system. It begins with an exploration of how development activities differ when using off-the-shelf components as opposed to developing system components as original code. This serves as a basis for the formulation of an off the-shelf estimation model called COCOTS that is carried through several iterations, each intended to improve upon its predecessor. The dissertation concludes with a comparison of the fidelity of COCOTS with COCOMO II, limitations of available project data that impacted the calibration of COCOTS, and suggestions for further investigation that might improve the fidelity of COCOTS itself and also expand and deepen the general understanding of the benefits and risks of using off-the-shelf components in software system development.

Mohammed Al-Said, "Detecting Model Clashes During Software Systems Development," PhD Dissertation, Department of Computer Science, University of Southern California, December 2003 (pdf)

Software models are widely used during software systems development. Software developers use the models to better understand the system users' requirements, develop candidate solutions, and validate their decisions. Currently, the set of existing software models is large. Model clashes occur when models with inconsistent assumptions are used at the same time during the development of a software dependent system. Model clashes occur frequently, impede product development, and may cause project failure if not detected and avoided in a timely manner. But outsides of product model clashes (architecture, structure, traceability gaps), relatively little has been done to address them.

This thesis lays out the background of the model clashes problem which includes formally defining a model-clash, formally demonstrating that model clashes exist between software models, and listing the causes of model clashes. It describes a new method for identifying the models' assumptions and the clashes among these assumptions. This method was validated by applying it against a mixture of commonly used software models and the result was used to assess the ability of MBASE (Model Based system Architecting and Software Engineering) to identify and avoid model clashes. A semi-automated software tool was developed based on this method.

To demonstrate the value of detecting and avoiding model clashes, this thesis establishes the relationship between the consequences of model clashes and the risk in software projects. Using data from 35 completed software projects, it shows that each clash has an impact, which can be defined as the product of the clash's occurrence probability and severity. This thesis shows that the clash impact is directly related to the project risk exposure.

Finally, this thesis describes an experiment involving 18 software projects, where a process was applied to detect, analyze, resolve, and avoid model clashes. The results demonstrate the distribution of the different clash types and each type contribution to the project risk. The results also validate the hypothesis: detecting and avoiding model clashes during software systems development improves the software projects' outcome.

Pongtip Aroonvatanaporn, "Shrinking the Cone of Uncertainty With Continuous Assessment for Software Team Dynamics in Design and Development" PhD Dissertation, Department of Computer Science, University of Southern California, August 2012 (pdf)

Team synchronization and stabilization are essential especially for large software projects. Unfortunately, little is usually done to assess and reduce the uncertainties and knowledge gaps that exist within the project. As the project progresses through its life cycle, the team can gain more information about the project and the team's capabilities. These necessary data can be obtained through performing assessments on the team and the project. However, these assessment procedures are often complex, discouraging, and difficult to analyze; an effective framework and tool support can greatly enhance the process. Hence, with improved assessment methods, software project teams can quickly gather the necessary data, determine the actions needed to improve performance, and achieve an improved project outcome in the end. Furthermore, uncertainties in projects can significantly impact the cost and schedule estimations, which determine the scopes of the projects and the resources required to complete them. However, developing accurate and realistic estimates requires a high level of experience, expertise, and historical data. Oftentimes, once the resources have been estimated, little is done to reduce the uncertainties in the estimations as the project progresses through its life cycle. With improved software tracking and estimation mechanisms, teams can develop more accurate estimations, allowing the goals of the project outcome to be monitored and achieved with the available resources. This research investigates the appropriateness of current project tracking and team assessment methodologies, and develops a new assessment framework to effectively improve team synchronization and stabilization as well as project effort tracking and estimation. The COnstructive Team Improvement Process MOdel (COTIPMO) is an assessment framework developed to effectively achieve this by enabling software development teams to quickly track their project progress, continuously assess the team's performance, and make adjustments to the project estimates as necessary, thus shrinking the cone of uncertainty for software projects.

Jongmoon Baik, "The Effects of CASE Tools on Software Development Effort," PhD Dissertation, Department of Computer Science, University of Southern California, December 2000 (pdf)

CASE (Computer Aided Software Engineering) tools have played a critical role in improving software productivity and quality by assisting tasks in software development processes since the 1970’s. Several parametric software cost models adopt “use of software tools” as one of the environmental factors that affect software development productivity. However, most software development teams use CASE tools that are assembled over time and adopt new tools without establishing formal evaluation criteria. Several software cost models assess the productivity impacts of CASE tools based just on breadth of tool coverage without considering other productivity dimensions such as degree of integration, tool maturity, and user support. This dissertation provides an extended set of tool rating scales based on the completeness of tool coverage, the degree of tool integration, and tool maturity/user support. It uses these scales to refine the way in which CASE tools are effectively evaluated within COCOMO (COnstructive COst MOdel) II. In order to find a best fit of weighting values for the extended set of tool rating scales in the extended research model, a Bayesian approach is adopted to combine two sources of (expert-judged and data-determined) information to increase prediction accuracy. The research model using the extended three TOOL rating scales is validated by using cross-validation methodologies such as data splitting and bootstrapping.

Yimin Bao, "A Flexible Integration Framework for Software Tool Interoperability," PhD Dissertation, Department of Computer Science, University of Southern California, August 1996 (pdf)

Current trend of constructing new systems from collections of pre-existing third-party tools and commercial off-the-shelf (COTS) software presents serious challenges to existing integration technology. The unique constraints of third-party software, the interactive, autonomous nature of most of today's software applications, and the increasingly emphasized new integration requirements are among the major reasons for causing this difficulty.

This dissertation presents a flexible integration framework which has general applicability for pre-existing tools and COTS software, supports users to easily change the way software interact with each other (thus supporting flexible integration, system evolution and component reusability), and is easily programmable by the end-users. The framework consists of four major substrates: (1) A dynamic interoperability model which separates interaction relationship from tools and provides implicit communication and dynamic binding for managing independence and flexibility of the interaction relationship. This forms the conceptual base for our framework; (2) A black box integration approach called implicit tool encapsulation which extends the traditional integration interfaces to graphical user interface and system interface, and uses passive, implicit, system-centered mechanism for tool encapsulation; (3) A general methodology for tool integration which is derived from the dynamic interoperability model and implicit tool encapsulation approach, and is applicable for pre-existing third-party tool and COTS software; (4) A language called Tool Integration Language (TIL) which has the capability of accessing the functionality of tools through their user interface as well as other interfaces, and it can be used to describe the user's interaction between tools and the interaction between tools themselves. This provides necessary integration language for our framework and de-couples the integration policy from the mechanism.

A prototype system--Tool Integration Server System (TISS) has been constructed which supports TIL glue code generation and execution, and provides flexible integration mechanisms for our framework. By instantiating the general integration methodology with TIL language and TISS system, it generates a flexible and practical integration approach for integrating pre-existing, GUI-based, interactive tools without any source code modification.

Jesal Bhuta, "A Framework for Intelligent Assessment and Resolution of Commercial-Off-The-Shelf Product Incompatibilities," PhD Dissertation, Department of Computer Science, University of Southern California, August 2007 (pdf)

Software systems today are frequently composed from prefabricated commercial components that provide complex functionality and engage in complicated interactions. Such projects that utilize multiple commercial-off-the-shelf (COTS) products often confront interoperability conflicts resulting in budget and schedule overruns. These conflicts occur because of the incompatible assumptions made by developers during the development of these products. Identification of such conflicts and planning strategies to resolve them is critical for developing such systems under budget and schedule constraints. Unfortunately, acquiring information to perform interoperability analysis is a time-intensive process. Moreover, increase in the number of COTS products available to fulfill similar functionality leads to hundreds of COTS product combinations, further complicating the COTS interoperability assessment activity.

This dissertation motivates, presents and validates an intelligent assessment and resolution framework for Commercial-Off-The-Shelf (COTS) incompatibilities. The framework can be used to perform high-level and automated interoperability assessment to filter out COTS product combinations whose integration will not be feasible within project constraints. The framework efficiently and effectively captures knowledge on COTS product interoperability and allows a user to automatically leverage this knowledge to perform interoperability assessment. The framework elements have been utilized to develop an interoperability assessment tool – Integration Studio.

This framework is empirically validated using controlled experiments and project implementations in 25 projects from small, medium and large network centric systems from diverse business domains. The empirical evidence consistently indicates an increase in interoperability assessment productivity by about 50% and accuracy by 20% in small and medium systems.

Yuriy Brun, "Self-Assembly for Discreet, Fault-Tolerant, and Scalable Computation on Internet-Sized Distributed Networks," PhD Dissertation, Department of Computer Science, University of Southern California, March 2008 (pdf)

When engineers compare biological and software systems, the former come out ahead in the majority of dimensions.  For example, the human body is far more complex, better suited to deal with faulty components, more resistant to malicious agents such as viruses, and more adaptive to environmental changes than your favorite operating system.  Thus it follows that we, the engineers, may be able to build better software systems than the ones we build today by borrowing technologies from nature and injecting them into our system design process. 

In this dissertation, I present an architectural style and accompanying implementation support for building distributed software systems that allow large networks, such as the Internet, to solve computationally intensive problems.  This architectural style, the tile style, is based on a nature's system of crystal growth, and thus inherits some of nature's dependability, fault and adversary tolerance, scalability, and security.  The tile style allows one to distribute computation onto a large network in a way that guarantees that unless someone controls a large fraction of the network, they cannot learn the private data within the computation or force the computation to fail.  These systems are highly scalable, capable of dealing with faulty and malicious nodes, and are discreet since every sufficiently small group of nodes knows neither the problem nor the data. 

The tile style is based on a formal mathematical model of self-assembly.  In order to leverage this model to build software, I define the notion of self-assembling computation and develop systems that compute functions such as adding, multiplying, factoring, and solving NP-complete problems SubsetSum and SAT.  For each system, I prove its correctness, compute its probability of successful computation, and show that its running time and tileset size are asymptotically optimal. 

I use the mathematical nature of the tile assembly model to present a formal mathematical analysis of the tile style, proving that software systems built using this style are discreet, fault- and adversary-tolerant, and scalable.  I further implement a tile-style system and use it to distribute computation to empirically evaluate the tile style's utility.

Yue Chen, "Software Security Economics and Threat Modeling Based on Attack Path Analysis," PhD Dissertation, Department of Computer Science, University of Southern California, November 2007 (pdf)

The thesis presents the Threat Modeling Method Based on Attack Path Analysis (T-MAP) which quantifies security threats by calculating the total severity weights of relevant attacking paths for Commercial Off The Shelf (COTS) based systems. Further security economic analysis enabled by T-MAP is demonstrated. Compared to existing approaches, T-MAP is sensitive to system stakeholder value priorities and organizational IT environment. It distills the technical details of thousands of relevant software vulnerabilities into management-friendly numbers at a high-level; it systematically establishes the traceability and consistency from management-level organizational value propositions to technical-level security threats and corresponding mitigation strategies. In its trial usage in a large IT organization, T-MAP demonstrated promising strength in prioritizing and estimating security investment cost-effectiveness, as well as in evaluating the security performance of COTS systems. The steps of using T-MAP to optimize the cost-effectiveness of software patching, user account control and firewall are demonstrated through case studies. A system stakeholder value sensitive Firewall rule generation method based on T-MAP is introduced. In addition, a software tool that automates T-MAP has been developed.

Zhihao Chen, "Reduced-Parameter Modeling for Cost Estimation Models," PhD Dissertation, Department of Computer Science, University of Southern California, May 2006 (pdf)

In this research, parametric software cost estimation models and their related calibration methods have been analyzed, especially for the COCOMO model and the Bayesian calibration approach. This research combines machine learning techniques and tatistical techniques. With this approach, the prediction powers of the COCOMO parametric software cost model are shown to be significantly improved while the variability is decreased with respect to the dataset being analyzed. This research studies not only the accuracy but also the variances of the model and the variables. It can improve the confidence of people who use software cost estimation models, show the prediction power of software cost estimation models after calibration, and make it easier and better to perform software data collection and analysis. However, the research also identifies risks in using the approach, such as dropping parameters that will vary on future projects. This research provides methods that can help an organization to reason about the relationship between the characteristics of the organization and its projects' software development costs and schedules. The methods can thus help the organization to make more cost-effective development decisions and investment decisions. The research also provides new insights on how to combine calibration, stratification, hold-out, and machine learning techniques to produce more accurate parametric models for particular organizations or situations.

Sunita Chulani, "Bayesian Analysis of Software Costs and Quality Models," PhD Dissertation, Department of Computer Science, University of Southern California, May 1999 (pdf)

Software cost and quality estimation has become an increasingly important field due to the increasingly pervasive role of software in today's world. In spite of the existence of about a dozen software estimation models, the field continues to remain not-too-well-understood, causing growing concerns in the software-engineering community.

In this dissertation, the existing techniques that are used for building software estimation models are discussed with a focus on the empirical calibration of the models. It is noted that traditional calibration approaches (especially the popular multiple-regression approach) can have serious difficulties when used on software engineering data that is usually scarce, incomplete, and imprecisely collected. To alleviate these problems, a composite technique for building software models based on a mix of data and expert judgement is discussed. This technique is based on the well understood and widely accepted Bayes' theorem that has been successfully applied in other engineering domains including to some extent in the software-reliability engineering domain. But, the Bayesian approach has not been effectively exploited for building more robust software estimation models that use a variance-balanced mix of project data and expert judgement.

The focus of this dissertation is to show the improvement in accuracy of the cost estimation model (COCOMO II) when the Bayesian approach is employed versus the multiple regression approach. When the Bayesian model calibrated using a dataset of 83 datapoints is validated on a dataset of 161 datapoints (all datapoints are actual completed software projects collected from Commercial, Aerospace, Government and non-profit organizations), it yields a prediction accuracy of PRED(.30) = 66% (i.e. 106 or 66% of the 161 datapoints are estimated within 30% of the actuals). Whereas the pure-regression based model calibrated using 83 datapoints when validated on the same 161 project dataset yields a poorer accuracy of PRED(.30) = 44%.

A quality model extension of the COCOMO II model, namely COQUALMO, is also discussed. The development of COQUALMO from its onset enables one to understand how a comprehensive modeling methodology can be used to build effective software estimation models using the Bayesian framework elaborated in this dissertation.

Bradford Clark, "The Effects of Software Process Maturity on Software Development Effort," PhD Dissertation, Department of Computer Science, University of Southern California, August 1997 (pdf)

This research examines the effects of Software Process Maturity, using the Software Capability Maturity Model, version 1.1, on software development effort. The technical challenge in this research is determining how much change in effort is due solely to changing Process Maturity when this change generally occurs concurrently with changes to other factors that also influence software development effort. Six mathematical models used in this research support the conclusion that for one hundred twelve projects increasing Process Maturity one level results in a 15% to 21% reduction in effort.

George Edwards, "Automated Synthesis of Domain-Specific Model Interpreters," PhD Dissertation, Department of Computer Science, University of Southern California, August 2010 (pdf)

Software architecture models are specifications of the principal design decisions about a software system that primarily govern its structure, behavior, and quality. Architecture models provide a blueprint for how the system is implemented, serve as a basis for experimentation with and rationalization of design decisions, and enable the automation of software engineering tasks. Domain-specific languages (DSLs) are modeling languages that are defined and customized for a particular family of problems or systems (the domain). DSLs are able to concisely and intuitively express software architecture models because they natively include the design abstractions that are most useful and natural for the system under development.

However, because DSLs have non-standard semantics that cannot be known a priori, leveraging architecture models specified in a DSL for automated quality analysis, code generation, and other activities requires the implementation of specialized tools, called model interpreters, that perform these functions. Implementation and maintenance of domain-specific model interpreters are widely recognized as difficult and complex tasks that incur a significant burden on software engineers and reduce the appeal of the DSL approach.

In this dissertation, I describe XTEAM, an integrated set of processes, notations, tools, and designs that enable the automated synthesis of model interpreters for DSLs. The key elements of XTEAM are (1) semantics embedded in the DSL specification (called a metamodel), (2) a metamodel interpreter that automatically synthesizes transformation rules derived from those semantics, and (3) a reusable model interpreter framework that applies those transformation rules in an efficient and structured way to domain-specific models. The impact of XTEAM is that software engineers can use a DSL and perform automated model analysis, code generation, and other types of model transformation and manipulation without having to develop custom tools, greatly reducing the time, effort, expense, and risk associated with domain-specific modeling. XTEAM has been and continues to be used successfully on a number of software engineering projects, and experimental evaluation of XTEAM indicates that, even when pessimistic assumptions are made, XTEAM automates what would otherwise require multiple person-months of effort.

Alexander Egyed , "Heterogeneous View Integration and Its Automation," PhD Dissertation, Department of Computer Science, University of Southern California, August 2000 (pdf)

Software systems are characterized by unprecedented complexity. One effective means of dealing with that complexity is to consider a system from a particular perspective, or view (e.g., architecture or design diagram). Views enable software developers to reduce the amount of information they have to deal with at any given time. They enable this by utilizing a divide-and-conquer strategy that allows large-scale software development problems to be broken up into smaller, more comprehensible pieces. Individual development issues can then be evaluated without the need of access to the whole body of knowledge about a given software system. The major drawback of views is that development concerns cannot truly be investigated by themselves, since concerns tend to affect one another. Successful and precise product development supported via multiple views requires that common assumptions and definitions are recognized and maintained in a consistent fashion. In other words, having views with inconsistent assumptions about a system's expected environment reduces their usefulness and possibly renders invalid solutions based on them.

Developing software systems therefore requires more than what general-purpose software development models can provide today. Development is about modeling, solving, and interpreting, and in doing so a major emphasis is placed on mismatch identification and reconciliation within and among diagrammatic and textual views. Our work introduces a view integration framework and demonstrates how its activities enable view comparison in a more scalable and reliable fashion. Our framework extends the comparison activity with mapping and transformation to define the 'what' and the 'how' of view integration. We will demonstrate the use of our framework on the Unified Modeling Language (UML), which has become a de-facto standard for object-oriented software development. In this context we will describe causes of model inconsistencies among UML views, and show how integration techniques can be applied to identify and resolve them in a more automated fashion. Our framework is tool supported.

Jared Fortune, "Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0)," PhD Dissertation,Department of Industrial and Systems Engineering, University of Southern California, December 2009 (pdf)

The Constructive Systems Engineering Cost Model (COSYSMO) is an open, parametric model developed at the University of Southern California Center for Systems and Software Engineering (USC-CSSE) as a tool for estimating the amount of systems engineering effort for a large-scale system. This dissertation presents an improvement to COSYSMO by introducing the ability to account for systems engineering reuse through 1) a framework for the reuse process and 2) a refined characterization of systems engineering reuse.

Complex systems have reached the point where reuse is the norm rather than the exception. In an effort to address increasing complexity while maintaining manageability, systems engineers frequently leverage (or reuse) previously developed systems engineering products (i.e. requirements, test plans, interfaces, etc.) as a means of reducing development schedule, system cost, or risk; however, the effects of reuse have historically been difficult to quantify, and therefore justify. Through past and on-going work at USC-CSSE in systems engineering cost modeling as well as the close relationship between USC-CSSE and its industry affiliates, the need for a systems engineering reuse cost model was identified. COSYSMO 2.0 addresses this need by providing systems engineers with a tool that enables the quantification and rationalization of the effect of reuse. The model has broad usability for and direct applicability to practitioners, and has been validated by both subject matter expert opinion and across over 40 projects at a large and diversified systems engineering organization.

The intellectual contributions of this dissertation are not solely limited to COSYSMO 2.0 as an improvement to COSYSMO and a model for estimating the effect of systems engineering reuse, but also include a) the identification of research gaps in the field of systems engineering, b) the characterization of systems engineering reuse through industry-accepted definitions, and c) the development of a framework and heuristics for how systems engineering organizations can approach and manage reuse.

Cristina Gacek, "Detecting Architectural Mismatches During Systems Composition," PhD Dissertation, Department of Computer Science, University of Southern California, December 1998 (PARTS: body, appendix WHOLE: pdf)

The USC Architect's Automated Assistant (AAA) tool and method version 0.1 [Abd-Allah 1996] provides a capability for early detection of software architectural style mismatches among four architectural styles: Main-Subroutine, Pipe-and-Filter, Event-Based, and Distributed Processes. For these four styles, mismatch detection is based on a set of seven conceptual features distinguishing each style, and a set of bridging connectors characterizing compositions among the four styles. However, it was a significant open question whether these conceptual features and connectors were sufficient to characterize composition of other architectural styles.

The work presented here formalizes some additional architectural styles--namely Blackboard, Closed-Loop Feedback Control, Logic Programming, Real-Time, Rule-Based, Transactional Database, and Internet Distributed Entities styles--and extends the mismatch analysis capability to cover interactions of the original four styles with the new ones. The analysis results tested various hypotheses, such as the extensibility of the conceptual feature framework for mismatch detection, and the sufficiency of the original seven conceptual features to characterize the broader set of styles and their composition. In our work we found that the underlying conceptual feature framework could work to cover a broader range of styles and systems, with some extensions. However, the conceptual feature set and the underlying Z-language formal model were not sufficient to cover the full range of styles and systems interactions. We have developed extensions to the conceptual feature set and Z formal model to cover the full set of compositional interactions analyzed. Additionally, we provide means for checking each and every mismatch at the model level, including the dynamic ones, as well as a fully operational tool. We also provide an initial discussion of a more formal basis for detecting and classifying architectural conceptual features, thus providing a formal framework for extending the models.

LiGuo Huang, "Software Quality Analysis: A Value-Based Approach," PhD Dissertation, Department of Computer Science, University of Southern California, December 2006 (pdf)

Quality is a major challenge for all complex software systems. Some important attributes of software quality include reliability, availability, safety, security, survivability, performance, accuracy, etc. These have long been requirements of aerospace and defense systems. Now, equally challenging requirements are being placed on “everyday systems” that increasingly provide the infrastructure for our daily lives such as commercial, e-business and embedded systems. They are subject to modest, usually tacit, often stakeholder-specific quality requirements. And it is important that we can define and meet a software system’s quality requirements to be fit for its purpose.

However, there are many views of software quality. And there also exist many ways to suboptimize its achievement and/or misallocate limited project resources using incomplete views. In addition, complex missions or projects usually involve a large and heterogeneous group of stakeholders with various (often time-varying) quality priorities and different (often conflicting) needs. This makes both one-size-fits-all quality metrics and software development processes driven by such metrics risky to use. This also points to the need for better frameworks to define, reason about and achieve quality attributes.

Based on these observations, the primary goal of this dissertation is to develop a Value-Based Software Quality Analysis framework that integrates the stakeholder/value approach into quality attribute definitions, metrics, models and development processes aiming at achieving the appropriate quality levels for software systems. This framework pays explicit attention to business values that a software system generates for its success-critical stakeholders. It helps us to understand the nature of quality and to achieve the stakeholder mutually satisfactory quality requirements. It addresses the above problems in four aspects: 1) value-based definitions of software quality attributes; 2) value-based quality metrics; 3) Value-Based Software Quality Model (VBSQM) to reason about the Return On Investment (ROI) of quality and to perform combined risk analyses; and 4) Value-Based Software Quality Achievement (VBSQA) process.

Finally, the VBSQM and VBSQA process have been applied and found to be effective on three diverse software systems with different value profiles: a USC-CSE e-service project, the NASA/USC Inspector SCRover project, and a real-world ERP software development project in industry.

Hoh Peter In, "Conflict Identification and Resolution for Software Attribute Requirements," PhD Dissertation, Department of Computer Science, University of Southern California, December 1998 (pdf)

A critical success factor in requirements engineering involves determining and resolving conflicts among candidate system requirements proposed by multiple stakeholders. Many software projects have failed due to requirements conflicts among the stakeholders.

The WinWin system developed at USC provides an approach for resolving requirements conflicts among the stakeholders. The WinWin system provides a framework for negotiation between the stakeholders to identify and resolve these conflicts. However, such systems do not scale well for large software projects containing many requirements.

Based on an analysis of the options for addressing this problem, I have focused on semiautomated tools and techniques for identifying and resolving conflicts among software quality attributes. I have developed two prototype support tools, QARCC and S-COST, which expand the capabilities of the WinWin system. QARCC focuses on software architecture strategies for achieving quality attribute objectives. S-COST focuses on tradeoffs among software cost, functionality, and other quality attributes. I have also developed portions of underlying theories and models which serve as the basis for the prototype tools.

Finally, I evaluated the theories, models, and tools with the results of WinWin negotiations, such as the CS577 15-project samples.

Apurva Jain, "Theory of Value-Based Software Engineering," PhD Dissertation, Department of Computer Science, University of Southern California, December 2007 (pdf)

What is the primary thesis of this research?
The activity of developing software systems is not an end, but a means to an end for the people who directly or indirectly depend on them. Taking such a view thus implies that software systems must be engineered such that they help people meet their ends. The primary focus of this study is to show how decisions about software systems can be made such that they are better aligned to realize the values (ends) of its stakeholders (people).

Note that this is consistent with Webster’s definition of “engineering:”
(1) the activities or function of an engineer
(2a) the application of science and mathematics by which the properties of matter and the sources of energy in nature are made useful to people
(2b) the design and manufacture of complex products <software engineering>

The main definition (2a) puts making things useful to people (i.e., satisfying their value propositions at the center of what an engineering discipline should do.

Research on software systems has been roughly split into two aspects – process and product. Product research is mainly focused on the “what is produced” – generally technology focused, and addresses problems mostly within the ambit of computer science. These may include inventing technologies to improve a software system’s quality attributes, such as performance and security, or identifying better approaches to implementing existing or new technologies. For example, Web Services was invented in response to the growing interoperability needs among software systems.

Process research instead addresses “how the product is being produced” – in particular, the managerial and administrative aspects of developing software systems. The focus is on the practices that guide the development of a software system from its conceptualization through its evolution to a possibly much larger system such as the kind that serves an enterprise. Examples of research within the process area may include methods for decision making, life cycle choice, cost and schedule estimation, design and analysis, and risk management.

This study integrates product and process research. It goes beyond both in addressing why the product is being produced, and how well it needs to perform. It makes an inquiry into the current practices related to decision making in thinking about and constructing or acquiring software systems. With a few exceptions, it shows how most current practices fall short in addressing the full set of stakeholder values (disconnected from stakeholder values), and how current practices use an insufficient unit of analysis that does not include the context in which the software has to transition (disconnected from context). Some of these practices are discussed next to show how value-based and context-connected approaches are better. Others will be described in Chapter 2 – Literature review.

In addressing these issues, this study describes a proof of concept theory for developing software systems that uses stakeholder values as its unit of analysis. A significant contribution of this study is the explanation of the theoretical constructs used in creating a new theory, and a process framework that helped in making an initial assessment of its sufficiency and fitness with respect to a set of criteria.

Since this research deals with imprecise and situation‐varying human values, a combination of analytic and qualitative approaches was chosen for assessment of its findings. Future work to strengthen this research should include quantitative assessments and further extensions for each of the key theoretical constructs.

Hasan Kitapci, "Formalizing Informal Stakeholder Inputs Using Gap-Bridging Methods," PhD Dissertation, Department of Computer Science, University of Southern California, December 2007 (pdf)

For many software and systems developers, Requirements Engineering is one of the biggest problems they face. Without an accurate, consistent, and complete requirements specification it is very difficult to develop, change, and maintain software. It is now a generally accepted and frequently stated fact that one of the major causes of software development failures is poor requirements engineering. The CHAOS Surveys by the Standish Group report that 15% of the projects fail outright, and 51% are late, run over budget or provide reduced functionality. On average only 54% of the initial project requirements are delivered to the client. Inadequate specification of system requirements is considered to be one of the main causes for project failure.

Good--sufficiently complete, consistent, well-understood, and testable--requirements are a prerequisite for successful projects. Without understanding what the stakeholders--people who have some demands on the product, and hence must be consulted in the requirements gathering activity--really wants and needs, and writing these requirements in a concise, understandable and testable manner, projects not only won't lead to what the stakeholders wanted, but also they probably will fail.

Often, the stakeholders who can contribute the most value and realism to system requirements are the ones with the least time available to participate in the process. Negotiations are crucial in the early stages of the lifecycle to develop a set of mutually satisfactory requirements. We have found the EasyWinWin requirement negotiation method and tools to be an effective way to involve such busy stakeholders. It avoids time-consuming group wordsmithing, but results in relatively informal win-win agreements (e.g., "24/7" or "anytime, anywhere" vs. more precise availability and accessibility requirements). The transition to semi-formal and formal representations is still a challenging problem. While the EasyWinWin improves the identification and negotiation of stakeholders' requirements, the transition to more formal system requirements and further project artifacts is based on intuition and experience.

Requirements problems are present to a certain degree in almost all projects despite the fact that there are available techniques that could address them. The approach of our research is to integrate gap-bridging methods--template, NLP, keyword analysis, formal experts, and inspection--with requirements engineering to leverage the benefits of these methods in order to create a better requirements engineering process. The basic idea is that orchestrating gap-bridging methods whenever and wherever possible through the requirements engineering process could eliminate requirements related problems. This framework helps consistently elicit the customer needs, save time and cost, and trace the requirements through the entire development life cycle.

We enhance the existing tool EasyWinWin, to improve the results of requirements negotiation process and further to define the requirements specifications with the characteristics of good requirements taken into account. Beginning with the identification of requirements leading to the analysis and negotiation of requirements to the actual generation of requirements specification, the framework combines all requirements engineering activities in order to eliminate problems and provide a good foundation for the later artifacts of software development.

Supannika Koolmanojwong, "The Incremental Commitment Spiral Model Process Patterns for Rapid-Fielding Projects," PhD Dissertation, Department of Computer Science, University of Southern California, December 2010 (pdf)

To provide better services to customers and not to be left behind in a competitive business environment, a wide variety of ready-to-use software and technologies are available for one to grab and go and build up software systems at a very fast pace. Rapid fielding plays a major role in developing software systems to provide a quick response to the organization. This research investigates the appropriateness of current software development processes and develops new software development process guidelines, focusing on four process patterns: Use single NDI, NDI-intensive, Services-intensive, and Architected Agile. Currently, there is no single software development process model that is applicable to all four process patterns, but the Incremental Commitment Spiral Model (ICSM) can help a new project converge on a process that fits the project process scenario. The output of this research has been implemented as an Electronic Process Guide for USC Software Engineering students to use as a guideline to develop real-client Software Engineering course projects. An empirical study has been conducted to assess the suitability of the newly developed process as compared to results data from previous course projects. Subject to sources of variability in the nature of the projects, the assessment confirmed that the process selection guidelines led project teams to choose the most appropriate process pattern, and that the performance of the project teams choosing inappropriate processes produced less satisfactory results.

Alexander Lam, "Architecture and Application of an Autonomous Robotic Software Engineering Technology Tested (SETT)," PhD Dissertation, Department of Computer Science, University of Southern California, May 2008 (pdf)

This research provides a new way to develop and apply a new form of software: software engineering technology testbeds designed to evaluate alternative software engineering technologies, and to accelerate their maturation and transition into project use. Software engineering technology testbeds include not only the specifications and code, but also the package of instrumentation, scenario drivers, seeded defects, experimentation guidelines, and comparative effort and defect data needed to facilitate technology evaluation experiments.

The requirements and architecture to build a software engineering technology testbed has been developed and applied to evaluate a wide range of technologies. The technologies evaluated came from the fields of architecture, testing, state-model checking, and operational envelopes.

The testbed evaluation showed (1) that certain technologies were complementary and cost-effective to apply to mission-critical systems; (2) that the testbed was cost-effective to use by researchers; (3) that collaboration in testbed use by researchers and the practitioners resulted in actions to accelerate technology maturity and transition into project use; and (4) that the software engineering technology testbed's requirements and architecture were suitable for evaluating technologies and to accelerate their maturation and transition into project use.

Jo Ann Lane, "Imapacts of System of System Management Strategies on System of System Capability Engineering Effort," PhD Dissertation, Department of Industrial and Systems Engineering, University of Southern California, May 2009 (pdf)

Today’s need for more complex, more capable systems in a short timeframe is leading more organizations towards the integration of new and existing systems with commercial-off-the-shelf (COTS) products into network-centric, knowledge-based systems of systems (SoS). With this approach, system development processes to define the new architecture, identify sources to either supply or develop the required components, and eventually integrate and test these high level components are evolving and are being referred to as SoS Engineering (SoSE). In recent years, the systems engineering (SE) community has struggled to decide if SoSE is really different from traditional SE and, if it is different, how does it differ. Recent research and case studies [DoD, 2008] have confirmed that there are indeed key differences and that traditional SE processes are not sufficient for SoSE. However, as with any engineering discipline, how and how much SoSE differs depends on several factors. This research further investigated SoSE through the study of several large-scale SoSE programs and several SE programs that were considered part of one or more SoSs to identify key SoSE strategies and how these strategies differed based on SoS characteristics and constituent-systems. The results of these investigations were then captured in a system dynamics model that allows one to explore SoSE options with respect to engineering effort and return on SoSE investment. Two SoS capability development strategies (with and without an SoSE team to guide capability development) were compared and used to assess the value-added of the SoSE team with respect to total SE effort expended to engineer an SoS capability. It is clear from both the Office of the Secretary of Defense (OSD) pilot studies [DoD, 2008] and the system dynamics model analysis conducted as part of this research that there exist conditions under which investments in SoSE have positive and negative returns on investment. This dissertation provides the first quantitative determination of these conditions, and points out directions for future research that would strengthen the results.

DeWitt T. Latimer IV, "Effectiveness of Engineering Practices for the Acquisition and Employment of Robotic Systems," PhD Dissertation, Department of Computer Science, University of Southern California, May 2008 (pdf)

This thesis is concerned with the identification of engineering practices that most influence the ability of an organization to successfully acquire and employ a robot. Of specific interest are the matches or mismatches between our technical efforts and achieving robotic systems that are suitable for the intended purpose. From a survey of engineers (n=18) who have advised or performed the acquisition of robots, candidate relations between engineering practices and system success metrics are proposed. Those relationships are then evaluated against a 5 case studies and one mini-study to examine more closely how the practices are implemented as specific engineering methods in context. From those observations, a series project feasibility rationales are proposed to aid engineers and managers evaluate the feasibility of their robotic system acquisition.

Joo H. Lee, "Promoting Student Feedback in the Classroom," PhD Dissertation, Department of Computer Science, University of Southern California, May 2003 (pdf)

One form of distance education is when remote students are viewing a lecture either by television or webcast. A major challenge is to assist these remote students so they feel a part of the class. Several technologies have been proposed for mitigating the physical separation including: video teleconferencing, chat rooms, and remote application sharing.

This work focuses on computer mediated interactive distance education systems. The framework is TIDE (Technology for Interactive Distance Education) system that facilitates feedback from the students to the instructor. In contrast to most other research on distance education that focuses on enhancing the flow of information from the instructors to the students and/or from the educational content to the students, the TIDE system is intended to help the flow of information from the students to the instructor. This work provides a new learning paradigm-- the computer-mediated cooperative information transfer learning paradigm where the learning takes place as the result of the process whereby technology mediates between instructor and students to aid effective communication.

In this thesis I describe the implementation of the TIDE system including system architecture, system protocol, graphical user interface design, and three feedback mechanisms incorporated within. Then I describe a series of experiments that made use of the TIDE system in the studio classrooms available at USC's Distance Education Network during 2001/2002 with sixteen students and four instructors. The results from the experiments and analysis of the survey are used to evaluate the effectiveness of the TIDE system and three feedback mechanisms.

June Sup Lee, "Designing and Building Protocols for Web Applications," PhD Dissertation, Department of Computer Science, University of Southern California, August 1999 (pdf)

This thesis is about three topics, (i) software architectures, (ii) building application-level protocols, and (iii) transforming legacy software onto the web. In part (i) I describe an architectural style in which the computation is divided between the client and server. I call this style the 3-tier Web architecture. I provide a formal specification for the 3-tier Web architecture. One important aspect of this formal specification is how it assists in identifying various elements to consider when implementing an instance of this architecture. I present two instances of the 3-tier Web architecture--Web-COCOMO and Web-WinWin, which I have designed at the Center for Software Engineering at USC. While designing and building these example Web applications, I discovered that building the application-level protocol is one of the key issues in building Web applications. In part (ii) I focus down within the 3-tier Web architecture on the question of creating application-level protocols. I design and build a tool which helps a programmer generate such a protocol using RPC elements. The tool is novel in several ways, including the fact that it generates a complete and consistent protocol which is also efficient. In part (iii) of this thesis I apply the 3-tier Web architecture and my protocol tool to the problem of transforming legacy software systems to the Web. I identify the key elements in the process and show how the architecture and my tool can be successfully used.

Keun Lee, "Development and Evaluation of Value-Based Review (VBR) Methods," PhD Dissertation, Department of Computer Science, University of Southern California, May 2006 (pdf)

Reviewing is a key activity that can find defects at an early stage of system and software development. Since it is often cheaper to fix defects at an early stage, reviewing is a good technique for improving both product quality and project cost effectiveness. Currently, there are many review techniques proposed and many experiments have been performed to compare the review techniques. However, to data there have been no review techniques or experiments that have focused explicitly on the relative business value or mission value of the artifacts being reviewed.

In this dissertation, I provide Value-based review techniques adding cost effectiveness and value of each issue into review processes, and report on an experiment on Value-based review. I developed a set of VBR checklists with issues ranked by success-criticality, and a set of VBR processes prioritized by issue criticality and stakeholder-negotiated product capability priorities. The experiment involved 28 independent verification and validation (IV&V) subjects (full-time working professionals taking a distance learning course) reviewing specifications produced by 18 real-client, full-time student e-services projects. The IV&V subjects were randomly assigned to use either the VBR approach or the previous value-neutral checklist-based reading (CBR) approach that had been used in the course. The difference between groups was not statistically significant for number of issues reported, but was statistically significant for number of issues per review hour, total issue impact in terms of criticality and priority, and cost effectiveness in terms of total issue impact per review hour. For the latter, the VBRs were roughly twice as cost-effective as the CBRs.

The dissertation also covers threats to validity and limitations of the experiment. Threats to validity were present but appear to have been adequately addresses. The main limitation of the experiment was its coverage of reviews by individuals as compared to groups. For reviewers by groups, it is likely that combinations of VBR and risk-driven forms of perspective-based review (PBR) approaches would be most cost-effective. This and other topics are attractive candidates for further research.

Ming June Lee, "Foundations of the WinWin Requirements Negotiation System," PhD Dissertation, Department of Computer Science, University of Southern California, August 1996 (pdf)

Requirements Engineering (RE) constitutes an important part of Software Engineering. The USC WinWin requirements negotiation system addresses critical issues in requirements engineering including (1) multi-stakeholder considerations, (2) change management, and (3) groupware support. The WinWin approach to date has primarily involved exploratory prototyping. The system is now converging on a relatively stable set of artifacts and relationships. This makes it feasible and important to formalize these artifacts and relationships to provide a solid scientific framework for the WinWin system. This is the focused problem addressed by the research presented in this paper.

Qi Li, "VALUE-BASED, DEPENDENCY-AWARE INSPECTION AND TEST PRIORITIZATION", PhD Dissertation, Department of Computer Science, University of Southern California, December 2012 (pdf)

As two of the most popular defect removal activities, Inspection and Testing are of the most labor-intensive activities in software development life cycle and consumes between 30% and 50% of total development costs according to many studies. However, most of the current defect removal strategies treat all instances of software artifacts as equally important in a value-neutral way; this becomes more risky for high-value software under limited funding and competitive pressures.
In order to save software inspection and testing effort to further improve affordability and timeliness while achieving acceptable software quality, this research introduces a value-based, dependency-aware inspection and test prioritization strategy for improving the lifecycle cost-effectiveness of software defect removal options. This allows various defect removal types, activities, and artifacts to be ranked by how well they reduce risk exposure. Combining this with their relative costs enables them to be prioritized in terms of Return On Investment (ROI) or Risk Reduction Leverage (RRL). Furthermore, this strategy enables organizations to deal with two types of common dependencies among items to be prioritized. This strategy will help project managers determine “how much software inspection/testing is enough?” under time and budget constraints. Besides, a new metric Average Percentage of Business Importance Earned (APBIE) is proposed to measure how quickly testing can reduce the quality uncertainty and earn the relative business importance of the System Under Test (SUT).
This Value-Based, Dependency-Aware Inspection and Testing strategy has been empirically studied and successfully applied on a series of case studies within different prioritization granularity levels: (1). Prioritizing artifacts to be reviewed in 21 graduate- level, real-client software engineering course projects; (2). Prioritizing testing scenarios to be applied in an industrial project at the acceptance testing phase in Galorath, Inc.; (3). Prioritizing software features to be functionally tested in an industrial project in the China-NFS company; (4). Prioritizing test cases to be executed in 18 course projects. All the comparative statistics analysis from the four case studies show positive results from applying the Value-Based, Dependency-Aware strategy.

Ray Madachy, "A Software Project Dynamics Model for Process Cost, Schedule and Risk Assessment," PhD Dissertation, Department of Industrial and Systems Engineering, University of Southern California, December 1994 (pdf)

A dynamic model of an inspection-based software lifecycle process has been developed to support quantitative evaluation of the process. In conjunction with a knowledge-based method that has been developed for cost estimation and project risk assessment, these modeling techniques can support project planning and management, and aid in process improvement.

The model serves to examine the effects of inspection practices on cost, schedule and quality throughout the lifecycle. It uses system dynamics to model the interrelated flows of tasks, errors and personnel throughout different development phases and is calibrated to industrial data. It extends previous software project dynamics research by examining an inspection-based process with an original model, integrating it with the knowledge-based method for risk assessment and cost estimation, and using an alternative modeling platform.

While specific enough to investigate inspection practices, it is sufficiently general to incorporate changes for other phenomena. It demonstrates the effects of performing inspections or not, the effectiveness of varied inspection policies, and the effects of other managerial policies such as manpower allocation. The dynamic effects are tracked throughout the time history of a project to show resource usage, task completions and defect trends per phase. Cumulative metrics and development tradeoffs for decision making are also presented.

The knowledge-based method has been implemented on multiple platforms. As an extension to COCOMO, it aids in project planning by identifying, categorizing, quantifying and prioritizing project risks. It also detects cost estimate input anomalies and provides risk control advice in addition to conventional cost and schedule calculation. It extends previous work by focusing on risk assessment, incorporating substantially more rules, going beyond standard COCOMO, performing quantitative validation, and providing a user-friendly interface. The method is being used and enhanced in industrial environments as part of an integrated capability to assist in system acquisition, project planning and risk management.

Several types of validation tests are performed against industrial data, existing theory and other prediction models, and practitioners are used to evaluate the model. The results indicate a valid model that can be used for process evaluation and project planning, and serve as a framework for incorporating other dynamic process factors.

Sam Malek, "A User-Centric Approach for Improving a Distributed Software System's Deployment Architecture," PhD Dissertation, Department of Computer Science, University of Southern California, August 2007 (pdf)

The quality of service (QoS) provided by a distributed software system depends on many system parameters, such as network bandwidth, reliability of links, frequencies of software component interactions, etc. A distributed system's allocation of software components to hardware nodes (i.e., deployment architecture) can have a significant impact on its QoS. At the same time, often times there are many deployment architectures that provide the same functionality in large-scale software systems. Furthermore, the impact of deployment architecture on the QoS dimensions (e.g., availability, latency) of the services (functionalities) provisioned by the system could vary. In fact, some QoS dimensions may be conflicting, such that a deployment architecture that improves one QoS dimension, degrades another dimension.

In this dissertation, we motivate, present, and evaluate a framework aimed at finding the most appropriate deployment architecture with respect to multiple, and possibly conflicting, QoS dimensions. The framework provides a formal approach to modeling the problem, and a set of generic algorithms that can be tailored and instantiated for improving a system’s deployment architecture. The framework relies on system users’ (desired) degree of satisfaction with QoS improvements to resolve trade-offs between conflicting QoS dimensions. The framework is realized on top of an integrated tool suite, which further aids reusability and cross-evaluation of the solutions.

This dissertation is evaluated empirically on a large number of simulated representative scenarios. Various aspects of the framework have also been evaluated on two real distributed systems. The dissertation concludes with several open research questions that will frame our future work.

Ali Afzal Malik, "Quantitative and Qualitative Analyses of Requirements Elaboration for Early Software Size Estimation," PhD Dissertation, Department of Computer Science, University of Southern California, August 2010 (pdf)

Software size is one of the most influential inputs of a software cost estimation model. Improving the accuracy of size estimates is, therefore, instrumental in improving the accuracy of cost estimates. Moreover, software size and cost estimates have the highest utility at the time of inception. This is the time when most important decisions e.g. budget allocation, personnel allocation, etc. are taken. The dilemma, however, is that only high-level requirements for a project are available at this stage. Leveraging this high-level information to produce an accurate estimate of software size is an extremely challenging task.

Requirements for a software project are expressed at multiple levels of detail during its life cycle. At inception, requirements are expressed as high-level goals. With the passage of time, goals are refined into shall statements and shall statements into use cases. This process of progressive refinement of a project’s requirements, referred to as requirements elaboration, continues till source code is obtained. This research analyzes the quantitative and qualitative aspects of requirements elaboration to address the challenge of early size estimation.

A series of four empirical studies is conducted to obtain a better understanding of requirements elaboration. The first one lays the foundation for the quantitative measurement of this abstract process by defining the appropriate metrics. It, however, focuses on only the first stage of elaboration. The second study builds on the foundation laid by the first. It examines the entire process of requirements elaboration looking at each stage of elaboration individually. A general process for collecting and processing multi-level requirements to obtain elaboration data useful for early size estimation is described. Application of this general process to estimate the size of a new project is also illustrated.

The third and fourth empirical studies are designed to tease out the factors determining the variation in each stage of elaboration. The third study focuses on analyzing the efficacy of COCOMO II cost drivers in predicting these variations. The fourth study performs a comparative analysis of the elaboration data from two different sources i.e. small real-client e-services projects and multiple versions of an industrial process management tool.

Chris Mattmann, "Software Connectors for Highly Distributed and Voluminous Data-Intensive Systems," PhD Dissertation, Department of Computer Science, University of Southern California, December 2007 (pdf)

Data-intensive systems and applications transfer large volumes of data and metadata to highly distributed users separated by geographic distance and organizational boundaries. A dominating factor in these large volume data transfers is the selection of the appropriate software connector that satisfies user constraints on the required data distribution scenarios. This task is typically accomplished by consulting “gurus” who rely on their intuitions, at best backd by anecdotal evidence.

In this dissertation we motivate, present and evaluate a software architecture-based systematic framework for selecting software connectors based on eight key dimensions of data distribution that we use to represent the data distribution scenarios. Our framework, dubbed DISCO, accurately, efficiently, and reliably captures a guru’s domain knowledge and allows a user to automatically leverage that knowledge to drive connector selection. In addition, DISCO affords a user the ability to validate a guru’s domain knowledge against actual performance measurements of the connectors in the areas of efficiency, scalability, dependability and consistency.

We provide a set of models, algorithms, techniques and tools to represent data distribution scenarios, classify and select connectors and explore the trade off space when architecting large scale data distribution systems. To date, 13 real-world connectors across four connector families have been explored using our framework. We validate our framework empirically and qualitatively, employing 30 data distribution scenarios gleaned from three real-world projects spanning planetary science, cancer research and earth science at NASA’s Jet Propulsion Laboratory. We use a number of measures of accuracy including precision, recall and error rate. We also provide theoretical performance analysis of our connector selection algorithms. We report empirical performance measurements of the 13 connectors and use the data to revise and validate our precision measurements. In addition to our validation, we have integrated DISCO as a “plug-in” piece to an independently-developed COTS interoperability assessment framework, providing more feedback for a second use-case of the tool. We conclude the dissertation with a set of open research questions that will frame our future work.

Nikunj Mehta, "Composing Style-Based Software Architectures from Architectural Primitives," PhD Dissertation, Department of Computer Science, University of Southern California, December 2004 (pdf)

Software architectures provide high-level abstractions to deal with the complexity and richness of large-scale software systems. Often, similar architectural organization is found in different software systems. Architectural styles are an approach to exploit such similarities between different architectures. A style is useful in the design of architectures due to guarantees of desirable stylistic properties across architectures based on that style. Styles are also believed to bring economies of scale in the design and implementation of architectures. Although there are many systematic techniques for describing and selecting styles, there is considerably less underpinning for the systematic design and construction of style-based architectures.

This dissertation motivates, presents, and validates a methodology for the composition of style-based software architectures from architectural primitives. This methodology, embodied as the Alfa framework, is a collection of related techniques that support the expression of stylistic constraints as well as architectural organization through the use of architectural primitives. Not only does the Alfa framework ensure the conformance of stylistic constraints in architectures using those styles, but it is also amenable to scalable implementation of architectural designs. Moreover, our methodology is comprehensive in its scope: it supports the data, structural, topological, interaction, and behavioral concerns in styles and architectures. The unique aspects of Alfa are: explicit recognition of architectural primitives that underlie a wide range of styles and architectures; type checking of data in styles and architectures, to ensure validity of architectural compositions; effective analysis of architectures for conformance to style(s) used in them, to preserve stylistic properties; and scalable implementation of architectural primitives and their composition, to support architectural prototyping.

The key proposition of the dissertation is demonstrated through a case study that applies Alfa to distributed systems. The dissertation is validated analytically by deducing the algorithmic complexity of Alfa’s conformance analysis; and quantitatively, by measuring the scalability of Alfa-based architectural implementations in terms of memory usage and processing time. The dissertation is concluded by inferring the consequences of Alfa on the design, analysis, and implementation of software systems.

Marija Mikic-Rakic, "Software Architectural Support for Disconnected Operation in Distributed Environments," PhD Dissertation, Department of Computer Science, University of Southern California, December 2004 (pdf)

In distributed and mobile environments, the connections among the hosts on which a software system is running are often unstable. As a result of connectivity losses, the overall availability of the system decreases. The distribution of software components onto hardware nodes (i.e., deployment architecture) may be ill-suited for the given target hardware environment and may need to be altered to improve the software system's availability. The critical difficulty in achieving this task lies in the fact that determining a software system's deployment that will maximize its availability is an exponentially complex problem.

In this dissertation, we motivate, present, and evaluate an automated, flexible, software architecture-based solution for disconnected operation that increases the availability of the system during disconnection. We provide a set of models, algorithms, techniques, and tools for improving a distributed, mobile system's availability via redeployment, by enabling the system to (1) monitor its operation; (2) estimate its deployment architecture; and (3) effect that architecture automatically. While the focus of this dissertation research is on improving system availability, our methodology is extensible to support other non-functional properties that may be relevant in a given distributed system.

This dissertation is evaluated empirically, by conducting a series of benchmark tests and applying the methodology to example applications. The dissertation concludes with a set of open research questions that will frame our future work.

Ramin Moazeni, "Incremental Development Productivity Decline", PhD Dissertation, Department of Computer Science, University of Southern California, August 2014 (pdf)

Software production is on the critical path of increasingly many program abilities to deliver effective operational capabilities. This is due to the number, complexity, independence, interdependence, and software-intensiveness of their success-critical components and interfaces. The estimation parameters and knowledge bases of current software estimation tools are generally good for stable, standalone, single increment development. However, they do not fully account for the degrees of program and software dynamism, incrementality, coordination, complexity, and integration. These phenomena tend to decrease software productivity relative to the cost model estimates made for the individual software components and for the overall systems, but it is difficult to estimate by how much.

Incremental software development generally involves either adding, modifying, or deleting parts of the code in the previous increments. This means that if a useful system is to be built, the maintenance that will have to go into previous increments will take away productivity from the later ones.

This research tests hypotheses about a phenomenon called Incremental Development Productivity Decline (IDPD) that may be more or less present in incremental software projects of various categories.

Incremental models are now being used by many organizations in order to reduce development risks while trying to deliver releases of the product on time. It has become the most common method of software development with characteristics that influence the productivity of projects.

Different ways of measuring productivity are presented and evaluated in order to come to a definition or set of definitions that is suitable to these categories of projects. Data from several sources has been collected and analyzed, and hypotheses tested about the degree of IDPD and its variation by increment and category. The results indicated the existence of an IDPD phenomenon, that its magnitude varies by application category, but that it tended to vary from increment to increment.

Vu Nguyen, "Improved Size and Effort Estimation Models for Software Maintenance," PhD Dissertation, Department of Computer Science, University of Southern California, December 2010 (pdf)

Accurately estimating the cost of software projects is one of the most desired capabilities in software development organizations. Accurate cost estimates not only help the customer make successful investments but also assist the software project manager in coming up with appropriate plans for the project and making reasonable decisions during the project execution. Although there have been reports that software maintenance accounts for the majority of the software total cost, the software estimation research has focused considerably on new development and much less on maintenance.

In this dissertation, an extension to the well-known model for software estimation, COCOMO II, is introduced for better determining the size of maintained software and improving the effort estimation accuracy of software maintenance. While COCOMO II emphasizes the cost estimation of software development, the extension captures various characteristics of software maintenance through a number of enhancements to the COCOMO II size and effort estimation models to support the cost estimation of software maintenance.

Expert input and an industry data set of eighty completed software maintenance projects from three software organizations were used to build the model. A number of models were derived through various calibration approaches, and these models were then evaluated using the industry data set. The full model, which was derived through the Bayesian analysis, yields effort estimates within 30% of the actuals 51% of the time, outperforming the original COCOMO II model when it was used to estimate these projects by 34%. Further performance improvement was obtained when calibrating the full model to each individual program, generating effort estimates within 30% of the actuals 80% of the time.

Allen Peter Nikora, "Software System Defect Content Prediction from Development Process and Product Characteristics," PhD Dissertation, Department of Computer Science, University of Southern California, May 1998 (pdf)

Society has become increasingly dependent on software controlled systems (e.g., banking systems, nuclear power station control systems, and air traffic control systems). These systems have been growing in complexity – the number of lines of source code in the Space Shuttle, for instance, is estimated to be 10 million, and the number of lines of source code that will fly aboard Space Station Alpha has been estimated to be up to 100 million. As we become more dependent on software systems, and as they grow more complex, it becomes necessary to develop new methods to ensure that the systems perform reliably.

One important aspect of ensuring reliability is being able to measure and predict the system’s reliability accurately. The techniques currently being applied in the software industry are largely confined to the application of software reliability models during test. These are statistical models that take as their input failure history data (i.e., time since last failure, or number of failures discovered in an interval), and produce estimates of system reliability and failure intensity. To better control a system’s quality, we need the ability to measure the system’s reliability prior to test, when it is possible to influence the development process and change the system’s structure.

We develop a model for predicting the rate at which defects are inserted into a system, using measured changes in a system’s structure and development process as predictors, and show how to:
Estimate the number of residual defects in any module at any time.
Determine whether additional resources should be allocated to finding and repairing defects in a module.

In order to calibrate the model and estimate the number of remaining defects in a system, it is necessary to accurately identify and count the number of defects that have been introduced into a system. We develop a set of rules that can be used to count the number of defects that are present in the system, based on observed changes that have been made to the system as a result of repair actions.

Monvorath Phongpaibul, "Experimental and Analytical Comparison Between Pair Development and Software Development With Fagan's Inspection," PhD Dissertation, Department of Computer Science, University of Southern California, December 2007 (pdf)

Peer review is one of the essential activities in software quality assurance since peer reviews can detect and remove defects in the early stages of the software development life cycle. Removing defects early reduces the cost of defect rework later. Selecting a peer review methodology (e.g., inspection, walkthrough, checklist-based, defect-based, function-based, perspective-based, usage-based, value-based) to execute in a software project is difficult. The developers have to understand the commonalities and differences of each methodology. They need to know the relative strengths and weaknesses of these practices. However, very few studies have compared the commonalities and differences of each peer review methodology and none of the studies have shown an empirical comparison between pair programming and software inspection.

Software inspection and pair programming are effective verification techniques. Software inspection is one of the best practices in traditional software development while pair programming is one of the best practices in agile development. Numerous studies have shown the success of software inspection in large-scale software development over the past three decades. Although Pair Programming (PP) is a newer approach and less structured, it has had a strong impact on the success of agile software development projects over the past five years.

This dissertation aims to identify the critical factors that impact the cost-effectiveness of either pair programming/development or inspection and provide the decision framework to help the developers select the most effective technique under given conditions. To compare both techniques, four classroom experiments and one industry experiment were conducted in Thailand and US. The development effort and effect of quality were investigated with some additional calendar time comparisons.

Roshanak Roshandel, "Calculating Architectural Reliability Via Modeling and Analysis," PhD Dissertation, Department of Computer Science, University of Southern California, August 2005 (pdf)

Modeling and estimating software reliability during testing is useful in quantifying the quality of the software systems. However, such measurements applied late in the development process leave too little to be done to improve the quality and dependability of the software system in a cost-effective way. Reliability, an important dependability attribute, is defined as the probability that the system performs its intended functionality under specified design limits. We argue that reliability models must be built to predict the system reliability throughout the development process, and specifically when exact context and execution profile of the system is unknown, or when the implementation artifacts are unavailable. In the context of software architectures, various techniques for modeling software systems and specifying their functionality have been developed. These techniques enable extensive analysis of the specification, but typically lack quantification. Additionally, their relation to dependability attributes of the modeled software system is unknown.

In this dissertation, we present a software architecture-based approach to predicting reliability. The approach is applicable to early stages of development when the implementation artifacts are not yet available, and exact execution profile is unknown. The approach is two fold: first, the reliability of individual components is predicted via a stochastic reliability model built using software architectural artifacts. The uncertainty associated with the execution profile is modeled using Hidden Markov Models, which enable probabilistic modeling with unknown parameters. The overall system reliability is obtained compositionally as a function of the reliability of its constituent components, and their complex interactions. The interactions form a causal network that models how reliability at a specific time in a system's execution is affected by the reliability at previous time steps.

We evaluate our software architecture-based reliability modeling approach to demonstrate that reliability prediction of software systems architectures early during the development life-cycle is both possible and meaningful. The coverage of our architectural analyses, as well as our defect classification is evaluated empirically. The component-level and system-level reliability prediction methodology is evaluated using sensitivity, uncertainty, and complexity, and scalability analyses.

Chiyoung Seo, "Prediction of Energy Consumption Behavior in Component-Based Distributed Systems," PhD Dissertation, Department of Computer Science, University of Southern California, August 2008 (pdf)

Component-based distributed systems are increasingly deployed onto heterogeneous embedded platforms with limited battery resources. In this domain, it is crucial to provide an effective approach that enables an engineer to build energy-efficient component-based distributed systems throughout the entire development cycle spanning system design-time, construction-time, and runtime. To address this problem, this dissertation first presents a framework that (1) facilitates early estimation of the energy consumption induced by an architectural style in a component-based distributed system, and (2) consequently enables an engineer to use energy consumption estimates along with other quality attributes in determining the most appropriate style for a given distributed system during design-time.

As the second part of the solution, this dissertation provides a framework that allows the system engineer to estimate the energy consumption of a component-based distributed system implemented in Java at the level of its components, both during system construction-time and during runtime. We also present several practical applications of the framework, which can be effected either at system constructiontime or at runtime for reducing the distributed system’s energy consumption and consequently increasing the lifetime of the system’s key services.

Our overall approach, composed of the above two estimation frameworks, has been evaluated empirically on a large number of distributed application scenarios, and produced very accurate energy consumption estimates. The dissertation concludes with several open research questions that will frame our future work.

Thomas Tan, "Domain-Based Effort Distribution Model for Software Cost Estimation", PhD Dissertation, Department of Computer Science, University of Southern California, August 2012 (pdf)

In software cost estimation, effort allocation is an important and usually challenging task for project management. Due to the Cone of Uncertainty effect on overall effort estimation and lack of representative effort distribution data, project managers often find it difficult to plan for staffing and other team resources. This often leads to risky decisions to assign too few or too many people to complete software lifecycle activities. As a result, projects with inaccurate resource allocation will generally experience serious schedule delay or cost overrun, which has been the outcome of 44% of the projects reported by the Standish Group [Standish, 2009].

Due to lack of data, most effort estimation models, including COCOMO II, use a one-size-fits-all distribution of effort by phase and activity. The availability of a critical mass of data from U.S. Defense Department software projects on effort distribution has enabled me to test several hypotheses that effort distributions vary by project size, personnel capability, and application domains. This dissertation will summarize the analysis approach, describe the techniques and methodologies used, and report the results. The key results were that size and personnel capability were not significant sources of effort distribution variability, but that analysis of the influence of application domain on effort distribution rejected the null hypothesis that the distributions do not vary by domains, at least for the U.S. Defense Department sector. The results were then used to produce an enhanced version of the COCOMO II model and tool for better estimation of the effort distributions for the data-supported domains.

Ricardo Valerdi, "The Constructive Systems Engineering Cost Model (COSYSMO)," PhD Dissertation, Department of Computer Science, University of Southern California, August 2005 (pdf)

As organizations develop more complex systems, increased emphasis is being placed on Systems Engineering (SE) to ensure that cost, schedule, and performance targets are met. Correspondingly, the failure to adequately plan and fund the systems engineering effort appears to have contributed to a number of cost overruns and schedule slips, especially in the development of complex aerospace systems. This has resulted in a recent increased emphasis on revitalizing systems engineering in government and commercial organizations.

This dissertation presents a parametric model that can help people reason about their decisions related to systems engineering. COSYSMO, the Constructive Systems Engineering Cost Model, is an “open” model that contains eighteen parameters: four size drivers and fourteen effort multipliers. It is built on a framework similar to its wellknown predecessor, COCOMO II, and integrates accepted systems engineering standards to define its scope. Funded by industry affiliates, the model focuses on large-scale systems for military applications that employ a disciplined approach to systems engineering. Data was collected from six aerospace companies in the form of expert opinion and historical project data to define and calibrate the model. In reduced form, the model yields a PRED(30) of 50% for programs within a defined productivity range. In principle, the model should apply similarly to commercial systems engineering, but there is a lack of data to test this hypothesis.

The ultimate contributions of this dissertation can be found in at least two major areas: (a) in the theoretical and methodological domain of systems modeling in the quest of a more quantitative cost estimation framework, and (b) in advancing the state of practice in the assessment and tracking of systems engineering in the development of large aerospace systems

Ricardo Valerdi, "The Constructive Systems Engineering Cost Model (COSYSMO)," PhD Dissertation, Department of Computer Science, University of Southern California, August 2005 (pdf)

As organizations develop more complex systems, increased emphasis is being placed on Systems Engineering (SE) to ensure that cost, schedule, and performance targets are met. Correspondingly, the failure to adequately plan and fund the systems engineering effort appears to have contributed to a number of cost overruns and schedule slips, especially in the development of complex aerospace systems. This has resulted in a recent increased emphasis on revitalizing systems engineering in government and commercial organizations.

This dissertation presents a parametric model that can help people reason about their decisions related to systems engineering. COSYSMO, the Constructive Systems Engineering Cost Model, is an “open” model that contains eighteen parameters: four size drivers and fourteen effort multipliers. It is built on a framework similar to its wellknown predecessor, COCOMO II, and integrates accepted systems engineering standards to define its scope. Funded by industry affiliates, the model focuses on large-scale systems for military applications that employ a disciplined approach to systems engineering. Data was collected from six aerospace companies in the form of expert opinion and historical project data to define and calibrate the model. In reduced form, the model yields a PRED(30) of 50% for programs within a defined productivity range. In principle, the model should apply similarly to commercial systems engineering, but there is a lack of data to test this hypothesis.

The ultimate contributions of this dissertation can be found in at least two major areas: (a) in the theoretical and methodological domain of systems modeling in the quest of a more quantitative cost estimation framework, and (b) in advancing the state of practice in the assessment and tracking of systems engineering in the development of large aerospace systems

Dan Wu, "Security Functional Requirements Analysis for Developing Secure Software," PhD Dissertation, Department of Computer Science, University of Southern California, May 2007 (pdf)

Research experience shows that security needs to be considered from the beginning of software development life cycle to avoid expensive rework and reduce potential security vulnerabilities. Hence, defining the right set of security functional requirements (SFRs) and evaluated assurance level (EAL) becomes a critical task for developers when developing secure software. Much effort has been put into creating industry standards to provide a shared common base for stakeholders with concerns on security. One of the industry standards, which is used widely in both industry and government sides in many countries, is Common Criteria (CC). However, one of the drawbacks of Common Criteria is the inefficiency of use. Moreover, with limited project information in the early lifecycle phase, it is hard for developers with less security experience to select the right security requirements from what are defined in CC. Extensions on it and experiences from empirical studies on using it are demanded to achieve a better and more efficient use of CC, which also benefits developers by saving their effort on security functional requirements definition.

A thorough analysis has been done on a dataset consisted by the Security Target (ST) files of 242 security products published on common criteria portal website. A mapping between security objectives and SFRs is presented, which can save much development effort by reduce the range of candidate SFRs when developers know the project's security objectives in the early phases. In the cases when developers only know the product domain of this project, SFR patterns for nine different domains of security products are presented based on the statistic result from the published 242 security products, which can be customized or directly used for particular security application. The analysis result of correlations among SFR classes defined in CC and correlations among security objectives provide a good guidance for developers in designing the architecture of security products. A trend shows that EAL tends to increase when the number of SFRs increases. It is not strongly proved by the current dataset, but shows a research direction for further discussion and explorations in the future.

To validate the correctness of the mapping scheme between security objectives and SFRs, each of the ST files is reviewed to find out the consistency and difference between the presented mapping scheme with the actual selected SFRs in 242 security products with certain security objectives. A method is presented to evaluate the effectiveness of these security patterns, which can be used as a factor for developers when to consider applying the patterns for actual use.

Ye Yang, "Composable Risk-Driven Processes for Developing Software Systems from Commercial-Off-The-Shelf (COTS) Products," PhD Dissertation, Department of Computer Science, University of Southern California, December 2006 (pdf)

Research experience has suggested that software processes should be thought of as a kind of software, which can be developed into composable component pieces that can be executed to perform different software lifecycle activities. General experience has indicated that the activities conducted while developing COTS-based applications (CBA) differ greatly from those conducted in traditional custom development. The primary research questions addressed in this dissertation are (1) Can these activity differences be characterized and statistically analyzed? (2) If so, can the primary CBA activity classes be organized into a decision framework for projects developing CBA’s? The resulting research provides a value-based composable set of processes for CBAs that includes an associated Process Decision Framework (PDF), a set of Composable Process Elements (CPEs), and a COCOTS Risk Analyzer.

A composable process implies the ability to construct a specific process from a higher level and broader process framework and a set of reusable process elements. The PDF is a recursive, re-entrant configuration structure, and establishes the relationships among a mix of the CPEs and other process fragments which are extended from the risk-driven WinWin Spiral model. The CPEs includes Assessment, Tailoring, and Glue code development/integration, which are the three primary sources of effort due to CBA development considerations, indicated by empirical analysis on both large industry and small campus e-services CBA projects. Each CPE is a defined, repeatable workflow. While the framework provides a composition basis to support developers for navigating through the option space in developing CBAs, the three process elements establish the basic constituents for composing COTS processes based on common process patterns identified in empirical studies. A technique named COCOTS Risk Analyzer, is also developed and implemented to aid the optimization of process decisions via risk based prioritization strategy. All together, the proposed solution supports flexible composition of process elements with respect to evolving stakeholders’ value propositions, COTS market, and risk considerations.

To validate the value-based set of processes, experiments have been designed and performed on student projects at USC graduate level software engineering class in Fall 2004 and Fall 2005 semesters. The evaluation results show that applying the value-based processes significantly improves the team performance.

Copyright 2015 The University of Southern California

The written material, text, graphics, and software available on this page and all related pages may be copied, used, and distributed freely as long as the University of Southern California as the source of the material, text, graphics or software is always clearly indicated and such acknowledgement always accompanies any reuse or redistribution of the material, text, graphics or software; also permission to use the material, text, graphics or software on these pages does not include the right to repackage the material, text, graphics or software in any form or manner and then claim exclusive proprietary ownership of it as part of a commercial offering of services or as part of a commercially offered product.