|
PhD Dissertations
Ahmed Abd-Allah, "Composing Heterogeneous Software Architectures," PhD Dissertation, Department of Computer Science, University of Southern California, August 1996 (pdf) A persistent problem in software engineering is how to put complex software systems together out of smaller subsystems, the problem of software composition. The emergence of software architectures and architectural styles has introduced a higher level of abstraction at which we can create and compose software systems. We examine the problem of providing formal semantics to the composition of different architectural styles within software systems, i.e. the problem of composing heterogeneous architectures. We describe a model of pure styles, and a model of their composition.
Our model of pure styles is highlighted by a uniform representation for describing many different styles. An architectural style space of major conceptual features is introduced which allows new styles to be rapidly incorporated into the model, including commercial-off-the-shelf packages which embody a specific style(s). We show a disciplined approach to the process of architectural composition, and show how architecture mismatches can be generated during composition. Finally, we describe a prototype tool which is built on top of the models.
Chris Abts , "Extending the COCOMO II Software Cost Model to Estimate Effort and Schedule for Software Systems Using Commercial-off-the-Shelf (COTS) Software Components: The COCOTS Model," PhD Dissertation, Department of Industrial and Systems Engineering, University of Southern California, May 2004 (pdf)
This dissertation discusses changes in software development practice that have occurred since the 1970s. In particular, the focus is on the rise of the "off-the-shelf"approach whereby large software systems are constructed out of a mix of original code and adapted code as well as pre-existing "black box" components for which the developer has no access to the source code. While certainly used prior to 1980, this technique of using black box elements has taken on greatly increased importance in recent years as one way to manage development risks as the overall complexity of software has grown.
Software estimation models, however, have generally not kept apace with off-the-shelf techniques. This includes the widely-used COCOMO software estimation model. While recently updated as the COCOMO II estimation model to reflect other changes in software development since its original publication in 1981, it still is unable to accommodate the use of black box off-the-shelf components in its estimation framework.
The focus of this dissertation then is the investigation of a potential extension of COCOMO II that is intended to provide an estimation capability for any off-the-shelf elements being designed into a software system. It begins with an exploration of how development activities differ when using off-the-shelf components as opposed to developing system components as original code. This serves as a basis for the formulation of an off the-shelf estimation model called COCOTS that is carried through several iterations, each intended to improve upon its predecessor. The dissertation concludes with a comparison of the fidelity of COCOTS with COCOMO II, limitations of available project data that impacted the calibration of COCOTS, and suggestions for further investigation that might improve the fidelity of COCOTS itself and also expand and deepen the general understanding of the benefits and risks of using off-the-shelf components in software system development.
Mohammed Al-Said, "Detecting Model Clashes During Software Systems Development," PhD Dissertation, Department of Computer Science, University of Southern California, December 2003 (pdf)
Software models are widely used during software systems development. Software developers use the models to better understand the system users' requirements, develop candidate solutions, and validate their decisions. Currently, the set of existing software models is large. Model clashes occur when models with inconsistent assumptions are used at the same time during the development of a software dependent system. Model clashes occur frequently, impede product development, and may cause project failure if not detected and avoided in a timely manner. But outsides of product model clashes (architecture, structure, traceability gaps), relatively little has been done to address them.
This thesis lays out the background of the model clashes problem which includes formally defining a model-clash, formally demonstrating that model clashes exist between software models, and listing the causes of model clashes. It describes a new method for identifying the models' assumptions and the clashes among these assumptions. This method was validated by applying it against a mixture of commonly used software models and the result was used to assess the ability of MBASE (Model Based system Architecting and Software Engineering) to identify and avoid model clashes. A semi-automated software tool was developed based on this method.
To demonstrate the value of detecting and avoiding model clashes, this thesis establishes the relationship between the consequences of model clashes and the risk in software projects. Using data from 35 completed software projects, it shows that each clash has an impact, which can be defined as the product of the clash's occurrence probability and severity. This thesis shows that the clash impact is directly related to the project risk exposure.
Finally, this thesis describes an experiment involving 18 software projects, where a process was applied to detect, analyze, resolve, and avoid model clashes. The results demonstrate the distribution of the different clash types and each type contribution to the project risk. The results also validate the hypothesis: detecting and avoiding model clashes during software systems development improves the software projects' outcome.
Jongmoon Baik, "The Effects of CASE Tools on Software Development Effort," PhD Dissertation, Department of Computer Science, University of Southern California, December 2000 (pdf)
CASE (Computer Aided Software Engineering) tools have played a critical role in improving software productivity and quality by assisting tasks in software development processes since the 1970’s. Several parametric software cost models adopt “use of software tools” as one of the environmental factors that affect software development productivity. However, most software development teams use CASE tools that are assembled over time and adopt new tools without establishing formal evaluation criteria. Several software cost models assess the productivity impacts of CASE tools based just on breadth of tool coverage without considering other productivity dimensions such as degree of integration, tool maturity, and user support. This dissertation provides an extended set of tool rating scales based on the completeness of tool coverage, the degree of tool integration, and tool maturity/user support. It uses these scales to refine the way in which CASE tools are effectively evaluated within COCOMO (COnstructive COst MOdel) II. In order to find a best fit of weighting values for the extended set of tool rating scales in the extended research model, a Bayesian approach is adopted to combine two sources of (expert-judged and data-determined) information to increase prediction accuracy. The research model using the extended three TOOL rating scales is validated by using cross-validation methodologies such as data splitting and bootstrapping.
Yimin Bao, "A Flexible Integration Framework for Software Tool Interoperability," PhD Dissertation, Department of Computer Science, University of Southern California, August 1996 (pdf)
Current trend of constructing new systems from collections of pre-existing third-party tools and commercial off-the-shelf (COTS) software presents serious challenges to existing integration technology. The unique constraints of third-party software, the interactive, autonomous nature of most of today's software applications, and the increasingly emphasized new integration requirements are among the major reasons for causing this difficulty.
This dissertation presents a flexible integration framework which has general applicability for pre-existing tools and COTS software, supports users to easily change the way software interact with each other (thus supporting flexible integration, system evolution and component reusability), and is easily programmable by the end-users. The framework consists of four major substrates: (1) A dynamic interoperability model which separates interaction relationship from tools and provides implicit communication and dynamic binding for managing independence and flexibility of the interaction relationship. This forms the conceptual base for our framework; (2) A black box integration approach called implicit tool encapsulation which extends the traditional integration interfaces to graphical user interface and system interface, and uses passive, implicit, system-centered mechanism for tool encapsulation; (3) A general methodology for tool integration which is derived from the dynamic interoperability model and implicit tool encapsulation approach, and is applicable for pre-existing third-party tool and COTS software; (4) A language called Tool Integration Language (TIL) which has the capability of accessing the functionality of tools through their user interface as well as other interfaces, and it can be used to describe the user's interaction between tools and the interaction between tools themselves. This provides necessary integration language for our framework and de-couples the integration policy from the mechanism.
A prototype system--Tool Integration Server System (TISS) has been constructed which supports TIL glue code generation and execution, and provides flexible integration mechanisms for our framework. By instantiating the general integration methodology with TIL language and TISS system, it generates a flexible and practical integration approach for integrating pre-existing, GUI-based, interactive tools without any source code modification.
Jesal Bhuta, "A Framework for Intelligent Assessment and Resolution of Commercial-Off-The-Shelf Product Incompatibilities," PhD Dissertation, Department of Computer Science, University of Southern California, August 2007 (pdf)
Software systems today are frequently composed from prefabricated commercial components that provide complex functionality and engage in complicated interactions. Such projects that utilize multiple commercial-off-the-shelf (COTS) products often confront interoperability conflicts resulting in budget and schedule overruns. These conflicts occur because of the incompatible assumptions made by developers during the development of these products. Identification of such conflicts and planning strategies to resolve them is critical for developing such systems under budget and schedule constraints. Unfortunately, acquiring information to perform interoperability analysis is a time-intensive process. Moreover, increase in the number of COTS products available to fulfill similar functionality leads to hundreds of COTS product combinations, further complicating the COTS interoperability assessment activity.
This dissertation motivates, presents and validates an intelligent assessment and resolution framework for Commercial-Off-The-Shelf (COTS) incompatibilities. The framework can be used to perform high-level and automated interoperability assessment to filter out COTS product combinations whose integration will not be feasible within project constraints. The framework efficiently and effectively captures knowledge on COTS product interoperability and allows a user to automatically leverage this knowledge to perform interoperability assessment. The framework elements have been utilized to develop an interoperability assessment tool – Integration Studio.
This framework is empirically validated using controlled experiments and project implementations in 25 projects from small, medium and large network centric systems from diverse business domains. The empirical evidence consistently indicates an increase in interoperability assessment productivity by about 50% and accuracy by 20% in small and medium systems.
Yuriy Brun, "Self-Assembly for Discreet, Fault-Tolerant, and Scalable Computation on Internet-Sized Distributed Networks," PhD Dissertation, Department of Computer Science, University of Southern California, March 2008 (pdf)
When engineers compare biological and software systems, the former come out ahead in the majority of dimensions. For example, the human body is far more complex, better suited to deal with faulty components, more resistant to malicious agents such as viruses, and more adaptive to environmental changes than your favorite operating system. Thus it follows that we, the engineers, may be able to build better software systems than the ones we build today by borrowing technologies from nature and injecting them into our system design process.
In this dissertation, I present an architectural style and accompanying implementation support for building distributed software systems that allow large networks, such as the Internet, to solve computationally intensive problems. This architectural style, the tile style, is based on a nature's system of crystal growth, and thus inherits some of nature's dependability, fault and adversary tolerance, scalability, and security. The tile style allows one to distribute computation onto a large network in a way that guarantees that unless someone controls a large fraction of the network, they cannot learn the private data within the computation or force the computation to fail. These systems are highly scalable, capable of dealing with faulty and malicious nodes, and are discreet since every sufficiently small group of nodes knows neither the problem nor the data.
The tile style is based on a formal mathematical model of self-assembly. In order to leverage this model to build software, I define the notion of self-assembling computation and develop systems that compute functions such as adding, multiplying, factoring, and solving NP-complete problems SubsetSum and SAT. For each system, I prove its correctness, compute its probability of successful computation, and show that its running time and tileset size are asymptotically optimal.
I use the mathematical nature of the tile assembly model to present a formal mathematical analysis of the tile style, proving that software systems built using this style are discreet, fault- and adversary-tolerant, and scalable. I further implement a tile-style system and use it to distribute computation to empirically evaluate the tile style's utility.
Yue Chen, "Software Security Economics and Threat Modeling Based on Attack Path Analysis," PhD Dissertation, Department of Computer Science, University of Southern California, November 2007 (pdf)
The thesis presents the Threat Modeling Method Based on Attack Path Analysis (T-MAP) which quantifies security threats by calculating the total severity weights of relevant attacking paths for Commercial Off The Shelf (COTS) based systems. Further security economic analysis enabled by T-MAP is demonstrated. Compared to existing approaches, T-MAP is sensitive to system stakeholder value priorities and organizational IT environment. It distills the technical details of thousands of relevant software vulnerabilities into management-friendly numbers at a high-level; it systematically establishes the traceability and consistency from management-level organizational value propositions to technical-level security threats and corresponding mitigation strategies. In its trial usage in a large IT organization, T-MAP demonstrated promising strength in prioritizing and estimating security investment cost-effectiveness, as well as in evaluating the security performance of COTS systems. The steps of using T-MAP to optimize the cost-effectiveness of software patching, user account control and firewall are demonstrated through case studies. A system stakeholder value sensitive Firewall rule generation method based on T-MAP is introduced. In addition, a software tool that automates T-MAP has been developed.
Zhihao Chen, "Reduced-Parameter Modeling for Cost Estimation Models," PhD Dissertation, Department of Computer Science, University of Southern California, May 2006 (pdf) In this research, parametric software cost estimation models and their related calibration methods have been analyzed, especially for the COCOMO model and the Bayesian calibration approach. This research combines machine learning techniques and tatistical techniques. With this approach, the prediction powers of the COCOMO parametric software cost model are shown to be significantly improved while the variability is decreased with respect to the dataset being analyzed. This research studies not only the accuracy but also the variances of the model and the variables. It can improve the confidence of people who use software cost estimation models, show the prediction power of software cost estimation models after calibration, and make it easier and better to perform software data collection and analysis. However, the research also identifies risks in using the approach, such as dropping parameters that will vary on future projects. This research provides methods that can help an organization to reason about the relationship between the characteristics of the organization and its projects' software development costs and schedules. The methods can thus help the organization to make more cost-effective development decisions and investment decisions. The research also provides new insights on how to combine calibration, stratification, hold-out, and machine learning techniques to produce more accurate parametric models for particular organizations or situations.
Sunita Chulani, "Bayesian Analysis of Software Costs and Quality Models," PhD Dissertation, Department of Computer Science, University of Southern California, May 1999 (pdf)
Software cost and quality estimation has become an increasingly important field due to the increasingly pervasive role of software in today's world. In spite of the existence of about a dozen software estimation models, the field continues to remain not-too-well-understood, causing growing concerns in the software-engineering community.
In this dissertation, the existing techniques that are used for building software estimation models are discussed with a focus on the empirical calibration of the models. It is noted that traditional calibration approaches (especially the popular multiple-regression approach) can have serious difficulties when used on software engineering data that is usually scarce, incomplete, and imprecisely collected. To alleviate these problems, a composite technique for building software models based on a mix of data and expert judgement is discussed. This technique is based on the well understood and widely accepted Bayes' theorem that has been successfully applied in other engineering domains including to some extent in the software-reliability engineering domain. But, the Bayesian approach has not been effectively exploited for building more robust software estimation models that use a variance-balanced mix of project data and expert judgement.
The focus of this dissertation is to show the improvement in accuracy of the cost estimation model (COCOMO II) when the Bayesian approach is employed versus the multiple regression approach. When the Bayesian model calibrated using a dataset of 83 datapoints is validated on a dataset of 161 datapoints (all datapoints are actual completed software projects collected from Commercial, Aerospace, Government and non-profit organizations), it yields a prediction accuracy of PRED(.30) = 66% (i.e. 106 or 66% of the 161 datapoints are estimated within 30% of the actuals). Whereas the pure-regression based model calibrated using 83 datapoints when validated on the same 161 project dataset yields a poorer accuracy of PRED(.30) = 44%.
A quality model extension of the COCOMO II model, namely COQUALMO, is also discussed. The development of COQUALMO from its onset enables one to understand how a comprehensive modeling methodology can be used to build effective software estimation models using the Bayesian framework elaborated in this dissertation.
Bradford Clark, "The Effects of Software Process Maturity on Software Development Effort," PhD Dissertation, Department of Computer Science, University of Southern California, August 1997 (pdf)
This research examines the effects of Software Process Maturity, using the Software Capability Maturity Model, version 1.1, on software development effort. The technical challenge in this research is determining how much change in effort is due solely to changing Process Maturity when this change generally occurs concurrently with changes to other factors that also influence software development effort. Six mathematical models used in this research support the conclusion that for one hundred twelve projects increasing Process Maturity one level results in a 15% to 21% reduction in effort.
Alexander Egyed , "Heterogeneous View Integration and Its Automation," PhD Dissertation, Department of Computer Science, University of Southern California, August 2000 (pdf)
Software systems are characterized by unprecedented complexity. One effective means of dealing with that complexity is to consider a system from a particular perspective, or view (e.g., architecture or design diagram). Views enable software developers to reduce the amount of information they have to deal with at any given time. They enable this by utilizing a divide-and-conquer strategy that allows large-scale software development problems to be broken up into smaller, more comprehensible pieces. Individual development issues can then be evaluated without the need of access to the whole body of knowledge about a given software system. The major drawback of views is that development concerns cannot truly be investigated by themselves, since concerns tend to affect one another. Successful and precise product development supported via multiple views requires that common assumptions and definitions are recognized and maintained in a consistent fashion. In other words, having views with inconsistent assumptions about a system's expected environment reduces their usefulness and possibly renders invalid solutions based on them.
Developing software systems therefore requires more than what general-purpose software development models can provide today. Development is about modeling, solving, and interpreting, and in doing so a major emphasis is placed on mismatch identification and reconciliation within and among diagrammatic and textual views. Our work introduces a view integration framework and demonstrates how its activities enable view comparison in a more scalable and reliable fashion. Our framework extends the comparison activity with mapping and transformation to define the 'what' and the 'how' of view integration. We will demonstrate the use of our framework on the Unified Modeling Language (UML), which has become a de-facto standard for object-oriented software development. In this context we will describe causes of model inconsistencies among UML views, and show how integration techniques can be applied to identify and resolve them in a more automated fashion. Our framework is tool supported.
Cristina Gacek, "Detecting Architectural Mismatches During Systems Composition," PhD Dissertation, Department of Computer Science, University of Southern California, December 1998 (PARTS: body, appendix WHOLE: pdf)
The USC Architect's Automated Assistant (AAA) tool and method version 0.1 [Abd-Allah 1996] provides a capability for early detection of software architectural style mismatches among four architectural styles: Main-Subroutine, Pipe-and-Filter, Event-Based, and Distributed Processes. For these four styles, mismatch detection is based on a set of seven conceptual features distinguishing each style, and a set of bridging connectors characterizing compositions among the four styles. However, it was a significant open question whether these conceptual features and connectors were sufficient to characterize composition of other architectural styles.
The work presented here formalizes some additional architectural styles--namely Blackboard, Closed-Loop Feedback Control, Logic Programming, Real-Time, Rule-Based, Transactional Database, and Internet Distributed Entities styles--and extends the mismatch analysis capability to cover interactions of the original four styles with the new ones. The analysis results tested various hypotheses, such as the extensibility of the conceptual feature framework for mismatch detection, and the sufficiency of the original seven conceptual features to characterize the broader set of styles and their composition. In our work we found that the underlying conceptual feature framework could work to cover a broader range of styles and systems, with some extensions. However, the conceptual feature set and the underlying Z-language formal model were not sufficient to cover the full range of styles and systems interactions. We have developed extensions to the conceptual feature set and Z formal model to cover the full set of compositional interactions analyzed. Additionally, we provide means for checking each and every mismatch at the model level, including the dynamic ones, as well as a fully operational tool. We also provide an initial discussion of a more formal basis for detecting and classifying architectural conceptual features, thus providing a formal framework for extending the models.
LiGuo Huang, "Software Quality Analysis: A Value-Based Approach," PhD Dissertation, Department of Computer Science, University of Southern California, December 2006 (pdf)
Quality is a major challenge for all complex software systems. Some important attributes of software quality include reliability, availability, safety, security, survivability, performance, accuracy, etc. These have long been requirements of aerospace and defense systems. Now, equally challenging requirements are being placed on “everyday systems” that increasingly provide the infrastructure for our daily lives such as commercial, e-business and embedded systems. They are subject to modest, usually tacit, often stakeholder-specific quality requirements. And it is important that we can define and meet a software system’s quality requirements to be fit for its purpose.
However, there are many views of software quality. And there also exist many ways to suboptimize its achievement and/or misallocate limited project resources using incomplete views. In addition, complex missions or projects usually involve a large and heterogeneous group of stakeholders with various (often time-varying) quality priorities and different (often conflicting) needs. This makes both one-size-fits-all quality metrics and software development processes driven by such metrics risky to use. This also points to the need for better frameworks to define, reason about and achieve quality attributes.
Based on these observations, the primary goal of this dissertation is to develop a Value-Based Software Quality Analysis framework that integrates the stakeholder/value approach into quality attribute definitions, metrics, models and development processes aiming at achieving the appropriate quality levels for software systems. This framework pays explicit attention to business values that a software system generates for its success-critical stakeholders. It helps us to understand the nature of quality and to achieve the stakeholder mutually satisfactory quality requirements. It addresses the above problems in four aspects: 1) value-based definitions of software quality attributes; 2) value-based quality metrics; 3) Value-Based Software Quality Model (VBSQM) to reason about the Return On Investment (ROI) of quality and to perform combined risk analyses; and 4) Value-Based Software Quality Achievement (VBSQA) process.
Finally, the VBSQM and VBSQA process have been applied and found to be effective on three diverse software systems with different value profiles: a USC-CSE e-service project, the NASA/USC Inspector SCRover project, and a real-world ERP software development project in industry.
Hoh Peter In, "Conflict Identification and Resolution for Software Attribute Requirements," PhD Dissertation, Department of Computer Science, University of Southern California, December 1998 (pdf)
A critical success factor in requirements engineering involves determining and resolving conflicts among candidate system requirements proposed by multiple stakeholders. Many software projects have failed due to requirements conflicts among the stakeholders.
The WinWin system developed at USC provides an approach for resolving requirements conflicts among the stakeholders. The WinWin system provides a framework for negotiation between the stakeholders to identify and resolve these conflicts. However, such systems do not scale well for large software projects containing many requirements.
Based on an analysis of the options for addressing this problem, I have focused on semiautomated tools and techniques for identifying and resolving conflicts among software quality attributes. I have developed two prototype support tools, QARCC and S-COST, which expand the capabilities of the WinWin system. QARCC focuses on software architecture strategies for achieving quality attribute objectives. S-COST focuses on tradeoffs among software cost, functionality, and other quality attributes. I have also developed portions of underlying theories and models which serve as the basis for the prototype tools.
Finally, I evaluated the theories, models, and tools with the results of WinWin negotiations, such as the CS577 15-project samples.
Hasan Kitapci, "Formalizing Informal Stakeholder Inputs Using Gap-Bridging Methods," PhD Dissertation, Department of Computer Science, University of Southern California, December 2007 (pdf)
For many software and systems developers, Requirements Engineering is one of the biggest problems they face. Without an accurate, consistent, and complete requirements specification it is very difficult to develop, change, and maintain software. It is now a generally accepted and frequently stated fact that one of the major causes of software development failures is poor requirements engineering. The CHAOS Surveys by the Standish Group report that 15% of the projects fail outright, and 51% are late, run over budget or provide reduced functionality. On average only 54% of the initial project requirements are delivered to the client. Inadequate specification of system requirements is considered to be one of the main causes for project failure.
Good--sufficiently complete, consistent, well-understood, and testable--requirements are a prerequisite for successful projects. Without understanding what the stakeholders--people who have some demands on the product, and hence must be consulted in the requirements gathering activity--really wants and needs, and writing these requirements in a concise, understandable and testable manner, projects not only won't lead to what the stakeholders wanted, but also they probably will fail.
Often, the stakeholders who can contribute the most value and realism to system requirements are the ones with the least time available to participate in the process. Negotiations are crucial in the early stages of the lifecycle to develop a set of mutually satisfactory requirements. We have found the EasyWinWin requirement negotiation method and tools to be an effective way to involve such busy stakeholders. It avoids time-consuming group wordsmithing, but results in relatively informal win-win agreements (e.g., "24/7" or "anytime, anywhere" vs. more precise availability and accessibility requirements). The transition to semi-formal and formal representations is still a challenging problem. While the EasyWinWin improves the identification and negotiation of stakeholders' requirements, the transition to more formal system requirements and further project artifacts is based on intuition and experience.
Requirements problems are present to a certain degree in almost all projects despite the fact that there are available techniques that could address them. The approach of our research is to integrate gap-bridging methods--template, NLP, keyword analysis, formal experts, and inspection--with requirements engineering to leverage the benefits of these methods in order to create a better requirements engineering process. The basic idea is that orchestrating gap-bridging methods whenever and wherever possible through the requirements engineering process could eliminate requirements related problems. This framework helps consistently elicit the customer needs, save time and cost, and trace the requirements through the entire development life cycle.
We enhance the existing tool EasyWinWin, to improve the results of requirements negotiation process and further to define the requirements specifications with the characteristics of good requirements taken into account. Beginning with the identification of requirements leading to the analysis and negotiation of requirements to the actual generation of requirements specification, the framework combines all requirements engineering activities in order to eliminate problems and provide a good foundation for the later artifacts of software development.
Alexander Lam, "Architecture and Application of an Autonomous Robotic Software Engineering Technology Tested (SETT)," PhD Dissertation, Department of Computer Science, University of Southern California, May 2008 (pdf)
This research provides a new way to develop and apply a new form of software: software engineering technology testbeds designed to evaluate alternative software engineering technologies, and to accelerate their maturation and transition into project use. Software engineering technology testbeds include not only the specifications and code, but also the package of instrumentation, scenario drivers, seeded defects, experimentation guidelines, and comparative effort and defect data needed to facilitate technology evaluation experiments.
The requirements and architecture to build a software engineering technology testbed has been developed and applied to evaluate a wide range of technologies. The technologies evaluated came from the fields of architecture, testing, state-model checking, and operational envelopes.
The testbed evaluation showed (1) that certain technologies were complementary and cost-effective to apply to mission-critical systems; (2) that the testbed was cost-effective to use by researchers; (3) that collaboration in testbed use by researchers and the practitioners resulted in actions to accelerate technology maturity and transition into project use; and (4) that the software engineering technology testbed's requirements and architecture were suitable for evaluating technologies and to accelerate their maturation and transition into project use.
DeWitt T. Latimer IV, "Effectiveness of Engineering Practices for the Acquisition and Employment of Robotic Systems," PhD Dissertation, Department of Computer Science, University of Southern California, May 2008 (pdf)
This thesis is concerned with the identification of engineering practices that most influence the ability of an organization to successfully acquire and employ a robot. Of specific interest are the matches or mismatches between our technical efforts and achieving robotic systems that are suitable for the intended purpose. From a survey of engineers (n=18) who have advised or performed the acquisition of robots, candidate relations between engineering practices and system success metrics are proposed. Those relationships are then evaluated against a 5 case studies and one mini-study to examine more closely how the practices are implemented as specific engineering methods in context. From those observations, a series project feasibility rationales are proposed to aid engineers and managers evaluate the feasibility of their robotic system acquisition.
Joo H. Lee, "Promoting Student Feedback in the Classroom," PhD Dissertation, Department of Computer Science, University of Southern California, May 2003 (pdf)
One form of distance education is when remote students are viewing a lecture either by television or webcast. A major challenge is to assist these remote students so they feel a part of the class. Several technologies have been proposed for mitigating the physical separation including: video teleconferencing, chat rooms, and remote application sharing.
This work focuses on computer mediated interactive distance education systems. The framework is TIDE (Technology for Interactive Distance Education) system that facilitates feedback from the students to the instructor. In contrast to most other research on distance education that focuses on enhancing the flow of information from the instructors to the students and/or from the educational content to the students, the TIDE system is intended to help the flow of information from the students to the instructor. This work provides a new learning paradigm-- the computer-mediated cooperative information transfer learning paradigm where the learning takes place as the result of the process whereby technology mediates between instructor and students to aid effective communication.
In this thesis I describe the implementation of the TIDE system including system architecture, system protocol, graphical user interface design, and three feedback mechanisms incorporated within. Then I describe a series of experiments that made use of the TIDE system in the studio classrooms available at USC's Distance Education Network during 2001/2002 with sixteen students and four instructors. The results from the experiments and analysis of the survey are used to evaluate the effectiveness of the TIDE system and three feedback mechanisms.
June Sup Lee, "Designing and Building Protocols for Web Applications," PhD Dissertation, Department of Computer Science, University of Southern California, August 1999 (pdf)
This thesis is about three topics, (i) software architectures, (ii) building application-level protocols, and (iii) transforming legacy software onto the web. In part (i) I describe an architectural style in which the computation is divided between the client and server. I call this style the 3-tier Web architecture. I provide a formal specification for the 3-tier Web architecture. One important aspect of this formal specification is how it assists in identifying various elements to consider when implementing an instance of this architecture. I present two instances of the 3-tier Web architecture--Web-COCOMO and Web-WinWin, which I have designed at the Center for Software Engineering at USC. While designing and building these example Web applications, I discovered that building the application-level protocol is one of the key issues in building Web applications. In part (ii) I focus down within the 3-tier Web architecture on the question of creating application-level protocols. I design and build a tool which helps a programmer generate such a protocol using RPC elements. The tool is novel in several ways, including the fact that it generates a complete and consistent protocol which is also efficient. In part (iii) of this thesis I apply the 3-tier Web architecture and my protocol tool to the problem of transforming legacy software systems to the Web. I identify the key elements in the process and show how the architecture and my tool can be successfully used.
Keun Lee, "Development and Evaluation of Value-Based Review (VBR) Methods," PhD Dissertation, Department of Computer Science, University of Southern California, May 2006 (pdf)
Reviewing is a key activity that can find defects at an early stage of system and software development. Since it is often cheaper to fix defects at an early stage, reviewing is a good technique for improving both product quality and project cost effectiveness. Currently, there are many review techniques proposed and many experiments have been performed to compare the review techniques. However, to data there have been no review techniques or experiments that have focused explicitly on the relative business value or mission value of the artifacts being reviewed.
In this dissertation, I provide Value-based review techniques adding cost effectiveness and value of each issue into review processes, and report on an experiment on Value-based review. I developed a set of VBR checklists with issues ranked by success-criticality, and a set of VBR processes prioritized by issue criticality and stakeholder-negotiated product capability priorities. The experiment involved 28 independent verification and validation (IV&V) subjects (full-time working professionals taking a distance learning course) reviewing specifications produced by 18 real-client, full-time student e-services projects. The IV&V subjects were randomly assigned to use either the VBR approach or the previous value-neutral checklist-based reading (CBR) approach that had been used in the course. The difference between groups was not statistically significant for number of issues reported, but was statistically significant for number of issues per review hour, total issue impact in terms of criticality and priority, and cost effectiveness in terms of total issue impact per review hour. For the latter, the VBRs were roughly twice as cost-effective as the CBRs.
The dissertation also covers threats to validity and limitations of the experiment. Threats to validity were present but appear to have been adequately addresses. The main limitation of the experiment was its coverage of reviews by individuals as compared to groups. For reviewers by groups, it is likely that combinations of VBR and risk-driven forms of perspective-based review (PBR) approaches would be most cost-effective. This and other topics are attractive candidates for further research.
Ming June Lee, "Foundations of the WinWin Requirements Negotiation System," PhD Dissertation, Department of Computer Science, University of Southern California, August 1996 (pdf)
Requirements Engineering (RE) constitutes an important part of Software Engineering. The USC WinWin requirements negotiation system addresses critical issues in requirements engineering including (1) multi-stakeholder considerations, (2) change management, and (3) groupware support. The WinWin approach to date has primarily involved exploratory prototyping. The system is now converging on a relatively stable set of artifacts and relationships. This makes it feasible and important to formalize these artifacts and relationships to provide a solid scientific framework for the WinWin system. This is the focused problem addressed by the research presented in this paper.
Ray Madachy, "A Software Project Dynamics Model for Process Cost, Schedule and Risk Assessment," PhD Dissertation, Department of Industrial and Systems Engineering, University of Southern California, December 1994 (pdf)
A dynamic model of an inspection-based software lifecycle process has been developed to support quantitative evaluation of the process. In conjunction with a knowledge-based method that has been developed for cost estimation and project risk assessment, these modeling techniques can support project planning and management, and aid in process improvement.
The model serves to examine the effects of inspection practices on cost, schedule and quality throughout the lifecycle. It uses system dynamics to model the interrelated flows of tasks, errors and personnel throughout different development phases and is calibrated to industrial data. It extends previous software project dynamics research by examining an inspection-based process with an original model, integrating it with the knowledge-based method for risk assessment and cost estimation, and using an alternative modeling platform.
While specific enough to investigate inspection practices, it is sufficiently general to incorporate changes for other phenomena. It demonstrates the effects of performing inspections or not, the effectiveness of varied inspection policies, and the effects of other managerial policies such as manpower allocation. The dynamic effects are tracked throughout the time history of a project to show resource usage, task completions and defect trends per phase. Cumulative metrics and development tradeoffs for decision making are also presented.
The knowledge-based method has been implemented on multiple platforms. As an extension to COCOMO, it aids in project planning by identifying, categorizing, quantifying and prioritizing project risks. It also detects cost estimate input anomalies and provides risk control advice in addition to conventional cost and schedule calculation. It extends previous work by focusing on risk assessment, incorporating substantially more rules, going beyond standard COCOMO, performing quantitative validation, and providing a user-friendly interface. The method is being used and enhanced in industrial environments as part of an integrated capability to assist in system acquisition, project planning and risk management.
Several types of validation tests are performed against industrial data, existing theory and other prediction models, and practitioners are used to evaluate the model. The results indicate a valid model that can be used for process evaluation and project planning, and serve as a framework for incorporating other dynamic process factors.
Chris Mattmann , "Software Connectors for Highly Distributed and Voluminous Data-Intensive Systems," PhD Dissertation, Department of Computer Science, University of Southern California, December 2007 (pdf)
Data-intensive systems and applications transfer large volumes of data and metadata to highly distributed users separated by geographic distance and organizational boundaries. A dominating factor in these large volume data transfers is the selection of the appropriate software connector that satisfies user constraints on the required data distribution scenarios. This task is typically accomplished by consulting “gurus” who rely on their intuitions, at best backd by anecdotal evidence.
In this dissertation we motivate, present and evaluate a software architecture-based systematic framework for selecting software connectors based on eight key dimensions of data distribution that we use to represent the data distribution scenarios. Our framework, dubbed DISCO, accurately, efficiently, and reliably captures a guru’s domain knowledge and allows a user to automatically leverage that knowledge to drive connector selection. In addition, DISCO affords a user the ability to validate a guru’s domain knowledge against actual performance measurements of the connectors in the areas of efficiency, scalability, dependability and consistency.
We provide a set of models, algorithms, techniques and tools to represent data distribution scenarios, classify and select connectors and explore the trade off space when architecting large scale data distribution systems. To date, 13 real-world connectors across four connector families have been explored using our framework. We validate our framework empirically and qualitatively, employing 30 data distribution scenarios gleaned from three real-world projects spanning planetary science, cancer research and earth science at NASA’s Jet Propulsion Laboratory. We use a number of measures of accuracy including precision, recall and error rate. We also provide theoretical performance analysis of our connector selection algorithms. We report empirical performance measurements of the 13 connectors and use the data to revise and validate our precision measurements. In addition to our validation, we have integrated DISCO as a “plug-in” piece to an independently-developed COTS interoperability assessment framework, providing more feedback for a second use-case of the tool. We conclude the dissertation with a set of open research questions that will frame our future work.
Nikunj Mehta, "Composing Style-Based Software Architectures from Architectural Primitives," PhD Dissertation, Department of Computer Science, University of Southern California, December 2004 (pdf)
Software architectures provide high-level abstractions to deal with the complexity and richness of large-scale software systems. Often, similar architectural organization is found in different software systems. Architectural styles are an approach to exploit such similarities between different architectures. A style is useful in the design of architectures due to guarantees of desirable stylistic properties across architectures based on that style. Styles are also believed to bring economies of scale in the design and implementation of architectures. Although there are many systematic techniques for describing and selecting styles, there is considerably less underpinning for the systematic design and construction of style-based architectures.
This dissertation motivates, presents, and validates a methodology for the composition of style-based software architectures from architectural primitives. This methodology, embodied as the Alfa framework, is a collection of related techniques that support the expression of stylistic constraints as well as architectural organization through the use of architectural primitives. Not only does the Alfa framework ensure the conformance of stylistic constraints in architectures using those styles, but it is also amenable to scalable implementation of architectural designs. Moreover, our methodology is comprehensive in its scope: it supports the data, structural, topological, interaction, and behavioral concerns in styles and architectures. The unique aspects of Alfa are: explicit recognition of architectural primitives that underlie a wide range of styles and architectures; type checking of data in styles and architectures, to ensure validity of architectural compositions; effective analysis of architectures for conformance to style(s) used in them, to preserve stylistic properties; and scalable implementation of architectural primitives and their composition, to support architectural prototyping.
The key proposition of the dissertation is demonstrated through a case study that applies Alfa to distributed systems. The dissertation is validated analytically by deducing the algorithmic complexity of Alfa’s conformance analysis; and quantitatively, by measuring the scalability of Alfa-based architectural implementations in terms of memory usage and processing time. The dissertation is concluded by inferring the consequences of Alfa on the design, analysis, and implementation of software systems.
Marija Mikic-Rakic, "Software Architectural Support for Disconnected Operation in Distributed Environments," PhD Dissertation, Department of Computer Science, University of Southern California, December 2004 (pdf)
In distributed and mobile environments, the connections among the hosts on which a software system is running are often unstable. As a result of connectivity losses, the overall availability of the system decreases. The distribution of software components onto hardware nodes (i.e., deployment architecture) may be ill-suited for the given target hardware environment and may need to be altered to improve the software system's availability. The critical difficulty in achieving this task lies in the fact that determining a software system's deployment that will maximize its availability is an exponentially complex problem.
In this dissertation, we motivate, present, and evaluate an automated, flexible, software architecture-based solution for disconnected operation that increases the availability of the system during disconnection. We provide a set of models, algorithms, techniques, and tools for improving a distributed, mobile system's availability via redeployment, by enabling the system to (1) monitor its operation; (2) estimate its deployment architecture; and (3) effect that architecture automatically. While the focus of this dissertation research is on improving system availability, our methodology is extensible to support other non-functional properties that may be relevant in a given distributed system.
This dissertation is evaluated empirically, by conducting a series of benchmark tests and applying the methodology to example applications. The dissertation concludes with a set of open research questions that will frame our future work.
Allen Peter Nikora, "Software System Defect Content Prediction from Development Process and Product Characteristics," PhD Dissertation, Department of Computer Science, University of Southern California, May 1998 (pdf)
Society has become increasingly dependent on software controlled systems (e.g., banking systems, nuclear power station control systems, and air traffic control systems). These systems have been growing in complexity – the number of lines of source code in the Space Shuttle, for instance, is estimated to be 10 million, and the number of lines of source code that will fly aboard Space Station Alpha has been estimated to be up to 100 million. As we become more dependent on software systems, and as they grow more complex, it becomes necessary to develop new methods to ensure that the systems perform reliably.
One important aspect of ensuring reliability is being able to measure and predict the system’s reliability accurately. The techniques currently being applied in the software industry are largely confined to the application of software reliability models during test. These are statistical models that take as their input failure history data (i.e., time since last failure, or number of failures discovered in an interval), and produce estimates of system reliability and failure intensity. To better control a system’s quality, we need the ability to measure the system’s reliability prior to test, when it is possible to influence the development process and change the system’s structure.
We develop a model for predicting the rate at which defects are inserted into a system, using measured changes in a system’s structure and development process as predictors, and show how to:
Estimate the number of residual defects in any module at any time.
Determine whether additional resources should be allocated to finding and repairing defects in a module.
In order to calibrate the model and estimate the number of remaining defects in a system, it is necessary to accurately identify and count the number of defects that have been introduced into a system. We develop a set of rules that can be used to count the number of defects that are present in the system, based on observed changes that have been made to the system as a result of repair actions.
Monvorath Phongpaibul, "Experimental and Analytical Comparison Between Pair Development and Software Development With Fagan's Inspection," PhD Dissertation, Department of Computer Science, University of Southern California, December 2007 (pdf)
Peer review is one of the essential activities in software quality assurance since peer reviews can detect and remove defects in the early stages of the software development life cycle. Removing defects early reduces the cost of defect rework later. Selecting a peer review methodology (e.g., inspection, walkthrough, checklist-based, defect-based, function-based, perspective-based, usage-based, value-based) to execute in a software project is difficult. The developers have to understand the commonalities and differences of each methodology. They need to know the relative strengths and weaknesses of these practices. However, very few studies have compared the commonalities and differences of each peer review methodology and none of the studies have shown an empirical comparison between pair programming and software inspection.
Software inspection and pair programming are effective verification techniques. Software inspection is one of the best practices in traditional software development while pair programming is one of the best practices in agile development. Numerous studies have shown the success of software inspection in large-scale software development over the past three decades. Although Pair Programming (PP) is a newer approach and less structured, it has had a strong impact on the success of agile software development projects over the past five years.
This dissertation aims to identify the critical factors that impact the cost-effectiveness of either pair programming/development or inspection and provide the decision framework to help the developers select the most effective technique under given conditions. To compare both techniques, four classroom experiments and one industry experiment were conducted in Thailand and US. The development effort and effect of quality were investigated with some additional calendar time comparisons.
Roshanak Roshandel, "Calculating Architectural Reliability Via Modeling and Analysis," PhD Dissertation, Department of Computer Science, University of Southern California, August 2005 (pdf)
Modeling and estimating software reliability during testing is useful in quantifying the quality of the software systems. However, such measurements applied late in the development process leave too little to be done to improve the quality and dependability of the software system in a cost-effective way. Reliability, an important dependability attribute, is defined as the probability that the system performs its intended functionality under specified design limits. We argue that reliability models must be built to predict the system reliability throughout the development process, and specifically when exact context and execution profile of the system is unknown, or when the implementation artifacts are unavailable. In the context of software architectures, various techniques for modeling software systems and specifying their functionality have been developed. These techniques enable extensive analysis of the specification, but typically lack quantification. Additionally, their relation to dependability attributes of the modeled software system is unknown.
In this dissertation, we present a software architecture-based approach to predicting reliability. The approach is applicable to early stages of development when the implementation artifacts are not yet available, and exact execution profile is unknown. The approach is two fold: first, the reliability of individual components is predicted via a stochastic reliability model built using software architectural artifacts. The uncertainty associated with the execution profile is modeled using Hidden Markov Models, which enable probabilistic modeling with unknown parameters. The overall system reliability is obtained compositionally as a function of the reliability of its constituent components, and their complex interactions. The interactions form a causal network that models how reliability at a specific time in a system's execution is affected by the reliability at previous time steps.
We evaluate our software architecture-based reliability modeling approach to demonstrate that reliability prediction of software systems architectures early during the development life-cycle is both possible and meaningful. The coverage of our architectural analyses, as well as our defect classification is evaluated empirically. The component-level and system-level reliability prediction methodology is evaluated using sensitivity, uncertainty, and complexity, and scalability analyses.
Ricardo Valerdi, "The Constructive Systems Engineering Cost Model (COSYSMO)," PhD Dissertation, Department of Computer Science, University of Southern California, August 2005 (pdf)
As organizations develop more complex systems, increased emphasis is being placed on Systems Engineering (SE) to ensure that cost, schedule, and performance targets are met. Correspondingly, the failure to adequately plan and fund the systems engineering effort appears to have contributed to a number of cost overruns and schedule slips, especially in the development of complex aerospace systems. This has resulted in a recent increased emphasis on revitalizing systems engineering in government and commercial organizations.
This dissertation presents a parametric model that can help people reason about their decisions related to systems engineering. COSYSMO, the Constructive Systems Engineering Cost Model, is an “open” model that contains eighteen parameters: four size drivers and fourteen effort multipliers. It is built on a framework similar to its wellknown predecessor, COCOMO II, and integrates accepted systems engineering standards to define its scope. Funded by industry affiliates, the model focuses on large-scale systems for military applications that employ a disciplined approach to systems engineering. Data was collected from six aerospace companies in the form of expert opinion and historical project data to define and calibrate the model. In reduced form, the model yields a PRED(30) of 50% for programs within a defined productivity range. In principle, the model should apply similarly to commercial systems engineering, but there is a lack of data to test this hypothesis.
The ultimate contributions of this dissertation can be found in at least two major areas: (a) in the theoretical and methodological domain of systems modeling in the quest of a more quantitative cost estimation framework, and (b) in advancing the state of practice in the assessment and tracking of systems engineering in the development of large aerospace systems
Dan Wu, "Security Functional Requirements Analysis for Developing Secure Software," PhD Dissertation, Department of Computer Science, University of Southern California, May 2007 (pdf)
Research experience shows that security needs to be considered from the beginning of software development life cycle to avoid expensive rework and reduce potential security vulnerabilities. Hence, defining the right set of security functional requirements (SFRs) and evaluated assurance level (EAL) becomes a critical task for developers when developing secure software. Much effort has been put into creating industry standards to provide a shared common base for stakeholders with concerns on security. One of the industry standards, which is used widely in both industry and government sides in many countries, is Common Criteria (CC). However, one of the drawbacks of Common Criteria is the inefficiency of use. Moreover, with limited project information in the early lifecycle phase, it is hard for developers with less security experience to select the right security requirements from what are defined in CC. Extensions on it and experiences from empirical studies on using it are demanded to achieve a better and more efficient use of CC, which also benefits developers by saving their effort on security functional requirements definition.
A thorough analysis has been done on a dataset consisted by the Security Target (ST) files of 242 security products published on common criteria portal website. A mapping between security objectives and SFRs is presented, which can save much development effort by reduce the range of candidate SFRs when developers know the project's security objectives in the early phases. In the cases when developers only know the product domain of this project, SFR patterns for nine different domains of security products are presented based on the statistic result from the published 242 security products, which can be customized or directly used for particular security application. The analysis result of correlations among SFR classes defined in CC and correlations among security objectives provide a good guidance for developers in designing the architecture of security products. A trend shows that EAL tends to increase when the number of SFRs increases. It is not strongly proved by the current dataset, but shows a research direction for further discussion and explorations in the future.
To validate the correctness of the mapping scheme between security objectives and SFRs, each of the ST files is reviewed to find out the consistency and difference between the presented mapping scheme with the actual selected SFRs in 242 security products with certain security objectives. A method is presented to evaluate the effectiveness of these security patterns, which can be used as a factor for developers when to consider applying the patterns for actual use.
Ye Yang, "Composable Risk-Driven Processes for Developing Software Systems from Commercial-Off-The-Shelf (COTS) Products," PhD Dissertation, Department of Computer Science, University of Southern California, December 2006 (pdf)
Research experience has suggested that software processes should be thought of as a kind of software, which can be developed into composable component pieces that can be executed to perform different software lifecycle activities. General experience has indicated that the activities conducted while developing COTS-based applications (CBA) differ greatly from those conducted in traditional custom development. The primary research questions addressed in this dissertation are (1) Can these activity differences be characterized and statistically analyzed? (2) If so, can the primary CBA activity classes be organized into a decision framework for projects developing CBA’s? The resulting research provides a value-based composable set of processes for CBAs that includes an associated Process Decision Framework (PDF), a set of Composable Process Elements (CPEs), and a COCOTS Risk Analyzer.
A composable process implies the ability to construct a specific process from a higher level and broader process framework and a set of reusable process elements. The PDF is a recursive, re-entrant configuration structure, and establishes the relationships among a mix of the CPEs and other process fragments which are extended from the risk-driven WinWin Spiral model. The CPEs includes Assessment, Tailoring, and Glue code development/integration, which are the three primary sources of effort due to CBA development considerations, indicated by empirical analysis on both large industry and small campus e-services CBA projects. Each CPE is a defined, repeatable workflow. While the framework provides a composition basis to support developers for navigating through the option space in developing CBAs, the three process elements establish the basic constituents for composing COTS processes based on common process patterns identified in empirical studies. A technique named COCOTS Risk Analyzer, is also developed and implemented to aid the optimization of process decisions via risk based prioritization strategy. All together, the proposed solution supports flexible composition of process elements with respect to evolving stakeholders’ value propositions, COTS market, and risk considerations.
To validate the value-based set of processes, experiments have been designed and performed on student projects at USC graduate level software engineering class in Fall 2004 and Fall 2005 semesters. The evaluation results show that applying the value-based processes significantly improves the team performance.
Copyright 2006 The University
of Southern California
The written material, text,
graphics, and software available on this page and all related
pages may be copied, used, and distributed freely as long as the
University of Southern California as the source of the material,
text, graphics or software is always clearly indicated and such
acknowledgement always accompanies any reuse or redistribution
of the material, text, graphics or software; also permission to
use the material, text, graphics or software on these pages does
not include the right to repackage the material, text, graphics
or software in any form or manner and then claim exclusive proprietary
ownership of it as part of a commercial offering of services or
as part of a commercially offered product.
|
|
