Research Paper

Complex Engineered Systems Design

Verification Based on Assume-Guarantee
Reasoning
Hoda Mehrpouyan,1 ,∗ Dimitra Giannakopoulou,2 Guillaume Brat,2 Irem Y. Tumer,3 and Chris Hoyle3

1 Department of Computer Science, College of Engineering, Boise State University, Boise, ID 83702
2 NASA Ames Research Center, Moffett Field, CA 94035
3 Complex Engineered Systems Design Lab, School of Mechanical, Industrial, and Manufacturing Engineering, Oregon State University,

Corvallis, OR

Received 12 June 2014; Revised 14 June 2016; Accepted 14 September 2016, after one or more revisions
Published online in Wiley Online Library (wileyonlinelibrary.com).
DOI 10.1002/sys.21368

ABSTRACT
System verification is one of the most critical tasks into the process of engineered system design. This
process is time-consuming and prone with errors when a limited set of scenarios is evaluated to guarantee
the correct functionality of the system. Therefore, novel design approaches and tools based on a rigorous
framework for analysis, verification, and testing are very much needed. This paper provides such a frame-
work where system properties are verified and modeled with respect to the assumptions on the environment
where components and (sub)systems’ performances are guaranteed under these assumptions. To validate
the proposed approach, this paper provides a case study to demonstrate how the proposed methodology
reduces design complexity and presents a formal argument to assess the quality of the design. C⃝ 2016
Wiley Periodicals, Inc. Syst Eng 19: 461–476, 2016

Key words: verification and validation; model-based systems engineering (MBSE); systems of systems
(SoS); other application sectors

1. INTRODUCTION liner jet and A380 superjumbo projects [Saporito, 2013]. De-
sign flaws were cited as basic issues in the A380 design and
With increasing complexity in the design of complex en- production, which resulted in a variety of glitches such as
gineered systems such as aerospace, maritime, nuclear, and engine blow-up, failure of the backup brakes, and discovery
major civil infrastructure systems, the cost and time required of cracks on the wings of the planes. Boeing 787 Dreamliner
for design and development are growing at an unsustainable faced similar issues, as described in the national transporta-
rate. For instance, Boeing and Airbus experienced significant tion safety board reports of the fire incident in the auxiliary
delays and cost overruns in delivering their latest 787 Dream- power unit (APU) on a Japan Airlines 787 flight from Boston
on January 2013. The report concluded that a design flaw
∗ Author to whom all correspondence should be addressed (e-mail: ho- might be the root cause of the fire. In another case, a Japan
[email protected]) Airline Dreamliner flight from Boston faced a considerable
delay after a fuel leak resulting from a faulty valve. The design
Systems Engineering Vol. 19, No. 6, 2016 flaws and equipment malfunction cost $5 billion for Boeing
C⃝ 2016 Wiley Periodicals, Inc.

461
462 MEHRPOUYAN ET AL.

on top of the $1 billion compensation claim from airlines, and Hudak, 2006], Modelica [Fritzson, 2010], Ptolemy [Buck
such as United Airlines and Air India. et al., 1994], MATLAB/Simulink [Simulink and Natick,
The design complexity of these types of safety critical 1993], and SysML [Friedenthal, Moore, and Steiner, 2011].
systems present various challenges for their safety assessment These tools are used to model the functionality and architec-
process. In order to improve the design and development ture of the system design, then simulation is carried out to ver-
process, manufacturing companies are increasingly relying ify the design. However, most of the simulation experiments
on simulations to understand the unexpected behavior of the are designed to evaluate a limited set of scenarios in order to
design to improve both robustness and performance of sys- deal with the system complexity. The effects of this informal
tem [ElMaraghy, 2009]. Based on his research, Foster [2013] and incomplete verification is the possibility that a nontested
concluded that verification process (i.e., establishing that the scenario could result in unexpected behavior and catastrophic
design, if implemented, would provide the desired function- system failure. To address the incomplete verification of de-
ality) consumes over 60% of the design time. In addition signs via simulation, formal methods have been proposed to
to longer verification time which results in project cost in- increase the confidence level. Formal verification enables the
crease, a survey conducted by Collett International Research evaluation of safety properties at different levels of abstrac-
Inc. [C.I.R. Inc., 2002] revealed that while the traditional tions (i.e., component, subsystem, system), guaranteeing the
verification approaches, that is, Preliminary Hazard Analysis systems’ behavior in every possible scenario. It is important
(PHA), Failure Mode and Effect Analysis (FMEA), are still to note that even formal verification is as good as the abstract
applicable, they are not sufficient; that is, they do not take models and properties that one verifies on such models.
into the consideration the whole complex network of rela- One of the objectives of the verification process is to make
tionships, between events, components, and the environments sure that the design complies with safety requirements. In
where the components operate in. order to satisfy most regulatory guidelines and safety stan-
Therefore, the aim of this paper is to provide a frame- dards, designers must develop a safety case to prove the safety
work for the effective use of formal methods in the early justification of a design. These cases should represent all
verification of safety requirements in safety critical systems. potential hazards and appropriate steps be taken to rectify
The proposed framework allows for automatic generation of the situation. These types of safety documents usually include
fault trees, and exhaustive safety property verification with safety specifications, results of failure and risk analysis, veri-
the help of model checking algorithms. The approach is based fication approach, and results of all the verification activities.
on assume-guarantee compositional reasoning, which verifies Figure 1 depicts the general view of the verification process.
global safety properties of the system by verifying local prop-
erties of each component [Giannakopoulou, Pasareanu, and
Barringer, 2002; Cobleigh, Giannakopoulou, and Păsăreanu,
2003; Alur, Madhusudan, and Nam, 2005; Chaki et al., 2005]. 2.1. Design Tools for Safety and Reliability
Therefore, the verification of large systems is made possible Analysis
through the verification of each system components sepa- This section provides a review of the scope and limitation of
rately, while using assumptions about each components en- currently available system safety and reliability tools with re-
vironment. While compositional reasoning based on assump- spect to verification of complex system designs. One of these
tions and guarantees is popular in other domains [see, e.g., methods is the risk assessment matrix [Siu, 1994], which
Henzinger, Minea, and Prabhu, 2001; Kazhamiakin, Pistore, categorizes risks in the form of probability and severity (loss
and Roveri, 2004; Baresi et al., 2007], we are not aware of of function per unit time). In this approach, a matrix based
any assumption-guarantee style reasoning to verify the safety on the importance of severity versus the probability of failure
properties of complex engineered systems. occurrence is created. However, one limitation of this method
The remainder of this paper is structured as follows: Sec- is its lack of ability to identify failure and its propagation path.
tion 2 presents the background and related research on fail- Another initial reliability study is possible through the use
ure analysis techniques in the early stages of system design, of PHA [Roland and Moriarty, 2009], which identifies and
while discussing their strengths and weaknesses. In addition, provides a database of failures and failure propagation paths.
the definition of assume-guarantee reasoning (AGR) and its PHA uses risk assessment matrix to assess the risk of each
commonly used terminologies and operators are addressed identified failure. The limitation of PHA is its lack of ability
in Section 2. In Section 3, an overview of the step-by-step to evaluate risks of aggregated hazards or simultaneous design
implementation of the AGR algorithm on the components of failure modes. In addition, the risk analysis of a complex
the design architectures is explained. Section 4 outlines the system design will result in large and costly implementation.
application of the proposed methodology in the analysis and FMEA [Do, 1980] is a bottom–up approach that investigates
verification of the safety properties of the satellite electrical failure modes of components and their effects on the rest of
power system (EPS) design. The paper ends with conclusions the system. FMEA provides an exhaustive analysis to identify
and future work. the single point of failures and their effects on the rest of the
system. The result of the analysis is used to increase relia-
bility, incorporate mitigation into the design, and optimize
2. BACKGROUND the design. The result of FMEA analysis can be added to
the PHA analysis of the design, since every failure mode
A variety of modeling approaches and tools are used in in- of each component is evaluated and additional information
dustry or in academia, for example, AADL [Feiler, Gluch, about the hazards resulting from the failures is obtained.

Systems Engineering DOI 10.1002/sys

 COMPLEX ENGINEERED SYSTEMS DESIGN VERIFICATION 463

Figure 1. Verification process in engineered system design.

However, FMEA is very costly in terms of resources, par- system architecture and verify its capability of meeting the
ticularly when implemented at the component level within requirements using abstract design information.
complex systems. Also, occurrences of simultaneous failures In addition, the traditional methodologies are mostly based
and multiple faults is not evaluated [Dhillon and Fashandi, on the evolution of the system through time, given an initial
1997; Tumer and Stone, 2001]. The completeness and cor- event and a set of forcing functions. These physics-based
rectness of the analysis is very much dependent on the expert models are based on the laws of physics from the outset.
knowledge. However, mathematical modeling of complex systems may
The next group of reliability analysis methods is based not be feasible since changes in operating conditions and
on the symbolic logic of the conceptual models of failure structural dynamics can affect the mathematical model, and it
scenarios within a design. The goal is to assess the probability makes it difficult or even impossible to develop mathematical
of failure occurrence in the system design. One of these meth- models for all real-life conditions. Therefore, this research
ods is the Reliability Block Diagram (RBD) [Čepin, 2011], takes advantage of automata learning and model checking
which divides the system into elements based on the func- approaches. Model checking is one of the approaches to for-
tional model of the system design, where each system element mal verification of finite-state hardware and software sys-
is assigned a reliability factor. Then a block diagram of the tems [Clarke, Grumberg, and Peled, 1999; Baier et al., 2008].
elements in a parallel, series, or the combination of parallel In this approach, a design will be modeled as a state tran-
and series is constructed. Each block represents a function or sition system with a finite number of states and a set of
an event in the system and each element’s failure mode is as- transitions. The design model is in essence a finite-state
sumed independent from the rest of the system. The reliability machine, and the fact that it is finite makes it possible to
factor may or may not be available for all the system design execute an exhaustive state-space exploration to prove that
elements and should be assigned by an expert who make it the design satisfies its requirements. Since there is an ex-
subjective and hard to validate. Another symbolic logic model ponential relationship between the number of states in the
is based on the Fault Tree Analysis (FTA) [Ericson and Ll, model and number of components that make up the sys-
1999], which studies the failure propagation path from the tem, the compositional reasoning approach is used to handle
point of start to the vulnerable components and assigns a the large state-space problem. Compositional verification is
severity factor to each failure model. One of the benefits of a promising approach for alleviating the state explosion in
using FTA is its ability to analyze the probability of simul- model checking. In practice, many systems are composed
taneous occurrence of failure within a complex systems. On of various processes running in parallel and interacting in
the other hand, the correct probabilistic evaluation requires complex ways. The safety specifications for such systems can
significant amount of resources. often be decomposed into safety properties that describe the
These traditional safety and reliability tools require de- behavior of small parts of the system. Utilizing the divide
tail information about components, their failure modes, and and conquer technique, if we can deduce that the system
probability of failures in order to complete their analysis. On satisfies each local property, and if we know that the com-
the other hand, conceptual design is a process of developing bination of the local properties implies the overall specifica-
behaviors and functions to create a design solution that meets tion, then we can conclude that the complete system satisfies
the design requirements. At this early stage of design, spe- the specification as well. The proposed framework supports
cific components have not yet been fully specified, therefore incremental system design and verification which provides
collecting data on failure probabilities of the components the abstraction required to reduce the inherent complexity
and how they propagate through the system model is very and to ensure design satisfies safety requirements, while ad-
challenging. The aim of this research is to provide an early dressing the challenges and limitations of traditional design
design framework that is able to automatically analyze the methods.

Systems Engineering DOI 10.1002/sys

464 MEHRPOUYAN ET AL.

2.2. Verification Based on Formal Methods requirements. Kurtoglu et al. [2005] and Kurtoglu and Tumer
[2007] present a framework for developing a functional model
In Henzinger and Sifakis [2006], Henzinger et al. cover the for the hardware components, while Wang and Cheng [2000]
advantages that formal verification offers over the above ap- suggest object-oriented programming for modularization and
proaches. In formal verification, system designers construct functional modeling of the software components. Functional
a precise mathematical model of the system under design, models provide the required information about the flow
so that extensive analysis is carried out to generate proof of of Energy, Material, and Signal (EMS) and data between
correctness. One of the well-established methods for auto- components throughout the system design. In functional
matic formal verification of the system is model checking, modeling [Stone and Wood, 2000; Hirtz et al., 2002], the
where a mathematical model of a system is constructed and EMS flows and functions are modeled using nouns and verbs,
verified with regards to specified properties. In model check- respectively, for example, store electricity, actuate electricity,
ing, the desired properties are defined in terms of tempo- etc. The functional model of the design is developed based
ral logic [Pnueli, 1977]. The defined logical formulae are on the hierarchical structure of functions and flows [Otto and
then used to prove that a system design meets safety re- Wood, 2001; Kurtoglu, Tumer, and Jensen, 2010]. Next, the
quirements and specifications. A model checker to establish structural model as a suitable design solution is developed.
assume-guarantee properties of components is called AGR The structural model describes different system components
[Giannakopoulou et al., 2002; Cobleigh et al., 2003; Alur and the EMS flow relationship between them. Using different
et al., 2005; Chaki et al., 2005]. The assume-guarantee ap- design solutions within the complex system design process,
proach is the evolution of compositional reasoning that in- various design concepts for a system are developed. While
terleaves abstraction and learning algorithm to perform au- all design concepts share the same functional description,
tomated compositional verification of complex systems. In they are implemented differently. They are different in
the AGR method, the system properties are verified and structure and behavior. Finally, the behavioral model of a
modeled with respect to the assumptions on the environ- component contains the nominal and failure states of the
ment where component and (sub)system performances are component, including transitions leading to these states.
guaranteed under these assumptions. The assumption gener- The behavioral model results from the relationship between
ation methodology uses compositional and hierarchical rea- input/output flows and the underlying first principles. Once
soning approaches via a compositional reachability analy- the behavioral models of the component are developed,
sis (CRA) [Cheung and Kramer, 1999] technique. CRA in- they are incorporated into the Labeled Transition Systems
crementally composes and abstracts the component models (LTSs) model by mapping them with their respective LTSs
into subsystem and, ultimately, a high-level system models. transitions.
Based on the AGR paradigm, assume-guarantee can be de-
fined as a pair of assumptions and guarantees which formally
describe:
3. METHODOLOGY
1. The context in which the system design is assumed to
be used. The contribution of this paper is developing an automated
2. The requirements which the system design demands to design verification framework to prove the correctness of the
guarantee correct operation. (It is important to note that complex engineered system design with regards to its func-
“guaranteeing the correct operation in an assumed envi- tional and safety properties. The proposed framework pro-
ronment” is only possible with a specified probability. vides information on the property violation of the composed
In this context, “guarantee” does not mean that system components during conceptual design, while identifying the
will always survive the assumed environment without failure propagation behavior. The automatic generation of
any failures, it means that system will survive at the failure propagation paths enables the system designers to
probability level in which one specifies, i.e., 0.9999.) better address the safety issues in the design.

AGR has been widely used in the computer science liter-

ature as a means for software verification. As a mathemati- 3.1. System Modeling
cal foundation for representing the engineering requirements,
this approach can be used for verification of complex engi- In the proposed approach, finite-state model of a system is an-
neered system design. In addition, the work focuses on safety alyzed to ensure satisfaction of safety properties that assure a
property specification and design verification at multiple ab- desired system behavior. Finite-state model is a representation
straction layers. of the system behavior that is generated in the form of Finite-
As discussed in the previous section, abstraction and State Process (FSP) [Rodrigues, 2000]. FSP is an algebraic
composition are the two most used principles in any system notation that is used to describe the component’s behavior
verification methodology for handling the complexity and (Table I). System designers create the FSP models which
analysis of engineered systems. When verifying complex sys- are designed to be machine readable, and thus provides an
tems, different approaches (e.g., model-based methods) use ideal language to specify abstract model of the component’s
hierarchical abstraction layers such as functional, structural, behavior or function. The developed FSP is then used through
and behavioral models to represent the system under study. A a modeling tool such as the Labeled Transition System Ana-
functional model is a representation of all the necessary func- lyzer (LTSA) [Magree, 1999] to provide compilation of FSPs
tions that the system must contain in order to meet the design into an LTS [Magree, 1999; Gilreath, 2001]. The LTS model is

Systems Engineering DOI 10.1002/sys

 COMPLEX ENGINEERED SYSTEMS DESIGN VERIFICATION 465

Table I. EPS Components and Their FSP Models

EPS System
Component Mode FSP Model
Battery Nominal (inflow[v:CUR] → outflow[v] → Battery)
Current sensor Nominal (inflow[v:CUR] → outflow[v] → Current sensor)
Voltage sensor Nominal (inflow[v:CUR] → outflow[v] → Voltage sensor)
Relay Nominal (inflow[v:CUR] → if (Open) then (outflow[v] → Relay else Relay))
Inverter Nominal (inflow[v:CUR] → outflow[v] → Inverter)
ac Resistor Nominal failure Operational = (inflow[v:CUR] → outflow[v] → Operational ∣ inflow[Spike] →
burn → BURNED), Burned = (inflow[CUR] → Burned)
dc Resistor Nominal failure same as ac resistor
Fan Nominal failure same as ac resistor

Figure 2. Parallel composition of primitive component and its safety property in LTS format.

expressed graphically by its alphabet, transition relation, and 1. A set S of states

states including single initial state (Fig. 2). The LTS of the 2. A set L of actions
system is constructed from the LTS of its subsystems, and is 3. A set → of transitions from one state to another.
verified against safety properties of the design requirements. 4. An initial state s0 ∈ S
In this research, the model checking algorithm is integrated as 5. 𝐓 = (S, L, →, s0 ).
part of the LTSA tool which performs exhaustive execution
of all system’s behavior to determine if the safety property
is violated or not. The LTSA takes advantage of the method
3.2. Parallel Composition of LTSs
called CRA and creates a reachability graph for the system
which contains information about the safety values of each In this section, the parallel composition operator [Garavel
state. A safety property then is checked by analyzing the et al., 1999] and the composition mechanism that assists
reachability graph, searching for paths on which the safety with compositional modeling of the component models are
property is violated. LTSs’ T is defined as: explained. The parallel composition operator enables both

Systems Engineering DOI 10.1002/sys

466 MEHRPOUYAN ET AL.

Figure 3. Structural model of the EPS.

associative and commutative composition; therefore the order The on-mode has the functional action of generating power,
of LTSs models that are composed together is insignificant. which can also result in overcurrent spikes. As illustrated
The parallel composition operator, denoted by “∥”, is a binary in Figure 3, the generated spike affects the ac resistor, fan,
operator that accepts two LTSs as an input argument. Based and dc resistor that are denoted by circles on the right-hand
on the definition of this operator, composed LTSs interact by side of the figure. These components are vulnerable to the
synchronizing on common actions (i.e., exchange of EMS) spike generated by the power source. Thus, safety properties
shared in their FSP models with interleaving of the remain- are needed to protect these vulnerable components and ensure
ing actions. Designing interacting components with LTSs is the proper operation of the whole system. The safety proper-
therefore sensitive to the selection of action names. In addi- ties define the types of failure that a component is vulnerable
tion, parallel composition is based on the model instantiation to and must be checked to ensure the failure state is not
which is defined by constructing a copy of an LTS model reached. There are three different paths 1. {A, B, C}, 2. {A, B,
where each transition label is prefixed by the name of the D}, and 3. {A, E} in which the generated spike from the power
instance. source can reach the three vulnerable components; these are
considered design flaws [Mehrpouyan et al., 2012].

3.3. EPS Design

3.4. Reusable Models and Binding Interfaces
Validation of the proposed verification framework is through
application to an EPS which is designed to provide power In the proposed system modeling approach, each component
to selected loads. In an aerospace vehicle, these loads usu- defines a scope for the transition in its behavioral model (e.g.,
ally include subsystems such as the propulsion, avionics, and an instance of the resistor is defined by the use of the com-
thermal management systems. The basic functionality that mand res:resistor). Instantiation permits the reuse of LTSs
EPS is required to provide is common to many aerospace during system modeling through multiple instantiations, for
applications such as power storage, power distribution, and example, ac and dc resistors behave in a similar fashion under
operation of loads [Poll et al., 2007]. Figure 3 displays the the influence when the power received from the environment
existing design of the EPS, containing a power source con- exceeds the resistor’s ability to dissipate the heat and therefore
nected through a series of relays to an inverter and several the resistor can be used for both components. Instantiation
loads consisting of a large fan, a Direct Current (dc) resistor creates unique labeling of transitions in the LTS models (e.g.,
and an Alternating Current (ac) resistor. In order to create each transition in the ac resistor’s behavioral model is labeled
an integrated health management environment, a sensor suite with a prefix of “ACres” and each transition in the dc resis-
is designed to enable monitoring of currents, voltages, and tor’s model is labeled with a prefix of “DCres.”
temperatures throughout the circuit. A series of four ac or In order to create the compositional model of the circuit
dc voltage sensors and three current transmitters measure the breaker and ac resistor, an instance of the circuit breaker is
voltage and current at different points throughout the circuit. defined by the use of the command cb:CircuitBreaker that is
This is the Modelica [Tiller, 2001] representation of the EPS composed with the previously defined instance ACres. How-
design. ever, the two LTSs do not have any alphabet in common so
In the EPS testbed, the power source component that is de- no synchronization is possible. For this reason, the binding
noted by a circle may have the operational modes on and off. between the two models is created by the use of the command

Systems Engineering DOI 10.1002/sys

 COMPLEX ENGINEERED SYSTEMS DESIGN VERIFICATION 467

Functional Model
System Model
Database of
Failure Modes

Structural Model
Component
(Chosen Design)
Behavioral Model

LTSA Verification Tool

FSP

LTS

Automated Model Checking

(Assume-Guarantee Reasoning)

No Does Design Yes

Meet Its
Requirements
?

1. Generate Assume-guarantee Pair 1. Generate Assume-guarantee Pair

2. Generate Failure Propagation Paths

Figure 4. An overview of the proposed verification process.

ACres.inflow/cb.outflow. The binding leads to a model where 1. A functional model is generated as a representation of
the circuit breaker’s output current is recognized as the ac re- all the necessary functions that the system must contain
sistor’s input current. As a result, the new label ACres.inflow, in order to meet the design requirements is used to create
is substituted with the old label, cb.outflow. With the bind- the structural model of the system.
ing command in place, the synchronization takes place 2. The structural model is used as a resource to gather
and the LTSs components have a common action to information about the different system components and
communicate. the EMS flow relationship between them to create a
In our context, properties are modeled as safety LTSs. A database of their failure states and safety properties and
safety LTS is an LTS that contains no failure states. When the behavioral models.
checking a property P, an error LTS denoted Perr is created, 3. Once the behavioral models of the component are devel-
which identifies possible violations with the failure state. oped, they are incorporated into the FSPs by mapping
the behavior models with their respective FSPs’ transi-
tions.
3.5. Verification Process 4. The LTS of the system is automatically constructed
The contribution of this paper is developing an automated from the FSP models that are developed by design en-
design verification framework to prove the correctness of the gineers.
complex engineered system design with regards to its func- 5. The LTSs are incrementally analyzed and abstracted
tional and safety properties. The proposed framework pro- through the use of a reachability graph to determine
vides information on the property violation of the composed whether the safety property is violated.
components during conceptual design, while identifying the 6. If the failure state of the component is reached, then the
failure propagation behavior. The automatic generation of failure propagation path is provided.
failure propagation paths enables the system designers to bet-
ter address the safety issues in the design. Figure 4 depicts The AGR paradigm requires exact identification of the
the relationship between system models and the verification component properties, which in this case are defined based
framework that either provides the proof of correctness or on the failed states, of the components and (sub)systems. In
failure propagation information. order to identify the failed states the effects of incoming EMS
The main steps to apply the AGR framework are summa- flows on the operation of the components is analyzed and
rized as follows: two generic states of nominal and failed are defined. The

Systems Engineering DOI 10.1002/sys

468 MEHRPOUYAN ET AL.

Figure 5. The algorithm that generates assumptions.

component’s state is recognized as nominal when a com- safety property P on a system with two components M1 and
ponent is operating with the performance and functionality M2 is defined as
intended by the system designer. On the other hand, failure Rule ASym:
state is defined as a component functioning in a way that was
not intended by the designer. 1 ∶ ⟨A⟩ M1 ⟨P⟩
2 ∶ ⟨tr ue⟩ M2 ⟨A⟩
⟨tr ue⟩ M1 ∥ M2 ⟨P⟩

3.6. Automated Model Checker Based on AGR In this rule, A denotes an assumption about the environment
of M1 . Note that the rule is not symmetric in its use of the
In the assume-guarantee paradigm, the formula is a triplet two components and does not support circularity. Despite its
⟨A ⟩M ⟨P⟩, where M is a component, P is a property, and simplicity, experience has shown it to be quite useful in the
A is an assumption about M’s environment. The formula is context of checking safety properties.
true if whenever M is part of a system satisfying A, then the The objective is to automatically generate the weakest as-
system must also guarantee P. sumptions for components and their compositions, so that the
Let M be a finite-state component (Table I) with Σ being assume-guarantee rule is derived in an incremental manner.
the set of its interaction points with the environment, and let In this research, the algorithm in Giannakopoulou, Pasare-
P be a safety property. Then there is a natural notion of the anu, and Cobleigh [2004] is used to generate approximate
weakest assumption Aw , such that ⟨Aw ⟩ M ⟨P⟩ holds, where assumptions and guarantees at the component, subsystem,
Aw = Σ. Aw characterizes all possible environments E under and system level. Figure 5 depicts the algorithm which is
which the property holds. based on model checking and machine learning techniques
It has been shown that, for any finite-state component M, to construct an initial assumption derived from component’s
the weakest assumption Aw exists and can be constructed behaviorial models. Model checking is used to make sure
algorithmically [Giannakopoulou et al., 2002]. An ideal Aw the generated assumption is safe and permissive based on
should precisely represent the component in all its intended the assume-guarantee rule. The first step of the algorithm is
usages. It should be safe, meaning that it should exclude to check for the first part of the Rule ASym: ⟨A⟩ M1 ⟨P⟩.
all problematic interactions, and permissive, in that it should If it is violated, it means that the assumption is too weak
include all the good interactions. so it does not prevent M1 from reaching its failure state.
A weakest assumption is generated so that it is both safe Based on the generated failure propagation path, the algo-
and permissive. Safety in this context means to restrict the be- rithm creates a new assumption which is stronger than the
haviors of the component to those that satisfy P. Permissive- previous one. The iteration continues until the first rule of
ness, on the other hand is concerned with restricting behaviors ⟨A⟩ M1 ⟨P⟩ is addressed. The next step is to check the second
only if necessary. Permissiveness is desirable, because Aw is rule ⟨tr ue⟩ M2 ⟨A⟩. If the rule holds then it is concluded that
then appropriate for deciding whether an environment E is ⟨tr ue⟩ M1 ∥ M2 ⟨P⟩, otherwise the failure propagation path
suitable for M (if E does not satisfy Aw , then E ∥ M does not is generated to provide the reason why component M2 is not
satisfy P). The simplest assume guarantee rule for checking a able to guarantee A . Then the counterexample is analyzed

Systems Engineering DOI 10.1002/sys

 COMPLEX ENGINEERED SYSTEMS DESIGN VERIFICATION 469

Sense Sense Sense Sense Actuate Convert

Voltage Voltage Voltage Voltage Elec. E Elec. E

Store Supply Actuate Actuate Actuate Condition Actuate Actuate Convert

Elec. E Elec. E Elec. E Elec. E Elec. E Elec. E Elec. E Elec. E Elec. E

Sense Sense Sense Actuate Convert

Current Current Current Elec. E Elec. E

Figure 6. Functional model of the electrical power system.

Table II. Composition of the EPS components Kramer, 1999] of a software system based on its architecture.
In general, the conceptual design of a complex engineered
EPS System Compositional Model system has a hierarchical structure and is modular [Woodard,
∥ Module1 = (acRes:(Vulnerable ∥ Perr )). 2006]. CRA incrementally computes and abstracts the be-
∥ Module2 = (fan:(Vulnerable ∥ Perr )). havior of composite components based on the behavior of
∥ Module3 = (dcRes:(Vulnerable ∥ Perr )). their immediate children in the hierarchy. The input language
∥ Module4 = (rel272:Relay)/acRes.inflow/rel272.outflow. “FSP” of the tool is a process-algebra style notation with LTS
∥ Module5 = (rel275:Relay)/fan.inflow/rel275.outflow. semantics.
∥ Module6 = (cs267:CurrentSensor)/rel272.inflow/cs267.outflow. A property is also expressed as an LTS, but with ex-
∥ Module7 = (cs267:CurrentSensor)/rel275.inflow/cs267.outflow. tended semantics, and is treated as an ordinary component
∥ Module8 = (vm256:VoltMeter)/cs267.inflow/vm256.outflow. during composition. Properties are combined with the com-
∥ Module9 = (inv2:Inverter)/vm256.inflow/inv2.outflow. ponents to which they refer. They do not interfere with
∥ Module10 = (rel284:Relay)/dcRes.inflow/rel284.outflow. system behavior unless they are violated. In the presence
∥ Module11 = (vm281:VoltMeter)/rel284.inflow/vm281.outflow. of violations, the properties introduced may reduce the
∥ Module12 = (cs281:CurrentSensor)/vm281.inflow/cs281.outflow. state space of the (sub)systems analyzed. As in our ap-
∥ Module13 = (vm242:VoltMeter)/inv2.inflow/vm242.outflow. proach, the LTSA framework treats components as open
∥ Module14 = (vm242:VoltMeter)/cs281.inflow/vm242.outflow. systems that may only satisfy some requirements in spe-
∥ Module15 = (rel244:Relay)/vm242.inflow/rel244.outflow. cific contexts. By composing components with their proper-
∥ Module16 = (vm240:VoltMeter)/rel244.inflow/vm240.outflow. ties, it postpones analysis until the system is closed, mean-
∥ Module17 = (cs240:CurrentSensor)/vm240.inflow/cs240.outflow. ing that all contextual behavior that is applicable has been
∥ Module18 = (bat2:Battery)/cs240.inflow/bat2.outflow. provided.

to realize if component M1 reaches its failure state or not. If 4. CASE STUDY

M1 does not reach its failure states then the assumption is too
strong and should be weakened and tested again. Otherwise, Continuing with the EPS design concept, the development of
it is concluded that the composition of M1 ∥ M2 violates LTSs implies direct mapping between the functional model
property P. (Fig. 6) and the structural architecture of the system, in
which specific component or (sub)system is selected to im-
plement the functional requirements in the actual system
3.6.1. Soundness and Completeness design.
Soundness of an assume-guarantee rule means that when- In order to construct the LTS model of the EPS design, all
ever its premises hold, its conclusion holds as well. Without internal state transitions of the components are presented in
soundness, we cannot rely on the correctness of conclusions the FSP language. The constant variables(factors that do not
reached by applications of the rule, which makes the rule use- change during the course of this experiment) and ranges are
less for verification. On the other hand, completeness states defined as follows:
that whenever the conclusion of the rule is correct, the rule
is applicable, that is, there exist suitable assumptions such
that the premises of the rule hold. While completeness is 1. const Low = 0
not needed to ensure correctness of proofs obtained by the 2. const Medium = 1
rule, it is important as a measure for the usability of the 3. const Spike = 2
rule [Giannakopoulou et al., 2002]. 4. const Open = 1
5. const Close = 0
6. range CUR = Low..Spike
3.7. The LTSA Tool
LTSA [Magee, Kramer, and Giannakopoulou, 1999; Magree, The notations 0, 1, and 2 are used to denote low, medium,
1999] is an automated tool that supports CRA [Cheung and and spike currents in the EPS components, respectively.

Systems Engineering DOI 10.1002/sys

470 MEHRPOUYAN ET AL.

Figure 7. Parallel composition in LTS format.

Figure 8. Assume-guarantee pair for the EPS design of Figure 3.

4.1. Primitive Components and Properties is in the range of low or medium and the resulting output
current also in the low or medium range; 2—the current inflow
As depicted in Table I, all internal state transitions of the prim- to the ac resistor is spiking which results in the state of “burn.”
itive components are presented in the FSP language. Each If the resistor is in the state of burn, no matter what input
component’s nominal behavioral model is incorporated into current level to the ac resistor, there is no output current past
the FSP code, therefore the resulting model contains no failure the ac resistor.
state. For example, while the ac resistor is in the operational The failure modes of components are represented by Perr .
mode (Table I), two transitions are possible, which are speci- The error LTSs are constructed to represent all the faulty
fied using the “OR” logical operator “|”. These two transitions transitions that lead to failure states. In order to model the
are defined as: 1—the current input into the ac resistor which failure mode of the three vulnerable components discussed in

Systems Engineering DOI 10.1002/sys

 COMPLEX ENGINEERED SYSTEMS DESIGN VERIFICATION 471

Figure 9. Model of the EPS with circuit breaker.

Table III. FSP Code for Circuit Breaker.

Circuit Breaker
Component Mode LTS Model
Circuit Breaker nominal (inflow[v:CUR] → if ( v < Spike) then (outflow[v] → CircuitBreaker) else CircuitBreaker) +
{outflow[Spike]}

Table IV. Composition of the EPS Components Table V. Generated Assume-Guarantee Pair of Figure 9

EPS System Compositional Model For EPS Design of Figure 9

Component Assumption Guarantee
∥ Module1 = (acRes:(Vulnerable ∥ Perr )).
∥ Module2 = (fan:(Vulnerable ∥ Perr )). Battery [0,1,2] [0,1,2]
∥ Module3 = (dcRes:(Vulnerable ∥ Perr )). 𝐶𝑖𝑟𝑐𝑢𝑖𝑡 𝐵𝑟𝑒𝑎𝑘𝑒𝑟236 [0,1,2] [0,1]
∥ Module4 = (rel272:Relay)/acRes.inflow/rel272.outflow. 𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟240 [0,1,2] [0,1,2]
∥ Module5 = (rel275:Relay)/fan.inflow/rel275.outflow. 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑆𝑒𝑛𝑠𝑜𝑟240 [0,1,2] [0,1,2]
∥ Module6 = (cs267:CurrentSensor)/rel272.inflow/cs267.outflow. 𝑅𝑒𝑙𝑎𝑦244 [0,1,2] [0,1,2]
∥ Module7 = (cs267:CurrentSensor)/rel275.inflow/cs267.outflow. 𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟242 [0,1,2] [0,1,2]
∥ Module8 = (cb266:CircuitBreaker)/cs267.inflow/cb266.outflow. 𝐶𝑖𝑟𝑐𝑢𝑖𝑡 𝐵𝑟𝑒𝑎𝑘𝑒𝑟262 [0,1,2] [0,1]
∥ Module9 = (vm256:VoltMeter)/cb266.inflow/vm256.outflow. 𝐼𝑛𝑣𝑒𝑟𝑡𝑜𝑟2 [0,1,2] [0,1,2]
∥ Module10 = (inv2:Inverter)/vm256.inflow/inv2.outflow. 𝐶𝑖𝑟𝑐𝑢𝑖𝑡 𝐵𝑟𝑒𝑎𝑘𝑒𝑟266 [0,1,2] [0,1]
∥ Module11 = (cb262:CircuitBreaker)/inv2.inflow/cb262.outflow. 𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟265 [0,1] [0,1]
∥ Module12 = (rel284:Relay)/dcRes.inflow/rel284.outflow. 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑡𝑒𝑥𝑡𝑆𝑒𝑛𝑠𝑜𝑟267 [0,1] [0,1]
∥ Module13 = (vm281:VoltMeter)/rel284.inflow/vm281.outflow. 𝑅𝑒𝑙𝑎𝑦272 [0,1] [0,1]
∥ Module14 = (cs281:CurrentSensor)/vm281.inflow/cs281.outflow. 𝑅𝑒𝑙𝑎𝑦275 [0,1] [0,1]
∥ Module15 = (cb280:CircuitBreaker)/cs281.inflow/cb280.outflow. 𝐴𝐶𝑅𝑒𝑠𝑖𝑠𝑡𝑜𝑟 [0,1] [0,1]
∥ Module16 = (vm242:VoltMeter)/cb262.inflow/vm242.outflow. 𝐹 𝑎𝑛 [0,1] [0,1]
∥ Module17 = (vm242:VoltMeter)/cb280.inflow/vm242.outflow. 𝐶𝑖𝑟𝑐𝑢𝑖𝑡 𝐵𝑟𝑒𝑎𝑘𝑒𝑟280 [0,1,2] [0,1]
∥ Module18 = (rel244:Relay)/vm242.inflow/rel244.outflow. 𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑆𝑒𝑛𝑠𝑜𝑟281 [0,1] [0,1]
∥ Module19 = (vm240:VoltMeter)/rel244.inflow/vm240.outflow. 𝑅𝑒𝑙𝑎𝑦284 [0,1] [0,1]
∥ Module20 = (cs240:CurrentSensor)/vm240.inflow/cs240.outflow. 𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟281 [0,1] [0,1]
∥ Module21 = (cb236:CircuitBreaker)/cs240.inflow/cb236.outflow. 𝐷𝐶𝑅𝑒𝑠𝑖𝑠𝑡𝑜𝑟 [0,1] [0,1]
∥ Module22 = (bat2:Battery)/cb236.inflow/bat2.outflow.
with its property that is reached by the illegal transition of
acres.burn.
design architecture of Figure 3, a generic property named Perr
is defined for all three components as below: 4.2. Compositional Model
property Perr = STOP + burn.
In order to create the compositional model of the EPS sys-
The LTSA tool represents failure state by −1 as it is depicted tem, the order of compositions is decided based on the
in Figure 2 in the compositional model of the ac resistor functional model of the design. Table II represents the

Systems Engineering DOI 10.1002/sys

472 MEHRPOUYAN ET AL.

Table VI. Generated Assume-Guarantee Pair (Remove Un- The result of EPS compositional verification concluded by
necessary Circuit Breakers) the AGR was that the “system and environment are incom-
patible.” The reason for this conclusion is that the designers of
Improved EPS Design the EPS system assumed normal operating condition for the
Component Assumption Guarantee system at all times. In normal condition, all three susceptible
Battery [0,1,2] [0,1,2] components receive nominal voltage and current, while any
𝐴𝐶𝑅𝑒𝑠𝑖𝑠𝑡𝑜𝑟 [0,1] [0,1] variation in load and distribution has an effect on the sys-
𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟240 [0,1,2] [0,1,2] tem. Therefore, the analyzed design is not considered fault
𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑆𝑒𝑛𝑠𝑜𝑟240 [0,1,2] [0,1,2] tolerant.
𝑅𝑒𝑙𝑎𝑦244 [0,1,2] [0,1,2] In addition to verifying the desired properties of the system
𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟242 [0,1,2] [0,1,2] design, the proposed methodology automatically computes
𝐼𝑛𝑣𝑒𝑟𝑡𝑜𝑟2 [0,1,2] [0,1,2] the required assume-guarantee pair for each component in the
𝐶𝑖𝑟𝑐𝑢𝑖𝑡 𝐵𝑟𝑒𝑎𝑘𝑒𝑟266 [0,1,2] [0,1] design to prove the global properties of the design under con-
𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟265 [0,1,2] [0,1,2] sideration. There are cases where no assume-guarantee pair
𝑅𝑒𝑙𝑎𝑦272 [0,1] [0,1] is generated by the verification algorithm because there is no
𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑆𝑒𝑛𝑠𝑜𝑟267 [0,1] [0,1] environment in which the design can be implemented safely.
𝑅𝑒𝑙𝑎𝑦275 [0,1] [0,1] Figure 8 represents the assume-guarantee pairs generated by
𝐹 𝑎𝑛 [0,1] [0,1] the AGR for each system element in the design, which implies
𝐶𝑖𝑟𝑐𝑢𝑖𝑡 𝐵𝑟𝑒𝑎𝑘𝑒𝑟280 [0,1,2] [0,1] that each component guarantees to output current flow of low
𝐶𝑢𝑟𝑟𝑒𝑛𝑡 𝑆𝑒𝑛𝑠𝑜𝑟281 [0,1] [0,1] or medium (0 or 1) iff they receive current inflow of low or
𝑅𝑒𝑙𝑎𝑦284 [0,1] [0,1] medium.
𝑉 𝑜𝑙𝑡 𝑀𝑒𝑡𝑒𝑟281 [0,1] [0,1] In the case of detecting safety violation in the sys-
𝐷𝐶𝑅𝑒𝑠𝑖𝑠𝑡𝑜𝑟 [0,1] [0,1] tem design, the verification framework returns a counterex-
ample, which provides information of the failure prop-
agation path. Only one counterexample is necessary to
compositional model of the EPS system for two types of prove that the design violates its properties. In the case
components. Those components that operate in nominal mode of EPS design, the failure propagation path starts from
such as “battery” and “curr entsensor240 ” in module18 are “battery” propagating through different components such as
composed by the creation of binding between them. The 𝐶𝑢𝑟𝑟𝑒𝑛𝑡𝑆𝑒𝑛𝑠𝑜𝑟240 , 𝑉 𝑜𝑙𝑡𝑀𝑒𝑡𝑒𝑟240 , 𝑅𝑒𝑙𝑎𝑦244 , 𝑉 𝑜𝑙𝑡𝑀𝑒𝑡𝑒𝑟242 ,
binding is modeled by the EMS flow between the two com- 𝐼𝑛𝑣𝑒𝑟𝑡𝑒𝑟2 , 𝑉 𝑜𝑙𝑡𝑀𝑒𝑡𝑒𝑟256 , 𝐶𝑢𝑟𝑟𝑒𝑛𝑡𝑆𝑒𝑛𝑠𝑜𝑟267 , 𝑅𝑒𝑙𝑎𝑦272 and
ponents which is represented by the use of the command reaching the “ac resistor” causing it to burn. The “inflow.2”
cs240 .in f low∕bat2 .out f low. The binding leads to the com- represents the existence of the spike in the incoming and
positional model where the battery’s output flow is recognized outgoing current flow in each component.
as curr entsensor240 ’s input flow. The second types of com-
ponents are those with failure states, which are required to
be composed with their defined properties before they can be 4.4. Design Based on the Result of Verification
considered for composition with other components, that is,
module1 through module3 in Table II. In order to correct the design flaws mentioned above, it is
required to add circuit breakers to the design as modeled in
Figure 9 to prevent the spike reaching the three vulnerable
components. The circles highlight the circuit breakers used to
4.3. Compositional Verification protect the ac resistor, fan, and dc resistor from an overcurrent
In order to verify the properties of the EPS system, the LTSA spike.
“compositional” algorithm is used. This algorithm imple- The operation of the circuit breaker is similar to that of
ments AGR in a learning framework to prove that the prop- an electrical switch, which is designed to protect an electri-
erties are satisfied or violated. The advantage of using model cal circuit from damage caused by overload or short circuit
checking and automata learning algorithm is its ability to per- (Table III). Therefore, the integration of circuit breakers in
form CRA in an exhaustive manner to search for violations of the design architecture prevents the resistors and fan from
design properties. In addition, the LTSA algorithm uses a spe- burning. The following equation represents the AGR rule of
cific form of learning algorithm based on minimization and triple type:
abstraction, which dramatically reduces the number of state
spaces required for analysis. For example, if the two modules 1. < {0..1} > 𝐴𝐶𝑅𝑒𝑠𝑖𝑠𝑡𝑜𝑟 < 𝐍𝐨 Burn >
of the EPS LTSs, for example, module18 and module17 are 2. < 𝑡𝑟𝑢𝑒 > 𝐶𝑖𝑟𝑐𝑢𝑖𝑡𝐵𝑟𝑒𝑎𝑘𝑒𝑟 < {0..1} >
analyzed in a monolithic manner (∥ Test = (Module18 ∥
Module17 ) the state space of this composition results in 16 < true > AC Resistor ∥ Circuit Breaker <No Burn>
states with 27 transitions as illustrated in Figure 7. Eventually,
the full monolithic composition of the EPS design results in (1) <–0..1˝> AC resistor <No Burn> is proven correct
approximately 232 × 109 states, however, with the proposed if circuit breaker satisfies the assumption that inflow current
method in this paper, the compositional analysis is completed to the ac resistor is always {0..1}, resulting in guaranteeing
in 2 seconds. property No Burn.

Systems Engineering DOI 10.1002/sys

 COMPLEX ENGINEERED SYSTEMS DESIGN VERIFICATION 473

Figure 10. Verification time of monolithic composition versus AGR approach.

The compositional model of the modified design is repre- tion approach with a monolithical approach (Fig. 10) which
sented in Table IV. The full monolithic composition of the represents linear growth for the proposed verification process,
modified EPS design results in 19 × 1012 states, however, while an exponential growth is predicted for the monolithical
with the proposed method in this paper, the compositional verification process.
analysis is completed in 2.5 seconds. The result of verification
is successful, implying that the “system and environment are
compatible” and all the design safety requirements are met.
It is important to note that the generated assume-guarantee 5. CONCLUSION AND FUTURE WORK
pairs, as depicted in Table V, restrict the current-inflow of
low and medium, [0, 1], for a smaller number of components, This work provides a framework for the effective use of for-
compared to the EPS design without the circuit breakers. The mal methods in the early verification of safety requirements in
reason for this is that the cir cuitbr eaker266 detects a fault safety critical systems. This is especially important in proving
condition and interrupt current flow from reaching the two the correctness of the system design, where it is critical to
vulnerable components (ac resistor and fan) in the top branch, guarantee that the known interactions between system com-
while the cir cuitebr eaker280 protects the lower branch. ponents do not violate any safety properties. With regard to
Therefore, any components before these two circuit break- the design of complex systems, high-level system require-
ers can accept the inflow current of low, medium, and spike. ments are decomposed into component and (sub)system re-
This is a good indication of the weakest assumptions gen- quirements which logically map to the architectural decompo-
erated by the proposed framework, guiding the designers in sition of the system. Therefore, proof of correctness through
their understanding of the design requirements. Based on the preverification of system components and compositional rea-
generated assumptions, the two circuit breakers (236 and 262) soning is made possible. The aim of compositional reasoning
are not required and therefore can be eliminated from the is to improve scalability of the design verification problem by
design. decomposing the original verification task into subproblems.
The second alternative design architecture of the EPS sys- The simplification is based on the AGR that results in approxi-
tem with removed circuit breakers has been verified as a mating the requirements which a component and (sub)system
safe design. Table VI illustrates the assume-guarantee pair places on its operational environment to satisfy safety prop-
generated for each design component. erties. The case study of the EPS design demonstrated the ca-
As illustrated, the generated assume-guarantee pair is crit- pability of the proposed verification methodology to perform
ical in choosing alternative design solutions and migrating virtual integration of system elements and proving system-
between different design architectures with relatively small level requirements from the constraints allocated on the com-
effort. In addition, the generated assume-guarantee pair en- ponents. As a result, a class of design flaws has been uncov-
ables the system designers to trace the requirements through- ered because of an integration failure that occurs when system
out the design architecture. The behavior of the system de- components satisfy their requirements in isolation but not at
sign is described in an assume-guarantee style specification: the system level.
a component guarantees certain set of behaviors, given that The proposed approach models the behavior of composite
its environment follows certain assumptions. components using LTS models of the primitive components
In addition, a performance case study is conducted by and their safety properties, which are based on the struc-
comparing the performance results of the proposed verifica- tural model provided by the Modelica model of the system

Systems Engineering DOI 10.1002/sys

474 MEHRPOUYAN ET AL.

design. In addition, a fully automated compositional verifi- L. Baresi, D. Bianculli, C. Ghezzi, S. Guinea, and P. Spoletini,
cation technique is used to determine the correctness of the Validation of web service compositions, IET Softw 1(6) (2007)
design with regards to its requirement and generate pairs of 219–232.
assume-guarantee using a learning algorithm. Experimental J.T. Buck, S. Ha, E.A. Lee, and D.G. Messerschmitt, Ptolemy: A
results showed the effectiveness of the compositional reason- framework for simulating and prototyping heterogeneous sys-
ing approach in reducing the complexity of the verification tems, Int. J of Comput Simulation, special issue on “Simulation
process by using modularity and abstraction. In addition, Software Development” vol.4, pp. 155–182, April, 1994.
we showed how a deductive verification tool such as LTSA N. Siu, “Risk assessment for dynamic systems: an overview”. Reli-
combined with LTS models can be used for verification of ability Engineering & System Safety. 1994 Dec 31;43(1):43–73.
finite-state hardware system designs. The compositional ver- M. Čepin, “Reliability block diagram,” in Assessment of Power
ification helps in breaking a large complex system design System Reliability, Springer London, 2011, pp. 119–123.
into smaller parts whose verification can be checked in order S. Chaki, E. Clarke, N. Sinha, and P. Thati, “Automated assume-
to prove that the safety property of the components and the guarantee reasoning for simulation conformance,” in Computer
(sub)system holds. The assume-guarantee approach which is Aided Verification, Springer, New York, NY, USA, 2005, pp.
based on a learning algorithm [Giannakopoulou et al., 2002], 241–246.
produces and refines assumptions depending on failure propa- S.C. Cheung and J. Kramer, Checking safety properties using
gation paths and queries, the verification process is assured to compositional reachability analysis, ACM Trans Softw Eng
terminate. In addition, the algorithm returns counterexamples Methodol 8(1) (1999), 49–78.
which include failure propagation information in the early E.M. Clarke, O. Grumberg, and D. Peled, Model checking, MIT
stages of conceptual design. Another advantage of the pro- Press, 1999.
posed approach for verification of engineered systems is its J. Cobleigh, D. Giannakopoulou, and C. Păsăreanu, “Learning as-
independence from human intervention and expert user in de- sumptions for compositional verification,” International Confer-
vising the appropriate assume-guarantee pair. The experiment ence on Tools and Algorithms for the Construction and Analysis
in this paper provided strong evidence in favor of this line of of Systems. Springer Berlin Heidelberg, 2003.
research. B. Dhillon and A. Fashandi, Safety and reliability assessment tech-
It must be noted that state explosion is an inherent lim- niques in robotics, Robotica 15(6) (1997), 701–708.
itation of model checking, therefore no single technique is D. Do, Procedure for performing a failure mode, effects and critical-
expected to be efficient for all kinds of systems. This is also ity analysis, 1980.
reflected by the fact that most of the existing model-checking H.A. ElMaraghy, “Changing and evolving products and systems–
tools support several approaches to model checking. The fu- models and enablers,” in Changeable and Reconfigurable Manu-
ture work will concentrate to build experience in order to facturing Systems, Springer, London, 2009, pp. 25–45.
determine what types of analyses are appropriate for what C.A. Ericson and C. Ll, “Fault tree analysis,” System Safety Confer-
kinds of systems. Specifically, we will focus on the design ence, Orlando, Florida. 1999.
verification and analysis of complex engineered systems with P.H. Feiler, D.P. Gluch, and J.J. Hudak, The architecture analysis
software controlling the hardware. The design of such sys- & design language (AADL): An introduction, Technical report,
tems requires collaboration between experts from different DTIC Document, 2006.
design domains to formally define their interacting behavior H. Foster, Prologue: The 2012 Wilson Research Group functional
and verify conflicting requirements or objectives. verification study, American Institute of Aeronautics & Astro-
In addition, future work expands the approach discussed in nautics, 2013.
this paper to examine the learning algorithm and its generated S. Friedenthal, A. Moore, and R. Steiner, A practical guide to
assumptions to determine the most reliable design architec- SysML: The systems modeling language, Elsevier, 2011.
ture of the redundant systems. It is our goal to investigate P. Fritzson, Principles of object-oriented modeling and simulation
different aspects of fault tolerant system design requirements with Modelica 2.1, John Wiley & Sons, 2010.
while taking into account automatic injection of multiple fail- H. Garavel, M. Sighireanu, et al., A graphical parallel composition
ures and reasoning about different types of recovery strate- operator for process algebras, Proceedings of International Con-
gies. As a result, the existing verification technique is required ference on Formal Description Techniques for Distributed Sys-
to be modified to include systems that exhibit probabilistic tems and Communication Protocols, and Protocol Specification,
behavior. The approach will be based on the multiobjective Testing, and Verification FORTE/PSTV, vol. 99, Citeseer, 1999,
probabilistic model checking. Properties of these models are pp. 185–202.
formally defined as probabilistic safety properties. D. Giannakopoulou, C.S. Pasareanu, and H. Barringer, Assumption
generation for software component verification, ASE ’02: Pro-
ceedings of the 17th IEEE International Conference on Auto-
REFERENCES mated Software Engineering, IEEE, 2002, pp. 3–12.
D. Giannakopoulou, C.S. Pasareanu, and J.M. Cobleigh, Assume-
R. Alur, P. Madhusudan, and W. Nam, “Symbolic compositional guarantee verification of source code with design-level assump-
verification by learning assumptions,” in Computer Aided Ver- tions, Proceedings of the 26th International Conference on Soft-
ification, Springer Berlin Heidelberg, 2005, pp. 289–292. ware Engineering, IEEE Computer Society, 2004, pp. 211–220.
C. Baier and J.-P. Katoen, and K.G. Larsen, Principles of model W.F. Gilreath, Concurrency state models and Java programs, Scal-
checking, MIT Press, 2008. able Comput Pract Exp 3(4), 2001.

Systems Engineering DOI 10.1002/sys

 COMPLEX ENGINEERED SYSTEMS DESIGN VERIFICATION 475

T.A. Henzinger, M. Minea, and V. Prabhu, “Assume-guarantee rea- Conferences and Computers and Information in Engineering
soning for hierarchical hybrid systems,” in Hybrid Systems: Conference, 2012.
Computation and Control, Springer Berlin Heidelberg, 2001, pp. K. Otto and K. Wood, Product design: Techniques in reverse engi-
275–290. neering, systematic design, and new product development, Prov-
T. Henzinger and J. Sifakis, The embedded systems design chal- idence, Rhode Island IEEE Computer Society, 2001.
lenge, in International Symposium on Formal Methods, Springer A. Pnueli, The temporal logic of programs, Proceedings of 18th
Berlin Heidelberg, 2006, pp. 1–15. Annual Symposium on Foundations of Computer Science, IEEE,
J. Hirtz, R.B. Stone, D.A. McAdams, S. Szykman, and K.L. Wood, A 1977, pp. 46–57.
functional basis for engineering design: Reconciling and evolv- S. Poll, A. Patterson-Hine, J. Camisa, D. Garcia, D. Hall, C. Lee,
ing previous efforts, Res Eng Des 13(2) (2002), 65–82. O.J. Mengshoel, C. Neukom, D. Nishikawa, J. Ossenfort, et al.,
C.I.R. Inc., 2002 IC/ASIC functional verification study, 2002. Advanced diagnostics and prognostics testbed, Proceedings of
R. Kazhamiakin, M. Pistore, and M. Roveri, A framework the 18th International Workshop on Principles of Diagnosis (DX-
for integrating business processes and business requirements, 07), 2007, pp. 178–185.
EDOC ’04: Proceedings of Eighth IEEE International Enter- R.W. Rodrigues, Formalising UML activity diagrams using finite
prise Distributed Object Computing Conference, IEEE, 2004, state processes, Proceedings of the 3rd Intl. Conf. on the Unified
pp. 9–20. Modeling Language, York, UK, Citeseer, 2000.
T. Kurtoglu, M. Campbell, C. Bryant, R. Stone, and D. McAdams, H.E. Roland and B. Moriarty, “Preliminary hazard analysis,” System
Deriving a component basis for computational functional synthe- Safety Engineering and Management, 2nd edition, John Wiley &
sis, ICED: Proceedings of International Conference on Engineer- Sons, Inc., 2009, pp. 206–212.
ing Design, vol. 5, 2005, pp. 15–18. B. Saporito, How Safe is the Boeing 787? http://business.time.
T. Kurtoglu and I.Y. Tumer, FFIP: A framework for early assess- com/2013/01/11/how-safe-is-the-boeing-787/, January 2013.
ment of functional failures in complex systems, Proceedings of M. Simulink and M. Natick, The Mathworks, Inc., Natick, MA,
International Conference on Engineering Design, 2007. 1993.
T. Kurtoglu, I.Y. Tumer, and D.C. Jensen, A functional failure rea- R.B. Stone and K.L. Wood, Development of a functional basis for
soning methodology for evaluation of conceptual system archi- design, J Mech Des 122 (2000), 359–370.
tectures, Res Eng Des 21(4) (2010), 209–234. M. Tiller, Introduction to physical modeling with Modelica, vol. 615,
J. Magee, J. Kramer, and D. Giannakopoulou, Behaviour analysis of Kluwer Academic Publishers, 2001.
software architectures, Softw Archit 12 (1999), 35–49. I.Y. Tumer and R.B. Stone, Analytical method for mapping function
J. Magree, Behavioral analysis of software architectures using to failure during high-risk component development, Proceedings
LTSA, Proceedings of the 1999 International Conference on Soft- of the Design Engineering Technical Conferences, 2001.
ware Engineering, IEEE, 1999, pp. 634–637. E.Y. Wang and B.H. Cheng, Formalizing the functional model within
H. Mehrpouyan, D.C. Jensen, C. Hoyle, I.Y. Tumer, and T. Kurtgolu, object-oriented design, Int J Softw Eng Knowl Eng 10(1) (2000),
A model-based failure identification and propagation frame- 5–30.
work for conceptual design of complex systems, Proceedings C.J. Woodard, Architectural strategy and design evolution in com-
of the ASME 2012 International Design Engineering Technical plex engineered systems, PhD thesis, Citeseer, 2006.

Hoda Mehrpouyan is currently an Assistant Professor in the Computer Science Department at Boise State University. She received her PhD
from Oregon State University in Mechanical Engineering in 2014 and her MS degree in software engineering and management from Linköping
University in 2011. She spent 7 years in industry as a System Delivery Consultant and Programmer Analyst prior to returning to academia. Her
research focuses on model-based systems engineering, resilience and safety analysis, information technology, and simulation and verification
to support the design of complex systems.

Dimitra Giannakopoulou is a Research Computer Scientist with the NASA Ames Research Center, and a member of the Robust Software
Engineering Group. Her work is concerned with applying modular and compositional formal verification techniques to autonomous systems
and architectures. Before joining Ames, she was a Research Associate with the Department of Computing, Imperial College, University of
London, UK, working on methods for the specification and automatic verification of distributed systems. She has graduated from the Dept of
Computer Engineering and Informatics, University of Patras, Greece. She holds an MSc with distinction from Imperial College, in “Foundations
of Advanced Information Technology”, and since March 1999, a PhD degree from Imperial College, University of London, thesis title “Model
Checking for Concurrent Software Architectures”.

Guillaume Brat is employed by Carnegie-Mellon University and he conduct research in software verification within the Robust Software
Engineering group in the Intelligent Systems Division at NASA Ames. He received an M.Sc. and Ph.D. from the ECE Departmentat The
University of Texas at Austin. He is Principal Systems Scientist at CMU Silicon Valley serving as an IPA at NASA Ames Research Center.
He has been the Assistant Area lead for Robust Software Engineering since October 2009. The group conducts research on new verification
and validation techniques, mostly based on formal methods. He is also Project Scientist for the System-wide Safety Assurance Technologies
(SSAT) project in the Aviation Safety program in ARMD. He is focusing on the largest theme of the project, the Assurance of Flight Critical
Systems (AFCS).

Systems Engineering DOI 10.1002/sys

476 MEHRPOUYAN ET AL.

Irem Tumer is a Professor of mechanical, industrial, and manufacturing engineering at Oregon State University, where she leads the Complex
Engineered System Design Laboratory, and Associate Dean for Research and Economic Development for the College of Engineering. Prior to
accepting a faculty position at Oregon State University, Dr. Tumer led the Complex Systems Design and Engineering group in the Intelligent
Systems Division at NASA Ames Research Center, where she worked from 1998 through 2006 as Research Scientist, Group Lead, and
Program Manager. She received her PhD in mechanical engineering from the University of Texas at Austin in 1998. Since moving to Oregon
State University in 2006, her funding has largely been through NSF, AFOSR, DARPA, and NASA.

Chris Hoyle is currently an Assistant Professor and Arthur Hitsman Faculty Scholar in design in the Mechanical Engineering Department at
Oregon State University. He received his PhD from Northwestern University in Mechanical Engineering in 2009 and his master’s degree in
mechanical engineering from Purdue University in 1994. He was previously a Design Engineer and an Engineering Manager at Motorola,
Inc., for 10 years before enrolling in the PhD program. His current research interests are focused upon decision making in engineering
design, with emphasis on the early design phase. His research contributions are to the field of decision-based design, specifically in linking
consumer preferences and enterprise-level objectives with the engineering design process. His areas of expertise are uncertainty propagation
methodologies, Bayesian statistics and modeling, stochastic consumer choice modeling, optimization, and design automation. He is coauthor
of the book Decision-Based Design: Integrating Consumer Preferences into Engineering Design.

Systems Engineering DOI 10.1002/sys

Copyright of Systems Engineering is the property of John Wiley & Sons, Inc. and its content
may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for
individual use.