Scribd
Upload
140 views

Module 5 Verification and Validation of Simulation Models

This document discusses verification and validation of simulation models. It defines verification as ensuring the conceptual model is accurately implemented in the computer program, while validation determines if the correct model is built by calibrating it with data. The key steps are: 1) Observe the real system and collect data on its behavior and components. 2) Develop a conceptual model and then implement it as an operational simulation model. 3) Verify the model by checking it matches the conceptual design and testing its outputs. 4) Validate the model by calibrating it with data, validating its assumptions match reality, and comparing its inputs/outputs to the real system. Validation ensures the model provides accurate predictions.

Uploaded by

Pradyumna A Kubear
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
140 views

Module 5 Verification and Validation of Simulation Models

This document discusses verification and validation of simulation models. It defines verification as ensuring the conceptual model is accurately implemented in the computer program, while validation determines if the correct model is built by calibrating it with data. The key steps are: 1) Observe the real system and collect data on its behavior and components. 2) Develop a conceptual model and then implement it as an operational simulation model. 3) Verify the model by checking it matches the conceptual design and testing its outputs. 4) Validate the model by calibrating it with data, validating its assumptions match reality, and comparing its inputs/outputs to the real system. Validation ensures the model provides accurate predictions.

Uploaded by

Pradyumna A Kubear
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 15

Module 5: Verification and Validation of

Simulation Models
Contents
Model-Building, Verification, and Validation
Verification of Simulation Models
Calibration and Validation
Purpose & Overview
Difference between Verification and
Validation
Verification Validation
• Verification is concerned with • Validation is concerned with
building the model correctly. building the correct model.
• It proceeds by the comparison of • It attempts to confirm that a
the conceptual model to the model is an accurate
computer representation that representation of the real
implements that conception. system.
• Verification would check the • Validation is achieved through
design doc and correcting the calibration of model.
spelling mistake.
Modeling-Building, Verification & Validation

Steps in Model-Building
•Real system
• Observe the real system
• Interactions among the components
• Collecting data on the behavior
Conceptual model Construction of a
conceptual model
•Simulation program Implementation
of an operational model
Verification
• Purpose: ensure the conceptual model is reflected accurately in the computerized
representation.
• Many common-sense suggestions, for example:
Have someone else check the model.
 Make a flow diagram that includes each logically possible action a system
can take when an event occurs.
Closely examine the model output for reasonableness under a variety of input
parameter settings.
Print the input parameters at the end of the simulation, make sure they have
not been changed inadvertently.
Make the operational model as self-documenting as possible.
If the operational model is animated, verify that what is seen in the animation
imitates the actual system.
Use the debugger.
Examination of Model Output for Reasonableness
• Two statistics that give a quick indication of model reasonableness are current
contents and total counts.

Current content: The number of items in each component of the system at a given
time.

Total counts: Total number of items that have entered each component of the system
by a given time.

• Compute certain long-run measures of performance, e.g. compute the long-run


server utilization and compare to simulation results.
Examination of Model Output for Reasonableness
• A model of a complex network of queues consisting of many service centers.

• If the current content grows in a more or less linear fashion as the simulation run
time increases, it is likely that a queue is unstable .

• If the total count for some subsystem is zero, indicates no items entered that
subsystem, a highly suspect occurrence.

• If the total and current count are equal to one, can indicate that an entity has
captured a resource but never freed that resource.
Calibration and Validation
Cont..
Danger during the calibration phase
• Typically few data sets are available, in the worst case only one, and the model is
only validated for these.
•Solution: If possible collect new data sets.
No model is ever a perfect representation of the system
• The modeler must weigh the possible, but not guaranteed, increase in model
accuracy versus the cost of increased validation effort.
Calibration and Validation

Naylor and Finger formulated a three-step approach for validation


Process:

1. Build a model that has high face validity

2. Validate model assumptions

3. Compare the model input-output transformations with the real system’s data
Validation: 1. High Face Validity
Ensure a high degree of realism:
• Potential users should be involved in model construction from its
conceptualization to its implementation.

Sensitivity analysis can also be used to check a model’s face validity.


• Example: In most queueing systems, if the arrival rate of customers were to
increase, it would be expected that server utilization, queue length and delays would
tend to increase.

For large-scale simulation models, there are many input variables and thus
possibly many sensitivity tests.
• Sometimes not possible to perform all of theses tests, select the most critical ones.
Validation: 2. Validate Model Assumptions
General classes of model assumptions:
• Structural assumptions: how the system operates.
• Data assumptions: reliability of data and its statistical analysis.
Bank example: customer queueing and service facility in a bank.
Structural assumptions
• Customer waiting in one line versus many lines
• Customers are served according FCFS versus priority
Data assumptions, e.g., interarrival time of customers, service times
for commercial accounts.
• Verify data reliability with bank managers
• Test correlation and goodness of fit for data
Validation: 3. Validate Input-Output Transformation
In this phase of the validation process, the model is viewed as an input –output
transformation – that is , the model accepts values of the input parameters and
transforms these inputs into output measures of performance.
Goal: Validate the model’s ability to predict future behavior
• The only objective test of the model.
• The structure of the model should be accurate enough to make good
predictions for the range of input data sets of interest.
• One possible approach: use historical data that have been reserved for
validation purposes only.
• Criteria: use the main responses of interest.
Bank Example
Bank Example: The Black Box

You might also like