Modeling and Simulation
George Box, "Robustness in the Strategy of Scientific Model Building." In Robustness in Statistics: Proceedings of a Workshop, 1979
" essentially, all models are wrong but some are useful."
Dr. Rita R. Colwell, Director, U.S. National Science Foundation, National Press Club, 1999
"Science used to be composed of two endeavors, theory and experiment. Now it has a third component: computer simulation which links the other two."
Greek Philosopher Plutarch
"to find fault is easy, to do better maybe difficult."
Agent Based Model Agent
(1) a computer program; (2) represents some real-world actor situated in some environment; (3) has inputs, such as perception of the environment or communication with other agents; (4) has outputs, such as interaction with environment or communication with other agents; (5) has purpose (achieve some goal, carry out some task); (6) has rules
What is the 7 step V&V process?
(1) formulate the problem; (2) collect data/construct assumptions document; (3) validate assumptions document; (4) program the model; (5) validate programmed model; (6) design/conduct/analyze experiment; (7) document and present results
Why is the importance of stochastic models to M&S?
(1) generation of random numbers and random variates; (2) statistical modeling of simulation input data; (3) statistical modeling of simulation output data; (4) techniques to enhance precision and confidence for a given number of trial runs
Why is ABM an interesting M&S methodology?
(1) structure (behavior) is emergent from agent interaction without explicitly being programmed; (2) agents have goals, beliefs, strategies, and act (these things can be modeled easily); (3) agent strategies and the environment are dynamic (changing/evolving over time; often this is so easily modeled)
Thomas Khun, "The Structure of Scientific Revolutions" Chicago: University of Chicago Press, 1962
(summary here)
What are some system dynamics tools available?
AnyLogic, Vensim, Stella
human strengths v. computer strengths
HS: recognize patterns, reason inductively, devise and apply strategies, adapt to change and unexpected events CS: calculate rapidly, store and recall information reliably, perform repetitive activities, maintain performance under heavy load for an extended period of time
event-driven (discrete-state system)
In discrete-state systems, discrete state changes occur only at certain points in time through instantaneous transitions; with each state transition, we associate an event. Further, we attribute the state transition to the occurrence of the event.
What are some other types of models?
Markov chains, queuing models, petri nets; the modeler must chose the appropriate type-this is the art of modeling [Fishwick, 1995]
What are some ABM tools?
Swarm, Repast, ACE
Why is the central limit theorem important?
The CLT provides a set of simple rules for determining the mean, variance, and shape of a distribution of sample means; distributions of sample means are used in all hypothesis test with means; we use the CLT to provide better, more accurate results.
Andrew Collins, "Which is Worse: Large-Scale Simulations or the 80% Solution?," SCS M&S Magazine, 2012
The analyst can begin to see the world through the lens of the technique they use (caution, this is much like theory); example= Jay Forrester's System Dynamics (Jay Forrester, Industrial Dynamics, Cambridge: MIT Press, 1961) in industrial problems, to urban development (Jay Forrester, Urban Dynamics, Cambridge: MIT Press 1969) and the world (Jay Forrester, World Dynamics, Cambridge: MIT Press 1971) use different techniques and their underlying assumptions for the same problem is fine if you take the instrumentalism (constructivist) view of science (Roy Bernard, "Decision Science or Decision-aid Science?", European Journal of Operational Research, 1993) looks at large scale simulation and the 80% solution which are in competition with one another
system
a combination of components that act together to perform a function not possible with any of the individual components.
simulation
a method for executing a model to extract data concerning model behavior
model
a physical, mathematical or otherwise logical representation of a system, entity, phenomenon or process.
attribute
a significant or defining property or characteristic of a model or simulation
event
a specific instantaneous action or occurrence which results in an instantaneous change of system state in a discrete state system.
deterministic system
a system in which all variables are deterministic (not random); they are defined/have assigned values; often used for short term training
stochastic system
a system in which one or more variables is a random variable; in this case, the system state becomes a random process and a probabilistic framework; can have deterministic values; used for long term planning
continuous-state system
a system in which the state space Q consists of vectors which can assume a continuum of real or complex values
discrete-state system
a system in which the state space Q consists of vectors which can assume only a discrete set of real or complex values; as a consequence changes occur at discrete time instants.
continuous-time system
a system in which the time variable is represented by a continuous variable t
discrete-time system
a system in which the time variable is represented by a sequence of discrete time values
random experiment
a well-defined experiment having different possible outcomes that cannot be predicted before hand
Dynamic V&V techniques
acceptance testing, Alpha testing, assertion checking, Beta testing, bottom-up testing, comparison testing, statistical techniques, structural testing, submodel/module testing, visualization, animation
system dynamics
an M&S methodology for investigating complex feedback systems; method uses Influence Diagrams and Causal Loop diagrams to display the qualitative effects of system feedback; this method uses Stock and Flow diagrams to display the quantitative behavior of individual system components; used in business systems, financial systems, and social systems
Informal V&V techniques
audit, desk checking, documentation checking, face validation, inspections, reviews, Turing tests, walkthroughs
perception
awareness of the elements of the environment through physical sensations
Static V&V techniques
cause-effect graphing, control analysis, data analysis, fault/failure analysis, interface analysis, semantic analysis, structural analysis, symbolic evaluation, syntax analysis, traceability assessment
continuous system
continuous state, time-driven system; often used to represent deterministic, physics-based systems; provides a view of system behavior over time
How does fidelity relate to scale?
depends on what you are modeling
discrete event system (DES)
discrete-state, event-driven system; easily accommodates uncertainty and variability; provides a view of the system behavior over time
What are the 3 M&S attributes?
fidelity, resolution, scale
How does fidelity relate to resolution?
if you have high fidelity you do not necessarily have high resolution
Formal V&V techniques
induction, inductive assertions, inference, logical deduction, Lambda calculus, predicate calculus, predicate transformation, proof of correctness
What are the 4 V&V techniques?
informal, static, dynamic, formal [Balci, 1998]
human factors (HF)
knowledge of human cognition and perception; basics of how to interface effectively with humans
Distributed Interactive Simulation (DIS)
mainly used for simulators and platform/entity level simulations; Principles of DIS: no central management; autonomous simulations, information exchange via PDU, autonomous perception of the situation; distributed cause-effect responsibilities; minimizing data traffic
stochastic models
many models contain parameters that are described by random variables; simulation of these models results in outputs that are so also random variables; analysis of these systems requires the application of concepts from the area of mathematics called probability and statistics
Why are humans important in M&S?
most simulations are designed to interface with a human user at some point in the M&S process; many complex systems contain humans as system components; the use of M&S to study complex systems requires an understanding of how to interface with and represent humans
instrumentalist view of science
not so much worried about finding the truth but gaining insight into the problem at hand, thus scientific theory is a useful instrument in understanding the world
accreditation
official certification by a responsible authority that a model is acceptable for a specific purpose [DoD, 1996]
time driven (continuous-state system)
one of the most common models for continuous-state systems is the differential equation; these models are derived using the physical laws governing the behavior of a system; the system response varies as a continuous function of time, even when there is no change int he system inputs. Thus, the system state appears to evolve simply because time advances.
agent based models
one or more agents interacting with one another and/or some environment; through a relatively simple set of rules, agents can achieve complex adaptive behavior that is not pre-programmed; agent's behavior emerges and evolves based on environmental inputs an interactions with other agents.
What are some Human Behavior Model approaches?
recognition primed decision-making, rational choice theory, subjective expected utility theory, bounded rationality, prospect theory
constructive simulation
simulation involving real people making inputs into a simulation that carries out those inputs by simulated people operating simulated systems [DoD, 1998]
live simulation
simulation involving real people operating real systems [DoD, 1998]
virtual simulation
simulation involving real people operating simulated systems [DoD, 1998]
continuous simulation
simulation of continuous-state, time-driven system; models usually deterministic and continuous-time; models often take the form of differential equations
discrete event simulation
simulation of discrete-state, event-driven systems; models usually stochastic and often continuous-time; models often take the form of queuing systems
scale
size of the overall scenario or event the simulation represents (aka level)
Monte Carlo simulation
stochastic simulation of real world systems modeled as random experiments; system behavior is modeled using probability distributions; physics of the system is not represented; utilize randomly generated parameter values; usually conduct multiple trials and perform statistical analysis; simulation is static
human/computer interfacing (HCI)
study of how to design effective and efficient interfaces between a computer simulation and a human operator
human behavior modeling (HBM)
study of how to model computationally the process of human decision making
fidelity
the accuracy of a model's representation or simulation's results; (aka validity); fidelity is relative to reality, representation and requirements
cognition
the act or process of knowing including both awareness and judgment
behavior
the actions or reactions of a person in response to external or internal stimuli
resolution
the degree of detail which which the real-world is simulated; the more the detail the higher the resolution; (aka granularity)
VV&A
the process of determining if a model is correct and usable; the process of developing and delimiting confidence that a model can be used for a specific purpose.
verification
the process of determining that a model's implementation accurately represents the developer's conceptual description and specification [DoD, 1996]; answers the question 'is it coded right?'
validation
the process of determining the degree to which a model (and data) is an accurate representation of the reals world from the perspective of the models' intended usage [DoD, 1996]; answers the question 'is the right thing coded?'
visualization
the use of computer graphics and visual techniques to provide a visual representation of simulation-related information
analysis
the use of mathematically based techniques to obtain information from data and to arrive at a good or optimal decisions in complex situations
modeling and simulation
the use of models and simulations, combined with analysis and visualization, to develop information for managerial or technical decisions and/or for training
How does resolution relate to scale?
these two items are independent
Roy Bernard, "Decision Science or Decision-aid Science?" European Journal of Operational Research, 1993
use different techniques and their underlying assumptions for the same problem is fine if you take the instrumentalism (constructivist) view of science
Central Limit Theorem
variable x(n) (iid); population distribution with mean mu and a standard deviation sigma; if n is sufficiently large, the distribution of the means (x bar) of n random samples will approximate a normal distribution with mu (x bar)=mu