Observation

What data are collected from the ABM for testing, understanding, and analyzing it, and how and when are they collected?

This concept describes how information from the ABM is collected and analyzed, which can strongly affect what users understand and believe about the model. This concept can also be another place to tie the model design back to the purpose stated in Element 1: once the model is built and running, how is it used to address its purpose? This concept is not intended to document how simulation experiments and model analyses are conducted, but instead to describe how information is collected from the model for use in such analyses. Observation is important because ABMs can be complex and produce many kinds of output: it is impossible to observe and analyze everything that happens in such a model so we must explain what results we do observe. Observation almost always includes summary statistics on the state of the agents and, perhaps, other entities such as spatial units and collectives. The ODD description needs to state how such statistics were observed: which state variables of which agents (e.g., were agents categorized?) were observed at what times, and how they were summarized. It is especially important to understand whether analyses considered only measures of central tendency (e.g., mean values of variables across agents) or also observed variability among agents, e.g., by looking at distributions of variable values across all agents.

Modelers sometimes also collect observations at the agent level, e.g., by selecting one or more agents and having them record their state over simulated time. Such observations can be useful for understanding behaviors that emerge in a model. The ability to legitimately compare simulation results to data collected in the real world can be a major observation concern, leading some modelers to simulate, in their ABM, the data collection methods used in empirical studies. This “virtual scientist” technique (modeling the data collector; Zurell et al. 2010) strives to understand the biases and uncertainties in the empirical data by reproducing them in an ABM where unbiased and accurate observations are also possible. Describe: • The key outputs of the model used for analyses and how they were observed from the simulations. Such outputs may be simple and straightforward (e.g., means of agent state variables observed once per simulated week), or fairly complex (e.g., the frequency with which the simulated population went extinct within 100 simulated years, out of 1000 model runs). • Any “virtual scientist” or other special techniques used to improve comparison of model results to empirical observations.

In ITC Evacuation Model

At the level of individual agent:

• The exit choice
• The pre-evacuation time
• The time of evacuation 
• If leaver or follower or officer
• The evacuation path of each individual agent

At the global level:

• The total evacuation time
• The number of agents
• The number of followers, officers, and leavers
• The number of evacuees per exit. 

• Graphical output on the model interface shows the habitat type of each cell, via cell color. Bird locations are also shown. The model randomly selects one bird per day and displays a trace of its movement during the day, so foraging patterns can be observed. Summary statistics on the bird population, pest insects, and other food insects, are provided via plots on the interface and output files. Virtual surveys are a special kind of observation designed for comparison of model results to field surveys of bird populations; they simulate the methods used by field biologists to estimate bird densities so that the results are affected by the same biases as the real surveys. This observation technique is fully described below (the virtual survey submodel).
• The model’s purpose is to study how potential management alternatives affect the persistence of dog populations, and one measure of a simulated population’s persistence is the probability that it goes extinct within a certain number of years. This probability of extinction can be estimated as the fraction of replicate simulations in which no dogs are alive at the end. How long these simulations are, and how many replicates are executed, are arbitrary observation decisions. Here, the dog population’s persistence is estimated as the fraction of 500 replicate simulations in which there are no dogs alive after 100 years.
• The model fits into what has been generically called a virtual ecologist approach (Zurell et al., 2010), where the diver (i.e. the “virtual ecologist”) observes and records virtual fish, while performing a simulated sampling method. In the end of the sampling method, the counts made by the diver are divided by the sample area to calculate estimated densities. In order to calculate the accuracy of the sampling procedure, as well as the bias due to non-instantaneous observation, the real density of every species in the model environment is registered, as well as the density of each species inside the sampling area at the start of the sampling method. For every species, the model outputs the real density, the estimated density (using the virtual ecologist) and bias due to non-instantaneous sampling. Bias is calculated as the difference between the estimated (non-instantaneous) density and the initial density inside the sample area, divided by the initial density (Ward-Paige et al., 2010). Therefore, a bias of 1 means that the diver counted on average one more fish per square meter for each fish it would have counted if sampling was instantaneous.
• The location (spawning area) and genome of each individual are recorded every generation. The changes in the genetic structure are assessed through the global, within-subnetwork (upstream or downstream to the dam) and paired (between two subpopulations) Fst index (Weir and Cockerham, 1984, Weir, 1996) before and after the addition of the dam. Global Fst is the component of the genetic variation due to differences among all subpopulations. Within-subnetwork Fst is the component of the genetic variation due to differences among the subpopulations within each subnetwork. The pairwise Fst is the inter-population level genetic variation between two subpopulations.