Validation

Researchers aim to determine if the model captures the macro-level patterns observed in the real world, indicating that the interactions and behaviors of the agents within the model are responsible for generating these patterns. This is done by comparing the model's output with empirical data and evaluating the degree of agreement.

During the validation process of an agent-based model several challenges and potential problems can arise:

1. Stochastic Nature: Both the model and the real-world system being analyzed are likely to involve randomness and uncertainty. 
2. Predictive vs. Retrodictive Capability: A model may be capable of generating plausible predictions about future scenarios but may struggle to accurately recreate known past states of the system. This discrepancy can occur due to the inherent complexity and non-linear dynamics of the real-world system.
3. Data Quality and Reliability: While the model itself may be accurate, the data available from the real-world system may be incomplete, erroneous, or subject to measurement errors. 
4. Path Dependency: Many simulations exhibit path dependency, meaning that the outcomes are sensitive to the specific initial conditions and the sequence of events during the simulation. 

In a fire evacuation model, predictive validation is the most difficult because:

• predictive validation focuses on how well the model predicts the future behavior of the system, which in this case, is the evacuation process under real-world conditions.
• Evacuation events are rare and unpredictable: Real evacuations due to emergencies are infrequent and often involve unique circumstances. This makes it difficult to obtain a large amount of high-quality data from real evacuations for comparison with the model's predictions.
• Ethical considerations: Deliberately causing an evacuation event to collect data would be unethical and impractical.

While other validation steps also have their challenges:

• Input validation: While it's true collecting data from a burning building is impossible, data on building layouts, materials, fire spread rates, and occupant capacities can be obtained from existing sources or controlled experiments (not involving actual fires).
• Process validation: Understanding pre-evacuation behavior can be challenging, but surveys, interviews, and observations from past evacuations can provide valuable insights into how people react at the start of an evacuation.
• Descriptive output validation: Evacuation patterns might vary depending on the situation, but data on existing evacuation drills or historical evacuation events (e.g., building evacuation times, congestion points) can be used for comparison with the model's descriptive outputs.

Model Validity = Adequate for Its Purpose

Trace Protocol = Transparent and Comprehensive Ecological modelling documentation

Input Validation

Are the input data to the model is meaningful? Quantitative and Qualitative data.

Critically reflect on this input their resources:

• References
• Data Sources
• Data Collection process

Process Validation

How the process reflect the real word for the model purpose.

• To assist the simpliflying assumption inherent to the model design.
• Which spatio temporal scale that is used, why not others?
• Entities that selected
• Which covered by stochastic elements?
• How to encoporate interactions?

Descriptive Output Validation

How well can the model output capture the features of the data used to built the model

Realworld pattern, which criteria to simulate this pattern.

Predictive Output Validation

Forecast the sample data not used for the model building or data only acquired later or for another case study.