At MISIM, we consider shadow models an integral component of any well-designed simulation. Their primary advantage? They inject a level of thoroughness into the validation process that is indispensable. Shadow models provide an essential quality assurance check that cannot be understated, ensuring that the underlying simulation model is both accurate and well understood.
In simple terms, a shadow model is a secondary model that is used in parallel to the primary model. The shadow model is typically a simpler version of the simulation in which variability and uncertainty have either been removed, or controlled, such that results between the two models can be compared on a like-for-like basis.
Shadow Models Provide Several Benefits
Validation Through Simplicity. Shadow models serve as a critical check for the primary model’s performance. By continually comparing back to the primary model, they keep watch on outcomes and help to validate results – ensuring they are not only accurate but also reliable.
The more complex the primary model, the greater the importance of having a shadow model. As more detail is added to a model, so too is added potential for inaccuracies. A shadow model acts as a safeguard, a secondary line of defense against such pitfalls. By systematically showing the input-output relationship between models is a match, and any differences explained, the primary model outcomes can be empirically validated, and risk of error is greatly reduced.
Insight and Clarity by Distilling Complexity. Their simplicity cuts through the noise, offering insights into the logic and behavior of the simulation. Key influences are laid bare, and their impacts can be quickly ranged.
Understanding which inputs have the greatest influence on outcomes is another benefit provided by shadow models. Large models contain many interactions, feedback loops, and conditional logic which create a complex web of relationships between inputs and outputs. Shadow models offer a way to distill relationships into clear cause and effect terms, allowing developers to discern the key levers that drive outcomes. This distilled perspective can assist with development by placing focus on the critical components of the system.
Intuitive Understanding. By creating a more intuitive link between inputs and outputs, shadow models enhance stakeholder understanding, bridging the gap between complex simulation data and actionable insights. Through the shadow model build process a greater understanding of the system at large is attained. Their simplicity can help provide an easy-to-understand way of communicating results with stakeholders and provide a direct link between intuition and results, increasing their confidence in the model.
Building and Implementing Shadow Models
From a practitioner perspective, structure and implementation are important; a common model input file should be used to power both models. This reduces the chance of input error due to double handling. When possible, the models should be developed independently – preferably by a different person – to reduce the risk of error duplication in both models. And always, rigorous testing and validation should be performed to ensure that the models are functioning as intended and producing consistent results. This includes not only checking the technical aspects of the models but also reviewing the underlying assumptions and methodologies to ensure they are sound and appropriate for the intended application.
Beyond a Shadow of a Doubt
Our overarching mission is to unlock value by crafting accurate digital replicas of the systems we model and leveraging the information they provide. In this quest, shadow models are an important component, ensuring our models are accurate and truly operating as intended.
