Date of Award
12-2023
Document Type
Thesis
Abstract
In simulation, fidelity has become a topic of interest in determining how well a simulation is able to represent its referent situation. In many cases, the true referent is the real-world scenario in which the system will exist. However, the fidelity of a simulation may be computed in comparison to other referents including other simulation models or tests. Several metrics have been proposed to evaluate a model based on qualitative or subjective parameters. These proposed metrics offer possible solutions for the quantification of model fidelity, however their inability to compare features relative importance greatly limits their applicability to models and introduces ambiguity in model evaluation. Frist, previously presented metrics are introduced and evaluated. A new metric is then proposed to address concerns presented in the existing metric evaluation. The proposed metric uses model accuracy to a referent case to both determine feature weights and total model fidelity. The proposed metric is then applied to a simulation case and the results are used to make model selection decisions given hypothetical application scenarios. The proposed relative metric aims to compare similar models’ level of fidelity with the end goal of aiding in model selection. By combining the proposed metric with model computational cost, decisions on feature fidelity and inclusion can be made to meet the needs of a given simulation’s application.
Recommended Citation
Hybl, Evan, "Evaluating Model Fidelity to Aid Model Selection" (2023). Honors College Theses. 38.
https://open.clemson.edu/hct/38