Validation of statistical and AI models

Summary Validation of AI and machine-learning models does not require a separate framework. It requires the existing framework applied with additional attention to the properties of these models.

Common ground Conceptual soundness, data quality, methodology, performance and governance apply to both. So do benchmarking and stability analysis.

Additional emphasis for ML Machine-learning models introduce specific validation concerns:

• Data drift and concept drift monitoring
• Feature importance stability
• Explainability sufficient for the decisions the model supports
• Fairness and bias testing where the model affects individuals
• Reproducibility of training runs

Explainability Explainability is a means to an end: the ability to defend decisions to customers, investigators, auditors and supervisors. The technique — SHAP, surrogate models, rule extraction — matters less than whether the resulting explanation is fit for purpose.

Champion-challenger Continuous challenger evaluation is more informative for ML than periodic full revalidation, because performance can degrade quickly.

Governance Ownership of ML models should be as clear as for statistical models. Diffuse ownership between data science, technology and risk is a common failure mode.

Limitations Validation cannot certify accuracy on unseen data. It provides assurance that the model is fit for its stated purpose within known limits.

Related expertise See [Model Risk and Validation](/expertise/model-risk).