Bayesian Model Selection

This page contains resources about Bayesian Model Selection and Bayesian Model Comparison.

Subfields and Concepts

 * Bayes Factor
 * Bayesian Model Evidence / Marginal Likelihood
 * Bayesian Occam's Razor
 * Minimum Message Length (MML)
 * Bayesian Model Averaging (in Ensemble Learning)
 * Reversible jump MCMC
 * Birth and death MCMC
 * Bayesian Model Combination
 * Type II Maximum Likelihood / Empirical Bayes / Evidence Approximation
 * Type II Maximum a posteriori (MAP)
 * Laplace Approximation
 * Approximate Bayesian Computation (ABC)
 * Akaike Information Criterion (AIC)
 * Bayesian Information Criterion (BIC)
 * Deviance Information Criterion (DIC)
 * Bayesian Predictive Information Criterion (BPIC)
 * Focused Information Criterion (FIC)

Books and Book Chapters

 * Theodoridis, S. (2015). "Chapter 12: Bayesian Learning" Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
 * Barber, D. (2012). "Chapter 12: Bayesian Model Selection". Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Murphy, K. P. (2012). "Section 5.3: Bayesian model selection". Machine Learning: A Probabilistic Perspective. MIT Press.
 * Claeskens, G., & Hjort, N. L. (2008). Model selection and model averaging. Cambridge: Cambridge University Press.
 * Bishop, C. M. (2006). "Section 3.4: Bayesian Model Comparison". Pattern Recognition and Machine Learning. Springer.
 * MacKay, D. J. (2003). "Chapter 28: Model Comparison and Occam’s Razor". Information Theory, Inference and Learning Algorithms. Cambridge University Press.

Scholarly Articles

 * Alston, C., Kuhnert, P., Choy, S. L., McVinish, R., & Mengersen, K. (2005). Bayesian model comparison: Review and discussion. International Statistical Insitute, 55th session.
 * Wasserman, L. (2000). Bayesian model selection and model averaging. Journal of mathematical psychology, 44(1), 92-107.
 * Kass, R. E., & Raftery, A. E. (1993), Bayes factors and model uncertainty, Technical Report 254, Department of Statistics, University of Washington

Tuturials

 * Bayesian Model Selection by Tom Minka

Other resources

 * Bayesian model comparison - Metacademy
 * Evidence approximation - Metacademy