This page contains resources about Probabilistic Graphical Models, Probabilistic Machine Learning and Probabilistic Models, including Latent Variable Models.

Bayesian and non-Bayesian approaches can either be used.

## Subfields and Concepts

See Category:Probabilistic Graphical Models for some of its subfields.

• Bayesian Networks (directed graphical models) - not  necessarily following a "Bayesian" approach
• Naive Bayes classifier (generative model)
• Bayesian Naive Bayes
• Tree Augmented Naive Bayes
• Logistic Regression (discriminative model)
• Gaussian Bayes Network / Gaussian Belief Net / Directed Gaussian Graphical Model
• Dynamic Bayesian Network
• Deep Belief Network
• Hierarchical Bayesian Model
• Stochastic Computation Graph
• Factor Analyzer
• Auto-Regressive Network / Fully-visible Bayes Network (FVBN)
• Variational Autoencoder (VAE)
• Markov Random Fields (undirected graphical models)
• Gibbs Random Field
• Gaussian MRF / Undirected Gaussian Graphical Model
• Lattice Model
• Potts Model
• Ising Model
• Hopfield Network
• Boltzmann Machine
• Restricted Boltzmann Machine
• Conditional Random Field
• Structural Support Vector Machine
• Deep Boltzmann Machine
• Associative Markov Network
• Maximum Entropy (Maxent) Model
• Structural Support Vector Machine (SSVM) / Max Margin Markov Network (M3net)
• Factor Graph
• Stochastic Models (Stochastic Processes, Random Fields, ...)
• Latent Variable Models (i.e. Partially Observed Probabilistic Models)
• Mixed Networks (i.e. both deterministic and probabilistic)
• Chain Graph / Mixed Graph (i.e. both directed and undirected edges)
• Structure Learning
• PC Algorithm
• Network Scoring
• Chow-Liu Trees
• Minimal I-Map
• Bayesian Model Selection
• Annealed Importance Sampling
• Sparsity promoting priors
• L2-regularization / Bayesian Ridge Regression / Gaussian prior
• L1-regularization / Bayesian LASSO / Laplace prior
• Spike and Slab / Bernoulli-Gaussian prior
• Inference in graphical models / Probabilistic Inference
• Exact Inference / Exact Marginalization
• Enumeration
• Variable Elimination Algorithm / Bucket Elimination
• Sum-Product Algorithm / Belief Propagation / Sum-Product Message Passing / Factor Graph propagation
• Max-Product Algorithm / Max-Product Belief Propagation / Max-Sum Algorithm
• Conditioning
• Junction Tree Algorithm / Clique Tree Propagation
• Forward-Backward Algorithm (used for HMM)
• Baum-Welch Algorithm (used for HMM)
• Viterbi Algorithm (used for HMM)
• Approximate Inference

## Books and Book Chapters

• Jordan, M. I. (TBA) An Introduction to Probabilistic Graphical Models. (draft)
• Bellot, D. (2016). Learning Probabilistic Graphical Models in R. Packt Publishing.
• Pfeffer, A. (2016). Practical probabilistic programming. Manning Publications Co.
• Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
• Theodoridis, S. (2015). Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
• Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 9: Graphs and Model Selection". Statistical learning with sparsity: the lasso and generalizations. CRC Press.
• Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
• Ankan, A., & Panda, A. (2015). Mastering Probabilistic Graphical Models Using Python. Packt Publishing Ltd.
• Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian Networks in R. Springer122, 125-127.
• Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
• Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
• Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
• Neal, R. M. (2012). Bayesian learning for neural networks (Vol. 118). Springer Science & Business Media.
• Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall.
• Alpaydin, E. (2010). "Chapter 16: Graphical Models". Introduction to machine learning. MIT Press.
• Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
• Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge University Press.
• Borgelt, C., Steinbrecher, M., & Kruse, R. R. (2009). Graphical Models - Representations for Learning, Reasoning and Data Mining. John Wiley & Sons.
• Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 9: Context-dependent Classification". Pattern Recognition. 4th Ed. Academic Press.
• Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning1(1-2), 1-305.
• Bishop, C. M. (2006). "Chapter 8. Graphical Models". Pattern Recognition and Machine Learning. Springer. pp. 359–422.
• Jordan, M. I. (2003). An Introduction to Probabilistic Graphical Models.
• Jordan, M. I., & Sejnowski, T. J. (Ed.). (2001). Graphical models: Foundations of neural computation. MIT Press.
• Cowell, R. G., D., A. Philip, L., Steffen L., & Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. Springer.
• Lauritzen, S. L. (1996). Graphical Models. Oxford University Press.
• Jensen, F. (1996). An Introduction to Bayesian Networks. Springer.
• Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
• Jordan, M. I. (Ed.). (1998). Learning in graphical models. Kluwer Academic Publishers.

## Scholarly Articles

• Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature521(7553), 452-459.
• Larrañaga, P., & Moral, S. (2011). Probabilistic graphical models in artificial intelligence. Applied soft computing11(2), 1511-1528.
• Airoldi, E. M. (2007). Getting Started in Probabilistic Graphical Models. PLoS Computational Biology, 3(12), e252.
• Wainwright, M. J., & Jordan, M. I. (2008). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305.
• Koller, D., Friedman, N., Getoor, L., & Taskar, B. (2007). 2 Graphical Models in a Nutshell. Statistical Relational Learning, 13.
• Silva, R., Scheine, R., Glymour, C., & Spirtes, P. (2006). Learning the structure of linear latent variable models. Journal of Machine Learning Research7(Feb), 191-246.
• Frey, B. J., & Jojic, N. (2005). A comparison of algorithms for inference and learning in probabilistic graphical models. IEEE Transactions on pattern analysis and machine intelligence27(9), 1392-1416.
• Jordan, M. I. (2004). Graphical Models. Statistical Science, 140-155.
• Jordan, M. I., & Weiss, Y. (2002). Graphical models: Probabilistic inference.The handbook of brain theory and neural networks, 490-496.