Probabilistic Graphical Model

This page contains resources about Probabilistic Graphical Models, Probabilistic Machine Learning and Probabilistic Models, including Latent Variable Models.

Bayesian and non-Bayesian approaches can either be used.

Subfields and Concepts
See Category:Probabilistic Graphical Models for some of its subfields.
 * Bayesian Networks (directed graphical models) - not   necessarily following a  "Bayesian" approach
 * Naive Bayes classifier (generative model)
 * Bayesian Naive Bayes
 * Tree Augmented Naive Bayes
 * Logistic Regression (discriminative model)
 * Gaussian Bayes Network / Gaussian Belief Net / Directed Gaussian Graphical Model
 * Dynamic Bayesian Network
 * Hidden Markov Model (HMM)
 * Linear Dynamical System / State Space Model
 * Kalman filter / Linear Gaussian State Space Model
 * Time Series Model
 * Deep Belief Network
 * Hierarchical Bayesian Model
 * Stochastic Computation Graph
 * Factor Analyzer
 * Auto-Regressive Network / Fully-visible Bayes Network (FVBN)
 * Variational Autoencoder (VAE)
 * Markov Random Fields (undirected graphical models)
 * Gibbs Random Field
 * Gaussian MRF / Undirected Gaussian Graphical Model
 * Lattice Model
 * Potts Model
 * Ising Model
 * Hopfield Network
 * Boltzmann Machine
 * Restricted Boltzmann Machine
 * Conditional Random Field
 * Structural Support Vector Machine
 * Deep Boltzmann Machine
 * Associative Markov Network
 * Maximum Entropy (Maxent) Model
 * Structural Support Vector Machine (SSVM) / Max Margin Markov Network (M3net)
 * Factor Graph
 * Stochastic Models (Stochastic Processes, Random Fields, ...)
 * Latent Variable Models (i.e. Partially Observed Probabilistic Models)
 * Continuous Latent Variable Models
 * Factor Analyzer
 * Gaussian Process Latent Variable Model (GPLVM)
 * Gauss-Markov Process
 * State Space Model
 * Kalman filter / Linear Gaussian SSM
 * Discrete Latent Variable Models
 * Latent Dirichlet Allocation
 * Hidden Markov Model
 * Mixture Model
 * Bayesian Model
 * Non-Bayesian Model
 * Mixed Networks (i.e. both deterministic and probabilistic)
 * Chain Graph / Mixed Graph (i.e. both directed and undirected edges)
 * Structure Learning
 * PC Algorithm
 * Network Scoring
 * Chow-Liu Trees
 * Minimal I-Map
 * Bayesian Model Selection
 * Annealed Importance Sampling
 * Sparsity promoting priors
 * L2-regularization / Bayesian Ridge Regression / Gaussian prior
 * L1-regularization / Bayesian LASSO / Laplace prior
 * Spike and Slab / Bernoulli-Gaussian prior
 * Inference in graphical models / Probabilistic Inference
 * Exact Inference / Exact Marginalization
 * Enumeration
 * Variable Elimination Algorithm / Bucket Elimination
 * Sum-Product Algorithm / Belief Propagation / Sum-Product Message Passing / Factor Graph propagation
 * Max-Product Algorithm / Max-Product Belief Propagation / Max-Sum Algorithm
 * Conditioning
 * Junction Tree Algorithm / Clique Tree Propagation
 * Forward-Backward Algorithm (used for HMM)
 * Baum-Welch Algorithm (used for HMM)
 * Viterbi Algorithm (used for HMM)
 * Approximate Inference
 * Deterministic / Structural: Variational Bayesian Inference (as Optimization)
 * Stochastic: Monte Carlo Inference / Sampling Inference / Particle-based Inference
 * Laplace Approximation

Video Lectures

 * Probabilistic Graphical Models by Daphne Koller
 * Machine Learning, Probability and Graphical Models by Sam Roweis - VideoLectures.Net
 * Graphical Models by Zoubin Ghahramani - VideoLectures.Net
 * Graphical Models by Cedric Archambeau - VideoLectures.Net
 * Introduction to Graphical Models for Data Mining by Arindam Banerjee - VideoLectures.Net
 * Bayesian Learning by Zoubin Ghahramani - VideoLectures.Net
 * Graphical modelling and Bayesian structural learning by Peter Green - VideoLectures.Net
 * Graphical Models by Christian Borgelt
 * Learning Bayesian Networks by Richard E. Neapolitan - VideoLectures.Net
 * Machine Learning, Probability and Graphical Models by Sam Roweis - VideoLectures.Net
 * Probabilistic Graphical Models by Sam Roweis - VideoLectures.Net

Lecture Notes

 * Probabilistic Graphical Models by Sargur Srihari
 * Probabilistic Graphical Models by David Sontag
 * Probabilistic Graphical Models by Andreas Krause
 * Probabilistic Graphical Models by Eric Xing
 * Probabilistic Graphical Models Course by Sargur Srihari
 * Foundations of Graphical Models by David M. Blei
 * Probabilistic Models of Discrete Data by David M. Blei
 * Probabilistic Modelling and Reasoning by Amos Storkey
 * COS597C: Advanced Methods in Probabilistic Modeling BY David M. Blei
 * CS228: Probabilistic Graphical Models by Stefano Ermon
 * CSC 2541: Topics in Machine Learning: Bayesian Methods for Machine Learning by Radford Neal
 * CSE 515T: Bayesian Methods in Machine Learning by Roman Garnett
 * CS 281A/Stat 241A: Statistical Learning Theory - Probabilistic Graphical Models by Michael Jordan
 * Unsupervised Learning by Lester Mackey
 * Probabilistic and Unsupervised Learning by Maneesh Sahani - Gatsby
 * Approximate Inference and Learning in Probabilistic Models by Maneesh Sahani - Gatsby
 * Machine Learning by Kevin Murphy
 * Topics in multivariate analysis: Probabilistic graphical models
 * Advanced Statistical Machine Learning by Stefanos Zafeiriou
 * Statistical Methods in Computer Science by Su-In Lee
 * Inference in Graphical Models by Sewoong Oh
 * COS513: Foundations of Probabilistic Modeling
 * Probabilistic reasoning and statistical inference by Daniel Lassiter

Books and Book Chapters

 * Jordan, M. I. (TBA) An Introduction to Probabilistic Graphical Models. (draft)
 * Bellot, D. (2016). Learning Probabilistic Graphical Models in R. Packt Publishing.
 * Pfeffer, A. (2016). Practical probabilistic programming. Manning Publications Co.
 * Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
 * Theodoridis, S. (2015). Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
 * Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 9: Graphs and Model Selection". Statistical learning with sparsity: the lasso and generalizations. CRC Press.
 * Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
 * Ankan, A., & Panda, A. (2015). Mastering Probabilistic Graphical Models Using Python. Packt Publishing Ltd.
 * Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian Networks in R. Springer, 122, 125-127.
 * Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
 * Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
 * Neal, R. M. (2012). Bayesian learning for neural networks (Vol. 118). Springer Science & Business Media.
 * Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall.
 * Alpaydin, E. (2010). "Chapter 16: Graphical Models". Introduction to machine learning. MIT Press.
 * Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
 * Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge University Press.
 * Borgelt, C., Steinbrecher, M., & Kruse, R. R. (2009). Graphical Models - Representations for Learning, Reasoning and Data Mining. John Wiley & Sons.
 * Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 9: Context-dependent Classification". Pattern Recognition. 4th Ed. Academic Press.
 * Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305.
 * Bishop, C. M. (2006). "Chapter 8. Graphical Models". Pattern Recognition and Machine Learning. Springer. pp. 359–422.
 * Jordan, M. I. (2003). An Introduction to Probabilistic Graphical Models.
 * Jordan, M. I., & Sejnowski, T. J. (Ed.). (2001). Graphical models: Foundations of neural computation. MIT Press.
 * Cowell, R. G., D., A. Philip, L., Steffen L., & Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. Springer.
 * Lauritzen, S. L. (1996). Graphical Models. Oxford University Press.
 * Jensen, F. (1996). An Introduction to Bayesian Networks. Springer.
 * Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
 * Jordan, M. I. (Ed.). (1998). Learning in graphical models. Kluwer Academic Publishers.

Scholarly Articles

 * Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature, 521(7553), 452-459.
 * Larrañaga, P., & Moral, S. (2011). Probabilistic graphical models in artificial intelligence. Applied soft computing, 11(2), 1511-1528.
 * Airoldi, E. M. (2007). Getting Started in Probabilistic Graphical Models. PLoS Computational Biology, 3(12), e252.
 * Wainwright, M. J., & Jordan, M. I. (2008). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305.
 * Koller, D., Friedman, N., Getoor, L., & Taskar, B. (2007). 2 Graphical Models in a Nutshell. Statistical Relational Learning, 13.
 * Silva, R., Scheine, R., Glymour, C., & Spirtes, P. (2006). Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7(Feb), 191-246.
 * Frey, B. J., & Jojic, N. (2005). A comparison of algorithms for inference and learning in probabilistic graphical models. IEEE Transactions on pattern analysis and machine intelligence, 27(9), 1392-1416.
 * Jordan, M. I. (2004). Graphical Models. Statistical Science, 140-155.
 * Jordan, M. I., & Weiss, Y. (2002). Graphical models: Probabilistic inference.The handbook of brain theory and neural networks, 490-496.

Tutorials

 * Graphical Models: Structure Learning by David Heckermann
 * Graphical Models: Parameter Learning by Zoubin Ghahramani
 * Heckerman's Bayes Net Learning Tutorial
 * A Brief Introduction to Graphical Models and Bayesian Networks by Kevin Murphy
 * An Introduction to Graphical Models by Michael Jordan
 * Bayesian Modelling in Machine Learning: A Tutorial Review
 * Bayesian Methods for Machine Learning - NIPS 2004
 * Probabilistic Modelling, Machine Learning, and the Information Revolution by Zoubin Ghahramani - 2012
 * Graphical Models by Zoubin Ghahramani - MLSS 2012
 * Graphical Models Lectures - 2015

Software

 * Edward: A library for probabilistic modeling, inference, and criticism - Python with TensorFlow
 * PRMLT - MATLAB Toolbox for the book of PRML by C. Bishop
 * pmtk3 - Probabilistic Modeling Toolkit for MLPP book by Murphy in Matlab/Octave (3rd edition)
 * pyprobml - Python code for MLPP book by K. Murphy
 * BRMLtoolbox - MATLAB and Julia code for the BRML book by D. Barber
 * PyBRML - Python code for the BRML book by D. Barber
 * Bayesian Probabilistic Matrix Factorization - MATLAB
 * Mens X Machina PGM Toolbox - MATLAB
 * UGM (undirected graphical models) - MATLAB
 * Module libpgm - Python
 * Graphical Models Toolkit (GMTK)
 * Bayesian Modeling and Monte Carlo Methods
 * SamIam
 * BNT - Bayes Net Toolbox in MATLAB
 * libDAI - C++
 * OpenGM - C++
 * Stan - Python (PyStan) and R (RStan) interfaces
 * PyMC3 - Python
 * Infer.NET - Developed by Microsoft Research
 * OpenBUGS - Bayesian Inference Using Gibbs Sampling

Other Resources

 * Comparison of software toolkits
 * Probabilistic Graphical Models wiki
 * Easier Plate Notation in Python using Daft - Python
 * Graphical Models - Notebook