Ioannis Kourouklides

This page contains resources about Probabilistic Graphical Models, Probabilistic Machine Learning and Probabilistic Models, including Latent Variable Models.

Bayesian and non-Bayesian approaches can either be used.

Subfields and Concepts[]

See Category:Probabilistic Graphical Models for some of its subfields.

  • Bayesian Networks (directed graphical models) - not  necessarily following a "Bayesian" approach
  • Markov Random Fields (undirected graphical models)
    • Gibbs Random Field
    • Gaussian MRF / Undirected Gaussian Graphical Model
    • Lattice Model
      • Potts Model
      • Ising Model
    • Hopfield Network
    • Boltzmann Machine
      • Restricted Boltzmann Machine
    • Conditional Random Field
    • Structural Support Vector Machine
    • Deep Boltzmann Machine
    • Associative Markov Network
    • Maximum Entropy (Maxent) Model
    • Structural Support Vector Machine (SSVM) / Max Margin Markov Network (M3net)
    • Factor Graph
  • Stochastic Models (Stochastic Processes, Random Fields, ...)
  • Latent Variable Models (i.e. Partially Observed Probabilistic Models)
  • Mixed Networks (i.e. both deterministic and probabilistic)
  • Chain Graph / Mixed Graph (i.e. both directed and undirected edges)
  • Structure Learning
    • PC Algorithm
    • Network Scoring
    • Chow-Liu Trees
    • Minimal I-Map
    • Bayesian Model Selection
    • Annealed Importance Sampling
    • Sparsity promoting priors
      • L2-regularization / Bayesian Ridge Regression / Gaussian prior
      • L1-regularization / Bayesian LASSO / Laplace prior
      • Spike and Slab / Bernoulli-Gaussian prior
  • Inference in graphical models / Probabilistic Inference
    • Exact Inference / Exact Marginalization
      • Enumeration
      • Variable Elimination Algorithm / Bucket Elimination
      • Sum-Product Algorithm / Belief Propagation / Sum-Product Message Passing / Factor Graph propagation
      • Max-Product Algorithm / Max-Product Belief Propagation / Max-Sum Algorithm
      • Conditioning
      • Junction Tree Algorithm / Clique Tree Propagation
      • Forward-Backward Algorithm (used for HMM)
      • Baum-Welch Algorithm (used for HMM)
      • Viterbi Algorithm (used for HMM)
    • Approximate Inference

Online Courses[]

Video Lectures[]

Lecture Notes[]

Books and Book Chapters[]

  • Jordan, M. I. (TBA) An Introduction to Probabilistic Graphical Models. (draft)
  • Bellot, D. (2016). Learning Probabilistic Graphical Models in R. Packt Publishing.
  • Pfeffer, A. (2016). Practical probabilistic programming. Manning Publications Co.
  • Koduvely, H. M. (2015). Learning Bayesian Models with R. Packt Publishing.
  • Theodoridis, S. (2015). Machine Learning: A Bayesian and Optimization Perspective. Academic Press.
  • Hastie, T., Tibshirani, R., & Wainwright, M. (2015). "Chapter 9: Graphs and Model Selection". Statistical learning with sparsity: the lasso and generalizations. CRC Press.
  • Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
  • Ankan, A., & Panda, A. (2015). Mastering Probabilistic Graphical Models Using Python. Packt Publishing Ltd.
  • Nagarajan, R., Scutari, M., & Lèbre, S. (2013). Bayesian Networks in R. Springer122, 125-127.
  • Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
  • Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
  • Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
  • Neal, R. M. (2012). Bayesian learning for neural networks (Vol. 118). Springer Science & Business Media.
  • Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall.
  • Alpaydin, E. (2010). "Chapter 16: Graphical Models". Introduction to machine learning. MIT Press.
  • Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
  • Darwiche, A. (2009). Modeling and reasoning with Bayesian networks. Cambridge University Press.
  • Borgelt, C., Steinbrecher, M., & Kruse, R. R. (2009). Graphical Models - Representations for Learning, Reasoning and Data Mining. John Wiley & Sons.
  • Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2008). "Chapter 9: Context-dependent Classification". Pattern Recognition. 4th Ed. Academic Press.
  • Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning1(1-2), 1-305.
  • Bishop, C. M. (2006). "Chapter 8. Graphical Models". Pattern Recognition and Machine Learning. Springer. pp. 359–422.
  • Jordan, M. I. (2003). An Introduction to Probabilistic Graphical Models.
  • Jordan, M. I., & Sejnowski, T. J. (Ed.). (2001). Graphical models: Foundations of neural computation. MIT Press.
  • Cowell, R. G., D., A. Philip, L., Steffen L., & Spiegelhalter, D. J. (1999). Probabilistic Networks and Expert Systems. Springer.
  • Lauritzen, S. L. (1996). Graphical Models. Oxford University Press.
  • Jensen, F. (1996). An Introduction to Bayesian Networks. Springer.
  • Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann.
  • Jordan, M. I. (Ed.). (1998). Learning in graphical models. Kluwer Academic Publishers.

Scholarly Articles[]

  • Ghahramani, Z. (2015). Probabilistic machine learning and artificial intelligence. Nature521(7553), 452-459.
  • Larrañaga, P., & Moral, S. (2011). Probabilistic graphical models in artificial intelligence. Applied soft computing11(2), 1511-1528.
  • Airoldi, E. M. (2007). Getting Started in Probabilistic Graphical Models. PLoS Computational Biology, 3(12), e252.
  • Wainwright, M. J., & Jordan, M. I. (2008). Graphical Models, Exponential Families, and Variational Inference. Foundations and Trends® in Machine Learning, 1(1-2), 1-305.
  • Koller, D., Friedman, N., Getoor, L., & Taskar, B. (2007). 2 Graphical Models in a Nutshell. Statistical Relational Learning, 13.
  • Silva, R., Scheine, R., Glymour, C., & Spirtes, P. (2006). Learning the structure of linear latent variable models. Journal of Machine Learning Research7(Feb), 191-246.
  • Frey, B. J., & Jojic, N. (2005). A comparison of algorithms for inference and learning in probabilistic graphical models. IEEE Transactions on pattern analysis and machine intelligence27(9), 1392-1416.
  • Jordan, M. I. (2004). Graphical Models. Statistical Science, 140-155.
  • Jordan, M. I., & Weiss, Y. (2002). Graphical models: Probabilistic inference.The handbook of brain theory and neural networks, 490-496.



See also[]

Other Resources[]