Machine Learning



This page contains resources about Pattern Recognition, Computational Statistics and Machine Learning in general.

More specific information is included in each subfield.

Subfields and Concepts
See Category:Machine Learning for some of its subfields.
 * Supervised Learning
 * Classification
 * Discriminative Model Vs Generative Model
 * Regression
 * Parametric Model Vs Nonparametric Model
 * Structured Learning
 * Generalized Linear Model (GLM or GLIM)
 * Support Vector Machine
 * Supervised Dimensionality Reduction
 * Adaptive Basis Function Model
 * Decision Tree Learning
 * Classification and Regression Tree (CART)
 * ID3 Algorithm
 * Artificial Neural Network
 * Feedforward Neural Network
 * Recurrent Neural Network
 * Radial Basis Function (RBF) Network
 * Kohonen Network
 * Supervised Ensemble Learning
 * Bayesian Averaging
 * Bagging
 * Boosting
 * Bayes Optimal Classifier
 * Decision Forest / Random Forest
 * Supervised Dictionary Learning
 * Supervised Deep Learning
 * Deep Belief Network
 * Unsupervised Learning
 * Clustering / Discrete Latent Variable Models
 * Unsupervised Dimensionality Reduction / Continuous Latent Variable Models
 * Manifold Learning (although supervised variants exist)
 * Autoencoder
 * Unsupervised Ensemble Learning
 * Unsupervised Dictionary Learning
 * Unsupervised Deep Learning
 * Deep Autoencoder
 * Deep Belief Network
 * Semi-supervised Learning
 * Active Learning
 * Inductive Learning
 * Supervised Learning
 * Semi-supervised Learning
 * Analytical Learning
 * Instance-based Learning
 * Lazy Learning
 * k-Nearest Neighbors (k-NN) Algorithm
 * Case-based Reasoning (CBR)
 * Eager Lerning
 * RBF Network
 * Kernel Machine
 * Decision Tree Learning
 * Backpropagation
 * Naive Bayes
 * Reinforcement Learning
 * Multi-Armed Bandit
 * Finite Markov Decision Process
 * Temporal-Difference Learning
 * Q-Learning
 * Adaptive Dynamic Programming
 * Deep Reinforcement Learning
 * Probabilistic Machine Learning
 * Bayesian Network (directed graphical models)
 * Markov Random Field (undirected graphical models)
 * Mixture Model
 * Stochastic Model
 * Bayesian Machine Learning
 * Variational Bayesian Learning
 * Statistical Learning Theory and Computational Learning Theory
 * Online Learning and Sequential Prediction
 * Applications
 * Computer Vision
 * Medical Imaging
 * Robotics
 * Natural Language Processing
 * Computational Finance
 * Bioinformatics

Video Lectures

 * Machine Learning by Andrew Ng - Coursera
 * Machine Learning by Pedro Domingos - Coursera
 * Neural Networks for Machine Learning by Geoffrey Hinton - Coursera
 * Practical Machine Learning by Jeff Leek - Coursera
 * NYU Course on Big Data, Large Scale Machine Learning by John Langford and Yann LeCun
 * Learning from Data by Yaser Abu-Mostafa
 * Introduction to Machine Learning by Barnabas Poczos and Alex Smola
 * Machine Learning by Nando de Freitas
 * Pattern Recognition by Fred A. Hamprecht (2011 2012 )
 * Machine Learning and Pattern Recognition by Charles Sutton (Lecture notes )
 * Machine Learning by Joachim M. Buhmann
 * Pattern Recognition by P.S.Sastry - NPTEL
 * CS188 Intro to AI by Pieter Abbeel - Edx
 * PASCAL Lecture Series - VideoLectures.Net

Introductory

 * CO395: Machine Learning by Maja Pantic - very introductory course
 * Prediction: Machine Learning and Statistics by Cynthia Rudin- very introductory course
 * Machine Learning by Michael Littman
 * SGN-2506: Introduction to Pattern Recognition by Jussi Tohka
 * CSCI1950-F: Introduction to Machine Learning by Erik Sudderth
 * CS 229: Machine Learning by Andrew Ng
 * CSC 411: Machine Learning and Data Mining by Aaron Hertzmann
 * CS 760: Machine Learning by David Page
 * CSE446: Machine Learning
 * Introduction To Machine Learning by David Sontag
 * CSC321: Introduction to Neural Networks and Machine Learning by Tijmen Tieleman - this might be a bit advanced for beginners
 * CSC2515: Introduction to Machine Learning by Geoffrey Hinton - this is very similar to the above
 * Introduction to Pattern Recognition by Sargur Srihari
 * Introduction to Machine Learning Course by Sargur Srihari - this might be a bit advanced for beginners
 * Introduction to Machine Learning by Shai Shalev-Shwartz - this might be a bit advanced for beginners
 * COS 511: Foundations of Machine Learning by Rob Schapire
 * CS 2750 Machine Learning by Milos Hauskrecht
 * Introductory Applied Machine Learning by Victor Lavrenko and Nigel Goddard
 * Machine Learning and Pattern Recognition by Yann LeCun
 * Machine Learning by Tom Mitchell
 * Machine Learning by Tommi Jaakkola
 * Machine Learning by Andrew Zisserman
 * Pattern Recognition and Analysis by Rosalind W. Picard
 * CSCE 666: Pattern Analysis Fall by Ricardo Gutierrez-Osuna
 * Pattern Recognition by Richard Zanibbi
 * Neural Networks and Pattern Recognition by Ömer Cengiz ÇELEBİ
 * Machine Learning by Carl Edward Rasmussen and Zoubin Ghahramani
 * Learning from Data by Amos Storkey
 * Machine Learning: Pattern Recognition by Gwenn Englebienne
 * Machine Learning by Tony Jebara
 * Machine Learning I by Le Song

Advanced

 * CS281: Advanced Machine Learning by Ryan Adams
 * CSC2535: Advanced Machine Learning by Geoffrey Hinton
 * Advanced Topics in Machine Learning by Andreas Krause
 * Advanced Topics in Machine Learning (Kernel Methods) by Arthur Gretton - Gatsby
 * Pattern Recognition by Ricardo Gutierrez-Osuna
 * Introduction to Pattern Recognition by Jason Corso
 * Pattern Recognition by Olga Veksler
 * Pattern Recognition by Charles Robertson
 * Pattern Recognition by Esa Alhoniemi
 * Advanced Machine Learning by Mehryar Mohri
 * Advanced Machine Learning by Tony Jebara
 * Machine Learning II by Le Song

Specialized

 * STA 663 Statistical Computing and Computation by Cliburn Chan and Janice McCarthy
 * Statistical Machine Learning from Data by Samy Bengio
 * STA 4273H: Statistical Machine Learning by Ruslan Salakhutdinov
 * CS59000: Statistical Machine Learning by Alan Qi
 * Statistical Machine Learning and Data Mining by Yee Whye Teh
 * Identification, Estimation, and Learning by Harry Asada
 * Unsupervised Learning by Zoubin Ghahramani
 * Probabilistic and Unsupervised Learning by Maneesh Sahani - Gatsby
 * Approximate Inference and Learning in Probabilistic Models by Maneesh Sahani - Gatsby
 * Reinforcement Learning by David Silver
 * Reinforcement Learning by Michael Herrmann
 * Pattern Recognition by Xia Hong
 * Pattern Recognition for Machine Vision by Bernd Heisele and Yuri Ivanov
 * CS 5785: Modern Analytics by Serge J. Belongie
 * Advanced Statistical Machine Learning by Stefanos Zafeiriou

Practical

 * Witten, I. H., & Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann.
 * Martinez, W. L., & Martinez, A. R. (2007). Computational statistics handbook with MATLAB. 2nd Ed. CRC press.
 * Martinez, W. L., Martinez, A. R., Martinez, A., & Solka, J. (2010). Exploratory data analysis with MATLAB. 2nd Ed. CRC Press.
 * Sammut, C., & Webb, G. I. (Eds.). (2011). Encyclopedia of Machine Learning. Springer Science & Business Media.
 * Han, J., Pei, J., & Kamber, M. (2011). Data mining: concepts and techniques. 3rd Ed. Morgan Kaufmann.
 * Conway, D., & White, J. (2012). Machine Learning for Hackers. O'Reilly Media.
 * McKinney, W. (2012). Python for data analysis: Data wrangling with Pandas, NumPy, and IPython. O'Reilly Media.
 * Rajaraman, A., & Ullman, J. D. (2012). Mining of Massive Datasets. Cambridge University Press.
 * Brownlee, J. (2013). Clever Algorithms: Statistical Machine Learning Recipes. Jason Brownlee.
 * Schutt, R., & O'Neil, C. (2013). Doing data science: Straight talk from the frontline. " O'Reilly Media, Inc.".
 * Battiti, R., & Brunato, M. (2014). The LION Way. Machine Learning Plus Intelligent Optimization. CreateSpace.
 * Zumel, N., Mount, J., & Porzak, J. (2014). Practical data science with R. Manning.
 * Nolan, D., & Lang, D. T. (2015). Data Science in R: A Case Studies Approach to Computational Reasoning and Problem Solving. CRC Press.
 * Davidson-Pilon, C. (2015). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional.
 * Elston, S. F. (2015). Data Science in the Cloud with Microsoft Azure Machine Learning and R. O'Reilly Media, Inc.
 * Kelleher, J. D., Mac Namee, B., & D'Arcy, A. (2015). Fundamentals of machine learning for predictive data analytics: algorithms, worked examples, and case studies. MIT Press.
 * Lantz, B. (2015). Machine Learning with R. 2nd Ed. Packt Publishing Ltd.
 * Yu-Wei, C. D. C. (2015). Machine Learning with R cookbook. Packt Publishing Ltd.
 * Raschka, S. (2015). Python Machine Learning. Packt Publishing Ltd.
 * Ankan, A., & Panda, A. (2015). Mastering Probabilistic Graphical Models Using Python. Packt Publishing Ltd.
 * Grus, J. (2015). Data Science from Scratch: First Principles with Python. O'Reilly Media.
 * Madhavan, S. (2015). Mastering Python for Data Science. Packt Publishing Ltd.
 * Zaccone, G. (2016). Getting started with TensorFlow. Packt Publishing Ltd.
 * VanderPlas, J. (2016). Python Data Science Handbook: Essential Tools for Working with Data. O'Reilly Media.
 * Müller, A. C., & Guido, S. (2016). Introduction to Machine Learning with Python: A Guide for Data Scientists. O'Reilly Media, Inc.
 * Wickham, H., & Grolemund, G. (2017). R for Data Science. O'Reilly Media.
 * Geron, A. (2017). Hands-on machine learning with Scikit-Learn and TensorFlow: concepts, tools, and techniques to build intelligent systems.O'Reilly Media.

Introductory

 * Mitchell, T. M. (1997). Machine Learning. McGraw Hill.
 * Smola, A., & Vishwanathan, S. V. N. (2008). Introduction to Machine Learning. Cambridge University Press.
 * Alpaydin, E. (2010). Introduction to Machine Learning. MIT Press.
 * Theodoridis, S., Pikrakis, A., Koutroumbas, K., & Cavouras, D. (2010). Introduction to Pattern Recognition: A Matlab Approach. Academic Press.
 * Abu-Mostafa, Y. S., Magdon-Ismail, M., & Lin, H. T. (2012). Learning From Data. AMLBook.
 * Kuhn, M., & Johnson, K. (2013). Applied Predictive Modeling. Springer.
 * Kruse, R., Borgelt, C., Klawonn, F., Moewes, C., Steinbrecher, M., & Held, P. (2013). Computational Intelligence: A Methodological Introduction. Springer Science & Business Media.
 * James, G., Witten, D., & Hastie, T. (2014). An Introduction to Statistical Learning: With Applications in R.
 * Aggarwal, C. C. (2015). Data Mining: The Textbook. Springer.
 * Blum, A., Hopcroft, J., & Kannan, R. (2015). Foundations of Data Science.
 * Nilsson, N. J. (2015). Introduction to machine learning. An early draft of a proposed textbook.

Advanced

 * Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
 * Theodoridis, S., Koutroumbas, K., (2009). Pattern Recognition, 4th Ed., Academic Press
 * Duda, R. O., Hart, P. E., & Stork, D. G. (2012). Pattern Classification. John Wiley & Sons.
 * Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foundations of Machine Learning. MIT press.
 * Shalev-Shwartz, S., & Ben-David, S. (2014). Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press.
 * Theodoridis, S. (2015). Machine Learning: A Bayesian and Optimization Perspective. Academic Press.

Specialized

 * Rojas, R. (1996). Neural Networks: A Systematic Introduction. Springer Science & Business Media.
 * Moon, T. K., &  Stirling, W. C. (2000). Mathematical methods and algorithms for signal processing. Pearson.
 * Webb, A. R. (2002). Statistical Pattern Recognition. 2nd Ed. John Wiley & Sons.
 * MacKay, D. J. (2003). Information Theory, Inference and Learning Algorithms. Cambridge University Press.
 * Kushner, H., & Yin, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications (Vol. 35). 2nd Ed. Springer Science & Business Media.
 * Taylor,J. S. & Cristianini, N. (2004). Kernel Methods for Pattern Analysis. Cambridge University Press.
 * Williams, C. K., & Rasmussen, C. E. (2006). Gaussian Processes for Machine Learning. MIT Press.
 * Vapnik, V. (2006). Estimation of dependences based on empirical data. Springer Science & Business Media.
 * Hastie, T., Tibshirani, R., Friedman, J., Hastie, T., Friedman, J., & Tibshirani, R. (2009). The Elements of Statistical Learning. 2nd Ed. New York: Springer.
 * Koller, D., & Friedman, N. (2009). Probabilistic Graphical Models. MIT Press.
 * Haykin, S. O. (2009). Neural Networks and Learning Machines. 3rd Ed. Pearson.
 * Gentle, J. E. (2009). Computational statistics. Springer.
 * Russell, S. J., & Norvig, P. (2010). "Part IV: Uncertain knowledge and reasoning". Artificial Intelligence: A Modern Approach. Prentice Hall.
 * Bühlmann, P., & Van De Geer, S. (2011). Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer Science & Business Media.
 * Givens, G. H., & Hoeting, J. A. (2012). Computational statistics. 2nd Ed. John Wiley & Sons.
 * Murphy, K. P. (2012). Machine Learning: A Probabilistic Perspective. MIT Press.
 * Barber, D. (2012). Bayesian Reasoning and Machine Learning. Cambridge University Press.
 * Bubeck, S. & Cesa-Bianchi, N. (2012). Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. Foundations and Trends® in Machine Learning, 5(1), 1-122. Now Publishers.
 * Jebara, T. (2012). Machine Learning: Discriminative and Generative (Vol. 755). Springer Science & Business Media.
 * Nielsen, M. A. (2015). Neural Networks and Deep Learning. Determination Press.
 * Goodman, N. D., & Tenenbaum, J. B. (2016). Probabilistic Models of Cognition. 2nd Ed. (link)
 * Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). Deep Learning. MIT Press.

Software

 * PRMLT - MATLAB Toolbox for the book of PRML by C. Bishop
 * pmtk3 - Probabilistic Modeling Toolkit for MLPP book by Murphy in Matlab/Octave (3rd edition)
 * pyprobml - Python code for MLPP book by K. Murphy
 * BRMLtoolbox - MATLAB and Julia code for the BRML book by D. Barber
 * PyBRML - Python code for the BRML book by D. Barber
 * PythonDataScienceHandbook - Python code for the PDSH book by by J. VanderPlas
 * OpenML
 * Microsoft R
 * BigML - Prediction and Analytics tasks under 16MB are free
 * Neural Network Toolbox - MATLAB
 * Torch7 - a scientific computing framework for Machine Learning algorithms (based on Lua)
 * Lush - an OOP language for large-scale numerical and graphic applications (based on Lisp)
 * TensorFlow - Google
 * CNTK - Microsoft
 * Pylearn2 - A Machine Learning research library
 * scikit-learn - Python
 * mlpy - Python
 * Orange - Data Visualization and Analysis
 * Matlab Machine Learning Toolboxes
 * mloss.org. Machine Learning open source software
 * April-ANN - A Pattern Recognizer In Lua with ANNs
 * Weka- Data mining software in Java
 * MLTK - Machine Learning Toolkit in Java
 * OpenNN
 * Bayesian Modeling and Monte Carlo Methods
 * The Lightspeed Matlab Toolbox
 * MCML - broad range support for Monte Carlo methods to implement Machine Learning applications
 * Orange - Visual programming language
 * MLPACK - C++
 * Shogun - C++ toolbox that offers interfaces for MATLAB,Octave, Python, R, Java, Lua, Ruby and C#(mainly for Kernel Machines)
 * autograd - Efficiently computes derivatives of numpy code (Python)
 * pySPACE - Signal Processing And Classification Environment (SPACE) in Python
 * dlib - C++ (with Python API)
 * Computational Statistics Toolbox - MATLAB
 * Exploratory Data Analysis (EDA) Toolbox - MATLAB
 * aimacode - Code for the AIMA book by Russell and Norvig
 * Core ML - Apple

Datasets

 * UCI Machine Learning Repository - a large collection of standard datasets for testing learning algorithms
 * DeepLearning.Net - a list of datasets that can be used for benchmarking Deep Learning algorithms
 * MLdata

Other Resources

 * Artificial Intelligence - Google Scholar Metrics (Top Publications)
 * Computer Vision and Pattern Recognition - Google Scholar Metrics (Top Publications)
 * Data Mining and Analysis - Google Scholar Metrics (Top Publications)
 * NIPS - A top-tier Conference in Machine Learning
 * ICML - A top-tier Conference in Machine Learning
 * Machine Learning Types - Medium
 * Video Tutorials - Youtube channel of 'Mathematical Monk'
 * Basic Concepts in Machine Learning
 * Awesome-Machine-Learning (Github) - A curated list of Machine Learning frameworks, libraries and software (by language)
 * Computational Statistics in Python (2016 version, Github)
 * Comparison of software toolkits
 * Software for Data Mining, Analytics, Data Science, and Knowledge Discovery - KDnuggets
 * Machine Learning and Statistical Learning in R
 * Metacademy - List of concepts in Machine Learning
 * Machine Learning, Statistical Inference and Induction - Notebook
 * A Course in Machine Learning by Hal Daumé III - textbook
 * Courses on Statistical Pattern Recognition - summary of 33 courses
 * FastML - List of Machine Learning courses online
 * Results and Errors Percentage on Standard Datasets - MNIST, CIFAR, Pascal VOC, etc.
 * Machine Learning Surveys - List of literature surveys, reviews, and tutorials on Machine Learning and related topics
 * Machine Learning on Google+ - online community
 * The Shape of Data - Data Mining and Machine Learning blog
 * Data Mining & Machine Learning - a mindmap diagram comparing the two areas
 * Basic Glossary of Machine Learning
 * Machine Learning with MATLAB - free ebook
 * References for Machine Learning - Wikia
 * Which machine learning algorithm should I use? - blog post
 * MACHINE LEARNING BASICS AND ITS TYPES - Blog post