Outline of machine learning

The following outline is provided as an overview of and topical guide to machine learning:

Machine learning – subfield of computer science^[1] (more particularly soft computing) that evolved from the study of pattern recognition and computational learning theory in artificial intelligence.^[1] In 1959, Arthur Samuel defined machine learning as a "Field of study that gives computers the ability to learn without being explicitly programmed".^[2] Machine learning explores the study and construction of algorithms that can learn from and make predictions on data.^[3] Such algorithms operate by building a model from an example training set of input observations in order to make data-driven predictions or decisions expressed as outputs, rather than following strictly static program instructions.

What type of thing is machine learning?

• An academic discipline
• A branch of science
  • An applied science
    • A subfield of computer science
      • A branch of artificial intelligence
      • A subfield of soft computing

Branches of machine learning

Subfields

• Computational learning theory – studying the design and analysis of machine learning algorithms.^[4]
• Grammar induction
• Meta learning

Cross-disciplinary fields

• Adversarial machine learning
• Predictive analytics
• Quantum machine learning
• Robot learning

Machine learning hardware

• Graphics processing unit
• Tensor processing unit
• Vision processing unit

Machine learning tools

• Comparison of deep learning software
  • Comparison of deep learning software/Resources

Proprietary frameworks

• Amazon Machine Learning
• DistBelief – replaced by TensorFlow

Open source frameworks

• Apache Singa
• MLPACK
• TensorFlow

Machine learning libraries

Machine learning methods

• Dimensionality reduction
  • CCA
  • Factor analysis
  • Independent component analysis (ICA)
  • Linear discriminant analysis (LDA)
  • Multidimensional scaling (MDS)
  • Non-negative matrix factorization (NMF)
  • Partial least squares regression (PLSR)
  • Principal component analysis (PCA)
  • Principal component regression (PCR)
  • Projection pursuit
  • Sammon mapping
  • t-distributed stochastic neighbor embedding (t-SNE)
• Ensemble learning
  • Boosting
  • Bootstrap aggregating (Bagging)
  • AdaBoost
  • Stacked Generalization (blending)
  • Gradient boosting machine (GBM)
  • Gradient boosted decision tree (GBRT)
  • Random Forest
• Instance-based algorithm
  • K-nearest neighbors algorithm (KNN)
  • Learning vector quantization (LVQ)
  • Self-organizing map (SOM)
• Regression analysis
  • Logistic regression
  • Ordinary least squares regression (OLSR)
  • Linear regression
  • Stepwise regression
  • Multivariate adaptive regression splines (MARS)
• Regularization algorithm
  • Ridge regression
  • Least Absolute Shrinkage and Selection Operator (LASSO)
  • Elastic net
  • Least-angle regression (LARS)
• Classifiers
  • Probabilistic classifier
    • Naive Bayes classifier
  • Binary classifier
  • Linear classifier
  • Hierarchical classifier

Supervised learning

• Supervised learning
  • AODE
  • Association rule learning algorithms
    • Apriori algorithm
    • Eclat algorithm
  • Case-based reasoning
  • Gaussian process regression
  • Gene expression programming
  • Group method of data handling (GMDH)
  • Inductive logic programming
  • Instance-based learning
  • Lazy learning
  • Learning Automata
  • Learning Vector Quantization
  • Logistic Model Tree
  • Minimum message length (decision trees, decision graphs, etc.)
    • Nearest Neighbor Algorithm
    • Analogical modeling
  • Probably approximately correct learning (PAC) learning
  • Ripple down rules, a knowledge acquisition methodology
  • Symbolic machine learning algorithms
  • Support vector machines
  • Random Forests
  • Ensembles of classifiers
    • Bootstrap aggregating (bagging)
    • Boosting (meta-algorithm)
  • Ordinal classification
  • Information fuzzy networks (IFN)
  • Conditional Random Field
  • ANOVA
  • Quadratic classifiers
  • k-nearest neighbor
  • Boosting
    • SPRINT
  • Bayesian networks
    • Naive Bayes
  • Hidden Markov models

Artificial neural network

• Artificial neural network
  • Autoencoder
  • Backpropagation
  • Boltzmann machine
  • Convolutional neural network
  • Deep learning
  • Hopfield network
  • Multilayer perceptron
  • Perceptron
  • Radial basis function network (RBFN)
  • Restricted Boltzmann machine
  • Recurrent neural network (RNN)
  • Self-organizing map (SOM)
  • Spiking neural network

Bayesian

• Bayesian statistics
  • Bayesian knowledge base
  • Naive Bayes
  • Gaussian Naive Bayes
  • Multinomial Naive Bayes
  • Averaged One-Dependence Estimators (AODE)
  • Bayesian Belief Network (BBN)
  • Bayesian Network (BN)

Decision tree

• Decision tree algorithm
  • Classification and regression tree (CART)
  • Iterative Dichotomiser 3 (ID3)
  • C4.5 algorithm
  • C5.0 algorithm
  • Chi-squared Automatic Interaction Detection (CHAID)
  • Decision stump
  • Conditional decision tree
  • ID3 algorithm
  • Random forest
  • SLIQ

Linear classifier

• Linear classifier
  • Fisher's linear discriminant
  • Linear regression
  • Logistic regression
  • Multinomial logistic regression
  • Naive Bayes classifier
  • Perceptron
  • Support vector machine

Unsupervised learning

• Unsupervised learning
  • Expectation-maximization algorithm
  • Vector Quantization
  • Generative topographic map
  • Information bottleneck method

Artificial neural network

• Artificial neural network
  • Feedforward neural network
    • Extreme learning machine
  • Logic learning machine
  • Self-organizing map

Association rule learning

• Association rule learning
  • Apriori algorithm
  • Eclat algorithm
  • FP-growth algorithm

Hierarchical clustering

• Hierarchical clustering
  • Single-linkage clustering
  • Conceptual clustering

Cluster analysis

• Cluster analysis
  • BIRCH
  • DBSCAN
  • Expectation-maximization (EM)
  • Fuzzy clustering
  • Hierarchical Clustering
  • K-means algorithm
  • K-means clustering
  • K-medians
  • Mean-shift
  • OPTICS algorithm

Anomaly detection

• Anomaly detection
• k-nearest neighbors classification (k-NN)
• Local outlier factor

Semi-supervised learning

• Semi-supervised learning
  • Generative models
  • Low-density separation
  • Graph-based methods
  • Co-training

Reinforcement learning

• Reinforcement learning
  • Temporal difference learning
  • Q-learning
  • Learning Automata
  • State-Action-Reward-State-Action (SARSA)

Deep learning

• Deep learning
  • Deep belief networks
  • Deep Boltzmann machines
  • Deep Convolutional neural networks
  • Deep Recurrent neural networks
  • Hierarchical temporal memory
  • Deep Boltzmann Machine (DBM)
  • Stacked Auto-Encoders

Others

• Data Pre-processing
• Online machine learning

Applications of machine learning

• Biomedical informatics
• Computer vision
• Customer relationship management –
• Data mining
• Email filtering
• Inverted pendulum – balance and equilibrium system.
• Natural language processing
  • Automatic summarization
  • Automatic taxonomy construction
  • Dialog system
  • Grammar checker
  • Language recognition
    • Handwriting recognition
    • Optical character recognition
    • Speech recognition
  • Machine translation
  • Question answering
  • Speech synthesis
  • Text simplification
• Pattern recognition
  • Facial recognition system
  • Handwriting recognition
  • Image recognition
  • Optical character recognition
  • Speech recognition
• Recommendation system
• Search engine

Machine learning problems and tasks

• Anomaly detection
• Association rules
• Bias-variance dilemma
• Classification
• Clustering
• Empirical risk minimization
• Feature engineering
• Feature learning
• Learning to rank
• Occam learning
• Online learning
• PAC learning
• Regression
• Reinforcement Learning
• Semi-supervised learning
• Statistical learning
• Structured prediction
  • Graphical models
    • Bayesian network
    • Conditional random field (CRF)
    • Hidden Markov model (HMM)
• Unsupervised learning
• VC theory

Machine learning research

• List of artificial intelligence projects
• List of datasets for machine learning research

History of machine learning

• Timeline of machine learning

Machine learning projects

• DeepMind
• Google Brain

Machine learning organizations

• Knowledge Engineering and Machine Learning Group

Machine learning venues

Machine learning conferences and workshops

• Artificial Intelligence and Security (AISec) (co-located workshop with CCS)
• Conference on Neural Information Processing Systems (NIPS)
• ECML PKDD
• International Conference on Machine Learning (ICML)

Machine learning journals

• Machine Learning
• Journal of Machine Learning Research (JMLR)
• Neural Computation

Persons influential in machine learning

• Alberto Broggi
• Andrei Knyazev
• Andrew McCallum
• Andrew Ng
• Armin B. Cremers
• Ayanna Howard
• Barney Pell
• Ben Goertzel
• Ben Taskar
• Bernhard Schölkopf
• Brian D. Ripley
• Christopher G. Atkeson
• Corinna Cortes
• Demis Hassabis
• Douglas Lenat
• Eric Xing
• Ernst Dickmanns
• Geoffrey Hinton – co-inventor of the backpropagation and contrastive divergence training algorithms
• Hans-Peter Kriegel
• Hartmut Neven
• Heikki Mannila
• Jacek M. Zurada
• Jaime Carbonell
• Jerome H. Friedman
• John D. Lafferty
• John Platt – invented SMO and Platt scaling
• Julie Beth Lovins
• Jürgen Schmidhuber
• Karl Steinbuch
• Katia Sycara
• Leo Breiman – invented bagging and random forests
• Lise Getoor
• Luca Maria Gambardella
• Léon Bottou
• Marcus Hutter
• Mehryar Mohri
• Michael Collins
• Michael I. Jordan
• Michael L. Littman
• Nando de Freitas
• Ofer Dekel
• Oren Etzioni
• Pedro Domingos
• Peter Flach
• Pierre Baldi
• Pushmeet Kohli
• Ray Kurzweil
• Rayid Ghani
• Ross Quinlan
• Salvatore J. Stolfo
• Sebastian Thrun
• Selmer Bringsjord
• Sepp Hochreiter
• Shane Legg
• Stephen Muggleton
• Steve Omohundro
• Tom M. Mitchell
• Trevor Hastie
• Vasant Honavar
• Vladimir Vapnik – co-inventor of the SVM and VC theory
• Yann LeCun – invented convolutional neural networks
• Yasuo Matsuyama
• Yoshua Bengio
• Zoubin Ghahramani

See also

• Outline of artificial intelligence
  • Outline of computer vision
  • Outline of natural language processing
• Outline of robotics

Further reading

• Trevor Hastie, Robert Tibshirani and Jerome H. Friedman (2001). The Elements of Statistical Learning, Springer. ISBN 0-387-95284-5.
• Pedro Domingos (September 2015), The Master Algorithm, Basic Books, ISBN 978-0-465-06570-7
• Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar (2012). Foundations of Machine Learning, The MIT Press. ISBN 978-0-262-01825-8.
• Ian H. Witten and Eibe Frank (2011). Data Mining: Practical machine learning tools and techniques Morgan Kaufmann, 664pp., ISBN 978-0-12-374856-0.
• David J. C. MacKay. Information Theory, Inference, and Learning Algorithms Cambridge: Cambridge University Press, 2003. ISBN 0-521-64298-1
• Richard O. Duda, Peter E. Hart, David G. Stork (2001) Pattern classification (2nd edition), Wiley, New York, ISBN 0-471-05669-3.
• Christopher Bishop (1995). Neural Networks for Pattern Recognition, Oxford University Press. ISBN 0-19-853864-2.
• Vladimir Vapnik (1998). Statistical Learning Theory. Wiley-Interscience, ISBN 0-471-03003-1.
• Ray Solomonoff, An Inductive Inference Machine, IRE Convention Record, Section on Information Theory, Part 2, pp., 56-62, 1957.
• Ray Solomonoff, "An Inductive Inference Machine" A privately circulated report from the 1956 Dartmouth Summer Research Conference on AI.

References

External links

• Data Science: Data to Insights from MIT (machine learning)
• International Machine Learning Society
• Popular online course by Andrew Ng, at Coursera. It uses GNU Octave. The course is a free version of Stanford University's actual course taught by Ng, whose lectures are also available for free.
• mloss is an academic database of open-source machine learning software.