Machine learning
{{for|the journal|Machine Learning (journal)}}
{{portalpar|Artificial intelligence}}
As a broad subfield of [[artificial intelligence]], '''machine learning''' is concerned with the design and development of [[algorithm]]s and techniques that allow computers to "learn". At a general level, there are two types of learning: [[Inductive reasoning|inductive]], and [[Deductive reasoning|deductive]].  Inductive machine learning methods extract rules and patterns out of massive data sets.

The major focus of machine learning research is to extract information from data automatically, by computational and statistical methods. Hence, machine learning is closely related not only to [[data mining]] and [[statistics]], but also [[theoretical computer science]].

==Applications==
Machine learning has a wide spectrum of applications including [[natural language processing]], [[syntactic pattern recognition]], [[search engines]], [[diagnosis|medical diagnosis]], [[bioinformatics]], [[brain-machine interfaces]] and [[cheminformatics]], detecting [[credit card fraud]], [[stock market]] analysis, classifying [[DNA sequence]]s, [[speech recognition|speech]] and [[handwriting recognition]], [[object recognition]] in [[computer vision]], [[strategy game|game playing]] and [[robot locomotion]].

== Human interaction ==
Some machine learning systems attempt to eliminate the need for human intuition in the analysis of the data, while others adopt a collaborative approach between human and machine. Human intuition cannot be entirely eliminated since the designer of the system must specify how the data is to be represented and what mechanisms will be used to search for a characterization of the data. Machine learning can be viewed as an attempt to automate parts of the [[scientific method]]{{Fact|date=May 2008}}. 

Some statistical machine learning researchers create methods within the framework of [[Bayesian statistics]].

== Algorithm types ==<!-- This section is linked from [[Learning]] -->
Machine learning [[algorithm]]s are organized into a [[taxonomy]], based on the desired outcome of the algorithm. Common algorithm types include:

* [[Supervised learning]] &mdash; in which the algorithm generates a function that maps inputs to desired outputs. One standard formulation of the supervised learning task is the [[statistical classification|classification]] problem: the learner is required to learn (to approximate) the behavior of a function which maps a vector <math>[X_1, X_2, \ldots X_N]\,</math> into one of several classes by looking at several input-output examples of the function.
* [[Unsupervised learning]] &mdash; An agent which models a set of inputs: labeled examples are not available.
* [[Semi-supervised learning]] &mdash; which combines both labeled and unlabeled examples to generate an appropriate function or classifier.
* [[Reinforcement learning]] &mdash; in which the algorithm learns a policy of how to act given an observation of the world.  Every action has some impact in the environment, and the environment provides feedback that guides the learning algorithm.
* [[Transduction (machine learning)|Transduction]] &mdash; similar to supervised learning, but does not explicitly construct a function: instead, tries to predict new outputs based on training inputs, training outputs, and test inputs which are available while training.
* [[Leaning to learn]] &mdash; in which the algorithm learns its own [[inductive bias]] based on previous experience.

The computational analysis of machine learning algorithms and their performance is a branch of [[theoretical computer science]] known as [[computational learning theory]].

== Machine learning topics ==
:''This list represents the topics covered on a typical machine learning course.''
{{MultiCol}}
;Prerequisites
*[[Bayesian theory]]

;Modeling [[conditional probability|conditional probability density functions]]<nowiki>:</nowiki> [[Regression analysis|regression]] and [[Statistical classification|classification]]
*[[Artificial neural network]]s
*[[Decision tree]]s
*[[Gene expression programming]]
*[[Genetic algorithms]]
*[[Genetic programming]]
*[[Holographic associative memory]]
*[[Inductive Logic Programming]]
*[[Kriging|Gaussian process regression]]
*[[Linear discriminant analysis]]
*[[Nearest neighbor (pattern recognition)|K-nearest neighbor]]
*[[Minimum message length]]
*[[Perceptron]]
*[[Quadratic classifier]]
*[[Radial basis function network]]s
*[[Support vector machine]]s

;Algorithms for estimating model parameters:
*[[Dynamic programming]]
*[[Expectation-maximization algorithm]]

;Modeling [[probability density function]]s through [[generative model]]s:
*[[Graphical model]]s including [[Bayesian network]]s and [[Markov network|Markov random fields]]
*[[Generative topographic map]]

{{ColBreak}}
;Approximate inference techniques
*[[Monte Carlo method]]s
*[[Variational Bayes]]
*[[Variable-order Markov model]]s
*[[Variable-order Bayesian network]]s
*[[Loopy belief propagation]]

;Optimization
*Most of methods listed above either use [[Optimization (mathematics)|optimization]] or are instances of optimization algorithms

;Meta-learning (ensemble methods)
*[[Boosting]]
*[[Bootstrap aggregating]]
*[[Random forest]]
*[[Weighted majority algorithm]]

;Inductive transfer and learning to learn
*[[Inductive transfer]]
*[[Reinforcement learning]]
*[[Temporal difference learning]]
*[[Monte-Carlo method]]
{{EndMultiCol}}

== See also ==
<div style="-moz-column-count:3; column-count:3;">
* [[Autonomous robot]]
* [[Computational intelligence]]
* [[Fuzzy logic]]
* [[Inductive logic programming]]
* [[Intelligent system]]
* [[Journal of Machine Learning Research]]
* [[List of important publications in computer science#Machine learning|Important publications in machine learning (computer science)]]
* [[List of numerical analysis software]]
*[[MLMTA]] Machine Learning: Models, Technologies & Applications
* [[Multi-label classification]]
* [[NIPS|Neural Information Processing Systems (NIPS)]] (conference)
* [[Neural network software]]
* [[Pattern recognition]]
* [[Predictive analytics]]
* [[Weka (machine learning)|WEKA]] Open-source machine learning framework for pattern classification, regression, and clustering.
</div>

== References ==
{{Nofootnotes|date=February 2008}}

== Further reading ==
* Ethem Alpaydın (2004) ''Introduction to Machine Learning (Adaptive Computation and Machine Learning)'', MIT Press, ISBN 0262012111
* Christopher M. Bishop (2007) ''Pattern Recognition and Machine Learning'', Springer ISBN 0-387-31073-8.
* Ryszard S. Michalski, Jaime G. Carbonell, Tom M. Mitchell (1983), ''Machine Learning: An Artificial Intelligence Approach'', Tioga Publishing Company, ISBN 0-935382-05-4.
* Ryszard S. Michalski, Jaime G. Carbonell, Tom M. Mitchell (1986), ''Machine Learning: An Artificial Intelligence Approach, Volume II'', Morgan Kaufmann, ISBN 0-934613-00-1.
* Yves Kodratoff, Ryszard S. Michalski (1990), ''Machine Learning: An Artificial Intelligence Approach, Volume III'', Morgan Kaufmann, ISBN 1-55860-119-8.
* Ryszard S. Michalski, George Tecuci (1994), ''Machine Learning: A Multistrategy Approach'', Volume IV, Morgan Kaufmann, ISBN 1-55860-251-8.
* Bhagat, P. M. (2005). ''Pattern Recognition in Industry'', Elsevier. ISBN 0-08-044538-1.
* Bishop, C. M. (1995). ''Neural Networks for Pattern Recognition'', Oxford University Press. ISBN 0-19-853864-2.
* Richard O. Duda, Peter E. Hart, David G. Stork (2001) ''Pattern classification'' (2nd edition), Wiley, New York, ISBN 0-471-05669-3.
* Huang T.-M., Kecman V., Kopriva I. (2006), [http://learning-from-data.com Kernel Based Algorithms for Mining Huge Data Sets, Supervised, Semi-supervised, and Unsupervised Learning],  Springer-Verlag, Berlin, Heidelberg, 260 pp. 96 illus., Hardcover, ISBN 3-540-31681-7.
* KECMAN Vojislav (2001), [http://support-vector.ws LEARNING AND SOFT COMPUTING, Support Vector Machines, Neural Networks and Fuzzy Logic Models], The MIT Press, Cambridge, MA, 608 pp., 268 illus., ISBN 0-262-11255-8.
* MacKay, D. J. C. (2003). ''[http://www.inference.phy.cam.ac.uk/mackay/itila/ Information Theory, Inference, and Learning Algorithms]'', Cambridge University Press.  ISBN 0-521-64298-1.
* Mitchell, T. (1997). ''Machine Learning'', McGraw Hill. ISBN 0-07-042807-7.
* Ian H. Witten and Eibe Frank "Data Mining: Practical machine learning tools and techniques" Morgan Kaufmann ISBN 0-12-088407-0.
* Sholom Weiss and Casimir Kulikowski (1991). ''Computer Systems That Learn'', Morgan Kaufmann.  ISBN 1-55860-065-5.
* Mierswa, Ingo and Wurst, Michael and [[Ralf Klinkenberg|Klinkenberg, Ralf]] and Scholz, Martin and Euler, Timm: ''YALE: Rapid Prototyping for Complex Data Mining Tasks'', in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-06), 2006.
* Trevor Hastie, Robert Tibshirani and Jerome Friedman (2001). ''[http://www-stat.stanford.edu/~tibs/ElemStatLearn/ The Elements of Statistical Learning]'', Springer. ISBN 0387952845.
* Vladimir Vapnik (1998). ''Statistical Learning Theory''. Wiley-Interscience, ISBN 0471030031.

==External links==
* [http://www.machinelearning.org/ International Machine Learning Society]
* [http://www.ml.cmu.edu/ Machine Learning Department at CMU]
* [http://www.mli.gmu.edu/ Machine Learning and Inference Laboratory at George Mason University]
* [http://www.anc.ed.ac.uk/machine-learning/ Machine Learning Group University of Edinburgh]
* [http://www.cs.iastate.edu/~honavar/Courses/cs673/machine-learning-courses.html Index of Machine Learning Courses]
* [http://www.kmining.com/info_conferences.html Kmining List of machine learning, data mining and KDD scientific conferences]
* [http://scholarpedia.org/article/Encyclopedia_of_Computational_Intelligence The Encyclopedia of Computational Intelligence]
* [http://mloss.org/about/ Machine Learning Open Source Software]
* [http://www.patternrecognition.co.za/tutorials.html Machine Learning Tutorials]

[[Category:Computer vision]]
[[Category:Machine learning]]
[[Category:Learning]]
[[Category:Cybernetics]]

[[ar:تعلم آلي]]
[[bg:Машинно самообучение]]
[[ca:Aprenentatge automàtic]]
[[cs:Strojové učení]]
[[de:Maschinelles Lernen]]
[[et:Masinõppimine]]
[[el:Μηχανική Μάθηση]]
[[es:Aprendizaje automático]]
[[fa:یادگیری ماشینی]]
[[fr:Apprentissage automatique]]
[[ko:기계 학습]]
[[it:Apprendimento automatico]]
[[he:למידה חישובית]]
[[lt:Sistemos mokymasis]]
[[nl:Machinaal leren]]
[[ja:機械学習]]
[[pl:Uczenie maszynowe]]
[[pt:Aprendizagem de máquina]]
[[ru:Машинное обучение]]
[[sl:Strojno učenje]]
[[fi:Koneoppiminen]]
[[sv:Maskininlärning]]
[[th:การเรียนรู้ของเครื่อง]]
[[vi:Học máy]]
[[uk:Машинне навчання]]
[[zh:机器学习]]