Machine Learning

Artificial intelligence Portal

Machine learning is an area of artificial intelligence concerned with the development of techniques which allow computers to "learn". More specifically, machine learning is a method for creating computer programs by the analysis of data sets. Machine learning overlaps heavily with statistics, since both fields study the analysis of data, but unlike statistics, machine learning is concerned with the algorithmic complexity of computational implementations. Many inference problems turn out to be NP-hard, so part of machine learning research is the development of tractable approximate inference algorithms.

Machine learning has a wide spectrum of applications including search engines, medical diagnosis, detecting credit card fraud, stock market analysis, classifying DNA sequences, speech and handwriting recognition, game playing and robot locomotion.

Human interaction

Some machine learning systems attempt to eliminate the need for human intuition in the analysis of the data, while others adopt a collaborative approach between human and machine. Human intuition cannot be entirely eliminated since the designer of the system must specify how the data are to be represented and what mechanisms will be used to search for a characterization of the data. Machine learning can be viewed as an attempt to automate parts of the scientific method. Some machine learning researchers create methods within the framework of Bayesian statistics.

Algorithm types

Machine learning algorithms are organized into a taxonomy, based on the desired outcome of the algorithm. Common algorithm types include:

• supervised learning --- where the algorithm generates a function that maps inputs to desired outputs. One standard formulation of the supervised learning task is the classification problem: the learner is required to learn (to approximate the behavior of) a function which maps a vector into one of several classes by looking at several input-output examples of the function.
• unsupervised learning --- which models a set of inputs: labeled examples are not available.
• semi-supervised learning --- which combines both labeled and unlabeled examples to generate an appropriate function or classifier.
• reinforcement learning --- where the algorithm learns a policy of how to act given an observation of the world. Every action has some impact in the environment, and the environment provides feedback that guides the learning algorithm.
• transduction --- similar to supervised learning, but does not explicitly construct a function: instead, tries to predict new outputs based on training inputs, training outputs, and new inputs.
• learning to learn --- where the algorithm learns its own inductive bias based on previous experience.

The performance and computational analysis of machine learning algorithms is a branch of statistics known as computational learning theory.

Machine learning topics

This list represents the topics covered on a typical machine learning course.

• Modeling conditional probability density functions: regression and classification
  • Artificial neural networks
  • Decision trees
  • Gene expression programming
  • Genetic Programming
  • Gaussian process regression
  • Linear discriminant analysis
  • k-Nearest Neighbor
  • Minimum message length
  • Perceptron
  • Radial basis functions
  • Support vector machines
• Modeling probability density functions through generative models:
  • Expectation-maximization algorithm
  • Graphical models including Bayesian networks and Markov Random Fields
  • Generative Topographic Mapping
• Appromixate inference techniques:
  • Markov chain Monte Carlo method
  • Variational methods
• Optimization: most of methods listed above either use optimization or are instances of optimization algorithms.

See also

• Artificial intelligence
• Computational intelligence
• Data mining
• Pattern recognition
• Important publications in machine learning (computer science)
• Important publications in machine learning (statistics)
• Autonomous robot
• Inductive logic programming

Bibliography

• Bishop, C. M. (1995). Neural Networks for Pattern Recognition, Oxford University Press. ISBN 0198538642
• Richard O. Duda, Peter E. Hart, David G. Stork (2001) Pattern classification (2nd edition), Wiley, New York, ISBN 0471056693.
• MacKay, D. J. C. (2003). Information Theory, Inference, and Learning Algorithms, Cambridge University Press. ISBN 0521642981
• Mitchell, T. (1997). Machine Learning, McGraw Hill. ISBN 0070428077
• Sholom Weiss and Casimir Kulikowski (1991). Computer Systems That Learn, Morgan Kaufmann. ISBN 1-55860-065-5

External links

General resources

• UCI description
• MLnet Mailing List
• Index of Machine Learning Courses
• Kmining List of machine learning, data mining and KDD scientific conferences
• Book "Intelligent Systems and their Societies" by Walter Fritz
• Links from Open Directory Project
• Eruditionhome Great directory site for machine learning.
• MLpedia – wiki dedicated to machine learning.

Journals and Conferences

• Journal of Machine Learning Research
• Machine Learning Journal
• Machine Learning papers @ CiteSeer
• NIPS: Neural Information Processing Systems
• ICML: International Conference on Machine Learning

Research groups

• Machine Learning and Biological Computation Group @ University of Bristol
• Statistical Multimedia Learning Group @ University of British Columbia
• Machine Learning @ Cornell University
• Intelligent Data Analysis Group @ Fraunhofer FIRST, Berlin
• Machine Learning and Natural Language Processing @ University of Freiburg
• Machine Learning @ The Hebrew University
• Center for Computational Intelligence, Learning and Discovery @ Iowa State University
• Machine Learning Systems Group @ the Jet Propulsion Laboratory, California Institute of Technology
• Department of Knowledge Technologies @ Jozef Stefan Institute
• Machine Learning Group @ Université Libre de Bruxelles
• Department of Empirical Inference @ Max Planck Institute for Biological Cybernetics, Tübingen
• Machine Learning and Applied Statistics @ Microsoft Research
• Machine Learning Group @ University of Toronto
• Machine Learning and Data Mining in Bioinformatics Group @ TU München

Software

• SPIDER is a complete machine learning toolbox for MATLAB.
• PRTools is another complete package similar to SPIDER and implemented in MATLAB. SPIDER seems to have more native support and functions for kernel methods, but PRTools has a slightly larger variety of other machine learning tools. PRTools has an accompanying textbook and much better documentation. Both SPIDER and PRTools are available freely for non-commercial applications.
• Orange is a machine learning suite with Python scripting and a visual programming interface.
• YALE is a powerful and free tool for Machine Learning and Data Mining.
• Weka Machine Learning Software
• MATLAB, by The MathWorks, has toolbox support for many machine learning tools. The Bioinformatics toolbox includes Support Vector Machines and KNN classifiers. The Statistics toolbox includes linear discriminant and decision tree classification. The Neural Network toolbox is a complete set of tools for implementing Neural Networks (PRTools relies on it for its neural network classifiers). New methods for classifier performance evaluation and cross validation make MATLAB more attractive for machine learning.
• MLC++ is a library of C++ classes for supervised machine learning
• MDR is an open-source software package for detecting attribute interactions using the multifactor dimensionality reduction (MDR) method.