Difference between revisions of "Machine learning"

From Lesswrongwiki
Jump to: navigation, search
(This article could use more attention, but I tried to quickly fix the biggest problems)
 
Line 1: Line 1:
 
{{wikilink}}
 
{{wikilink}}
'''Machine Learning''' refers to the general field of study that deals with automated statistical learning and pattern detection by non-biological systems. It can be seen as a classical sub-domain of artifical intelligence that specifically deals with data analysis, modeling and prediction through the knowledge extracted from the previous (training) samples. As a multi-disciplinary area, it has borrowed concepts and ideas ranging from pure mathematics to cognitive science, all the while trying to exhaustively describe learning systems.
+
'''Machine Learning''' refers to the general field of study that deals with automated statistical learning and pattern detection by non-biological systems. It can be seen as a sub-domain of artificial intelligence that specifically deals with modeling and prediction through the knowledge extracted from training data. As a multi-disciplinary area, it has borrowed concepts and ideas from other areas like pure mathematics and cognitive science.
  
== Most common algorithms ==
+
== Understanding different machine learning algorithms ==
When considering the most used algorithms in Machine Learning, we can take on multiple approaches to describe its subdivisions. It's possible to create a simple taxonomy based on their most prominent characteristics and implementations, such as the suggested by Lotte and colleagues:
+
The most widely used distinction is between unsupervised (e.g. k-means clustering, principal component analysis) vs supervised (e.g. Support Vector Machines, logistic regression) methods.  The first approach identifies interesting patterns (e.g. clusters and latent dimensions) in unlabeled training data, whereas the second takes labeled training data and tries to predict the label for unlabeled data points from the same distribution.
  
Firstly, the most widely distinction is usually made between the ''generative'' or unsupervised (k-NN, k-means clustering, …) vs ''discriminative'' or supervised (Support Vector Machines, Linear Discriminant Analysis, …) ones – while the first is able to spontaneously generate different categories based purely on the data structure, the second kind is only able of distinguishing previously learned classes (through the feeding of correctly identified data). This is probably the most prominent distinction between Machine Learning methods.
+
Another important distinction relates to the bias/variance tradeoff -- some machine learning methods are are capable of recognizing more complex patterns, but the tradeoff is that these methods can overfit and generalize poorly if there's noise in the training data -- especially if there's not much training data available.
  
These same algorithms can be seen as ''static'' (such as simple Neural Networks like Perceptrons), disregarding the temporal\sequential characteristics of the data, or ''dynamic'' (Hidden Markov Chains or Recurrent Neural Networks, for instance), able to account for those temporal dynamics and treating time series.
+
There are also subfields of machine learning devoted to operating on specific kinds of data.  For example, Hidden Markov Models and recurrent neural networks operate on time series data.  Convolutional neural networks are commonly applied to image data.
  
The third and last big difference refers to its sensitivity to the variance within the data, that is, the algorithm’s ability to model the training data - either very closely or more loosely (which in turn influences its ability to generalize).
+
== Applications ==
 +
The use of machine learning has been widespread since its formal definition in the 50’s. The ability to make predictions based on data has been extensively used in areas such as analysis of financial markets, natural language processing and even brain-computer interfaces. Amazon’s product suggestion system makes use of training data in the form of past customer purchases in order to predict what customers might want to buy in the future.
  
== Applications examples ==
+
In addition to its practical usefulness, machine learning has also offered insight into human cognitive organization. It seems likely machine learning will play an important role in the development of [[artificial general intelligence]].
The use of Machine Learning has been widespread since its formal definition in the 50’s. The ability to explore the data structure and make predictions based on previous behavior has been extensively used in areas such as market analysis, natural language processing or even brain-computer interfaces. Amazon’s titles suggestion, for instance, is an example of a deep and recursive system for modeling previous buys and generating possible hypothesis from that data.
 
 
 
Besides the technological advantages of this ability to probe large amounts of data and aid in research as simple tools, the development and study of machine learning methods has also lead to substantial insights into the human cognitive organization. At the same time, although limited in its essence, it seems machine learning will give important contributions to the development of an [[artificial general intelligence]].
 
  
 
== Further Reading & References==
 
== Further Reading & References==
  
*[http://robotics.stanford.edu/~nilsson/MLBOOK.pdf Stanford's introduction to Machine Learning]
+
*[https://web.stanford.edu/~hastie/ElemStatLearn/ The Elements of Statistical Learning]
*Ghahramani, Z. (2004).Unsupervised Learning. In: Bousquet, O., von Luxburg, U. and Raetsch, G. Advanced Lectures in Machine Learning. Lecture Notes in Computer Science, 3176, 72-112. Berlin: Springer-Verlag
 
*Lotte, F., Congedo, M.,  Lécuyer, A., Lamarche, F., Arnaldi, B. (2007). A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces. Journal of Neural Engineering, 4, 1-13
 
*Ng, A. & Jordan, M. (2002). On generative versus discriminative classifiers: a comparison of logistic regression and naive Bayes. Proc. Advances in Neural Information Processing
 
*Rabiner, L. R., (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE, 77, 257–286
 
*Rubinstein, Y. D., & Hastie T. (1997). Discriminative versus informative learning. Proc. 3rd Int. Conf. on Knowledge Discovery and Data Mining
 
  
 
==See Also==
 
==See Also==
  
*[[Decision theory]]
 
*[[Friendly AI]]
 
*[[Game theory]]
 
 
*[[Prediction]]
 
*[[Prediction]]

Latest revision as of 18:53, 22 December 2019

Smallwikipedialogo.png
Wikipedia has an article about

Machine Learning refers to the general field of study that deals with automated statistical learning and pattern detection by non-biological systems. It can be seen as a sub-domain of artificial intelligence that specifically deals with modeling and prediction through the knowledge extracted from training data. As a multi-disciplinary area, it has borrowed concepts and ideas from other areas like pure mathematics and cognitive science.

Understanding different machine learning algorithms

The most widely used distinction is between unsupervised (e.g. k-means clustering, principal component analysis) vs supervised (e.g. Support Vector Machines, logistic regression) methods. The first approach identifies interesting patterns (e.g. clusters and latent dimensions) in unlabeled training data, whereas the second takes labeled training data and tries to predict the label for unlabeled data points from the same distribution.

Another important distinction relates to the bias/variance tradeoff -- some machine learning methods are are capable of recognizing more complex patterns, but the tradeoff is that these methods can overfit and generalize poorly if there's noise in the training data -- especially if there's not much training data available.

There are also subfields of machine learning devoted to operating on specific kinds of data. For example, Hidden Markov Models and recurrent neural networks operate on time series data. Convolutional neural networks are commonly applied to image data.

Applications

The use of machine learning has been widespread since its formal definition in the 50’s. The ability to make predictions based on data has been extensively used in areas such as analysis of financial markets, natural language processing and even brain-computer interfaces. Amazon’s product suggestion system makes use of training data in the form of past customer purchases in order to predict what customers might want to buy in the future.

In addition to its practical usefulness, machine learning has also offered insight into human cognitive organization. It seems likely machine learning will play an important role in the development of artificial general intelligence.

Further Reading & References

See Also