The what, The when and The how
- Arthur Samuel (1959) said that machine learning is the field of study that gives computers the ability to learn without being explicitly programmed. The goal of machine learning is to develop programs that learn from experience, automatically improve the performance and adapt to new environments over time
Machine learning techniques are well suited for real life problems that use methods to extract useful information from complex and intractable problems in less time
They can be tolerant to data that is inaccurate, partially incorrect or uncertain
These methods can be used to construct models and make predictions
- Machine learning is programming computers to optimize a performance criterion using example data or past experience
- There is no need to “learn” to calculate payroll
Learning is used when
- Human expertise does not exist (navigating on Mars)
- Humans are unable to explain their expertise (speech recognition)
- Solution changes in time (routing on a computer network)
- Solution needs to be adapted to particular cases (user biometrics)
The why: Applications
- Financial Banks
- Stock Market
- The World Wide Web
The why: Applications. More
Machine learning is preferred approach to
- Speech recognition, Natural language processing
- Computer Vision
- Robot Control
- Computational Biology
This trend is accelerating
- Improved Machine Learning Algorithms
- Improved data capture, networking, faster computers
- Software too complex to write by hand
- New sensors/IO devices
- Demand for self-customization to user, environment
- Supervised learning makes predictions when the outcome variable is available while training models whereas in unsupervised learning attempts to search for relevant patterns while outcome variable is unknown.
- Examples of supervised learning are spam prediction, fault prediction. In supervised learning, classification techniques such as decision tree, neural networks, and support vector machines are used.
- In unsupervised learning, clustering methods are used to identify patterns from un-labeled samples. There is no such supervisor and we only have input data. The aim is to find the regularities in the input. There is a structure to the input space such that certain patterns occur more often than others, and we want to see what generally happens and what does not. In statistics, this is called density estimation.
Types of Machine Learning
- Regression: Continuous valued outputs
- Classification: Discrete valued outputs
- Regression algorithms include linear regression, ridge regression and so on
- Classification algorithms include logistic regression, support vector machines, decision trees and so on
Types of Supervised Learning
Clustering: The idea behind clustering is to make divisions, or clusters based on some commonality, some underlying patterns etc.
- K-Means Clustering – clustering your data points into a number (K) of mutually exclusive clusters. A lot of the complexity surrounds how to pick the right number for K.
Hierarchical Clustering – clustering your data points into parent and child clusters. You might split your customers between younger and older ages, and then split each of those groups into their own individual clusters as well.
In the case of hierarchical clustering, we want the data to be split in a tree-like fashion with leaves and nodes, instead of several scattered clusters. We can build these in two ways - top-down and bottom-up.
Types of Unupervised Learning
- Gathering data
- Preparing that data
- Choosing the model
- Hyper-parameter tuning
Steps in building a machine learning system
- There are two types of data: discrete (non-metric) and continuous (metric)
- Discrete type of data is of categorical or ordinal type. For example, whether a component is faulty or is not faulty
- In contrast, continuous (metric) of data represents amount or magnitude such as lines of source code
Types of data
Non-metric (discrete) measurements can have either of the following scales:
- Nominal: When scale can be divided into categories of classes and there is no ranking or ordering in the categories of classes. Eg: Class x can be “faulty” or “not faulty”
- Ordinal: This scale can also be divided into categories of classes. However, these categories have ranking or ordering. Eg: The impact of a fault on the system can be “high”, “medium” or “low”
The metric (continuous) scale can be divided into the following:
- Interval: On any part of the scale, the difference between two adjacent points is equal. The scale has no arbitrary zero point. Eg: Temperatures. In this scale, one value cannot be represented as a multiple of another due to absence of arbitrary zero point
- Ratio: They are like interval scales with an absolute zero point and one value can be expressed as a multiple of another. Eg: Weight
- Absolute: This scale only represents counts. Eg: the number of faults encountered during inspections
Digression: Semi-supervised and Reinforcement Learning
Apart from the categories mentioned above, there are two other fields of machine learning: semi-supervised and reinforcement learning.
1. Reinforcement learning has two components: an agent and an environment. The agent selects and executes an action according to a policy, observes its impact on the environment (and is either rewarded or penalized), and then changes its actions appropriately.
2. Semi-supervised learning is a combination of supervised and unsupervised learning. In this case, only a few samples in your data have labels (as in supervised learning); the rest is unlabeled (as in unsupervised learning). The model uses the labeled samples for learning, and generalizes further and identifies underlying patterns etc. using the unlabeled samples.
Here are some resources for you to study machine learning from, and also applying it to various problems.
- Machine Learning: Andrew NG, Coursera. (MUST)
- Blogs: wildml’s deep learning tutorials (not for beginners) are beautiful. Also www.machinelearningmastery.com is the numero uno blog for all things ML.
- Software: Install Anaconda. Spyder will come installed with it; that’s the one which you’ll be writing code in. There will be other places, obviously (like Pycharm) – but this is the most convenient and useful
- Some of the popular frameworks that are in use for machine learning are scikit-learn, pandas, Tensorflow, Keras, Theano and PyTorch.
Datasets: I use these places for my data:
Machine Learning: The what, the when and the how
By Anjali Bhavan