What is Machine Learning?

...Machine Learning

Standard algorithms need a hard coded logic for input and output... On the contrarty machine learning models develops the logic using the input and output data

Input

Output

Standard algorithm has the function:

Machine Learning algorithm estimates the function

(y = f(x)) given the data:

0 : 0

1 : 2

n : 2 * n

Hypothesis: f(x) = 2 * x

Supervised and Unsupervised Learning

Supervised Learning vs. Unsupervised Learning

In Supervised Learning, we have a predefined input and output. Example - for training a sentiment analysis system, we have the text and the sentiment of each text

In Unsupervised Learning, we only have the input data and a defined action to take on them. Example - Clustering, Anomaly Detection, Dimensionality Reduction, etc.

Neural Networks

...Neural Networks

a beautiful, highly flexible and generic biologically-inspired programming paradigm which enables a computer to learn from observational data

Neural Network Architecture

Components

• ​Inputs
• Bias Inputs
• Weights - W
• Hidden layer activation 
• Output layer activation

Significance of Weights(Network Parameter)

Activation Functions

Significance of Biases

Forward Propagation

Back Propagation

Backpropagation - Logic

Loss(Cost) Function

Classification - Cross Entropy Loss:

Regression - Mean Squared Error:

Convex Optimization

Neural Network - Equations

Forward Propagation

Backward Propagation

Loss Function

Suggested Reading: 

Neural Networks

• A Neural Network in 11 Lines of Python
• http://neuralnetworksanddeeplearning.com/chap2.html
• Calculus on Computational Graphs
• Convex Optimization Overview

Introduction to Theano

Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently.

• Built on top of numpy
• Symbolic Expressions
• Automatic Differentiation
• In built integration for GPU Computation
• Python Interface
• Shared Variables

Theano Syntax

import numpy
import theano
import theano.tensor as T
from theano import pp

x = T.dscalar('x')
y = x ** 2
gy = T.grad(y, x)
f = theano.function([x], gy)
f(4)
# array(8.0)

Suggested Reading:

Theano

• DeepLearning.net Theano Tutorial
• Speading up your neural network with Theano

Deep Neural Networks

Suggested Reading:

Deep Learning Models

• Regularization and Activation Functions in Neural Nets
• Learning Deep Architectures
• Deep Learning with Neural Networks
• Deep Learning - An MIT Press Book
• Practical recommendations for gradient-based training of deep architectures