Machine Learning

Dr. Sergey Kosov

Your Teacher

University Lecturer Data Engineering

Computer Science & Electrical Engineering

Dr. Sergey Kosov

What is Machine Learning?

(Machine) Learning is any process by which a system improves performance from experience

-Herbert Simon

Herbert Alexander Simon

June 1916 - Feb. 2001

An economist, political scientist and cognitive psychologist

Turing Award 1975

Nobel Prize in Economics 1978

Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it learn for themselves.

Why Machine Learning?

Develop systems that can automatically adapt and customise themselves to individual users
Discover new knowledge from large databases (data mining)
Ability to mimic human and replace certain monotonous tasks - which require some intelligence
Develop systems that are too difficult / expensive to construct manually because they require specific detailed skills or knowledge tuned to a specific task (knowledge engineering bottleneck)

Why Now?

Flood of available data (especially with the advent of the Internet)
Increasing computational power
Growing progress in available algorithms and theory developed by researchers
Increasing support from industries

General Information

up to 28 Lectures

up to 14 Practical Tutorials
up to 7 Home assignments

MACHINE LEARNING COURSE

by teaching assistants

Voluntary

Mandatory elective course for M.Sc. Data Engineering students

MAIN FOCUS

Modern supervised machine learning techniques:

Generative and Discriminative classification models
Probabilistic Graphical models

ORIENTATION

The course is highly oriented to practical applications:

Study of main software libraries: OpenCV, DGM and TensorFlow
Development of own machine learning techniques using C++ and/or Python
Study of the underlying optimisation algorithms

Introduction to main modern Machine Learning techniques with examples

Supervised Learning

Unsupervised Learning

Reinforcement Learning

History of Machine Learning

PROBABILISTIC GRAPHICAL MODELS

DISCRIMINATIVE MODELS

GENERATIVE MODELS

INTRODUCTION

Repeat the basics of Probability Theory

e.g. product rule, Bayes law, etc.

4 Lectures

PROBABILISTIC GRAPHICAL MODELS

DISCRIMINATIVE MODELS

GENERATIVE MODELS

INTRODUCTION

p(a|b) = \frac{p(b|a)\cdot p(a)}{p(b)}

Repeat the basics of Probability Theory

e.g. product rule, Bayes law, etc.

4 Lectures

Introduction to main modern Machine Learning techniques with examples

Supervised Learning

Unsupervised Learning

Reinforcement Learning

History of Machine Learning

PROBABILISTIC GRAPHICAL MODELS

DISCRIMINATIVE MODELS

GENERATIVE MODELS

INTRODUCTION

Theory

Bayes Model
Gaussian Model
Gaussian Mixture Model (GMM)

Repeat The Math

Mutivariate Gaussian Distribution

6 Lectures & 4 Tutorials

Practice

Maximum Likelihood algorithm
Sequential algorithm for GMM parameters estimation

PROBABILISTIC GRAPHICAL MODELS

DISCRIMINATIVE MODELS

GENERATIVE MODELS

INTRODUCTION

13 Lectures & 7 Tutorials

Theory

k-Nearest-Neighbours
Support Vector Machines
Decision Trees & Random Forests
Artificial Neural Networks
Perceptron
Deep Neural Networks

Practice

KD-Trees data structure
Reservoir sampling algorithm
Backpropagation algorithm

PROBABILISTIC GRAPHICAL MODELS

DISCRIMINATIVE MODELS

GENERATIVE MODELS

INTRODUCTION

5 Lectures & 3 Tutorials

Theory

Introduction to Graphical Models
Markov Random Fields
Conditional Random Fields

Application

Image semantic segmentation
Optical Flow reconstruction
Sequential data classification

Practical Assignments

Build your own Machine Learning algorithms with DGM library

Practical Assignments

Build your own Machine Learning algorithms with DGM library

DIRECT GRAPHICAL MODELS

A cross-platform open-source C++ library, which accompanies the course materials and may serve as a basis for your future projects

Web: http://research.project-10.de/dgm

GitHub: https://github.com/Project-10/DGM

Practical Assignments

Build your own Machine Learning algorithms with DGM library

DIRECT GRAPHICAL MODELS

A cross-platform open-source C++ library, which accompanies the course materials and may serve as a basis for your future projects

int main()
{
	auto	nodeTrainer = CTrainNode::create(Bayes, nStates, nFeatures);
	auto	graphKit    = CGraphKit::create(GraphType::dense, nStates);
	CMarker	marker(DEF_PALETTE_6);
	CCMat	confMat(nStates);

	// ========================= Training =========================
	nodeTrainer->addFeatureVecs(train_fv, train_gt);
	nodeTrainer->train();

	// ====================== Filling the Graph ===================
	Mat nodePotentials = nodeTrainer->getNodePotentials(test_fv);
	graphKit->getGraphExt().setGraph(nodePotentials);
	graphKit->getGraphExt().addDefaultEdgesModel(100.0f, 3.0f);
	graphKit->getGraphExt().addDefaultEdgesModel(test_fv, 300.0f, 10.0f);

	// ========================= Decoding =========================
	vec_byte_t optimalDecoding = graphKit->getInfer().decode(100);

	// ======================== Evaluation ========================
	Mat solution(imgSize, CV_8UC1, optimalDecoding.data());
	confMat.estimate(test_gt, solution);
	char str[255];
	sprintf(str, "Accuracy = %.2f%%", confMat.getAccuracy());
	printf("%s\n", str);

	return 0;
}

Web: http://research.project-10.de/dgm

GitHub: https://github.com/Project-10/DGM