Dreams

• Servers (+GPU)
• Team
  • Data Engineer(s)
  • DevOps
  • Assessors
  • Senior DS
• Data Infrastructure
• Enough time
• $$$

ML from sh*t & sticks

• No hardware
• No money
• No data (almost)
• You are alone
• Open Source
• You know what you need to solve

ML from sh*t & stick

f : X → Y

Data

Model

Calculator

DATA

What's kind of data?

• text
• image
• audio
• video
• graph
• numberic/categorial

DATA

DATA

Open-Source

• https://github.com/awesomedata/awesome-public-datasets​
• https://www.kaggle.com/datasets
• https://registry.opendata.aws/
• https://cloud.google.com/public-datasets/
• https://archive.ics.uci.edu/ml/index.php
• http://academictorrents.com/
• http://dataportals.org/

DATA

Labeling

• "Market" platform
• Self-hosted platform

DATA

Labeling: "Market" platform

DATA

Labeling: "Market" platform

• Toloka
• MTurk
• Figure Eight

DATA

Labeling: "Market" platform

• Fast
• Simple
• Cheap

• Low quality
• NDA issues
• Required "gold questions", "exams", etc

DATA

Labeling: Self-hosted platform

• PyBossa
• Prodigy
• MTsar

DATA

Labeling: Self-hosted platform

• SUPER FLEXIBLE
• Self-hosted

• Manual setup
• Required Assessors

DATA

Augmentation

DATA

Augmentation

Image / Video

• Crop
• Flip
• Rotate
• Color shift
• Blur
• Color filters
• etc

Text

• Synonyms 's/w1/w2'
• Back translation
• Drop/Insert random word
• Cut/Glue texts

Audio

• Noise
• Pitch
• Speed

DATA

Augmentation

Image / Video

Text / Audio

• torchvision.transforms
• keras.preprocessing.image 
• /mdbloice/Augmentor
• /aleju/imgaug

• /makcedward/nlpaug (?)
• ✋✋ 

MODEL

Data

Predictions

Model

MODEL

Data

BASELINE

Predictions

MODEL

Why so dummy model?

I want BOOSTING DEEP LEARNING!

• Fast (preparing & training)
• Good first approximation
• Less chance for overfitting
• Still works

MODEL

Embed your data

Confucius

MODEL

Data

Embeddings

Encoder

Your model

Predictions

MODEL

MODEL

Where can I find a "encoders"?

Here

MODEL

Pre-trained models

NLP

• /huggingface/transformers
• /RaRe-Technologies/gensim
• /fastai/fastai
• /allenai/allennlp

MODEL

Pre-trained models

CV

• torchvision.models
• keras.applications
• /Cadene/pretrained-models.pytorch

MODEL

MOAR MODELS

CALCULATOR

Laptop is OK,

but ....

CALCULATOR

CALCULATOR

https://www.floydhub.com/

https://www.paperspace.com/

https://colab.research.google.com/

CALCULATOR

CALCULATOR

CONCLUSION

• Simple better than complex
• Fail fast
• LABEL YOUR DATA
• Re-use OSS data/models
• Take free stuff

CONCLUSION

Recipe

1. Understand your task
2. Collect data
3. Label
4. Augment
5. Embed
6. Create a simple model
7. ...
8. PROFIT!