lbumentations: fast and flexible image augmentations

Vladimir Iglovikov

Sr. Computer Vision Engineer at Lyft

Ph.D. In Physics

Kaggle GrandMaster

Computer Vision Tasks

Input

Output

Need for the data

Problems with the labeled data

Expensive to collect
Hard to label
Legal or privacy problems

Augmentations

Synthetically increase dataset.

Augmentations

Synthetically increase dataset.
Make network invariant to transforms (rotations, brightness, flips, etc).

Augmentations

Synthetically increase dataset.
Make network invariant to transforms (rotations, brightness, flips, etc).
Work as a regularizer.

Augmentations

Augmentations vs Heavier architecture

lbumentations

Used in almost all winning solutions to CV problems at Kaggle.
Used in academia: 32+ citations
Used in industry

Who works on Albumentations?

Alexander Buslaev

Mapbox, Belarus

Kaggle Master

Vladimir Iglovikov

Lyft, USA

Kaggle Grandmaster

Alex Parinov

X5, Russia

Kaggle Master

Eugene Khvedchenia

Jumio, Ukraine

Kaggle Grandmaster

Mikhail Druzhinin

Simicon, Russia

and many contributors at Github

CPU becomes bottleneck => we want fast transforms

We want different targets to be transformed in sync

Original

Transformed

import albumentation as A

strong = albu.Compose([
    A.HorizontalFlip(p=0.5),
    A.RandomSizedCrop((700, 900), 600, 600),
    A.RGBShift(p=0.35),
    A.Blur(blur_limit=11, p=0.31415),
    A.RandomBrightness(p=0.2019),
    A.CLAHE(p=0.666),
], p=0.1984)

image
bounding boxes
mask
keypoints

Pixel-level transforms

38+ transforms

Original

RandomRain

RandomSnow

RandomFog

Spatial Tranforms

Support masks, bounding boxes, keypoints

33+ transforms

CropNonEmptyMaskIfExists,

Lambda

Key points

CropNonEmptyMaskIfExists,

Lambda

Multiple targets

We want to apply the same transform to:

N images
M masks
K bounding boxes
L keypoints

simultaneously.

import albumentations as A

aug = A.Compose(transformations, 
		p=0.42, 
		additional_targets={
    'image1': 'image',    
    ...
    'imageN': 'image',
    'bboxes1': 'bboxes',    
    ...
    'bboxesM': 'bboxes',
    'keypoints1': 'keypoints',    
    ...
    'keypointsK': 'keypoints',
    'mask1': 'mask',    
    ...
    'maskL': 'mask'
    })
    
    
    transformed = aug(image=img1, image2=img1, ....)

transform = A.Compose([
    A.RandomCrop(768, 768),
    A.OneOf([
        A.RGBShift(), 
        A.HueSaturationValue()
    ]),
])

# to yaml
A.save(transform, '/tmp/transform.yml', data_format='yaml')

# to json
A.save(transform, '/tmp/transform.json')

# to python dictionary
transform_dict = A.to_dict(transform)

Serialization / Deserialization

transform = A.Compose([
    A.RandomCrop(768, 768),
    A.OneOf([
        A.RGBShift(), 
        A.HueSaturationValue()
    ]),
])

# to yaml
A.save(transform, 'transform.yml', data_format='yaml')

# to json
A.save(transform, 'transform.json')

# to python dictionary
transform_dict = A.to_dict(transform)

# Load from yaml
loaded_transform = A.load('transform.yml', data_format='yaml')

# Load from json
loaded_transform = A.load('transform.json')

# Load from dictionary
loaded_transform = A.from_dict(transform_dict)

Documentation

https://github.com/albu/albumentations/tree/master/notebooks

https://albumentations.readthedocs.io

Deep Learning Competitions

https://github.com/albu/albumentations/blob/master/docs/hall_of_fame.rst

Topcoder Urban3D Challenge

Image

Mask

Instance Segmentation

1st place

Alexander Buslaev

Augmentations:

OpticalDistortion, Grid, Flip, ShiftScaleRotate, Transpose, HueSaturationValue

Topcoder: Spacenet 3 challenge

Semantic Segmentation

1st place

Alexander Buslaev

Augmentations:

Flip, ShiftScaleRotate, Transpose, HueSaturationValue

Kaggle: Carvana Image Masking Challenge

Semantic Segmentation

1st place

Vladimir Iglovikov, Alexander Buslaev,

Artem Sanakoyeu

Augmentations:

HorizontalFlip, ShiftScaleRotate, RandomBrightness, HueSaturationValue

Input

Output

Kaggle: Data Science Bowl 2018

Instance Segmentation

1st place

Alexander Buslaev,

Selim Seferbekov

Viktor Durnov

Augmentations:

CLAHE, IAASharpen, IAAEmboss, IAAAdditiveGaussianNoise, ToGray, InvertImg, RandomRotate90, Flip, Transpose, MotionBlur, MedianBlur, Blur, RandomContrast, RandomBrightness, ShiftScaleRotate, OpticalDistortion, GridDistortion, ElasticTransform, IAAPerspective, IAAPiecewiseAffine, HueSaturationValue, ChannelShuffle

Input

Output