Image Optimization + Machine Learning

Developer Advocate
at
@SchibstedSpain


#Web #CSS #Javascript

#Animation #SVG #PostCSS
#Optimization #Performance

Joan León

Table of Contents

  1. Introduction
  2. Do my images need to be optimized?
  3. How do I choose an image format?
  4. Ok, I need optimize my images
  5. I need validate the optimization
  6. Machine Learning
  7. Resources to improve your knowledge in Image Optimization

Introduction

Images on the web

Images on the web

Images on the web

Image Optimization

Do my images need to be optimized?

YES!

Online tools

WebPageTest

WebPageTest | Cloudinary

Developer Tools

Developer Tools

Developer Tools

Lighthouse

CLI Tools | psi

CLI Tools | lighthouse

How do I choose an image format?

Image Format

Image Format

JPEG

PNG

GIF

WebP

JPEG XR

BPG

Image Format | JPEG

Image Format | JPEG

Image Format | JPEG

Image Format | JPEG

Image Format | JPEG

The advantages of Progressive JPEGs

Image Format | JPEG

The advantages of Progressive JPEGs

Image Format | JPEG

The disadvantages of Progressive JPEGs

Progressive JPEGs can be slower to decode than baseline JPEGs.

Progressive JPEGs are also not always smaller.

Some users may consider progressive loading to be a disadvantage.

Image Format | WebP

Image Format | WebP

Netflix, Amazon, Quora, Yahoo, Walmart, Ebay, The Guardian, Fortune, and USA Today, all compress and serve images with WebP for browsers which support it.

Image Format | WebP

Ok, I need optimize my images

Image Optimization

Designer Responsibility

DR

Image Optimization | App

Image Optimization | App

Image Optimization | App

Image Optimization | App

Zopfli
PNGOUT
OptiPNG
AdvPNG
PNGCrush
JPEGOptim
Jpegtran
Guetzli
Gifsicle

jpeg-recompress
WebP
MozJPEG
pngquant

ImageMagick

Image Optimization | CLI

Original | 766Kb

jpeg-recompress

Quality 80 | 550Kb | 1.619s

Quality 70 | 378Kb | 1.241s

Image Optimization | CLI

Original | 766Kb

convert (ImageMagick)

Quality 80 | 712Kb | 0.342s

Quality 70 | 535Kb | 0.277s

Image Optimization | CLI

Original | 766Kb

MozJPEG

Quality 80 | 633Kb | 1.215s

Quality 70 | 378Kb | 0.808s

Image Optimization | CLI

Original | 766Kb

Guetzli

Quality 90 | 705Kb | 3m 58.52s

Quality 84 | 639Kb | 4m 3.33s

Image Optimization | CLI

Image Optimization | CLI

Imagemin + plugins

Imagemin + plugins

Image Optimization | CLI

Imagemin + WebP

Image Optimization | CLI

Image Optimization | Service

Image Optimization | Schibsted

I need validate the optimization

Designer

Image validation

SSIM

Butteraugli

==

The Structural Similarity

Tools for measuring perceived differences between images

Image validation

Image validation

SSIM

Image validation

Butteraugli

Image validation

Image validation

Machine Learning

Hype keyword

Image Optimization | Machine Learning

Image Optimization | Machine Learning

Bathrooms
Storage rooms
Stairs
Hallway
Garages

Image Optimization | Machine Learning

Chroma Subsample

Image Optimization | Machine Learning

Chroma Subsample

Original | 185Kb

4:2:1 | 183Kb

4:2:0 | 172Kb

Image Optimization | Machine Learning

Chroma Subsample

Original | 171Kb

4:2:1 | 164Kb

4:2:0 | 156Kb

Image Optimization | Machine Learning

Chroma Subsample + blur

4:2:0 | blur 1x1 | 136Kb

4:2:0 | 156Kb

Machine Learning | Models

Im2Txt by Google.

Im2Txt or, the Show and Tell model, is a deep neural network that learns how to describe the content of images.

YOLO by Redmon, Joseph and Farhadi, Ali.

You only look once (YOLO) is a state-of-the-art, real-time object detection system that can run on images or videos.

Machine Learning | Yolo

$ ./darknet detect cfg/yolov3.cfg yolov3.weights data/dog.jpg
layer     filters    size              input                output
    0 conv     32  3 x 3 / 1   416 x 416 x   3   ->   416 x 416 x  32  0.299 BFLOPs
    1 conv     64  3 x 3 / 2   416 x 416 x  32   ->   208 x 208 x  64  1.595 BFLOPs
    .......
  105 conv    255  1 x 1 / 1    76 x  76 x 256   ->    76 x  76 x 255  0.754 BFLOPs
  106 yolo
Loading weights from yolov3.weights...Done!
data/dog.jpg: Predicted in 17.682495 seconds.
bicycle: 99%
truck: 92%
dog: 100%

Machine Learning | Yolo

Machine Learning | COCO

COCO: Common Objects in Context by Microsoft.

Machine Learning | COCO

Machine Learning | COCO

Original | 247Kb

4:2:0 | blur 0.5 | 104Kb

-143Kb

Machine Learning | I want a COCO

{
    "format": "jpg",
    "actions": [
    {
    "blur_areas": [
        {
            "width": 400,
            "top": 100,
            "height": 400,
            "sigma": 10,
            "left": 500
        }
        ]
        },
        {
            "resize": {
            "width": 300,
            "fit": {
            "type": "clip"
        }
        }
        }
    ],
    "version": "2017-06",
    "quality": 90
}

Machine Learning | YAMS + COCO

+

Comin soon

Machine Learning | Image Responsability

Resources to improve your knowledge in Image Optimization

Improve your knowledge

Improve your knowledge

Chroma Subsampling

Improve your knowledge

Using the <picture> Tagpicture

Improve your knowledge

Follow José Manuel Perez

Engineer at @Spotify and Web Perf geek

Questions?

Thanks

Joan León

Image Optimization + Machine Learning

By Joan León

Image Optimization + Machine Learning

Como profesionales del desarrollo web, nos procupamos de las buenas prácticas en Javascript y CSS, de la semántica en HTML, de la accesibilidad de nuestro contenido, pero ¿qué pasa con la imágenes? Las imágenes representan hasta el 60% del contenido de los sites, desde julio de 2017 en un site promedio de 3.0Mb, 1.7Mb son imágenes. Ahora que sabemos que deberíamos dedicar tiempo a optimizar las imágenes, veamos los diferentes niveles de optimización, cómo automatizarlos y cómo con la ayuda del Machine Learning conseguir reducir un 15-17% el peso de algunas imágenes. En esta charla veremos herramientas para el análisis de las imágenes, la optimización, la validación (aka Tests) y cómo reducir el peso de alguna imágenes con la ayuda del reconocimiento de objetos con Machine Learning. Un frontend hablando de Machine Learning, ¿qué puede salir mal? 🤗

  • 3,225