Visual Regression Testing

https://github.com/visualtesting

Eli White

Why Visual Regression Testing?

Importance Hierarchy

CSS / SASSlint → mock browser → selenium → manual (visual)  visual (automatic) → visual (manual)

Quality Screenshots

Multiple browsers (Chrome, IE, Safari, Firefox)

 

Multiple resolutions (css breakpoints)

Testing a full page

Testing a single unit

Question...

What about dynamic sections? 

3 options.

Quality Image Diffs

ImageMagick is the common solution (slow, manual installation)

 

Done as a service: Applitools

 

Done in a browser: resemble.js (canvas)

 

Done in node: node-resemble

...uses a canvas implementation written with ImageMagick

 

Pure javascript implementation

Reviewing

If reviewing diffs is optional, it won't be done

 

Require manual review, but low bar to accept

...

"Approve Button"

Admin Panel

Engineering Goals

• Take screenshots in different languages (js / ruby / obj-c)
• Different Source Repository hosts (Github / Stash / Bitbucket)
• Store images in different places (fs, amazon, git)
• Diff against specified builds
• Different tools to generate the diff images

Plugins all the way down

Plugins all the way down

• visualtesting-storage-fs
• visualtesting-storage-s3
• visualtesting-storage-git
• visualtesting-diff-imagemagick
• visualtesting-diff-resemble
• visualtesting-diff-applitools
• visualtesting-service-github
• visualtesting-service-stash
• visualtesting-service-bitbucket

What keeps me up at night?

Speed: n-pages * m-browsers * k-builds 

Luckily there are lots of ways to parallelize this

What's next?

Some code cleanup

 

Lets start using it on some ideal projects (roswell)

 

Documentation

 

Try using it with native apps

 

 

Thanks

Special thanks to Christian Bromann

http://github.com/VisualTesting

<-- Seriously. I need a new name and logo. I stole this off Google Images