End-To-End Test Your Web Security

Gleb Bahmutov

Sr Director of Engineering, Mercari US

https://slides.com/bahmutov/e2e-testing-security

Global temperature anomaly 2022 https://climate.nasa.gov/vital-signs/global-temperature/

survival is possible* but we need to act now

  • change your life
  • join an organization

@bahmutov

rebellion.global www.wedonthavetime.org 

www.sierraclub.org 350.org www.greenpeace.org 

www.sunrisemovement.org

Speaker: Gleb Bahmutov PhD

@bahmutov

gleb.dev

github.com/bahmutov

glebbahmutov.com/blog

C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing

www.youtube.com/glebbahmutov

Gleb Bahmutov

Sr Director of Engineering

glebbahmutov.com/blog testing posts

Lots Of Testing

https://slides.com/bahmutov/decks/mercari

https://cypress.tips/courses

gleb.dev

Enter your username / password / credit card

gleb.dev

gleb.dev

You can have your

website hacked but only once.

Security, security, security

gleb.dev

Open Worldwide Application Security Project (OWASP)

https://owasp.org/www-project-top-ten/

XSS / code injection attacks

gleb.dev

Code injection attacks

=

confusion between data and code

gleb.dev

<html>
  <head>
    <style>...</style>
    <script src="https://code.jquery.com/3.5.0/jquery.min.js"></script>
    <script>
      ...
    </script>
  </head>
  <body>
    <h1>Chat</h1>
    <ul>
      <li>Chat message 1</li>
      <li>Chat message 2</li>
    </ul>
  </body>
</html>
<html>
  <head>
    <style>...</style>
    <script src="https://code.jquery.com/3.5.0/jquery.min.js"></script>
    <script>
      ...
    </script>
  </head>
  <body>
    <h1>Chat</h1>
    <ul>
      <li>Chat message 1</li>
      <li>Chat message 2</li>
    </ul>
  </body>
</html>

Data

<html>
  <head>
    <style>...</style>
    <script src="https://code.jquery.com/3.5.0/jquery.min.js"></script>
    <script>
      ...
    </script>
  </head>
  <body>
    <h1>Chat</h1>
    <ul>
      <li>Chat message 1</li>
      <li>Chat message 2</li>
    </ul>
  </body>
</html>

Code

Name:

John Doe

<span>{{ name }}<span>

<span>John Doe<span>

What about user input?

Name:

<script>alert('hi')</script>

<span>{{ name }}<span>

<span><script>alert('hi')</script><span>

innerText or innerHTML?

What about user input?

do you use eval(user input) anywhere???

Name:

<script>alert('hi')</script>

<span>{{ name }}<span>

<span><script>alert('hi')</script><span>

innerText or innerHTML?

What about user input?

do you use eval(user input) anywhere???

User input (data) is treated as code

innerText or innerHTML?

Dangerous, but has styling, elements, etc

Really dangerous if one user's content

can be viewed by another user

Will a script embedded in the user-entered content be executed (for another user)?

NEVER EVER EVER?

My Site

My content

 

 

 

 

 

 

 

 

User comments

Ads, etc

 

 

 

 

 

 

 

<script>stealStuff()</script>

http://excess-xss.com/

User input can come from:

  • input elements
  • database
  • URL
  • ...

Cross-Site Scripting attack (XSS)

gleb.dev

Let's Test It!

📝 details https://glebbahmutov.com/blog/csp-testing-using-cypress/

💾 code https://github.com/bahmutov/cypress-csp-example

gleb.dev

To test:

  • happy user path
  • unsuccessful injection
  • successful injection
  • injection prevention and reporting
// app.js
send$.addEventListener('click', () => {
  const message = message$.value
  const li = document.createElement('li')
  li.innerText = message
  messages$.appendChild(li)
  message$.value = ''
  send$.setAttribute('disabled', 'disabled')
})

// innerText (safe)

// cypress/e2e/spec.cy.js
it('adds a new message', () => {
  cy.visit('/')
  cy.contains('button', 'Send').should('be.disabled')
  cy.get('#message').type('Hello')
  cy.contains('button', 'Send').click()
  cy.get('#messages li')
    .should('have.length', 1).and('have.text', 'Hello')
  cy.get('#message').should('have.value', '')
  cy.contains('button', 'Send').should('be.disabled')
})

Cypress end-to-end test

https://docs.cypress.io/

// cypress/e2e/spec.cy.js
it('adds a new message', () => {
  cy.visit('/')
  cy.contains('button', 'Send').should('be.disabled')
  cy.get('#message').type('Hello')
  cy.contains('button', 'Send').click()
  cy.get('#messages li')
    .should('have.length', 1).and('have.text', 'Hello')
  cy.get('#message').should('have.value', '')
  cy.contains('button', 'Send').should('be.disabled')
})

Cypress end-to-end test

https://docs.cypress.io/

gleb.dev

Cypress tests commands slowed by 1 second each using cypress-slow-down plugin

Estimate: 3 points

Priority: P0

gleb.dev

// app.js
send$.addEventListener('click', () => {
  const message = message$.value
  const li = document.createElement('li')
  li.innerHTML = message
  messages$.appendChild(li)
  message$.value = ''
  send$.setAttribute('disabled', 'disabled')
})

// innerHTML (for now)

// cypress/e2e/spec.cy.js
it('adds a bold message', () => {
  cy.visit('/')
  cy.get('#message').type('<b>Important</b>')
  cy.contains('button', 'Send').click()
  cy.get('#messages li b')
    .should('have.length', 1)
    .and('have.text', 'Important')
})

gleb.dev

// cypress/e2e/spec.cy.js
it('adds a bold message', () => {
  cy.visit('/')
  cy.get('#message').type('<b>Important</b>')
  cy.contains('button', 'Send').click()
  cy.get('#messages li b')
    .should('have.length', 1)
    .and('have.text', 'Important')
})

Testing <script>... inputs

// cypress/e2e/spec.cy.js
it('injecting <script> tag does not work', () => {
  cy.on('window:load', (win) => cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message').type('Hello<script>console.log(`hacked`)</script>')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')
  cy.get('@log').should('not.have.been.called')
})

gleb.dev

// cypress/e2e/spec.cy.js
it('injecting <script> tag does not work', () => {
  cy.on('window:load', (win) => cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message').type('Hello<script>console.log(`hacked`)</script>')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')
  cy.get('@log').should('not.have.been.called')
})

gleb.dev

Testing <script>... inputs

Are we 100% safe?!

// cypress/e2e/spec.cy.js
it('injects XSS via img onerror attribute', () => {
  cy.on('window:load', (win) => 
        cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message')
    .type('Hello<img src="" onerror="console.log(`hacked`)" />')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')
  cy.get('@log').should('have.been.calledWith', 'hacked')
})

gleb.dev

// cypress/e2e/spec.cy.js
it('injects XSS via img onerror attribute', () => {
  cy.on('window:load', (win) => 
        cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message')
    .type('Hello<img src="" onerror="console.log(`hacked`)" />')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')
  cy.get('@log').should('have.been.calledWith', 'hacked')
})

Are we 100% safe?!

Are we 100% safe?!

No, we are not

Strip all but <b> and <i> tags... ?

// app.js
send$.addEventListener('click', () => {
  const message = message$.value
  const li = document.createElement('li')
  li.innerHTML = removeBadTags(message)
  messages$.appendChild(li)
  message$.value = ''
  send$.setAttribute('disabled', 'disabled')
})

gleb.dev

Strip all but <b> and <i> tags... ?

it('injects XSS via b onclick attribute', () => {
  cy.on('window:load', (win) => 
        cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message')
    .type('<b onclick="console.log(`hacked`)">Hello</b>')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li b', 'Hello').click()
  cy.get('@log').should('have.been.calledWith', 'hacked')
})

Strip all but <b> and <i> tags... ?

it('injects XSS via b onclick attribute', () => {
  cy.on('window:load', (win) => 
        cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message')
    .type('<b onclick="console.log(`hacked`)">Hello</b>')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li b', 'Hello').click()
  cy.get('@log').should('have.been.calledWith', 'hacked')
})

Will never happen to me

Angular 1 edition (dated)

Examples where we can embed script code into {{ }} template

{{constructor.constructor('alert(1)')()}}

fixed in Angular 1.2

from: http://www.slideshare.net/x00mario/an-abusive-relationship-with-angularjs

gleb.dev

Will never happen to me

{{ (_=''.sub).call.call({}[$='constructor'].getOwnPropertyDescriptor ( _.__proto__,$).value,0,'alert(1)')() }}

from: http://www.slideshare.net/x00mario/an-abusive-relationship-with-angularjs

Will never happen to me

{{ objectPrototype = ({})[['__proto__']]; objectPrototype[['__defineSetter__']]('$parent', $root.$$postDigest); $root.$$listenerCount[['constructor']] = 0; $root.$$listeners = [].map; $root.$$listeners.indexOf = [].map.bind; functionPrototype = [].map[['__proto__']]; functionToString = functionPrototype.toString; functionPrototype.push = ({}).valueOf; functionPrototype.indexOf = [].map.bind; foo = $root.$on('constructor', null); functionPrototype.toString = $root.$new; foo(); }} {{ functionPrototype.toString = functionToString; functionPrototype.indexOf = null; functionPrototype.push = null; $root.$$listeners = {}; baz ? 0 : $root.$$postDigestQueue[0]('alert(location)')(); baz = true;'' }} 

from: http://www.slideshare.net/x00mario/an-abusive-relationship-with-angularjs

Will never happen to me

{{ 'this is how you write a number properly. also, numbers are basically arrays.'; 0[['__proto__']].toString = [][['__proto__']].pop; 0[['__proto__']][0] = 'alert("TROLOLOLn"+document.location)'; 0[['__proto__']].length = 1; 'did you know that angularjs eval parses, then re-stringifies numbers? :)'; $root.$eval("x=0", $root); }}

from: http://www.slideshare.net/x00mario/an-abusive-relationship-with-angularjs

Lesson: you cannot sanitize your way out of <script> tags

At least: do not write the sanitizer yourself, use https://github.com/cure53/DOMPurify

gleb.dev

Estimate: 1 point

Priority: P2

gleb.dev

Solution: disable inline JavaScript

<body>
  <script>function willNotRun() {...}</script>
  <script src="assets/js/script.js"></script>
</body>

allowed from the right domain

never* allowed

gleb.dev

* or with specific exceptions

Attacker (probably) cannot change HTML / JavaScript on our server or replace a CDN library*

* tip: use exact library URLs and integrity checksums

gleb.dev

Content-Security-Policy (CSP)

<meta http-equiv="Content-Security-Policy" 
      content="script-src https://code.jquery.com 'self';">

Wide support for CSP (browsers, web apps)

https://cspvalidator.org/

gleb.dev

or even better

Content-Security-Policy: default-src 'self' example.com *.example.com

Content-Security-Policy (CSP)

gleb.dev

Content-Security-Policy: default-src 'self'; img-src *; media-src example.org example.net; script-src userscripts.example.com

Specify trusted sources for scripts, fonts, styles, images, media, frames, etc

@bahmutov

Set CSP Header

// server JS
const express = require('express')
const helmet = require('helmet')
const app = express()
app.use(
  helmet({
    contentSecurityPolicy: {
      directives: {
        defaultSrc: ["'self'"],
        // ughh, have to allow unsafe inline styles to be added by Cypress
        // https://github.com/cypress-io/cypress/issues/21374
        styleSrc: ["'self'", "'unsafe-inline'"],
        reportUri: ['/security-attacks'],
      },
    },
  }),
)

gleb.dev

// cypress.config.js
// https://on.cypress.io/experiments
// https://github.com/cypress-io/cypress/issues/1030
experimentalCspAllowList: ['default-src', 'script-src'],
  
// cypress/e2e/spec.cy.js
it('stops XSS', () => {
  cy.on('window:load', (win) => 
        cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message')
    .type('Hello<img src="" onerror="console.log(`hacked`)" />')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')
  cy.get('@log').should('not.be.called')
})
// cypress.config.js
// https://on.cypress.io/experiments
// https://github.com/cypress-io/cypress/issues/1030
experimentalCspAllowList: ['default-src', 'script-src'],
  
// cypress/e2e/spec.cy.js
it('stops XSS', () => {
  cy.on('window:load', (win) => 
        cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message')
    .type('Hello<img src="" onerror="console.log(`hacked`)" />')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')
  cy.get('@log').should('not.be.called')
})

CSP Violation error

CSP Violation Reports

gleb.dev

CSP Violation Reports

CSP Violation Reports

// server JS
const express = require('express')
const helmet = require('helmet')
const app = express()
app.use(
  helmet({
    contentSecurityPolicy: {
      directives: {
        defaultSrc: ["'self'"],
        // ughh, have to allow unsafe inline styles to be added by Cypress
        // https://github.com/cypress-io/cypress/issues/21374
        styleSrc: ["'self'", "'unsafe-inline'"],
        reportUri: ['/security-attacks'],
      },
    },
  }),
)

Test CSP violation reporting

// cypress/e2e/spec.cy.js
it('stops XSS and reports CSP violations', () => {
  cy.intercept('/security-attacks', {}).as('cspAttacks')

  cy.on('window:load', (win) => cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message').type('Hello<img src="" onerror="console.log(`hacked`)" />')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')

  cy.log('**XSS stopped and reported**')
  cy.wait('@cspAttacks').its('request.body').should('include', 'blocked')
  cy.get('@log').should('not.be.called')
})

Test CSP violation reporting

// cypress/e2e/spec.cy.js
it('stops XSS and reports CSP violations', () => {
  cy.intercept('/security-attacks', {}).as('cspAttacks')

  cy.on('window:load', (win) => cy.stub(win.console, 'log').as('log'))
  cy.visit('/')
  cy.get('#message').type('Hello<img src="" onerror="console.log(`hacked`)" />')
  cy.contains('button', 'Send').click()
  cy.contains('#messages li', 'Hello')

  cy.log('**XSS stopped and reported**')
  cy.wait('@cspAttacks').its('request.body').should('include', 'blocked')
  cy.get('@log').should('not.be.called')
})

gleb.dev

Code injection attacks are common

Use content security policy

Write tests to check your CSP directives

Verify the violation reporting works

gleb.dev

Learn More

💡 Trusted types experimental browser feature https://web.dev/trusted-types/

Gleb Bahmutov

gleb.dev

👏 Thank You 👏

End-To-End Test Your Web Security

https://slides.com/bahmutov/e2e-testing-security

👀 "The Big Dig: The story of America's most expensive road"

https://www.youtube.com/playlist?list=PLMQKK3_a14M3A-SQdVVWhOfOw8xRUuueJ

For All Boston / MA

peeps