SecondScreen for a national TV show with realtime pushes and crazy traffic

Michael - Schnitzel

Head Technology Amazee Labs

The Client

The Project(s)

Requirements

  • "News" Site during the Week
  • SecondScreen during Shows
  • 6~10 Shows pre-captured
  • 3 Live Shows

SecondScreen

Mobile
Fast
Pushes

Mobile

Drupal

Mobile First Design

Omega 4, SASS, Compass, SUSY

Live Mode

Normal Homepage with added Pane conditions

Admin Interface

NID, NID, NID

Push

Admin Interface Pushes

PubNub

Browser subscribes to Messages

Replaces necessary parts

Vote

Core Poll & Poll Anon

Session Cookie

Voting Cookie

Hackable™

every vote pushes

SMS Vote

UnHackable™

websms.at

  1. User submits poll
  2. User receives Text SMS with Code
  3. User enters Code
  4. Vote counted

FAST

8'000 requests / sec

Redis

Varnish
Monitoring

Hardware

  • 2x Varnish
  • 6x Webservers
  • 2x MySQL
  • 2x NFS
  • 2x Redis

REDIS

LRU is costly!

Varnish

Purging

Refreshing

sub vcl_recv {
  if (req.request == "REFRESH") {
      if (!client.ip ~ purge) { error 405 "Not allowed."; }
      set req.request = "GET";
      set req.hash_always_miss = true;
  }
}

acl purge {
    "127.0.0.1";
    "5.148.164.176"/28;
    "10.148.164.0"/24;
}
/usr/bin/curl -o /dev/null "http://dgst.srf.ch/" -X REFRESH --silent

Monitoring

Varnish

  • client requests
  • backend requests
  • most requested url in last 60s
  • Overall stats (hits per sec)

Servers

htop

Drupal

  • Syslog (watchdog)
  • NewRelic

Learnings

Tech

Generated Images don't have Cache Headers

image_style_deliver()

Ajax is POST

Varnish doesn't like

/ajax/ excluded in Varnish

rewrite Varnish vcl

Cache 404 and 403!

Learnings

Processes

Be there

Be flexible

Be proactive

Prepare as much as possible

testing, testing, testing

Results

deck

By Michael Schmid

deck

  • 2,023