...The shield that guards the realms of online courses

I pledge my life and honor to the Online Courses' Watch, for this night and all the nights to come.

The problem

Failed requests because Rails was unreachable

Main culprit is ...

Clients bombarding us with more requests than we can handle

The solution

Or one of them anyway

Redirect all spammy traffic to

Sorry Teachable

We need a better solution!

Hire Julien ...

and blame him when things go bad

But can we do better than that?

Certainly

Micro Caching!

Microcaching is an effective method for accelerating the delivery of dynamic content by caching it for very short periods of time.

The benefits of Microcaching are two‑fold: improves web performance by delivering content more quickly, and reduces the load on the origin servers.

Cache GET requests

after at
least 10 requests within a 30-second period to an endpoint by the same
user.

Cache is only used when consecutive requests are made within a very short amount of time

2 seconds if the response status is
- 200 OK
- 304 Not Modified
- or 404 Not Found
1 minute if the response status is
- 301 Moved Permanently.
1 second if it's anything else.

If Rails is dead, serve cached page instead of an error page

Results

Benchmark
1000 requests No Micro Caching
50 concurrently

Requests per second: 1.17

Percentage of the requests served within a certain time
50% 39
66% 43
75% 46
80% 49
90% 52
95% 59
98% 62
99% 69
100% 71

Benchmark
1000 requests With Micro Caching
50 concurrently

Requests per second: 17.36

Percentage of the requests served within a certain time
50% 0
66% 0
75% 0
80% 0
90% 0
95% 3
98% 40
99% 50
100% 56

17x RPM!

🎉🎉🎉

Next Steps

Cache content intentionally with longer retention periods
Use conditional get requests
Share the cache among all instances to reduce load on Rails even more
Make it easy to purge the cache manually using a webhook style
Cache pages with different retention periods