...The shield that guards the realms of online courses
I pledge my life and honor to the Online Courses' Watch, for this night and all the nights to come.
The problem
Failed requests because Rails was unreachable
Main culprit is ...
Clients bombarding us with more requests than we can handle
The solution
Or one of them anyway
Redirect all spammy traffic to
Sorry Teachable
We need a better solution!
Hire Julien ...
and blame him when things go bad
But can we do better than that?
Certainly
Micro Caching!
Microcaching is an effective method for accelerating the delivery of dynamic content by caching it for very short periods of time.
The benefits of Microcaching are two‑fold: improves web performance by delivering content more quickly, and reduces the load on the origin servers.
Cache GET requests
after at
least 10 requests within a 30-second period to an endpoint by the same
user.
Cache is only used when consecutive requests are made within a very short amount of time
-
2 seconds if the response status is
-
200 OK
-
304 Not Modified
-
or 404 Not Found
-
-
1 minute if the response status is
-
301 Moved Permanently.
-
-
1 second if it's anything else.
If Rails is dead, serve cached page instead of an error page
Results
Benchmark
1000 requests No Micro Caching
50 concurrently
Requests per second: 1.17
Percentage of the requests served within a certain time
50% 39
66% 43
75% 46
80% 49
90% 52
95% 59
98% 62
99% 69
100% 71
Benchmark
1000 requests With Micro Caching
50 concurrently
Requests per second: 17.36
Percentage of the requests served within a certain time
50% 0
66% 0
75% 0
80% 0
90% 0
95% 3
98% 40
99% 50
100% 56
17x RPM!
🎉🎉🎉
Next Steps
-
Cache content intentionally with longer retention periods
-
Use conditional get requests
- Share the cache among all instances to reduce load on Rails even more
- Make it easy to purge the cache manually using a webhook style
- Cache pages with different retention periods
...The shield that guards the realms of online courses I pledge my life and honor to the Online Courses' Watch, for this night and all the nights to come.