Improve Performance with External Dependencies

A Real World Case Study

The Task

Home page

Index page

Switch to the Real Feed Service

: (

Home page

Index page

Slow Backend

External Dependency

Feed with Limited Data

External Feed without Twitter

The Next Day

: (

Home page

Index page

External Request Cache Expiration

  • warm cache: < 500 ms
  • cold cache:    > 9000 ms

Cache expiry time: 1 hour

Retry Cache

   For any request to an external service:

  • If the cache is NOT expired
    • respond with the cached content
  • If the cache is expired,
    • send the request with a timeout limit
    • when timeout, return the previous cached content and keep the cache state as expired
    • when succeeds, activate and reset the cache with the successe response

A protect layer for external services

Homepage Robustness

Optional requests should NOT break the entire page

Performance Improvement

Before

After

Index Page Load Time

7~13 s

0 ~ 3 s

Index Page Timeout

w/ cold cache

never

Home Page Load Time

> 15 s

< 5 s

Home Page Timeout

w/ cold cache

never

External Service Resp Time

(cold cache)

~ 9 s

~ 9 s

Summary

  • Remove unnecessary data  from the requests
  • Put unstable external dependencies behind a protective cache layer
  • Primary functionities should be robust to errors from optional services
Made with Slides.com