Improve Performance with External Dependencies

A Real World Case Study

The Task

Home page

Index page

Switch to the Real Feed Service

: (

Home page

Index page

Slow Backend

External Dependency

Feed with Limited Data

External Feed without Twitter

The Next Day

: (

Home page

Index page

External Request Cache Expiration

• warm cache: < 500 ms
• cold cache:    > 9000 ms

Cache expiry time: 1 hour

Retry Cache

   For any request to an external service:

• If the cache is NOT expired
  • respond with the cached content
• If the cache is expired,
  • send the request with a timeout limit
  • when timeout, return the previous cached content and keep the cache state as expired
  • when succeeds, activate and reset the cache with the successe response

A protect layer for external services

Homepage Robustness

Optional requests should NOT break the entire page

Performance Improvement

Before

After

Index Page Load Time

7~13 s

0 ~ 3 s

Index Page Timeout

w/ cold cache

never

Home Page Load Time

> 15 s

< 5 s

Home Page Timeout

w/ cold cache

never

External Service Resp Time

(cold cache)

~ 9 s

~ 9 s

Summary

• Remove unnecessary data  from the requests

• Put unstable external dependencies behind a protective cache layer

• Primary functionities should be robust to errors from optional services