Microservices

- API Gateway & Netflix OSS

  • Number of items in the shopping cart

  • Order history

  • Customer reviews

  • Low inventory warning

  • Shipping options

  • Various recommendations,

  • Alternative purchasing options

  • Shopping Cart Service – Number of items in the shopping cart
  • Order Service – Order history
  • Catalog Service – Basic product information, such as its name, image, and price
  • Review Service – Customer reviews
  • Inventory Service – Low inventory warning
  • Shipping Service – Shipping options, deadlines, and costs drawn separately from the shipping provider’s API
  • Recommendation Service(s) – Suggested items

http://shopping.api.company.name/shopcarts

http://shipping.api.company.name/shipping/availables

http://inventory.api.company.name/invertory/101

http://recommendation.api.company.name/catalog/book

.......

  • Different clients need different data.
  • Client could make that many requests over a LAN.
  • Some services might use protocols that are not web-friendly.
  • Service instances and their locations (host+port) changes dynamically.
  • Difficult to refactor the microservices.

Using an API Gateway

  • request routing, composition, and protocol translation
  • authentication
  • monitoring
  • load balancing
  • caching
  • request shaping
  • management
  • .....

mobile/productdetails?productid=xxx

 Benefits

  • Insulates the clients from how the application is partitioned into microservices.
  • Insulates the clients from the problem of determining the locations of service instances.
  • Provides the optimal API for each client.
  • Reduces the number of requests/roundtrips. 
  • Simplifies the client by moving logic for calling multiple services from the client to API gateway.

 Drawbacks

  •  It is yet another highly available component that must be developed, deployed, and managed. 
  • Developers must update the API Gateway in order to expose each microservice’s endpoints. 

Netflix Experience

2,000,000,000 requests per day to the netflix API (2013)

Netflix API Redesign

If we can improve the interaction between the API and our UIs, we have a better chance of making more of our customers happier. 

one-size-fits-all (OSFA) REST API approach

- OSFA approach is that its emphasis is to make it convenient for the API provider, not the API consumer.

  • API development platform allows each UI team to create customized endpoints. So the request/response model can be optimized for each team’s UIs to account for unique or divergent device requirements.
  • To support the variability in our request/response model, we need a different kind of architecture.

Redefine the Border Between "Client" and "Server"

The traditional definition of "client code" is all code that lives on a given device or UI. "Server code" is typically defined as the code that resides on the server. The divide between the two is the network border. 

Redefine the Border Between "Client" and "Server"

  •  The adapter (currently written in Groovy) explodes that request out to a series of server-side calls that get the corresponding content.
  •  Once the adapter has some or all of its content, the adapter processes it for delivery, which includes pruning out unwanted fields, error handling and retries, formatting the response, and delivering the document header and body. 

Reduce Chattiness

Reduce Chattiness

threading, synchronization, thread-safety, concurrent data structures, non-blocking IO and other such concerns......

- The Other Microservices topic.

Design API Platform Netflix considers:

  • Distribute API Development

    • Each client application team should be capable of implementing and operating their own endpoints and the corresponding requests/responses. 

  • Mitigate Deployment Risks

    • The system must mitigate risks inherent in enabling rapid and frequent deployment by multiple teams with minimal coordination. 

  • Support Multiple Languages

    • Engineers implementing endpoints come from a wide variety of backgrounds with expertise including Javascript, Objective-C, Java, C, C#, Ruby, Python and others.

  • Distribute Operations​

    •  Each client team will now manage the deployment lifecycle of their own web service endpoints.

Architecture

  • [1] Dynamic Endpoints

  • [2] Endpoint Code Repository and Management

  • [3] Dynamic Polyglot JVM Language Runtime

  • [4 & 5] Asynchronous Java API + Reactive Programming Model

  • [6] Hystrix Fault Tolerance

  • [7] Backend Services and Dependencies

Architecture

MS1

MS2

MS3

MS4

http://10.2.131.11:8081/api/xxx

http://10.2.131.11:8082/api/xxx

http://10.2.151.99:8080/api/xxx

Architecture

MS1

MS2

MS3

MS4

Eureka Server

Eureka Client

Eureka Client

Register

Single Entry:

http://ms1/api/xxx

 

Architecture

MS1

MS2

MS3

MS4

Eureka Server

Eureka Client

Eureka Client

Discovery

Single Entry:

http://ms1/api/xxx

 

Edge Server(Zuul)

Ribbon Client

Ribbon Client

Single Entry:

http://ms2/api/xxx

 

List<Product> productList = restTemplate.getForObject("http://ms1/api/xxx", List.class);

Architecture

MS1

MS2

MS3

MS4

Eureka Server

Eureka Client

Eureka Client

Single Entry:

http://ms1/api/xxx

 

Amazon ELB

Edge Server(Zuul)

Edge Server(Zuul)

Ribbon Client

Ribbon Client

Single Entry:

http://ms2/api/xxx

 

/ps3/movies/1

Implement by Groovy

MS1

MS2

MS3

script repository

Architecture

MS1

MS2

MS3

MS4

Eureka Server

Eureka Client

Eureka Client

Single Entry:

http://ms1/api/xxx

 

Amazon ELB

Edge Server(Zuul)

Edge Server(Zuul)

Ribbon Client

Ribbon Client

Single Entry:

http://ms2/api/xxx

 

/ps3/movies/1

Implement by Groovy

MS1

MS2

MS3

Architecture

MS1

MS2

MS3

MS4

Eureka Server

Eureka Client

Eureka Client

Single Entry:

http://ms1/api/xxx

 

Amazon ELB

Edge Server(Zuul)

Edge Server(Zuul)

Ribbon Client

Ribbon Client

Single Entry:

http://ms2/api/xxx

 

/ps3/movies/1

Implement by Groovy

MS1

MS2

Fallback

Circuit Breaker Dashboard

Announcing Zuul:

Edge Service in the Cloud

Zuul, The Gatekeeper (http://ghostbusters.wikia.com/wiki/Zuul)

--The Netflix API is the front door to that system, supporting over 1,000 different device types and handing over 50,000 requests per second during peak hours.

 To handle all of these changes, as well as other challenges in supporting a complex and high-scale system, a robust edge service that enables rapid development, great flexibility, expansive insights, and resiliency is needed.

How Does Zuul Work

At the center of Zuul is a series of filters that are capable of performing a range of actions during the routing of HTTP requests and responses.

  • Type: most often defines the stage during the routing flow when the filter will be applied.
    • ​PRE
    • ROUTING
    • POST
    • ERROR
  • Execution Order: applied within the Type, defines the order of execution across multiple filters
  • Criteria: the conditions required in order for the filter to be executed
  • Action: the action to be executed if the Criteria are met
class DeviceDelayFilter extends ZuulFilter {

    def static Random rand = new Random()
    
    @Override
     String filterType() {
       return 'pre'
     }

    @Override
    int filterOrder() {
       return 5
    }

    @Override
    boolean shouldFilter() {
       return  RequestContext.getRequest().
                getParameter("deviceType")?equals("BrokenDevice"):false
    }

    @Override
    Object run() {
       sleep(rand.nextInt(20000)) //Sleep for a random number of seconds
                                  //between [0-20]
    }
}

http://stackoverflow.com/questions/28947549/what-does-netflix-zuul-do-more-than-regular-servlet-filters

Zuul provides a framework to dynamically read, compile, and run these filters

  • The source code for each filter is written to a specified set of directories on the Zuul server that are periodically polled for changes. Updated filters are read from disk, dynamically compiled into the running server, and are invoked by Zuul for each subsequent request.

Filter Types 

  • PRE include request authentication, logging debug info...
  • ROUTING This is where the origin HTTP request is built and sent using Apache HttpClient or Netflix Ribbon.
  • POST include adding standard HTTP headers to the response, gathering statistics and metrics, and streaming the response from the origin to the client....
  • ERROR filters execute when an error occurs during one of the other phases.

How Netflix Use Zuul

  • Authentication
  • Insights
  • Stress Testing
  • Canary Testing
  • Dynamic Routing
  • Load Shedding
  • Security
  • Static Response handling
  • Multi-Region Resiliency 

Because Zuul can add, change, and compile filters at run-time, system behavior can be quickly altered.

  • We add new routes, assign authorization access rules, and categorize routes all by adding or modifying filters.
  • And when unexpected conditions arise, Zuul has the ability to quickly intercept requests so we can explore, workaround, or fix the problem. 

http://techblog.netflix.com/2013/08/deploying-netflix-api.html

Insights

Zuul gives us a lot of insight into our systems, in part by making use of other Netflix OSS components. 

(Hystrix、Ribbon、 Turbine ....)

API Gateway

- the others

https://github.com/mfornos/awesome-microservices#api-gateways--edge-services

Open-source, Microservice & API Management Layer built on top of NGINX https://getkong.org

The StrongLoop API Gateway acts as an intermediary gateway between API consumers (clients) and backend providers (API servers) that externalizes, secures, and manages APIs. 

The apiman project brings an open source development methodology to API Management, coupling a rich API design & configuration layer with a blazingly fast runtime.

An open source, fast and scalable API management platform featuring an API gateway, analytics, developer portal and dashboard.

 

Netflix Zuul vs Nginx performance

http://blog.instea.sk/2015/04/netflix-zuul-vs-nginx-performance/

Reference

microservices-api-gateway

By JengHung Chen

Made with Slides.com

microservices-api-gateway

  • 4,669

More from JengHung Chen