Introduction to gRPC
by Michał Pawlik
code::dive 2019
A high performance, open-source universal RPC framework
An interprocess communication technique that is used for client-server based applications.
RPC
REST
JSON over HTTP/1.1
sounds similar to...
REpresentational State Transfer
Advantages
- Easy to learn
- Simple to implement
- Common
- Human readable serialization format
Drawbacks
- No documentation support built-in
- Requires boilerplate
- Bad support for versioning
- No type safety
- Inefficient format, CPU intensive compression
- No built-in support for pagination
- No valid way to trigger actions
Example OpenAPI definition
paths:
/pet:
post:
tags:
- "pet"
summary: "Add a new pet to the store"
description: ""
operationId: "addPet"
consumes:
- "application/json"
- "application/xml"
produces:
- "application/xml"
- "application/json"
parameters:
- in: "body"
name: "body"
description: "Pet object that needs to be added to the store"
required: true
schema:
$ref: "#/definitions/Pet"
responses:
405:
description: "Invalid input"
security:
- petstore_auth:
- "write:pets"
- "read:pets"
put:
tags:
- "pet"
summary: "Update an existing pet"
description: ""
operationId: "updatePet"
consumes:
- "application/json"
- "application/xml"
produces:
- "application/xml"
- "application/json"
parameters:
- in: "body"
name: "body"
description: "Pet object that needs to be added to the store"
required: true
schema:
$ref: "#/definitions/Pet"
responses:
400:
description: "Invalid ID supplied"
404:
description: "Pet not found"
405:
description: "Validation exception"
security:
- petstore_auth:
- "write:pets"
- "read:pets"
Sample from https://editor.swagger.io/
gRPC
Alternative to REST?
We'll see...
Handful of facts
gRPC Remote Procedure Calls
gRPC background
- Free and open source since 2015
- Built by Google - successor to internal RPC platform called Stubby
- Project of CNCF (Cloud Native Computing Foundation)
- Adopted in production by Google, Salesforce, Netflix, Spotify and more
- Uses protocol buffers for serialization
gRPC principles
- Based on services and messages
- Cross platform
- Built-in streaming support
- Pluggable security, metrics, health checks using interceptors
- Versioning support
- Layered - key facets of the stack must be able to evolve independently
Compared to other RPCs
- Asynchronous, non-blocking - supports distributed systems
- Very simple to define - just define and compile protobuf
- Message-based, no shared or mutable state
- Open source, based on standards
gRPC communication
Protobuf
Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler.
~developers.google.com
Let's try it out
Protobuf syntax
syntax = "proto3";
package example;
message Person {
string name = 1;
int32 id = 2;
string email = 3;
}
message Product {
int64 id = 1;
string name = 2;
double price = 3;
map<string,string> properties = 4;
}
message EmptyMessage {}
service ExampleService {
rpc RequestResponse(MyRequestMsg) returns (MyResponseMsg) {}
rpc ClientStreaming(stream MyRequestMsg) returns (MyResponseMsg) {}
rpc ServerStreaming(MyRequestMsg) returns (stream MyResponseMsg) {}
rpc BiDiStreaming(stream MyRequestMsgg) returns (stream MyResponseMsg) {}
}
Message definition
Tag numbers
Service definition
Sample service
- Define a product
- Expose product listing method
Sample service
// file name: shop.proto
syntax = "proto3";
package grpc_introduction;
message Product {
int64 id = 1;
string name = 2;
string description = 3;
double price = 4;
map<string,string> properties = 5;
}
message ProductsListRequest {}
service ShopServer {
rpc GetProducts (ProductsListRequest) returns (stream Product);
}
Implementation
Example in Go
Install dev tools
$ sudo dnf -y install protoc # or alike for your system
$ go get -u github.com/golang/protobuf/protoc-gen-go
Compile protobuf
$ protoc --go_out=. ./grpc_introduction/shop.proto
Import in code
package main
import (
pb "grpc_introduction"
"google.golang.org/grpc"
)
/* Rest of the code */
Output
package grpc_introduction
import (
fmt "fmt"
proto "github.com/golang/protobuf/proto"
math "math"
)
/*
... skipped ...
*/
type Product struct {
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
Description string `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
Price float64 `protobuf:"fixed64,4,opt,name=price,proto3" json:"price,omitempty"`
Properties map[string]string `protobuf:"bytes,5,rep,name=properties,proto3" json:"properties,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
Building server
type shopServer struct {
products []*pb.Product
}
func (s *shopServer) registerProduct(p *pb.Product) {
s.products = append(s.products, p)
}
func (s *shopServer) GetProducts(req *pb.ProductsListRequest, stream pb.ShopServer_GetProductsServer) error {
for _, p := range s.products {
if err := stream.Send(p); err != nil {
return err
}
}
log.Println("Products delivered")
return nil
}
func main() {
lis, _ := net.Listen("tcp", fmt.Sprintf(":%d", 9090))
log.Println("Listening on", lis.Addr())
grpcServer := grpc.NewServer()
svr := &shopServer{}
for i := 0; i < 20; i++ {
p := pb.Product{
Name: fmt.Sprintf("Product-%d", i),
Id: int64(i),
}
svr.registerProduct(&p)
}
pb.RegisterShopServerServer(grpcServer, svr)
grpcServer.Serve(lis)
}
Building client
func main() {
conn, _ := grpc.Dial("localhost:9090", grpc.WithInsecure())
client := pb.NewShopServerClient(conn)
ctx, _ := context.WithTimeout(context.Background(), 30*time.Second)
stream, _ := client.GetProducts(ctx, &pb.ProductsListRequest{})
products := make([]*pb.Product, 0)
for {
product, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
products := append(products, product)
}
log.Println(products)
}
Server build and run
$ go build
$ ./server
2019/09/17 11:06:36 Listening on [::]:9090
Client build and run
$ go build
$ ./client
2019/09/17 11:09:47 [name:"Product-0" id:1 name:"Product-1" id:2 name:"Product-2" id:3 name:"Product-3" id:4 name:"Product-4" id:5 name:"Product-5" id:6 name:"Product-6" id:7 name:"Product-7" id:8 name:"Product-8" id:9 name:"Product-9" id:10 name:"Product-10" id:11 name:"Product-11" id:12 name:"Product-12" id:13 name:"Product-13" id:14 name:"Product-14" id:15 name:"Product-15" id:16 name:"Product-16" id:17 name:"Product-17" id:18 name:"Product-18" id:19 name:"Product-19" ]
$ go build
$ ./server
2019/09/17 11:06:36 Listening on [::]:9090
2019/09/17 11:06:38 Products delivered
How about Python?
import grpc
import shop_pb2
import shop_pb2_grpc
channel = grpc.insecure_channel('localhost:9090')
stub = shop_pb2_grpc.ShopServerStub(channel)
products = stub.GetProducts(shop_pb2.ProductsListRequest())
for product in products:
print(product.name, product.id)
Compile protobuf
$ mkdir python-client
$ cp shop.proto python-client
$ cd python-client
$ virtualenv -p python3 .venv
$ pip install grpcio grpcio-tools
$ python -m grpc_tools.protoc -I=. --python_out=. --grpc_python_out=. shop.proto
# shop_pb2_grpc.py and shop_pb2.py are now generated in current directory
Implement client
Run it!
$ python main.py
Product-0 0
Product-1 1
Product-2 2
Product-3 3
Product-4 4
Product-5 5
Product-6 6
Product-7 7
Product-8 8
Product-9 9
Product-10 10
Product-11 11
Product-12 12
Product-13 13
Product-14 14
Product-15 15
Product-16 16
Product-17 17
Product-18 18
Product-19 19
Other languages?
-
Supported officially
-
C++
-
Java
-
Python
-
Dart
-
Objective C
-
Go
-
Ruby
-
Node.js
-
Dart
-
C#
-
PHP
-
-
And many more community driven
Summing up
Benefits of gRPC
- Protocol first
- Type safety
- Polyglot
- Easy to bootstrap
- Compared to JSON:
- Smaller messages
- less CPU-intensive
Thank you!
Questions?
Introduction to gRPC - Code Dive
By Michał
Introduction to gRPC - Code Dive
- 382