Take a REST with HTTP/2, Protobufs, and Swagger [feedly]

----
Take a REST with HTTP/2, Protobufs, and Swagger
// CoreOS Blog

At CoreOS we build and maintain a number of API-driven services across different domains, for example: container images in Quay, key/values in etcd, and cluster orchestration in Kubernetes. We're always looking for ways to make designing and building these APIs simpler and more efficient. In this post we explore a new Remote Procedure Call (RPC) framework based on protobufs and HTTP/2 called gRPC, that we think can make building and consuming APIs better.

Today, we are using gRPC in the etcd v3 API, the rkt API, and elsewhere in Tectonic, and we plan to use it in more projects soon. And CoreOS public APIs like Quay publish Swagger API documents that describe the REST interface, so that REST API bindings can be automatically generated for a variety of languages. Today we're going to dive into why we think gRPC is very useful and how it can be a bridge to the "REST" of the world.

One of the key reasons we chose gRPC is because it uses HTTP/2, enabling applications to present both a HTTP 1.1 REST+JSON API and an efficient gRPC interface on a single TCP port. This gives developers compatibility with the REST web ecosystem while advancing a new, high-efficiency RPC protocol. With Go 1.6 recently released, Go ships with a stable net/http2 package by default.

A gRPC application called EchoService

In this post we will build a small proof-of-concept gRPC application from a gRPC API definition, add a REST service gateway, and finally serve it all on a single TLS port. The application is called EchoService, and is the web equivalent of the shell command echo: the service returns, or "echoes", whatever text is sent to it.

First, let's define the arguments to EchoService in a protobuf message called EchoMessage, which includes a single field called value. We will define this message in a protobuf ".proto" file called service.proto. Here is our EchoMessage:

message EchoMessage {   string value = 1;  }  

In this same protobuf file, we define a gRPC service that takes this data structure and returns it:

service EchoService {    rpc Echo(EchoMessage) returns (EchoMessage) {    }  }  

Running this service.proto file through the Protocol Buffer compiler protoc generates a stub gRPC service in Go, along with clients in various languages. But gRPC alone isn't as useful as a service that also exposes a REST interface, so we won't stop with the gRPC service stub.

Next, we add the gRPC REST Gateway. This library will build a RESTful proxy atop the gRPC EchoService. To build this gateway, we add metadata to the EchoService proto to indicate that the Echo RPC maps to a RESTful POST method with all RPC parameters mapped to a JSON body. The gateway can map RPC parameters to URL paths and query parameters, but we omit those complications here for brevity.

service EchoService {    rpc Echo(EchoMessage) returns (EchoMessage) {      option (google.api.http) = {        post: "/v1/echo"        body: "*"      };    }  }  

In practical terms this means the gateway, once generated by protoc, can now accept a request from curl like this:

curl -X POST -k https://localhost:10000/v1/echo -d '{"value": "CoreOS is hiring!"}'  

The whole system so far looks like this, with a single service.proto file generating both a gRPC server and a REST proxy:

gRPC API with REST gateway

To bring this all together, the echo service creates a Go http.Handler to detect if the protocol is HTTP/2 and the Content-Type is "application/grpc" and sends such requests to the gRPC server. Everything else is routed to the REST gateway. The code looks something like this:

if r.ProtoMajor == 2 && strings.Contains(r.Header.Get("Content-Type"), "application/grpc") {  grpcServer.ServeHTTP(w, r)  } else {  otherHandler.ServeHTTP(w, r)  }  

To try it out, all you need is a working Go 1.6 development environment and the following simple incantations:

$ go get -u github.com/philips/grpc-gateway-example  $ grpc-gateway-example serve  

With the server running you can attempt requests on both HTTP 1.1 and gRPC interfaces:

grpc-gateway-example echo Take a REST from REST with gRPC  curl -X POST -k https://localhost:10000/v1/echo -d '{"value": "CoreOS is hiring!"}'  

One last bonus: You can browse the Swagger UI running at https://localhost:10000/swagger-ui/#!/EchoService/Echo if you have the server above running on your laptop.

gRPC/REST API Swagger document

We've taken a look at how to use gRPC to bridge to the world of REST. If you want to take a look at the complete project, check out the repo on GitHub. We think this pattern of using a single protobuf to describe an API leads to an easy to consume, flexible API framework, and we're excited to leverage it in more of our projects. If you have questions or want to learn more, join us in the CoreOS community.

Learn more about gRPC in the Bay Area

If you're in the Bay Area, join CoreOS and other developers at the March 1 gRPC Community Event in Mountain View. We will be giving a talk about how we use gRPC at CoreOS and will hand out some CoreOS stickers.

CoreOS is Hiring

Want to help design solid APIs, create compelling user experiences, and solve challenging systems problems? Learn about opportunities to become a part of the CoreOS team! We have offices in San Francisco, New York, and Berlin.

Special thanks to the Go team for net/http2, to the gRPC team, and to the grpc-gateway team for their work making gRPC awesome. Particular shoutouts to Qi Zhao and Brad Fitzpatrick for recently getting shared TLS ports for gRPC working in Go and to Yuki Yugui Sonoda and Andrew Z. Allen for the Swagger support in grpc-gateway.

----

Shared via my feedly reader

Sent from my iPhone