Traefik, Consul and Microservices in Go

Posted by Riomhaire Research on Friday, June 1, 2018

Microservices are very popular - but how do you scale them ? There are many alternative solutions to this computer science question, but the one’s today we will consider are a couple of common and popular pieces of software called ‘Traefik’ and ‘Consol’. This article is an alternative approch to the one we did in the Linkerd API Gateway Pattern.

The Problem.

The solution to scaling many instances of the same API from an API perspective (we wont be talking about repository backends such as databases within this article) resolve around the need for two things.

  1. How to find all the instances of a service you have.
  2. A way to route requests to those instances depending on criteria.

In this blog (1) is undertaken by Consul and (2) by Traefik. There are extra criteria you may want to consider:

  1. Health Checks - is a micro-service ‘healthy’ and available.
  2. Micro-services come and go … so its nice to do this this automatically reconfigure without having to reboot things.
  3. Some servers are more powerful than others and are able to handle more traffic so having more than one routing algorithm maybe nice to have.
  4. Scaling of registry and routing services.

What we want to build is a fault tolerant and scalable ‘greeting’ service … where we expose a couple endpoints which when called return either the message ‘Hello’ and info about the host handling the call, and a health endpoint so Consul can ensure that the service is still there.

Here is what we are trying to achieve:

The Greeting API

When an instance of one of our microservices starts up it registers with Consol (and de-registers on shutdown). When we invoke our API greeting service via a get on the ‘hello’ endpoint via the Traefik reverse proxy host, Traefik queries Consol for the service information and then selects one of the instances to invoke. That is basically what we are trying to achieve.

We will be running Traefik and Consol on one server and two instances of the greeting service on two others - four greeting instances in total.

Consul

Consul is a tool for discovering and configuring services in your infrastructure. Consul is basically a registry of serices with a number of useful features such as health checking, multi data-center support, service discovery and key-value store, and clustering amongst others. We won’t be going into how to configure for example a cluster or persistence - the consul website at consul.io has guides to do that. In the scenario above we will run a dev instance of Consul and once you have installed the binary you need to execute:

empire :: ~ » consul agent -dev -client 0.0.0.0 

You might need to use the ‘-bind’ option if you have more than one network interface. The client option allows you to point you browser at ‘http://:8500’ and receive information about the routes and services available and is quite useful (has an API as well):

The Consul Dashboard

Traefik

Traefik is a reverse proxy for HTTP and load balancer for use to deploy microservices (AKA Gateway API). Traefik supports a number of backends such as Docker, Swarm mode, Kubernetes, Marathon, Consul, Etcd, Rancher, Amazon ECS, amongst others to automatically manage its configuration.

empire :: ~ » traefik

The application looks for a file called ’traefik.toml’ in the current directory for its configuration. For development reasons we will try to keep this simple with only options for a dashboard (port 8888), the port to run (port 8090) on and information about how to talk to consol (its localhost port 8500).

################################################################
# Consul Catalog configuration backend
################################################################
debug = false

[entryPoints]
  [entryPoints.http]
  address = ":8090"

# Enable web configuration backend
[web]

# Web administration port
#
# Required
#
address = ":8888"

# Enable Consul Catalog configuration backend
[consulCatalog]

# Consul server endpoint
#
# Required
#
endpoint = "127.0.0.1:8500"

# Default domain used.
#
# Optional
#
#domain = "localhost"

# Expose Consul catalog services by default in traefik
#
# Optional
#
exposedByDefault = true

# Prefix for Consul catalog tags
#
# Optional
#
prefix = "traefik"

Traefik’s dashboard is similar to Consul’s but with slightly different data.

The Traefik Dashboard

Building The Greeting Application

So we have the basic infrastructure up and running. So what about the ‘greeting’ application ?

The first step is we have to decide how our URI for the greeting application is going to be constructed since it will define how the routing can be done. There are many ways to do this and the Meosif website is a good place to start for information on the subject. The pattern we like and use within Riomhaire follows this one:

        http(s)://<api-gateway-host>:<port>/api/<service-name>/<version>/<endpoint>

This pattern allows you to user path-prefix your routings. For our ‘greeting’ application we decided on URL’s in the form:

        http://empire:8090/api/greeting/v1/hello

The language of choice for us at the moment is ‘go’ and the full ‘consuldemo’ greeting application is based on the one given in the varunksaini blog post. If you know ‘go’ the following code is fairly simple:

package main

import (
	"flag"
	"fmt"
	"html"
	"log"
	"math/rand"
	"net/http"
	"os"
	"os/signal"
	"time"

	"github.com/gorilla/mux"
	consul "github.com/hashicorp/consul/api"
)

//Client provides an interface for getting data out of Consul
type Client interface {
	// Get a Service from consul
	Service(string, string) ([]string, error)
	// Register a service with local agent
	Register(string, int) error
	// Deregister a service with local agent
	DeRegister(string) error
}

type client struct {
	consul *consul.Client
}

//NewConsul returns a Client interface for given consul address
func NewConsulClient(addr string) (*client, error) {
	config := consul.DefaultConfig()
	config.Address = addr
	c, err := consul.NewClient(config)
	if err != nil {
		return &client{}, err
	}
	return &client{consul: c}, nil
}

// Register a service with consul local agent - note the tags to define path-prefix is to be used.
func (c *client) Register(id, name, host string, port int, path, health string) error {

	reg := &consul.AgentServiceRegistration{
		ID:      id,
		Name:    name,
		Port:    port,
		Address: host,
		Check: &consul.AgentServiceCheck{
			CheckID:       id,
			Name:          "HTTP API health",
			HTTP:          health,
			TLSSkipVerify: true,
			Method:        "GET",
			Interval:      "10s",
			Timeout:       "1s",
		},
		Tags: []string{
			"traefik.backend=" + name,
			"traefik.frontend.rule=PathPrefix:" + path,
		},
	}
	return c.consul.Agent().ServiceRegister(reg)
}

// DeRegister a service with consul local agent
func (c *client) DeRegister(id string) error {
	return c.consul.Agent().ServiceDeregister(id)
}

// Service return a service
func (c *client) Service(service, tag string) ([]*consul.ServiceEntry, *consul.QueryMeta, error) {
	passingOnly := true
	addrs, meta, err := c.consul.Health().Service(service, tag, passingOnly, nil)
	if len(addrs) == 0 && err == nil {
		return nil, nil, fmt.Errorf("service ( %s ) was not found", service)
	}
	if err != nil {
		return nil, nil, err
	}
	return addrs, meta, nil
}

func main() {
	consul := flag.String("consul", "localhost:8500", "Consul host")
	port := flag.Int("port", 10101, "this service port")
	flag.Parse()

	hostname, _ := os.Hostname()
	log.Println("Starting up... ", hostname, " consul host", *consul, " service  ", *port)

	// Register Service
	id := fmt.Sprintf("greeting-%v-%v", hostname, *port)
	consulClient, _ := NewConsulClient(*consul)
	health := fmt.Sprintf("http://%v:%v/api/greeting/v1/health", hostname, *port)
	consulClient.Register(id, "greeting", hostname, *port, "/api/greeting", health)

	router := mux.NewRouter().StrictSlash(true)

    // Define Health Endpoint
	router.Methods("GET").Path("/api/greeting/v1/health").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		str := fmt.Sprintf("{ 'status':'ok', 'host':'%v:%v' }", hostname, *port)
		fmt.Fprintf(w, str)
		log.Println("/api/greeting/v1/health called")
	})

    // The Hello endpoint for the greeting service
	router.Methods("GET").Path("/api/greeting/v1/hello/").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		str := fmt.Sprintf("Hello, %q at %v:%v\n", html.EscapeString(r.URL.Path), hostname, *port)
		rt := rand.Intn(100)
		time.Sleep(time.Duration(rt) * time.Millisecond)
		fmt.Fprintf(w, str)
		log.Println(str)
	})


    // De-register service at shutdown.
	c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt)
	go func() {
		for sig := range c {
			log.Println("Shutting Down...", sig)
			consulClient.DeRegister(id)
			log.Println("Done...Bye")
			os.Exit(0)
		}
	}()

	log.Fatal(http.ListenAndServe(fmt.Sprintf(":%v", *port), router))

}

Compiling the golang application and copying to the three machines warband, burgh and empire and then running an instance on each host by the simple command:

    consuldemo --consul "empire:8500"

after a few seconds you will start to see consul calling back the health endpoint:

2018/06/02 21:25:45 Starting up...  burgh  consul host empire:8500  service   10101
2018/06/02 21:25:49 /api/greeting/v1/health called
2018/06/02 21:25:59 /api/greeting/v1/health called

We are now at the point where we have three instances of the greeting microservice running on three hosts. Proving that we have load balancing and routing to the three instances we will open a terminal and fire up our old friend ‘curl’ and point it at the Traefik host and entry port:

warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at empire:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at burgh:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at warband:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at empire:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at burgh:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at warband:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at empire:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at burgh:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at warband:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at empire:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at burgh:10101
warband :: ~/Downloads » curl -L http://empire:8090/api/greeting/v1/hello
Hello, "/api/greeting/v1/hello/" at warband:10101
warband :: ~/Downloads » 

As you can see all hosts are hit and in order because the default load balancing algorithm is the classic ‘round robin’ one.

Conclusion and Comments

Setting up and scaling microservices using Consul and Traefik is fairly straight forward. There are a load of other things and options to play with with these two excellent package - but at least we have a place to start from.

Gary Leeson.