I’m very tempted to buy this https://www.etsy.com/listing/86833090/11x17-poster-robot-print-poster-monster

Programmable Infrastructure in Production with Kubernetes

This week I’m going to give a talk on automation and programmable infrastructure and I need a simple real-world example. I’ll use our Microbadger service, which we partially automated with our own custom scheduler. The code for our scheduler is open source and available at https://github.com/microscaling/microscaling. It scales by calling the K8s deployments API.

What does our service do?

Microbadger is a SaaS that lets you navigate the metadata for any public image on Docker Hub. You give us the image name, we get the data from Docker Hub, tidy it up and show it to you. So far, so standard.

How does it work?

Imagine a user requests a new image that isn’t already cached. There’s lots of data we want to show. Docker Hub gives us some of it very quickly and some takes much longer (in particular, a deep inspection that gets the download size and reverse engineers the docker command for each layer).

  • an AWS service (SQS for the queues).
  • A single-threaded Inspector service (Go) polls the Inspection Q for a new work item and requests the quick data from Docker Hub. It processes and caches the response and it also drops a request for the slower data onto the Size Q.
  • A single-threaded Size service (Go) polls the Size Q for a new work item and requests the slower data from DockerHub and processes and caches the response.

Behaviourally, what are the key features of this system?

  • The API service is vital, so needs to be fully resilient and fast (Go was useful for this).
  • The queues are also vital. We handle that by using a cloud queue service, which is expensive, but hey-ho. Note using SQS for our saved state keeps our k8s cluster stateless, which is easier to set up.
  • The Inspector service is urgent for new images, which are vital for our valuable new users.
  • The Size service is less urgent — if it’s unavailable for a while Microbadger won’t be as good but it’ll still work OK.

Ideal for programmable infrastructure!

  • the high priority service container identifier (we used the inspector service)
  • the low priority service container identifier (the size service)
  • It reduced the total amount of resources we used significantly, potentially enabling us to reduce our bill in future in a multi-tenant cluster, for example.

Philosophy

Money saving is good, but what interests us is the change in philosophy. In a monolithic world all of these components would have been inside a single huge service, which would probably have done all of this custom scheduling itself internally (monitor internal queues and start/stop threads).

  • Division of labour (specialisation of people and their services is more efficient)
  • Improved ability to leverage specialist 3rd party services (e.g. SQS)

SciFi author interested in tech, engineering, science, art, SF, economics, psychology, startups. Chaotic evil.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store