From AWS EC2 to Kubernetes on GCP

This is the story of how the Algorithmic Recommendations team at The New York Times moved from EC2 in AWS to Kubernetes on Google Cloud. I will explain how we moved about a dozen services under live load, what our architecture used to look like and how we transitioned to containers running on GKE.

I will give a detailed breakdown of how we migrated our production traffic system from VMs on AWS to Kubernetes, managed by GCP.

It’s a drama in three acts:

- Act 1: we happily live in AWS, serve recommendation traffic from EC2 instances, managed by Puppet, Jenkins, Consul, APIs written in Go but trouble looms on the horizon.

- Act 2: The world is in turmoil, the decision was made to move to GCP. We start to containerize our apps, move to a new Ci/CD pipeline, figure out Kubernetes and serve traffic out of AWS and GCP, multi-cloud by accident.

- Act 3: we finish the migration and productionize our Kubernetes setup, using Prometheus for monitoring, drone.io for the CI/CD pipeline and celebrate the end of our Jenkins and Puppet usage.

There will be actual code and config snippets, e.g. showing how used nginx as a reverse proxy between clouds. I’ll show how the new CI/CD pipeline works and how we gather and use metrics (in both the old and new world).

Comments

Comments are closed.