This is the story of how the Algorithmic Recommendations team at The New York Times moved from EC2 in AWS to Kubernetes on Google Cloud. I will explain how we moved about a dozen services under live load, what our architecture used to look like and how we transitioned to containers running on GKE.
I will give a detailed breakdown of how we migrated our production traffic system from VMs on AWS to Kubernetes, managed by GCP.
It’s a drama in three acts:
- Act 1: we happily live in AWS, serve recommendation traffic from EC2 instances, managed by Puppet, Jenkins, Consul, APIs written in Go but trouble looms on the horizon.
- Act 2: The world is in turmoil, the decision was made to move to GCP. We start to containerize our apps, move to a new Ci/CD pipeline, figure out Kubernetes and serve traffic out of AWS and GCP, multi-cloud by accident.
- Act 3: we finish the migration and productionize our Kubernetes setup, using Prometheus for monitoring, drone.io for the CI/CD pipeline and celebrate the end of our Jenkins and Puppet usage.
There will be actual code and config snippets, e.g. showing how used nginx as a reverse proxy between clouds. I’ll show how the new CI/CD pipeline works and how we gather and use metrics (in both the old and new world).