We’ve exited Kubernetes after 20 months. Here’s what the platform required and all the reasons that drove us away from it in the end.

This article is part of the “Goodbye K8s” series, looking at the reasons that eventually drove us away from one of the hottest platforms on the market:

1. Goodbye K8s! We Had So Many Plans!
2. Goodbye K8s! This Is Excellent!
3. Goodbye K8s! This Isn’t Working!
4. Goodbye K8s! Welcome Back, Old Friend!
5. Goodbye K8s! It’s Still a Business!
6. Goodbye K8s! Thanks for the Lessons!
7. Goodbye K8s! We’ll Meet Again!

A Financial Nightmare

With the multi-cloud technology stack sorted and the Ghost installation happily serving websites to the world, it was time to sit back, write articles — and wait for Bill Shock!

And boy, did it come. When the first bill arrived, we suddenly realised that running a two node GKE cluster (the way we’ve originally set it up) did end up costing us around £105 per month — that’s already including sustained usage discounts.

Now, that’s around £3.38 per day, as that particular month had 31 days. For a blog. That serves some websites. £3.38. Per. Day.

However, that was only for a two node GKE cluster.

For a three node cluster, our daily costs rose to over £5 per day. Adding up to over £150 per month.

For. A. Simple. Blog. Website.

Thankfully, the financial blow was much softened by Google’s generous offer to grant every new account USD 300 (around £230 at the time) in credits. Free to use on whatever you want over the course of 365 days. Later on, they changed that to 90 days.

However, that made no difference to us, as we burned through the entire pile of “free credits” in between 46 and 68 days, depending on how “adventurous” we felt with our K8s cluster setup.

A New Management World

All of a sudden, we found ourselves in a situation where we had to rely on a constant flow of free GCP credits to keep the blog running; paying for it ourselves was not an option. This meant that roughly every 40 to 60 days, we had to go through the same painstaking procedure.

Create a new GCP account. Unlock the roughly £230 of free credits (this required supplying real credit card details). Pass the updated account information to the deployment tooling. Re-deploy the entire stack. And at the end, update the DNS entries in AWS to point to the new installation.

However, it hardly ever worked that smoothly. GKE versions changed. Features became deprecated. We were suddenly stuck on legacy versions as new releases weren’t backwards compatible.

Deploying an entire new technology stack became a bit of a gamble every time we went for it. Trying to hit a moving target is a lot harder and less phun than you think.

Moreover, once we managed to get the new Ghost deployment working, we still had to migrate the content across. While Ghost has thankfully made that process fairly painless, there are still enough subtle nuances in the middle to drive someone mad (the man on this keyboard, for example).

Nonetheless, once fully up and running, the experience was really good which is in large parts due to Google’s CDN and the excellent Ghost server. But the latter was also part of a broader security concern as the server backend was exposed to the internet the entire time. The logs were telling us lots of interesting stories around bots “testing” our security.

It didn’t take a genius to figure out that this clearly wasn’t sustainable in the long run.

Time for a Realistic Assessment

The next article in this series describes our exit strategy. And how a realistic assessment of our technology stack and a relentless of focus on our original objectives resulted in some drastic simplifications.

Ultimately, this led us to abandoning K8s and reconnecting with an old friend we’ve never really lost sight of.