K8s best practice - building infrastructure that developers love

Friendly K8s Development

Why should developers not have a good user experience.

Put up the Skaffold

Take the burden out of rolling out local releases.

Friendly Kubernetes Development.

It’s a common problem; that shiny new Kubernetes cluster seems perfect to the team that built it, they followed DevOps principles and have Continuous Everything. They are deploying workloads using GitOps! It’s all really slick but there is still a disconnect between the local development loop and your CI process.

So what’s the problem?  Well it’s not necessarily CI as a process, although there can commonly be a very weak separation of stages within a pipeline which makes it difficult to replicate locally. The problem lies with the disconnect between CI and the local developer experience.

Why should developers get short changed on usability?

In most tech fields a lot of thought is given to the end users' experience.  Developers are an expensive resource, so optimising frequent time consuming tasks for them not only makes financial sense but can also help to attract talent. After all, if you get the impression you are going to spend a quarter of your time fighting with jenkins then that’s generally not an attractive proposition from a new employer!

Increasingly, developers are beginning to expect the same level of fluidity from internal systems that they can get from managed services such as Google Cloud Run or Google App Engine.  

This is hard to deliver with complex unmanaged infrastructure such as Kubernetes. The release process is multi staged and typically systems running in a cluster are broken down into several services.

Clearly there are lots of ways to approach developing software, but there are a few common problems.

Remote development environments have fixed versions of tooling, some teams will need to keep parity with production while others may be working on new functionality. It’s often not realistic to give every developer an entire cluster in the cloud to develop against; this limitation often delays testing of core system upgrades such as databases, or pipelines which can span many services and many releases before all components are ready.
  • Having to push everything to git to be queued for release to a remote development environment creates a bottleneck, devs need to be able to work locally in a production-like environment with a short feedback loop.
  • It’s not always practical to mock every endpoint your service will communicate with. Sometimes you need to be able to run a set of services, this can be complex when solely using tooling like docker and can lead to misconfiguration.
  • Environment configuration can sometimes be tightly coupled with the build pipeline for that environment this introduces uncertainty into the build process..
  • The tooling that the Platform Engineering teams write to deploy to your remote development environment generally focuses heavily on git release strategy, this means it is often left on the shelf for local work.  Devs are left to figure out the manual build steps themselves, this means time wasted at both ends.
  • Security flaws are not detected until the software gets to its release stage because developers likely do not have security scanning tools running in their local process..

What’s the solution?

Considering the developer experience from the get go is ideal but this is often not realistic, existing build pipelines and tooling are often hard to replace without interruption and this often requires your Platform Engineers to write much more complex deployment scripts.

Step up Skaffold

Skaffold can handle the entire CI process, no longer does your developer need to hand roll each release to their local environment. The local release process now just consists of a single command to start watching for code changes.

skaffold dev --port-forward 

Now, every time they save a change to a file a new container will be built, tagged and pushed to their local cluster. ( Minikube, K3s or indeed a preconfigured remote cluster. ) Manifests will be updated ( or can be an output to be pushed to a CD tool such as Argo or  Google Cloud Deploy ) and the change applied. Skaffold will even forward ports on localhost so they can see the change in a browser.

To give us one single defined build process from local iteration right through to production, Skaffold provides a Google Cloud Build container which just works out of the box,  so Operations can use skaffold configuration directly in the existing CI process with very little effort. 

As an added benefit Google Cloud Code is an IDE plugin available for most mainstream IDE’s which adds all the tooling your developers will need to get started. With drop-in tools available at both these key stages in the development pipeline skaffold is easy to integrate into your existing workflow.

Better still - if you want to try all this out with practically zero investment in time,  it all comes pre configured when you start a cloud shell instance from the GCP console; including a local kubernetes cluster!

Key Benefits

Skaffold streamlines the developer experience with Kubernetes, the key usability improvements are:

  • Single command to set up a development cluster, port forward and watch code for changes.
  • Single infrastructure configuration file for all environments.
  • Allows testing to be integrated into the local build process.
  • Support for various container and application build tooling (buildpacks, kaniko etc.)
  • Ability to directly deploy or render configuration for other CD tooling such as Argo.
  • Support for Helm and Kustomize.
  • Works seamlessly inside your CI pipeline.

Recent posts

How Santa prepares for Christmas using Google Cloud
{% video_player "embed_player" overrideable=False, type='scriptV4', hide_playlist=True,...
Ho, Ho, Ho Merry Christmas! 🎅
{% video_player "embed_player" overrideable=False, type='scriptV4', hide_playlist=True,...
CTS's Christmas Tip of the Week - It's a Gmail cracker

As it is the festive seasonwe thought we’d bring you a tip that will help you organise...

Recent posts

How Santa prepares for Christmas using Google Cloud
{% video_player "embed_player" overrideable=False, type='scriptV4', hide_playlist=True,...
Ho, Ho, Ho Merry Christmas! 🎅
{% video_player "embed_player" overrideable=False, type='scriptV4', hide_playlist=True,...
CTS's Christmas Tip of the Week - It's a Gmail cracker

As it is the festive seasonwe thought we’d bring you a tip that will help you organise...