For administrators

Software updates

10min

Introduction

General recommendation is to regularly check whether automated software are in working order and perform manual software updates as often as possible. Long time gaps between manual software update are likely to introduce multiple breaking changes at once, which could be hard to debug and remedy.

Updating Hosty

Hosty wrapped into a Docker image, that contains pre-installed binaries as well as Hosty CLI that is based on Composer. Here are the ways you can keep these up to date:

Docker image and pre-installed binaries

Hosty Docker image is build from lightweight Linux distribution called Alpine. It can be updated by bumping the version up using the environmental variable LINUX_ALPINE_VERSION in .env file. The available versions can be looked up on the releases page of Alpine.

Basic tools like git, docker, aws-cli, etc. are using the latest version available on each image build.

Tools like OpenTofu (Terraform), Kubernetes command line tool (kubectl) and Helm versions are updated by bumping the version up using the environmental variables OPEN_TOFU_VERSION, KUBECTL_VERSION and HELM_VERSION respectively.

Hosty CLI

To update Hosty CLI, which is based on Composer you can:

  • Execute make build and make cli in the terminal to SSH into Docker container.
  • Execute cd cli to change directory to the Hosty CLI source.
  • Execute composer outdated to check for oudated packages.
  • Execute compose update to update all the packages OR composer update example/package example/package2 to update specific packages.
  • This will update composer.json and composer.lock files, which you can commit into git repository.

Updating Terraform modules

Hosty blueprint utilizes number of terraform modules to optimize and streamline creation of resources on AWS. Within each module defintion there is a version input, which is used to set the version used by the blueprint. Above it, for each module, we've included a link to the latest available version.

How to update

  • Open terraform file that contains a module, for example /terraform/modules/cluster/eks.tf
  • You should see this resource block:
module "eks" { source = "terraform-aws-modules/eks/aws" # Versions can be found at https://registry.terraform.io/modules/terraform-aws-modules/eks/aws/latest. version = "20.14.0" ... }
  • You should be able to find the latest available version to this module, by visiting the link included in the comment above the version input of this module.
  • Update the version input of the module with a new value.
  • Execute tofu init , tofu plan and tofu apply

Recommendations

  • Before updating the module check the changelog to make sure, there are no breaking changes. If there are, change the module definition to account for them.
  • Updates to the modules are unlikely to affect the resources already deployed to AWS. If tofu plan indicates possible changes, make sure that they are expected and mentioned in the module's changelog.

Updating EKS cluster

Updating Kubernetes version for EKS cluster version is very straightforward, however it is advised to take some precautionary measures to prevent possible downtime. The Kubernetes version is controlled via a k8s_version variable in the /terraform/main.tf file and is defined separately for each cluster. The advised cluster update workflow for regular updates is:

  • Check EKS Available versions to check whether the are new versions available and for description of breaking changes in the changelog.
  • Create a new EKS cluster with the new version of Kubernetes for testing purposes.
  • Test whether projects work with the new Kubernetes version.
  • Once tests are successful, remove the newly created cluster.
  • Update the k8s_version input of the outdated cluster and execute tofu apply.

Automated EKS cluster node AMI updates

EKS cluster node groups use Bottlerocket OS through AWS managed AMIs (Amazon Machine Image). Bottlerocket OS is a free and open-source Linux-based operating system meant for hosting containers.

When the node group is deployed it uses the latest version of AMI available at that moment. As the new versions are released, these node groups need to be updated. By default it is a manual process, however we have automated this by using AWS Systems Manager Maintenance Windows.

Everyday, at 9am London time we've scheduled an automation task, that automatically updates AMIs of EKS node groups.

Automated project Docker image updates

Every project using Hosty blueprint specifies Docker images used for its containers in /.hosty/hosty.yaml file. The hosty build command will automatically pull the latest available versions for specified Docker images, with an option to override the version for testing and debugging purposes.