How to Install FluxCD on a Kubernetes Cluster (Step-by-Step Guide)

Overview

Hello “home-labers”, devops people and kubernetes adventurers. This post is a continuation of my new home-lab series and here I’ll be talking about the steps I took in order to install FluxCD on a Kubernetes cluster. In my case, I run a HA k3s cluster on my home-lab(link to the step by step guide on high availability k3s cluster setup - LINK) and I decided to go full GitOps and try to integrate all the best practices to deploy apps, manage infrastructure and provision resources. The first step for that was to choose a GitOps tool and for me it was FluxCD LINK.

Second step was to choose a Git version control and for me that was Github. FluxCD support many major Git platforms - Gitlab, Bitbucket, Github, Azure Devops, Gitea, AWS Code Commit etc, and they support them all and have documentation for the same - LINK.

Even though I installed the fluxcd on the k3s cluster, the same process applies to all Kubernetes distros(k8s, microk8s, rke, EKS, AKS, GKE)

Requirements:

• a running kubernetes cluster,
• fluxcd cli tool installed,
• git server/platform(I choose Github so I’ll be covering process for Github),

Homelab update: Installed the NFS CSI driver and connected it to my NAS and now dynamic storage provisioning LINK

Why fluxcd?

Main reason why I went with FluxCD is that it has a small footprint, so small resource usage was my first choice, which is my main goal for this home-lab, it’s modular and afterwards I can install plugins/other services as I go, and I can start using it right away even with basic install is enough which is just couple of controllers and that’s it. When you compare it with ArgoCD, it is more lighweight than ArgoCD. Although by default it does not come with a web gui dashboard like ArgoCD but it does offer web gui dashboard as instalable plugins.

Another reason was that in the past I mostly worked with ArgoCD and this was a perfect oportunity to try out and test FluxCD, so far I’m liking it.

Install fluxcd cli tool

First requirement is to install the FluxCD CLI tool. With it, we’ll do the initial installation and setup of FluxCD on the Kubernetes cluster and most of the FluxCD operation we’ll do with the same tool. Bellow are commands to install the FluxCD on Linux and MacOS:

Install on MacOS

brew install fluxcd/tap/flux

Install on Linux

curl -s https://fluxcd.io/install.sh | sudo bash

Install fluxcd on the cluster / FluxCD bootstrap

Next step is to install Fluxcd on a Kubernetes cluster. They actually call the process Flux bootstrap. Now, there are several options to run the FluxCD install - Personal option with the PAT(Personal Access Token), Organizations and SSH key version(Deploy keys as they call it) and we’ll explain those in a bit.

The PAT method requires that you on your Git account create the PAT token that has read/write permission to the git repo on which you plan commit GitOps/Kubernetes code and configs.

How to create a Github Personal Access Token(PAT)

We need to navigate to the developer settings in order to find the PAT menu.

Upper left corner with your profile icon -> Settings -> At the bottom of the left side menu -> Developer settings -> Personal Access tokens -> Fine grained token -> Generate new token.

This will start a token setup/configuration page where we need to define the longevity of the token, define to which repository will token have access and define repository permissions. The longevity selection is up to you - will you not have an expiration date or rotate/renew token occasionally. I strongly suggest to only define one repository to which token will have access and not all the repositories.

For the repository permissions, it’s enough to set - Administration -> Access: Read-only, Contents -> Access: Read and write and Metadata -> Access: Read-only.

Once you defined all the necessery properties, finalize the token creation. Then you’ll be presented with a token value and this is important, copy that value and strongly safekeep it since it will be displayed only once. If you loose it, you need to start over and create a new one.

The value of token parsed as the environment variable on your workstation:

export GITHUB_TOKEN=<gh-token>

Or add it in your .bashrc/.zshrc file and save it.

For the SSH deploy keys part, you just need to copy the contents of the .pub key from your .ssh folder and paste the content in the SSH and GPG keys menu also located on the left side menu in Settings

Bootstrap the cluster with FluxCD

The flux command bellow is for the personal Github account with a personal access token.

flux bootstrap github \
  --token-auth \
  --owner=my-github-username \
  --repository=my-repository-name \
  --branch=main \
  --path=clusters/my-cluster \
  --personal

This is one for Organizations(companies, organization running production etc) also with a PAT.

flux bootstrap github \
  --token-auth \
  --owner=my-github-organization \
  --repository=my-repository \
  --branch=main \
  --path=clusters/my-cluster

The last one here is for SSH connections, meaning - it uses SSH key pair, you need to generate beforehand(private and public key and public key must reside on Github) to connect to Github and read/write repository content.

flux bootstrap git \
  --url=ssh://[email protected]/<org>/<repository> \
  --branch=<my-branch> \
  --private-key-file=<path/to/ssh/private.key> \
  --password=<key-passphrase> \
  --path=clusters/my-cluster

Me personally I went with for now with the SSH install option.

NOTE - Please be aware when you run the fluxcd bootstrap command if you have kube context for several different clusters. FluxCD by default reads the kube config context and allways targets the default cluster context and fluxcd still doesn’t not have option to choose a speciffic kube config file so either make sure to select a proper default context or have a separate config file and export the file location as the KUBECONFIG environment variable.

Create fluxcd repo secret and allow access

Next is to add the repository itself to flux and create our git source of truth. This will tell Flux which repository and branch to monitor for changes.

flux create source git homelab \
  --url='ssh://[email protected]/username/homelab.git' \
  --private-key-file='/user/.ssh/github-private-key' \
  --password='private-key-password'
  --branch=main \

Even though I need to provide a password in this command, this acctually will be created as a secret and it won’t leave my cluster.

Create a repo kustomization to reconsile/sync

Lastly, I created a kustomization resources that will apply my changes and keep it in synced with git and infrastructure.

flux create kustomization homelab 
  --source=homelab 
  --path=./ 
  --prune=true 
  --validation=client 
  n--interval=5m

First commit and deploy

Now we’re ready to deploy. First step is to decide on the repository directory structure. Me personally I choose the monorepo approach, since this is my homelab and this structure will allow to have in one place my infrastructure configuration and apps/services.

As an example, I’m going to deploy Grafana on my cluster and I’ll be using the Helm chart for Grafana. So in order to tell Flux that I’m deploying a Helm chart, I’ve created a Grafana folder and in that folder I have two files - repository.yaml and release.yaml. Why two files? The repository file is to create Helmrepository resource and the Helm chart url. This tells Flux it’s a Helm resource so it knows it needs to use the Helm controller component to deploy the chart on the cluster. The release.yaml files is for to define values and configuration for the same Helm chart we want to deploy.

This is how the repository.yaml needs to look.

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: grafana
  namespace: monitoring
spec:
  interval: 24h
  url: https://grafana.github.io/helm-charts

And this is the example for the release.yaml files.

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: grafana
  namespace: monitoring
spec:
  interval: 30m
  chart:
    spec:
      chart: grafana
      version: "8.8.2"
      sourceRef:
        kind: HelmRepository
        name: grafana
        namespace: monitoring
      interval: 48h
  values:
    persistence:
      enabled: true
      type: pvc
      accessModes:
        - ReadWriteMany
      size: 4Gi
    initChownData:
      enabled: false

Checkout the docs for more repository strategies - LINK

After this, commit and push the files to your repository and run the following flux command to initiate the sync process between the Flux and git repo, or per FluxCD terms - reconsiliation.

flux reconsile kustomization homelab

End result:

By default, flux does not have webhook configured for git repository for automatic sync when pushing changes to the git repo. That needs to be configured separately. Until then, on each git change, you need to initiate the reconsile command.

Summary

And this concludes this article. What have we achieved/gained by installing fluxcd on a Kubernetes cluster? Well, this empowers you to adopt a GitOps approach, ensuring your cluster’s configuration and applications remain consistent, version-controlled, and automated. By leveraging FluxCD’s integration with Git repositories, you simplify application deployment, updates, and rollbacks, promoting reliability and scalability. This streamlined workflow not only reduces manual intervention but also enhances the overall management and operational efficiency of your Kubernetes environment.

Plus, by practising GitOps, you’ll learn another skill that will give you another tool you can put on your skillset belt.

Take care and thank you for your time!