This guide will walk you through the process of getting your Kubernetes cluster ready to be used on Komodo.

​
Supported Kubernetes deployments

  • Managed Kubernetes services (Amazon - EKS, Google Cloud - GKE)
  • On-prem clusters (Kubeadm, Rancher, RKE, K3s)

If you need help with anything below, please let us know on Discord! 😊

​
Prerequisites

  • A Kubernetes cluster with a publicly-accessible control plane endpoint
  • Each cluster node assigned a public (external) IP
  • Each cluster node labeled with the GPU type.

If your cluster has the NVIDIA GPU Operator installed or you are using GKE or Karpenter, your nodes already have the necessary GPU labels. No further action is required.

Currently supported GPU labels:

  • nvidia.com/gpu.product: automatically created by the Nvidia GPU operator
  • cloud.google.com/gke-accelerator: used by GKE clusters
  • karpenter.k8s.aws/instance-gpu-name: used by Karpenter

To check if your cluster is compatible, run:

echo "Checking nodes for external IP addresses..."
nodes=$(kubectl get nodes -o=jsonpath="{.items[*]['status.addresses', 'metadata.annotations']}" | tr -d " \t\n\r")
FOUND_EXTERNAL_IP_FROM_STATUS=false
FOUND_EXTERNAL_IP_FROM_ANNOTATION=false
if [[ $nodes == *"ExternalIP"* ]]; then
  FOUND_EXTERNAL_IP_FROM_STATUS=true
fi
if [[ $nodes == *"rke.cattle.io/external-ip"* ]]; then
  FOUND_EXTERNAL_IP_FROM_ANNOTATION=true
fi
FOUND_EXTERNAL_IPS=""
if [ "$FOUND_EXTERNAL_IP_FROM_STATUS" = true ] || [ "$FOUND_EXTERNAL_IP_FROM_ANNOTATION" = true ]; then
  FOUND_EXTERNAL_IPS="FOUND"
fi
echo "Checking nodes for GPU labels..."
output=$(kubectl get nodes --show-labels | awk -F'[, ]' '{for (i=1; i<=NF; i++) if ($i ~ /nvidia.com\/gpu.product=|cloud.google.com\/gke-accelerator=|karpenter.k8s.aws\/instance-gpu-name=/) print $i}')
if [ -z "$output" ]; then
  echo "No valid GPU labels found."
else
  echo "GPU Labels found:"
  echo "$output"
fi
if [ -n "$FOUND_EXTERNAL_IPS" ] && [ -n "$output" ]; then
  echo "Your cluster is ready for GPU workloads with Komodo AI"
else
  echo "Your cluster is not ready for GPU workloads with Komodo AI."
  echo "Contact us at hello@komodoai.dev or join our Discord server at https://discord.gg/baJGK6RKZC for help."
fi

Instructions for setting up NVIDIA GPU Operator on your cluster can be found here

​
Step 1: Create the Komodo Service Account

To create a service account that Komodo will use to launch your workloads on Kubernetes run:

kubectl apply -f https://raw.githubusercontent.com/komodoai/komo_docs/main/files/komodo.yaml

This will create resources in the default namespace. If you’d like to run your workloads in another namespace, download the YAML file above and change the namespace on every line with the comment Change to your namespace if using a different one

Copy this YAML to a file called service-account.yaml and then run:

​
Step 2: Get the values to connect cluster to Komodo

Run the below script to get the cluster control plane endpoint, Komodo service account token, and optionally, the cluster certificate authority data.

CLUSTER_CONTROL_PLANE_ENDPOINT=$(kubectl cluster-info  `kubectl config view | grep -m 1 "name" | awk '{ print $2 }'` | grep "control plane" | awk '{ print $NF }')
KOMODO_SA_SECRET=$(kubectl get secret komodo-secret --namespace default -o jsonpath='{.data.token}' | base64 -d)
CLUSTER_CA_DATA=$(kubectl config view --raw --minify --flatten -o jsonpath='{.clusters[0].cluster.certificate-authority-data}' 2>/dev/null)
if [ -z "$CLUSTER_CA_DATA" ]; then
  CLUSTER_CA_DATA="No Cluster Certificate Authority data found"
fi

echo "Cluster Control Plane endpoint:"
echo $CLUSTER_CONTROL_PLANE_ENDPOINT
echo "\n"
echo "Komodo Service Account Secret:"
echo $KOMODO_SA_SECRET
echo "\n"
echo "Cluster Certificate Authority data:"
echo $CLUSTER_CA_DATA

If no cluster certificate authority data was found, ask your IT administrator to obtain this data.

This is optional but highly recommended so that Komodo can verify your control plane’s TLS certificate.

​
Step 3: Connect your Kubernetes cluster to Komodo

Now that you have your control plane endpoint, service account token, and (optionally) your certificate authority data you’re ready to connect it to Komodo.

In the Komodo console, navigate to the Settings page, click Connect in the Kubernetes section and paste the values from Step 2 into the respective fields.

And that’s it! You can now launch your workflows on Komodo! Follow this tutorial to get started.

​
FAQ

GCPSupported Clouds