Before performing an upgrade, validate that the upgrade won’t revert any customization on your cluster. The steps to run a validation are detailed in here.

To check the current installed Kfuse version, run the following command:

helm list

Upgrade command

The upgrade command is the same as the install command specified in https://kloudfuse.atlassian.net/wiki/spaces/EX/pages/724238341. The command is copied here for reference. The latest released version is specified in the installation page.

helm upgrade --install -n kfuse kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse --version <SPECIFY VERSION HERE> -f custom_values.yaml

Upgrading to Kfuse version 2.2.3


  • Update the pinot zookeeper persistence (PVC) value to 32Gi. This is default value for this PVC now. Add the following snippet under pinot.zookeeper section:

persistence: size: 32Gi

Upgrading from Kfuse version 2.1 or earlier


  • Kloudfuse provided alerts organization has been updated for better maintenance. Make sure to remove old version.

# Connect to kfuse cluster and log in to catalog service pod kubens kfuse kubectl exec -it catalog-servicexxx -- bash # Remove older folders. python3 /catalog_service/catalog.py --remove_installed --list kloudfuse,kloudfuse_alerts,kubernetes_alerts --artifact_type alerts

Upgrading from Kfuse version 2.0.1 or earlier


  • The Kfuse-provisioned dashboard has been cleaned up. Run the following command:

kubectl -n kfuse exec -it kfuse-configdb-0 -- bash -c "PGDATABASE=alertsdb PGPASSWORD=\$POSTGRES_PASSWORD psql -U postgres -c 'delete from dashboard_provisioning where name='\''hawkeye-outliers-resources'\'';'; "

Upgrading from Kfuse version 1.3.4 or earlier


  • Note: Kfuse services will go offline during this process. Kloudfuse storage class configuration has been simplified to keep future releases/features in mind. This requires running the migrate_storage_class.sh script provided by Kloudfuse team.


    After running the script, ensure that the pvc’s storage class has kfuse-ssd, instead of kfuse-ssd-gcp or kfuse-ssd-aws.

    kubectl get pvc -n kfuse
  • Old alerts have to be removed. Kloudfuse alerts organization has changed with the introduction of additional alerts. New version does the organization automatically, however, the older alerts have to be removed.

    • Manually remove all alerts by navigating to the grafana tab and remove all alerts from kloudfuse_alerts and kubernetes_alerts folder.



  • Older kubernetes secret related configuration needs to be removed from the custom values.yaml file. Also kfuse-credentials secret can be removed.

auth: config: AUTH_TYPE: "google" AUTH_COOKIE_MAX_AGE_IN_SECONDS: 259200 auth: existingAdminSecret: "kfuse-credentials" existingSecret: "kfuse-credentials"
  • There is a schema change introduced in the traces table. Make sure to restart the Pinot servers after upgrade completes.

kubectl rollout restart sts -n kfuse pinot-server-realtime kubectl rollout restart sts -n kfuse pinot-server-offline


Upgrading from Kfuse version 1.3.2 or earlier


  • There is a schema change introduced in version 1.3.3. Make sure to restart the Pinot servers after upgrade completes.

kubectl rollout restart sts -n kfuse pinot-server-realtime kubectl rollout restart sts -n kfuse pinot-server-offline


Upgrading from Kfuse version 1.2.1 or earlier


  • Advance monitoring is available as an optional component in 1.3 and later releases. To enable:

    • Knight agent is required to be installed. Please review steps/settings here.

    • Additional agent settings required. Please review settings here.

  • Starting Kfuse version 1.3.0, Kfuse has added retention support using Pinot Minion framework. This feature requires changes to the existing Pinot Minion statefulset. The statefulset needs to be deleted prior to upgrade.

kubectl delete sts -n kfuse pinot-minion
  • Existing alerts have to be updated with new, efficient version. Please follow these steps to refresh the alerts.

    • Go to Kloudfuse UI → Alerts → Alert Rules

    • Using the filter panel on the left, expand “Component” item, and filter all the “Kloudfuse” and “Kubernetes” alerts to include only these. Remove each of these alerts from the filtered list. (post upgrade the new alerts will be installed automatically)

Upgrading from Kfuse version 1.1.1 or earlier

Cloud-Specific configurations

Starting Kfuse version 1.2.0, cloud-specific yamls (aws.yaml, gcp.yaml, azure.yaml) are not included in the chart anymore. The custom_values.yaml needs to include these configurations. Refer to https://kloudfuse.atlassian.net/wiki/spaces/EX/pages/793411613 and https://kloudfuse.atlassian.net/wiki/spaces/EX/pages/793378845. With Kfuse version 1.2.0, there is no need to pull the chart prior to installation. helm upgrade can be directly run with the Kfuse helm chart registry.

Database changes

A breaking change related to the number of postgresql servers installed as part of the Kfuse install was introduced after Kfuse version 1.1.0. Due to this, the stored alerts will be deleted if directly upgrading Kfuse. In order to retain the stored alerts, run the following pre and post upgrade steps below. Note that until the post-upgrade steps are executed, alerts & dashboards from pre-upgrade will not show up.


kubectl exec -n kfuse alerts-postgresql-0 -- bash -c 'PGPASSWORD=$POSTGRES_PASSWORD pg_dump -U postgres -F c alertsdb' > alertsdb.tar



kubectl cp -n kfuse alertsdb.tar kfuse-configdb-0:/tmp/alertsdb.tar kubectl exec -n kfuse kfuse-configdb-0 -- bash -c 'PGPASSWORD=$POSTGRES_PASSWORD pg_restore -U postgres -Fc --clean --if-exists -d alertsdb < /tmp/alertsdb.tar' kubectl delete pvc -n kfuse data-alerts-postgresql-0 kubectl delete pvc -n kfuse data-beffe-postgresql-0 kubectl delete pvc -n kfuse data-fpdb-postgresql-0

Upgrading from Kfuse version 1.0.4 or earlier

The Kfuse StorageClass resources need to be deleted prior to upgrade.

To check the current installed Kfuse version, run the following command:

helm list

To delete the storage class:

kubectl delete storageclass kfuse-ssd-aws kfuse-ssd-aws-gp3 kfuse-ssd-gcp

Otherwise, to upgrade just follow the install instructions as is.