Page Comparison

...

• In Kfuse version 2.6.5 or earlier

  • The pinot server realtime runs out of disk space if Pinot is unable to move the segments to the offline server. On some cases, the offline servers hit an exception, stops handling new messages and need to be restarted.

    Code Block
    kubectl rollout restart -n kfuse statefulset pinot-server-offline

  • The persistent disk attached to Pinot Server Realtime needs to be increased. Refer to https://kloudfuse.atlassian.net/wiki/spaces/EX/pages/edit-v2/786038817#Increasing-the-existing-PVC-size

• From Kfuse version 2.6.7 onwards, there is no need to resize the pinot server realtime disks. Follow the following steps.

  1. Restart pinot-server-offline.

  2. Edit pinot-server-realtime sts remove or set DISK_BALLOON _DISK env variable to false.

  3. Wait for pinot server realtime to start up and has complete moving segments to offline servers.

  4. Edit pinot-server-realtime sts to add back DISK_BALLOON _DISK env variable to true.

DeepStore access issues

...

If you find out that the PV usage has reached 100% and cannot be restarted gracefully, you need to increase the pvc size of pinot-realtime pvcs by 10% or so to accommodate the increased requirement and restart the pinot-server offline & realtime.

...