Prometheus Remote Write Config

Using an existing Prometheus server

Add http://<REPLACE KFUSE ADDRESS>/write as the remote_write endpoint (Note: use http or https accordingly).

To get the ingress IP for the kfuse stack, run: kubectl get svc | grep kfuse-ingress-nginx-controller. You can use the EXTERNAL-IP address or DNS name. Note that DNS name will not work if you have the kfuse stack installed on a different VPC or cluster other than the target cluster. You can use the IP address, however.

If the Prometheus server was installed using the standalone Prometheus helm chart, then add the following helm values and run helm upgrade.

serverFiles:
  prometheus.yml:
    remote_write:
      - url: http://<REPLACE KFUSE ADDRESS>/write
        queue_config:
          max_samples_per_send: 4000

Prometheus Operator

Helm Chart Values

If the Prometheus server was installed using Prometheus Operator or kube-prometheus-stack helm chart, then add the following helm values and run helm upgrade:

prometheus:
  remoteWrite:
    - queueConfig:
        maxSamplesPerSend: 4000
      url: http://<REPLACE KFUSE ADDRESS>/write

Prometheus CRD

Alternatively, the remote write config can be configured thru Prometheus CRD.

Edit the Prometheus CRD (kind: Prometheus) with the following remoteWrite entry:

remoteWrite:
  - queueConfig:
      maxSamplesPerSend: 4000
    url: http://<REPLACE KFUSE ADDRESS>/write

Replacing Prometheus Server

Prometheus server can also be replaced with Grafana Agent, which is lighter weight and can be configured to scrape and remote write metrics. The following helm chart can be used to install the agent. Ensure that the following helm values are used.