Prometheus Remote Write Config

Using an existing Prometheus server

Add http://<REPLACE KFUSE ADDRESS>/write as the remote_write endpoint (Note: use http or https accordingly).

To get the ingress IP for the kfuse stack, run: kubectl get svc | grep kfuse-ingress-nginx-controller. You can use the EXTERNAL-IP address or DNS name. Note that DNS name will not work if you have the kfuse stack installed on a different VPC or cluster other than the target cluster. You can use the IP address, however.

If the Prometheus server was installed using the Prometheus helm chart, then add the following helm values and run helm upgrade.

serverFiles:
  prometheus.yml:
    remote_write:
      - url: http://<REPLACE KFUSE ADDRESS>/write
        queue_config:
          max_samples_per_send: 4000

Prometheus Operator

Edit the Prometheus CRD (kind: Prometheus) with the following remoteWrite entry:

remoteWrite:
  - queueConfig:
      maxSamplesPerSend: 4000
    url: http://<REPLACE KFUSE ADDRESS>/write

Replacing Prometheus Server

Prometheus server can also be replaced with Grafana Agent, which is lighter weight and can be configured to scrape and remote write metrics. The following helm chart can be used to install the agent. Ensure that the following helm values are used.

config:
  prometheus:
    remoteWrite:
      url: http://<REPLACE KfUSE ADDRESS>/write
    configs:
      <copy existing scrape configs here>