Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Version History

« Previous Version 3 Current »

Kloudfuse 3.1.0 or higher supports ingesting metrics using Prometheus-Pushgateway

The Prometheus Pushgateway exists to allow ephemeral and batch jobs to expose their metrics to Prometheus. They can instead push their metrics to a Pushgateway and it then exposes these metrics to Prometheus.

Enable Pushgateway on Kloudfuse

You will need to update your values.yaml file used for Kfuse helm chart installation to enable the pushgateway:

global:
  orgId: "..."
  # ...
  pushgateway:
    enabled: true
  # ...

After updating the values.yaml, you can install/upgrade your Kfuse release using the following command:

helm upgrade --install  kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse --version 3.1.0 -f values.yaml

Once the upgrade is done, you will see the pod for kfuse-pushgateway spin up in your env.

Sending metrics to the Pushgateway

Using the Prometheus text protocol, pushing metrics is easy so no separate CLI is provided. Simply use a command-line HTTP tool like curl. Your favorite scripting language has most likely some built-in HTTP capabilities you can leverage here as well.

Note that in the text protocol, each line has to end with a line-feed character (aka 'LF' or '\n'). Ending a line in other ways, e.g. with 'CR' aka '\r', 'CRLF' aka '\r\n', or just the end of the packet, will result in a protocol error.

Pushed metrics are managed in groups, identified by a grouping key of any number of labels, of which the first must be the job label. The groups are easy to inspect via the web interface.

For implications of special characters in label values see the URL section below.

Examples:

  • Push a single sample into the group identified by {job="some_job"}:

      echo "some_metric 3.14" | curl --data-binary @- https://<KFUSE-DNS>/metrics/job/some_job

    Since no type information has been provided, some_metric will be of type untyped.

  • Push something more complex into the group identified by {job="some_job",instance="some_instance"}:

      cat <<EOF | curl --data-binary @- https://<KFUSE-DNS>/metrics/job/some_job/instance/some_instance
  # TYPE some_metric counter
  some_metric{label="val1"} 42
  # TYPE another_metric gauge
  # HELP another_metric Just an example.
  another_metric 2398.283
  EOF

    Note how type information and help strings are provided. Those lines are optional, but strongly encouraged for anything more complex.

For more information refer here

Once the metrics are flowing into pushgateway, our kfuse-scraper scrapes it to ingest metrics into the Kloudfuse database and they should appear on the Kfuse Metrics summary.

  • No labels