Kloudfuse 3.1.0
or higher supports ingesting metrics using Prometheus-Pushgateway
The Prometheus Pushgateway exists to allow ephemeral and batch jobs to expose their metrics to Prometheus. They can instead push their metrics to a Pushgateway and it then exposes these metrics to Prometheus.
Enable Pushgateway on Kloudfuse
You will need to update your values.yaml
file used for Kfuse helm chart installation to enable the pushgateway:
global: orgId: "..." # ... pushgateway: enabled: true # ...
After updating the values.yaml, you can install/upgrade your Kfuse release using the following command:
helm upgrade --install kfuse oci://us-east1-docker.pkg.dev/mvp-demo-301906/kfuse-helm/kfuse --version 3.1.0 -f values.yaml
Once the upgrade is done, you will see the pod for kfuse-pushgateway
spin up in your env.
Sending metrics to the Pushgateway
Using the Prometheus text protocol, pushing metrics is easy so no separate CLI is provided. Simply use a command-line HTTP tool like curl
. Your favorite scripting language has most likely some built-in HTTP capabilities you can leverage here as well.
Note that in the text protocol, each line has to end with a line-feed character (aka 'LF' or '\n'). Ending a line in other ways, e.g. with 'CR' aka '\r', 'CRLF' aka '\r\n', or just the end of the packet, will result in a protocol error.
Pushed metrics are managed in groups, identified by a grouping key of any number of labels, of which the first must be the job
label. The groups are easy to inspect via the web interface.
For implications of special characters in label values see the URL section below.
Examples:
Push a single sample into the group identified by
{job="some_job"}
:echo "some_metric 3.14" | curl --data-binary @- http://pushgateway.example.org:9091/metrics/job/some_job
Since no type information has been provided,
some_metric
will be of typeuntyped
.Push something more complex into the group identified by
{job="some_job",instance="some_instance"}
:cat <<EOF | curl --data-binary @- http://pushgateway.example.org:9091/metrics/job/some_job/instance/some_instance # TYPE some_metric counter some_metric{label="val1"} 42 # TYPE another_metric gauge # HELP another_metric Just an example. another_metric 2398.283 EOF
Note how type information and help strings are provided. Those lines are optional, but strongly encouraged for anything more complex.
For more information refer here