Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

What is Fluent Bit?

Fluent Bit is an open-source and multi-platform log processor tool which aims to be a generic Swiss knife for logs processing and distribution (verbatim from Fluent Bit’s documentation). For more information on Fluent Bit, refer to the documentation here.

Integrating Fluent Bit with Kloudfuse stack

Fluent Bit supports a variety of input and output plugins and parsers. For more information on the supported output plugins, see here. However, the Kloudfuse stack currently supports 2 output plugins:

  • DD (DataDog) plugin

  • HTTP plugin

You can install Fluent Bit using helm charts. Clone the charts repo from here by running:

git clone https://github.com/fluent/helm-charts.git

You’ll now need to configure the Fluent Bit installation by modifying charts/fluent-bit/values.yaml depending on the plugin you choose to use.

DD plugin integration

Add the following content to values.yaml, if you’re planning on using DD output plugin with Fluent Bit:

  outputs: |
    [OUTPUT]
        Name datadog
        Match <match_pattern>
        Host <kfuse_ingress_ip>
        dd_message_key log
        TLS on
        compress gzip
        apiKey random
        dd_tags <additional_tags>
        dd_source <dd_source_name>
        dd_service <dd_service_name>

Note that the Host IP/DNS name in the above configuration can be the internal or external Kloudfuse ingress IP, depending on how you’ve configured the kfuse stack.

All the settings/configurations up to TLS on are required for the kfuse stack to work properly. The rest of the settings are recommended.

HTTP plugin integration

If you’re not planning on using DD output plugin, then you can use the HTTP plugin to push the logs to kfuse stack. Add the following configuration to values.yaml for HTTP configuration.

    [OUTPUT]
        Name http
        Match <match_pattern>
        Host <kfuse_ingress_ip>
        Port 80
        URI /ingester/v1/fluent_bit
        Format msgpack

Note that the Host IP/DNS name in the above configuration can be the internal or external Kloudfuse ingress IP, depending on how you’ve configured the kfuse stack.

Currently msgpack is the only supported format for HTTP plugin. You can also not include this setting. The default format for HTTP plugin is msgpack.

Installing Fluent Bit helm chart

You will only need to edit the values.yaml file to configure either DD or HTTP plugin. You do not have to configure both.

After modifying values.yaml file, run the following two steps to install Fluent Bit:

helm repo add fluent https://fluent.github.io/helm-charts
helm install fluent-bit fluent/fluent-bit -f charts/fluent-bit/values.yaml

Wait for the Fluent Bit pods to come up, then verify that the logs are being pushed Kloudfuse stack by logging into the Kloudfuse UI.

  • No labels