Page Comparison

What is Fluent Bit?

Fluent Bit is an open-source and multi-platform log processor tool which aims to be a generic Swiss knife for logs processing and distribution (verbatim from Fluent Bit’s documentation). For more information on Fluent Bit, refer to the documentation here.

Integrating Fluent Bit with Kloudfuse stack

Fluent Bit supports a variety of input and output plugins and parsers. For more information on the supported output plugins, see here. However, the Kloudfuse stack currently supports 2 output plugins:

• DD (DataDog) plugin

• HTTP plugin

You can install Fluent Bit using helm charts. Clone the charts repo from here by running:

git clone https://github.com/fluent/helm-charts.git

You’ll now need to configure the Fluent Bit installation by modifying charts/fluent-bit/values.yaml depending on the plugin you choose to use.

...

Refer to installation instructions for Fluent Bit here. To configure Fluent Bit to send data to Kloudfuse stack, you can modify the agent’s config or values.yaml (if you’re going to install using helm).

You’ll need to configure HTTP output plugin to forward data to Kloudfuse.

HTTP plugin integration

Add the following content to values.yaml, if you’re planning on using DD output plugin with Fluent Bit:configuration to Fluent Bit agent’s config for HTTP configuration.

  outputs: |     [OUTPUT]
        Name datadoghttp
        Match <match_pattern>
        Host <kfuse_ingress_ip>
        dd_message_key logPort 443
        TLS on
        compress gzip
        apiKey random
        dd_tags <additional_tags>URI /ingester/v1/fluent_bit
        dd_source <dd_source_name>
        dd_service <dd_service_name>Format {json|json_lines|json_stream|msgpack}

HTTP plugin integration

...

Configuration

Kloudfuse UI allows you to filter log events based on log labels/tags. You’ll find the label selectors and filter on the left nav bar of the UI. To get a seamless experience with Kloudfuse while using Fluent Bit agent, we recommend the following configuration(s) or customization(s). Note that the fluent bit agent needs to be restarted for these to take effect.

Kubernetes Labels

Fluent Bit has a filter called kubernetes which will enrich the log event with Kubernetes metadata. Refer to the documentation on this filter here. If you have application deployed in a Kubernetes environment, we highly recommend enabling this filter for all those applications. Here’s an example configuration for this filter:

    [FILTER]
        Name kubernetes
        Match *          # Match everything
        Merge_Log On
        Keep_Log Off
        K8S-Logging.Parser On
        K8S-Logging.Exclude On

Cloud Labels

We recommend adding/configuring the aws filter to get the availability-zone and instance-type. Refer to the documentation for this filter here. At a minimum, we recommend the following configuration for aws filter:

    [OUTPUTFILTER]
        Name httpaws
        Match *      # Match <match_pattern>
 everything
        az true
        account_id true
       Host <kfuseec2_ingress_ip>instance_type true

This filter does not include the cluster name, however. We recommend adding modify filter to include the cluster name as a separate tag. Here's a sample config on how to do that:

    [FILTER]
        Name modify
   Port  80   Match *     URI /ingester/v1/fluent_bit# Match everything
        Add Format msgpack

cluster_name <CLUSTER_NAME>

Note that the

...

key, cluster_name in the above

...

Installing Fluent Bit helm chart

After modifying values.yaml file, run the following two steps to install Fluent Bit:

helm repo add fluent https://fluent.github.io/helm-charts
helm install fluent-bit fluent/fluent-bit -f charts/fluent-bit/values.yaml

Wait for the Fluent Bit pods to come up, then verify that the logs are being pushed Kloudfuse stack by logging into the Kloudfuse UIfilter is configurable too. If you use a key name other than cluster_name or clusterName, then add the following section under logs-parser section in your custom Kloudfuse’s values.yaml

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_cloud_cluster_name:
            - "$.<KEY_FOR_CLUSTER_NAME>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Static custom labels

To add static custom labels, use the modify filter. The example below adds a static custom label called tenant_name:

    [FILTER]
        Name modify
        Match *     # Match everything
        Add tenant_name <TENANT_NAME>

By default, any new static labels that you add here show up as log facets. If you want to make them a label/tag on Kloudfuse platform, add the following config under logs-parser section in your custom values.yaml.

 kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_additional_tags:
            - "$.tenant_name" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Log source

By default, Kloudfuse stack looks for container_name in the Fluent Bit payload as the log source. However, this will only be populated if the Fluent Bit agent is configured with kubernetes filter. If you want to Kloudfuse stack to use a different key as the log source, then include the following section under logs-parser section in your custom Kloudfuse’s values.yaml

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_source:
            - "$.<KEY_FOR_LOG_SOURCE>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Log message

Fluent Bit agent includes the log event message in log key in the payload. However this can be overriden in the agent configuration. You can customize key which Kloudfuse stack should look for to get the log event message. To customize this setting, include the following section under logs-parser section in your custom Kloudfuse’s values.yaml

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_msg:
            - "$.<MSG_KEY_FROM_AGENT_CONFIG>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="

Agent extracted key-value pairs

Fluent Bit supports various parsers to extract key value pairs from an unstructured log. For a full list of parsers, refer to the documentation here. By default, Kloudfuse will add all these key-value pairs to log facets, which can be filtered on the UI. Note that Kloudfuse cannot differentiate between these key-value pairs and any metadata fields, added by any filter other than kubernetes and aws. However, you can instruct Kloudfuse to track these key-value pairs as labels instead of log facets. To customize this setting, include the following section under logs-parser section in your custom Kloudfuse’s values.yaml

kf_parsing_config:
  config: |-
    - remap:
        args:
          kf_additional_tags:
            - "$.<PREFIX_KEY_FOR_AGENT_KV>" # must be JSONPath
        conditions:
          - matcher: "__kf_agent"
            value: "fluent-bit"
            op: "=="