Fluent Bit Collection of AWS ECS/ECS Fargate Container Logs
- Ayan Shah
- Roza Leyderman
- Swapnil Kulkarni
An Amazon Elastic Container Service (Amazon ECS) task definition allows you to specify only a single log configuration object for a given container. This limit means that you can forward logs to only a single destination. To forward logs to multiple destinations in Amazon ECS on Fargate, you can use FireLens.
Create a Custom Fluent Bit docker image
Create a custom Fluent Bit configuration file called logDestinations.conf with your choice of [OUTPUT] definitions. For example, the following configuration file includes configuration defined for KloudFuse (see Output for more examples):
[OUTPUT]
Name http
Match *
Host <Kloudfuse DNS>
Port 443
TLS on
TLS.debug 4
URI /ingester/v1/fluent_bit
Format {json|json_lines|json_stream|msgpack}Use the following Dockerfile example to create a Docker image with a custom Fluent Bit output configuration file:
FROM amazon/aws-for-fluent-bit:latest ADD logDestinations.conf /logDestinations.confTo use the Dockerfile that you created to create the custom fluent-bit Docker image, run the following command:
docker build -t custom-fluent-bit:latest .Push your image to the ECR Image repo where the ECS Containers will be pulling the images from
Deploy Fluent Bit Container in ECS
In your task definition, update the options for your FireLens configuration:
For more details ECS FluentBit task definition with an example see AWS Doc here