{"type":"doc","content":[{"type":"paragraph","content":[{"text":"You can use the filebeat agent and configure it to ingest files in a s3 bucket and ingest logs in Kloudfuse. Use the ","type":"text"},{"type":"inlineCard","attrs":{"url":"https://kloudfuse.atlassian.net/wiki/x/CYAEL"}},{"text":" documentation to setup the the Filebeat and configure the s3 bucket details as below,","type":"text"}]},{"type":"codeBlock","content":[{"text":"daemonset:\n enabled: false\ndeployment:\n enabled: true\n filebeatConfig:\n filebeat.yml: |\n filebeat.inputs:\n - type: aws-s3\n number_of_workers: 5\n bucket_arn: '<>'\n access_key_id: '<>' # Not needed in case of Node IAM Role\n secret_access_key: '<>' # Not needed in case of Node IAM Role\n endpoint: amazonaws.com\n default_region: \"${AWS_REGION:<>}\"\n fields:\n source: < # The logs will be visible with the same source.\n setup.ilm.enabled: false\n setup.template.enabled: false\n output.elasticsearch:\n hosts: [\"https://<>:443/ingester/api/v1/filebeat\"]","type":"text"}]},{"type":"paragraph"},{"type":"paragraph","content":[{"text":"If you are ingesting from S3 bucket with Node IAM role, you need to have following permissions added to the policy configured","type":"text"}]},{"type":"codeBlock","content":[{"text":"\t\t\t\t\"s3:ListAllMyBuckets\",\n\t\t\t\t\"s3:GetBucketLocation\",\n\t\t\t\t\"s3:ListBucket\",\n\t\t\t\t\"s3:GetObject\"","type":"text"}]},{"type":"paragraph","content":[{"text":"Once configured, the Filebeat will import the logs in the Kloudfuse endpoint configured.","type":"text"}]},{"type":"panel","attrs":{"panelType":"note"},"content":[{"type":"paragraph","content":[{"text":"Please reach out to the Kloudfuse team in case of any queries in the configuration.","type":"text"}]}]}],"version":1}

Browser not supported