Kloudfuse supports OpenTelemetry standard for Application Performance Management (APM) and distributed tracing. However, Kloudfuse allows you to keep your existing instrumentation and agents/collectors for distributed tracing. The following table lists the various supported instrumentation/SDKs and agents/collectors.
Instrumentation/SDK | Agent/Collector | Configuration |
---|---|---|
Datadog | Datadog Agent | Enable apm and point |
OpenTelemetry | OpenTelemetry collector | otlphttp exporter |
Elastic APM Agent (all languages) | No additional collector required | Point ELASTIC_APM_SERVER_URL environment variable to kloudfuse ingester endpoint |
OpenTelemetry (Otel) Collector:
Helm Installation:
Get the IP for the Kloudfuse endpoint with the following kubectl command (note: if kfuse stack is installed with a DNS or VPC Endpoint then use that address instead of IP address):
kubectl get svc -n kfuse | grep kfuse-ingress-nginx-controller-internal kfuse-ingress-nginx-controller-internal LoadBalancer 10.53.250.80 10.53.232.3 80:32716/TCP,443:30767/TCP 125m
Create a file called
otel-values.yaml
containing following (note the use of the IP address from above and therefore the tls.insecure set to true. By default this is always true) and replace the endpoint accordingly. Ensure thathttps
orhttp
is used depending if Kfuse has tls enabled or disabled.
image: repository: "otel/opentelemetry-collector-contrib" command: name: "otelcol-contrib" config: exporters: logging: verbosity: basic otlphttp: tls: insecure: true # add only if you're using insecure communication metrics_endpoint: https://<REPLACE KFUSE ADDRESS>/ingester/otlp/metrics traces_endpoint: https://<REPLACE KFUSE ADDRESS>/ingester/otlp/traces extensions: health_check: {} memory_ballast: size_in_percentage: 40 processors: batch: timeout: 10s memory_limiter: check_interval: 5s limit_percentage: 80 spike_limit_percentage: 25 resourcedetection: detectors: - env - eks - ec2 - gcp - aks - azure override: false timeout: 2s receivers: otlp: protocols: grpc: endpoint: ${env:MY_POD_IP}:4317 http: cors: allowed_origins: - http://* - https://* endpoint: 0.0.0.0:4318 service: extensions: - health_check - memory_ballast pipelines: metrics: exporters: - logging - otlphttp processors: - memory_limiter - batch - resourcedetection receivers: - otlp traces: exporters: - otlphttp processors: - batch - resourcedetection receivers: - otlp telemetry: metrics: address: ${MY_POD_IP}:8888 mode: daemonset nameOverride: otelcol ports: metrics: enabled: true otlp: enabled: true otlp-http: enabled: true presets: kubernetesAttributes: enabled: true
Please add any tolerations
& affinity
as required with your deployment.
Install collector with following steps using the created
otel-values.yaml
file
# Add otel repo and update helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts helm repo update # Install otel collector: helm install my-opentelemetry-collector open-telemetry/opentelemetry-collector -f otel-values.yaml
Note: The above instructions install the collector in daemonset mode. Please review the otel-collector documentation for additional modes and more configuration options.
Datadog Agent:
Configure the datadog agent with apm_dd_url set to the Kloudfuse ingester. You can find the Kloudfuse Platform ingress internal IP with the following kubectl command (note: if kfuse stack is installed with a DNS or VPC Endpoint then use that address instead of IP address):
kubectl get svc -n kfuse | grep kfuse-ingress-nginx-controller-internal kfuse-ingress-nginx-controller-internal LoadBalancer 10.53.250.80 10.53.232.3 80:32716/TCP,443:30767/TCP 125m
In the Datadog agent yaml, add the apm_config section with apm_dd_url pointing to the Kloudfuse ingester.
datadog: apm: enabled: true portEnabled: true socketEnabled: true agents: customAgentConfig: apm_config: enabled: true apm_dd_url: "http://10.53.250.80/ingester"