• We now display services and databases separately in their own tabs

• Services are now uniquely identified using combination of cloud and kubernetes labels. Service identity is carried over to the service and dependency maps as well as service details page. In addition, service identity is leveraged in setting APM and ASM alerts as well as when navigating to traces and errors from service details.

• Runtime metrics (JVM, Go, Python, etc.) for services based on detected telemetry language

• Significant enhancements to the Service Map (global view) and Service Dependency Map (in the Services details page) user experience leveraging service identity to uniquely identify services and databases.

• Ability to navigate from service to service details page

• Ability to size the node based on any of the RED metrics

• Various fixes and UX improvements

• Look and feel improvements to match rest of the features and screens

• Independent scrolling of sources and facets

• Easier selection of facets, and labels using toggle/All/only options

• Easier charting of facets based on data type directly from the Logs sidebar

• Uniform and easier editing of search filters across Logs and APM search bars

• Better color, fonts and sizes

• Uniform look and feel across various screens

• APM, Logs, and Metrics dashboards can be created, deleted and edited

• Dashboard import/export/copy functionality

• Dashboard template variables support

• Metrics metadata, including metric type, description and units

• Advanced Services Monitoring can be enabled for individual services from the APM services list

• Service details RED metrics charts have option to show anomalies when ASM is enabled

• Service details page also shows Kubernetes Infrastructure metrics on per host, and pod basis with outlier detection

• Auto-detection of deployment changes based on service version

• Service details page shows first seen time for each of the versions

• Service RED metrics and Execution Breakdown charts show markers for deployment

• Breakdown of execution time by down stream service and span type

• Overlay of deployment markers to co-relate deployment changes with service changes

• SLA daily/weekly/monthly report of RED metrics and APDEX

• Performance Report breaking down RED metrics and APDEX by span names for 24 hour period against 7 day average

• facet names can be searched and selected using typeahead (prefix matching) in logs search and logs analytics

• Improved APM search now allows editing of indivdiual eq, neq, regex eq and regex neq queries for low cardinality and hc attributes

• Simplified and improved editing of duration slider ranges

• Alert workflow has been simplified to make it easy to configure various kind of alerts including anomaly, change and threshold.

• Improved alert artifacts for System, Kubernetes and Logs integrations.

• Logs Analytics now supports Top List, Table and Pie Chart visualization in addition to the Time Series

• Facets can be aggregated or used in a group by independent of the source in which they appear.

• Ability to create alerts directly from the logs analytics screen for both queries and formulas.

• New aggregation functions - first, last, quantile, etc.

• Facet match/search now works across all the sources

• Logs term search and string searches are now faster due to various improvements in the indexing and caching in addition the streaming evaluation of the counts

• From the search bar you can now search for log lines containing a facet

• Ability to sort the logs search results table by custom columns

• Ability to chart numeric facets from the sidebar

• Sidebar now shows facet values sorted by their log lines count

• Improvements to charting interface with ability to jump to corresponding metrics exploration with support for more visualization types and comparison to previous periods.

• Added support for Top List, Table and Pie Chart visualization in addition to the Time Series

• Support for multiple queries and formulas

• Simplification of analytics UI to match logs analytics

• Ability to add analytics query to dashboard

• Support for analyzing and breaking down cardinality of various attributes (both indexed and non-indexed)

• Support for analyzing and breaking down cardinality of labels

• Various performance improvements.

• Support for threshold and anomaly based alerts for APM RED metrics.

• Support for Role Based Access Control.

Logs fingerprints can be sorted by ascending or descending log counts.

Logs sources and facets can now be searched in the sidebar.

APM dashboards show the breakdown of RED metrics by services.

• Services view sidebar loads faster.

• Service Detail view breakdown charts by Span Name supports selection through legend

• Link from a specific error group details to “Errors” page

Spinner is now displayed in the APM page for initial load also. Word wrap and UI distortion issues are fixed on many charts.

Term search is now the default search type for logs. Users can quote the search string to use the older string contains search. Term search is faster and more efficient in general.

Logs analytics screen is simplified and now much easier to use. You have option to add the queries to a dashboard.

You can now group fingerprints by multiple attributes. Earlier only grouping available was by source.

The external dependencies for APM services are shows in the service details page.

When you select a specific span from the trace details, you can now see the related logs based on various attributes such as, traceId, pod, etc. You can see the metrics related to the service or endpoint.

Queries for rate (for counter type) and histogram quantile (for histograms) are much faster now.

Spinner is now displayed in the APM pages while columns values are not yet available.

Infrastructure tab is now revamped with cleaner and slicker UI. In the related logs, you can now open logs details for any specific log line.

Query that run over the timeout or cause out of memory issues are terminated automatically. In addition, pods are restarted after hitting Out of Memory issue.

User can now perform analytics on Elastic APM errors globally as well as see error types, frequency and last occurrences for a given service.

Each input JSON log line is sorted internally by key names before indexing. This improves the storage efficiency and search speed by reducing number of unique patterns detected in logs stream.

Slice and dicing on APM errors as well as errors chart/table in the service details page.

Fixed various discrepancies in calculating RED metrics from the incoming spans and transactions for Elastic APM data.

Various performance improvements for auto-completion of metric names and labels as well as metric summarization.

ASM provides autonomic observability based on eBPF technology. kfuse-knight agent discovers and tracks all services and their interactions. ASM provides RED and USE metrics without any extra instrumentation or change to the application code. ASM also provides curated advanced alerts to detect anomalous and outliers behaviour in the services.

Spans can be queried now using TraceQL. In addition, you can view service map and flame-graph through Grafana.

Ability to skip auto extraction from JSON logs through logs parser configuration.

Charting was broken for grammar derived facets.

JSON message parsing has been optimized to reduce the CPU cost for logs parsing. This optimization is applicable to all log lines including structured JSON logs, embedded and partial JSON strings.

Span details now show stack-trace, local variables and context for Elastic APM. In addition to flame-graph spans are listed in a separate span list tab.

From the span details, user can now filter by (include/exclude) custom span attributes in addition to the standard OTel attributes.

Various fixes for Control Plane dashboards and alerts.

Filtering and navigation has been made uniform and more streamlined across all the streams including logs, traces, events and metrics.

Kloudfuse now has support for Service Level Objective. Users can set latency and availability SLOs for any service instrumented with distributed tracing.

Kloudfuse now has support for Single Sign On and support for various authorization methods including Google, Okta, Azure amongst others.

Kloudfuse alerting now supports Change, Outliers, Anomaly and Forecast alert types in addition to the threshold. Read more.

Kloudfuse catalog service enables support for migrating dashboards and alerts from external grafana to kfuse. Read more.

Logs parsing extracts, detects and color codes the data types of facets automatically to make it easier to work with large amount of logs data.

Logs parser pipeline stages can be configured through remap, relabel and transform actions/stages. This allows the users to configure and process logs data from any agent including fluent-bit, fluent-d, OTEL collector, DD-agent amongst others.

JSON message parsing has been optimized to reduce the CPU cost for logs parsing.. This optimizations apply not only to structured JSON logs but also to log lines containing embedded or partial JSON strings.

Apart from OTEL collector/format, Kloudfuse stack now supports Elastic APM and DD APM payload formats. The pipeline can be configured to drop/relabel various attributes as required.

Kloudfuse stack produces unified span derived metrics that can be configured to have arbitrary dimensions. If needed more span derived metrics can be produced by applying any filters and time/space aggregate to incoming data. The metric data can be retained independently of trace retention.

Users can download the full span data in two different formats; CSV format downloads only the columns shown in the UI whereas JSON format downloads all the attributes of incoming span that are stored by the stack.

Kloudfuse alerting now supports Change, Outliers, Anomaly and Forecast alert types in addition to the threshold. Read more.

The Infrastructure view to explore every Kubernetes resources now contains related logs, traces, metrics and events.

Control plane now contains information on forecasted disk and outliers resource usage capabilities for the Kloudfuse observability stack resources.

Log data is sorted by multiple keys (fingerprints, labels, timestamp); this results in more efficient disk storage and as a result better query perf.

Log queries can be saved as views to be referenced later or as adhoc dashboards to be used by various team members.

Logs events can be downloaded from kfuse UI in three different formats.

1. TXT: Raw log message as emitted by the application.

2. CSV: Comma separated log msg along with all the fields that are shown in the kfuse logs UI screen.

3. JSON: Full detailed log event with all the facets and labels associated with the log event.

The logs download is limited to 10MB.

This will be brought back in a later release with some enhancements.

On E2 machines metric segments used to take 4-10 minutes due to number of docs in each segment (~50M); We moved to columnar seal instead of row-by-row seal and seal times have come down by 50% or more.

kfuse stack can now ingest logs from Fluent-D directly. We support JSON and msg-pack format.

We now sort the log-lines on the disk based on their fingerprint. This results in better storage compression and improved grep and facet search performance.

We now support flame-graph view for incomplete traces. In certain customer environments we may not get a root-span or parts of traces may always be missing due to environment setup. Improve flame-graph visualization to render such traces properly.

Moved to dictionary based encoding for span durations and bigger segments to improve query speeds.

With the right instrumentation in place, Kloudfuse analytics can now do auto alerting and analysis. Auto alerting can be enabled easily for automatic monitoring of all Kubernetes services for anomalies on their RED metrics using Hawkeye. Auto analysis capability using Bullseye generates an analysis report with possible reasons for the alert (anomaly).