...
The detail view can be accessed by clicking on the log event detail icon that shows up when you hover on a log line, or by click+enter from the log list view. From the log event detail view, the user can filter using the fingerprint, log facets, and labels associated with the fingerprint.
...
...
As a result of Fingerprinting incoming log lines, we auto-detect tokens like numbers, IP address, duration, size, and UUID. These tokens are automatically assigned log facet names as _number_0, _ip_address_0, _duration_0
respectively. These auto-assigned log facet names can be renamed to something more readable by the user from the log event detail view. The renaming is scoped to a fingerprint.
...
Fingerprints View
Fingerprints view provides a concise view of the unique logs ingested by the stack. This bird’s eye view is very helpful when looking at the huge number of logs that are typically emitted by production systems. For example, by filtering the logs by severity level of ERROR, the user can easily highlight different kinds of error events that are being observed by the system without hunting for them one by one. The fingerprints can be searched just like log events. Moreover, the logs can be filtered to only include/exclude logs with the selected fingerprint. This serves as a smarter grep where you can quickly filter logs without finding unique strings to grep or grep -v by.
...
Log Analytics
Logs contain a lot of valuable information. Quite often developers instrument the application to log the various application metrics within the log line. Though convenient at development time this makes it harder to analyze the system in production as now the logs must be indexed properly to extract such metrics. Kloudfuse platform makes the metric generation from the log line fairly easy though due to its unique fingerprinting technology. Kloudfuse stack can auto-extract metric facets and highlight them under each source category. The “Log Analytics” view allows the users to select the numeric facets to be charted along with what aggregates to apply. The log lines containing the metric of interest can be filtered as its done in log search (or can be skipped as well). Range Aggregate allows time aggregation to be applied so that events from multiple log lines can be aggregated across everything or some common dimensions that can be selected using the “Grouping options”. These dimensions can include facets extracted from the log line or environment tags like pod_name, service name, etc. The chart is generated dynamically from the log lines and can be used for ad-hoc analysis during troubleshooting. For saving this metric refer to the next section.
...
Log facet selector
selector for log facet or count based metric to chart
Facet normalization function
function used to normalize the log facet to a numerical value
number - parse the log facet as a double value
count - normalize to 1 if the selected facet exists
duration - normalize a duration string to seconds. Valid time units are
ns, us (or "µs"), ms, s, m, h
. example: 1h30mbytes - normalize a size string to bytes. Valid size units are
KB, MB, GB, TB, PB, KiB, MiB, GiB, TiB, PiB
. example: 10MB
Range (time) aggregate: aggregate discrete points in time in time to produce one value per time-series and time-step. The aggregates are applied to log events that satisfy the log filters
Count based log metrics
rate : rate of log events at every time-step. i,e. count/time-step_seconds
count_over_time : count of log events at every time-step
Log Facet based log metrics
rate_counter : rate of monotonically increasing counter
sum_over_time
avg_over_time
max_over_time
min_over_time
first_over_time
last_over_time
stdvar_over_time
stddev_over_time
quantile_over_time
Range aggregate grouping
labels that define the time-series. log events are grouped by the labels and each group becomes a time-series
default grouping behavior is to group everything into one time series (except for rate and rate_counter which do not support grouping)
Vector (space) aggregate: Reduce the number of time series by aggregating across time-series at a given time step
sum
avg
min
max
stddev
stdvar
count
topk
bottomk
Vector aggregate grouping
labels that define the final time-series to collapse into. Must be a subset of the range aggregate grouping
default grouping behavior is to group everything into one time series
Generate chart button to chart the log-derived metric
Visualization type
Save metric icon
Log Analytics exploration workflow
Add any log filters as described in the Log Search View to filter down logs for charting
Count based log metrics
Choose
count_log_events
from the log facet selectorChoose
number
as the normalization functionChoose
rate
orcount_over_time
as the Range/time aggregation functionClick on
Generate chart
to chart the count based metric
...
Log facet metrics
Choose the log facet to chart from the log facet selector
Choose
number/bytes/duration
as the normalization function to normalize the facet value. Choosecount
to count the number of times the log facet appears in the time-stepChoose one of the Log facet based range aggregation function
Click on
Generate chart
...
Save Metrics
The metrics that are explored can be saved as well to keep them for longer retention or further analysis. To save the explored metric, “Save Metrics” button can be used. The user can enter a unique name for the metric along with the dimensions that need to be saved for the metric series. By default the UI selects the dimensions that were used for metric exploration. The saved metric is pushed to the in-built metric storage.
The metrics that are saved are listed in the “Metrics” view at the bottom of the page. Any saved metric that is no longer required can be deleted from this list. The user can explore the saved metric using standard kfuse metric exploration (by clicking on the icon) or through Grafana metric explorer. Support for exporting the metrics to an external metric system will be coming in the future.
...
Fingerprints View
Fingerprints view provides a concise view of the unique logs ingested by the stack. This bird’s eye view is very helpful when looking at the huge number of logs that are typically emitted by production systems. For example, by filtering the logs by severity level of ERROR, the user can easily highlight different kinds of error events that are being observed by the system without hunting for them one by one. The fingerprints can be searched just like log events. Moreover, the logs can be filtered to only include/exclude logs with the selected fingerprint. This serves as a smarter grep where you can quickly filter logs without finding unique strings to grep or grep -v by.
...
Log Analytics
Logs contain a lot of valuable information. Quite often developers instrument the application to log the various application metrics within the log line. Though convenient at development time this makes it harder to analyze the system in production as now the logs must be indexed properly to extract such metrics. Kloudfuse platform makes the metric generation from the log line fairly easy though due to its unique fingerprinting technology. Kloudfuse stack can auto-extract metric facets and highlight them under each source category. The “Log Analytics” view allows the users to select the numeric facets to be charted along with what aggregates to apply. The log lines containing the metric of interest can be filtered as its done in log search (or can be skipped as well). Range Aggregate allows time aggregation to be applied so that events from multiple log lines can be aggregated across everything or some common dimensions that can be selected using the “Grouping options”. These dimensions can include facets extracted from the log line or environment tags like pod_name, service name, etc. The chart is generated dynamically from the log lines and can be used for ad-hoc analysis during troubleshooting. For saving this metric refer to the next section.
...
Log facet selector
selector for log facet or count based metric to chart
Facet normalization function
function used to normalize the log facet to a numerical value
number - parse the log facet as a double value
count - normalize to 1 if the selected facet exists
duration - normalize a duration string to seconds. Valid time units are
ns, us (or "µs"), ms, s, m, h
. example: 1h30mbytes - normalize a size string to bytes. Valid size units are
KB, MB, GB, TB, PB, KiB, MiB, GiB, TiB, PiB
. example: 10MB
Range (time) aggregate: aggregate discrete points in time in time to produce one value per time-series and time-step. The aggregates are applied to log events that satisfy the log filters
Count based log metrics
rate : rate of log events at every time-step. i,e. count/time-step_seconds
count_over_time : count of log events at every time-step
Log Facet based log metrics
rate_counter : rate of monotonically increasing counter
sum_over_time
avg_over_time
max_over_time
min_over_time
first_over_time
last_over_time
stdvar_over_time
stddev_over_time
quantile_over_time
Range aggregate grouping
labels that define the time-series. log events are grouped by the labels and each group becomes a time-series
default grouping behavior is to group everything into one time series (except for rate and rate_counter which do not support grouping)
Vector (space) aggregate: Reduce the number of time series by aggregating across time-series at a given time step
sum
avg
min
max
stddev
stdvar
count
topk
bottomk
Vector aggregate grouping
labels that define the final time-series to collapse into. Must be a subset of the range aggregate grouping
default grouping behavior is to group everything into one time series
Generate chart button to chart the log-derived metric
Visualization type
Save metric icon
Log Analytics exploration workflow
Add any log filters as described in the Log Search View to filter down logs for charting
Count based log metrics
Choose
count_log_events
from the log facet selectorChoose
number
as the normalization functionChoose
rate
orcount_over_time
as the Range/time aggregation functionClick on
Generate chart
to chart the count based metric
...
Log facet metrics
Choose the log facet to chart from the log facet selector
Choose
number/bytes/duration
as the normalization function to normalize the facet value. Choosecount
to count the number of times the log facet appears in the time-stepChoose one of the Log facet based range aggregation function
Click on
Generate chart
...
Log Source Integration
Kloudfuse stack can ingest from a variety of agents and cloud services. The following lists the various sources and how to configure them.
...