Configure log based alerts

 

Logs stream supports threshold alerts.

Creating Alert

Step 1. Define the search query

  • Add any required log filters in the “Log Filters” search box

  • Log count based charting

    • Use the rate or count_over_time aggregate to chart the count of log lines based on the log filter

  • Log Facet based charting

    • Select a log facet to extract and chart. For instance, select a duration facet “took”.

    • Apply a normalization function like 'duration' to interpret the duration string as seconds

    • Choose a aggregation function to aggregate log events in time and generate a time-series

    • Add any grouping facets to “Group by” section. This can reduce the number of time-series

      For more details on Log derived metrics, see Log Derived Metrics section in Logs overview

Step 2: Populate condition and Evaluation

  • Populate the Condition section by defining the:

    • aggregate to be used on the query result from the drop-down.

    • query or expression from the drop-down

    • thresholds that should be breached for the alert to be firing

  • Populate the Evaluation section by defining the:

    • evaluation frequency that determines the frequency at which alert expression/query must be evaluated (must be a multiple of 10 seconds. For example, 1m, 30s, etc.) and

    • specify the duration for which the condition must be true before an alert fires

(Note: Once a condition is breached, the alert goes into the “Pending” state. If the condition remains breached for the duration specified in “For”, the alert transitions to the “Firing” state, otherwise it reverts to the “Normal” state)

Step 3: Populate Name and Title details

  • Choose the folder to which the alert definition should be saved. (If you need to create a separate folder, then create one using the “new folder” option in the drop-down menu).

  • Rule Name: set a descriptive name for the rule.

  • Group Name: Specify a group name. Rules within a group are run sequentially at regular intervals, with the same evaluation time.

  • Populate title and summary with variables to include additional information in the alert.

Step 4: Configure a contact point

  • Choose how notifications are sent to your teams (email, Slack, PagerDuty, etc). Choose an existing contact point from the drop-down menu for notifications when this alert fires, or create a new one. To configure a new contact point, please see details for each type of contact point in this section. Once done, click “Create Rule”