@@ -27,7 +27,7 @@ and the conditions that need to be met for the alert to change state and trigger
...
@@ -27,7 +27,7 @@ and the conditions that need to be met for the alert to change state and trigger
## Execution
## Execution
The alert rules are evaluated in the Grafana backend in a scheduler and query execution engine that is part
The alert rules are evaluated in the Grafana backend in a scheduler and query execution engine that is part
of core Grafana. Only some data soures are supported right now. They include `Graphite`, `Prometheus`,
of core Grafana. Only some data sources are supported right now. They include `Graphite`, `Prometheus`,
`InfluxDB` and `OpenTSDB`.
`InfluxDB` and `OpenTSDB`.
### Clustering
### Clustering
...
@@ -59,8 +59,8 @@ specify a query letter, time range and an aggregation function.
...
@@ -59,8 +59,8 @@ specify a query letter, time range and an aggregation function.
avg()OFquery(A,5m,now)ISBELOW14
avg()OFquery(A,5m,now)ISBELOW14
```
```
-`avg()` Controls how the values for **each** serie should be reduced to a value that can be compared against the threshold. Click on the function to change it to another aggregation function.
-`avg()` Controls how the values for **each** series should be reduced to a value that can be compared against the threshold. Click on the function to change it to another aggregation function.
-`query(A, 5m, now)` The letter defines what query to execute from the **Metrics** tab. The second two parameters defines the time range, `5m, now` means 5 minutes from now to now. You can also do `10m, now-2m` to define a time range that will be 10 minutes from now to 2 minutes from now. This is useful if you want to ignore the last 2 minutes of data.
-`query(A, 5m, now)` The letter defines what query to execute from the **Metrics** tab. The second two parameters define the time range, `5m, now` means 5 minutes from now to now. You can also do `10m, now-2m` to define a time range that will be 10 minutes from now to 2 minutes from now. This is useful if you want to ignore the last 2 minutes of data.
-`IS BELOW 14` Defines the type of threshold and the threshold value. You can click on `IS BELOW` to change the type of threshold.
-`IS BELOW 14` Defines the type of threshold and the threshold value. You can click on `IS BELOW` to change the type of threshold.
The query used in an alert rule cannot contain any template variables. Currently we only support `AND` and `OR` operators between conditions and they are executed serially.
The query used in an alert rule cannot contain any template variables. Currently we only support `AND` and `OR` operators between conditions and they are executed serially.
...
@@ -74,7 +74,7 @@ of another alert in your conditions, and `Time Of Day`.
...
@@ -74,7 +74,7 @@ of another alert in your conditions, and `Time Of Day`.
#### Multiple Series
#### Multiple Series
If a query returns multiple series then the aggregation function and threshold check will be evaluated for each series.
If a query returns multiple series then the aggregation function and threshold check will be evaluated for each series.
What Grafana does not do currently is track alert rule state **per series**. This has implications that is exemplified
What Grafana does not do currently is track alert rule state **per series**. This has implications that are detailed
in the scenario below.
in the scenario below.
- Alert condition with query that returns 2 series: **server1** and **server2**
- Alert condition with query that returns 2 series: **server1** and **server2**
...
@@ -89,8 +89,7 @@ we plan to track state **per series** in a future release.
...
@@ -89,8 +89,7 @@ we plan to track state **per series** in a future release.
### No Data / Null values
### No Data / Null values
Below you condition you can configure how the rule evaluation engine should handle queries that return no data or only null valued
Below your conditions you can configure how the rule evaluation engine should handle queries that return no data or only null values.
data.
No Data Option | Description
No Data Option | Description
------------ | -------------
------------ | -------------
...
@@ -100,23 +99,23 @@ Keep Last State | Keep the current alert rule state, what ever it is.
...
@@ -100,23 +99,23 @@ Keep Last State | Keep the current alert rule state, what ever it is.
### Execution errors or timeouts
### Execution errors or timeouts
The last option is how to handle execution or timeout errors.
The last option tells how to handle execution or timeout errors.
Error or timeout option | Description
Error or timeout option | Description
------------ | -------------
------------ | -------------
Alerting | Set alert rule state to `Alerting`
Alerting | Set alert rule state to `Alerting`
Keep Last State | Keep the current alert rule state, what ever it is.
Keep Last State | Keep the current alert rule state, what ever it is.
If you an unreliable time series store that where queries sometime timeout or fail randomly you can set this option
If you have an unreliable time series store from which queries sometime timeout or fail randomly you can set this option
t`Keep Last State` to basically ignore them.
to `Keep Last State` in order to basically ignore them.
## Notifications
## Notifications
In alert tab you can also specify alert rule notifications along with a detailed messsage about the alert rule.
In alert tab you can also specify alert rule notifications along with a detailed messsage about the alert rule.
The message can contain anything, information about how you might solve the issue, link to runbook etc.
The message can contain anything, information about how you might solve the issue, link to runbook, etc.
The actual notifications are configured and shared between multiple alerts. Read the
The actual notifications are configured and shared between multiple alerts. Read the
[Notifications]({{<relref"notifications.md">}}) guide for how to configure and setup notifications.
[notifications]({{<relref"notifications.md">}}) guide for how to configure and setup notifications.
## Alert State History & Annotations
## Alert State History & Annotations
...
@@ -129,7 +128,7 @@ submenu in the alert tab to view & clear state history.
...
@@ -129,7 +128,7 @@ submenu in the alert tab to view & clear state history.