Grafana ships with advanced support for Elasticsearch. You can do many types of
Grafana ships with advanced support for Elasticsearch. You can do many types of simple or complex Elasticsearch queries to
simple or complex elasticsearch queries to visualize logs or metrics stored in elasticsearch. You can
visualize logs or metrics stored in Elasticsearch. You can also annotate your graphs with log events stored in Elasticsearch.
also annotate your graphs with log events stored in elasticsearch.
## Adding the data source
## Adding the data source
1. Open the side menu by clicking the Grafana icon in the top header.
1. Open the side menu by clicking the the Grafana icon in the top header.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
3. Click the `+ Add data source` button in the top header.
4. Select *Elasticsearch* from the *Type* dropdown.
> NOTE: If this link is missing in the side menu it means that your current user does not have the `Admin` role for the current organization.
> NOTE: If you're not seeing the `Data Sources` link in your side menu it means that your current user does not have the `Admin` role for the current organization.
3. Click the `Add new` link in the top header.
4. Select `Elasticsearch` from the dropdown.
Name | Description
Name | Description
------------ | -------------
------------ | -------------
Name | The data source name, important that this is the same as in Grafana v1.x if you plan to import old dashboards.
*Name* | The data source name. This is how you refer to the data source in panels & queries.
Default | Default data source means that it will be pre-selected for new panels.
*Default* | Default data source means that it will be pre-selected for new panels.
Url | The http protocol, ip and port of you elasticsearch server.
*Url* | The HTTP protocol, IP, and port of your Elasticsearch server.
Access | Proxy = access via Grafana backend, Direct = access directly from browser.
*Access* | Proxy = access via Grafana backend, Direct = access directly from browser.
Proxy access means that the Grafana backend will proxy all requests from the browser, and send them on to the Data Source. This is useful because it can eliminate CORS (Cross Origin Site Resource) issues, as well as eliminate the need to disseminate authentication details to the Data Source to the browser.
Proxy access means that the Grafana backend will proxy all requests from the browser, and send them on to the Data Source. This is useful because it can eliminate CORS (Cross Origin Site Resource) issues, as well as eliminate the need to disseminate authentication to the browser.
Direct access is still supported because in some cases it may be useful to access a Data Source directly depending on the use case and topology of Grafana, the user, and the Data Source.
### Direct access
### Direct access
If you select direct access you must update your Elasticsearch configuration to allow other domains to access
If you select direct access you must update your Elasticsearch configuration to allow other domains to access
Elasticsearch from the browser. You do this by specifying these to options in your **elasticsearch.yml** config file.
Elasticsearch from the browser. You do this by specifying these to options in your **elasticsearch.yml** config file.
...
@@ -50,46 +45,83 @@ Elasticsearch from the browser. You do this by specifying these to options in yo
...
@@ -50,46 +45,83 @@ Elasticsearch from the browser. You do this by specifying these to options in yo
Here you can specify a default for the `time field` and specify the name of your elasticsearch index. You can use
Here you can specify a default for the `time field` and specify the name of your Elasticsearch index. You can use
a time pattern for the index name or a wildcard.
a time pattern for the index name or a wildcard.
### Elasticsearch version
Be sure to specify your Elasticsearch version in the version selection dropdown. This is very important as there are differences how queries are composed. Currently only 2.x and 5.x
are supported.
## Metric Query editor
## Metric Query editor
The Elasticsearch query editor allows you to select multiple metrics and group by multiple terms or filters. Use the plus and minus icons to the right to add / remove
The Elasticsearch query editor allows you to select multiple metrics and group by multiple terms or filters. Use the plus and minus icons to the right to add/remove
metrics or group bys. Some metrics and group by have options, click the option text to expand the the row to view and edit metric or group by options.
metrics or group by clauses. Some metrics and group by clauses haves options, click the option text to expand the row to view and edit metric or group by options.
## Series naming & alias patterns
You can control the name for time series via the `Alias` input field.
Pattern | Description
------------ | -------------
*{{term fieldname}}* | replaced with value of a term group by
*{{metric}}* | replaced with metric name (ex. Average, Min, Max)
*{{field}}* | replaced with the metric field name
## Pipeline metrics
## Pipeline metrics
If you have Elasticsearch 2.x and Grafana 2.6 or above then you can use pipeline metric aggregations like
Some metric aggregations are called Pipeline aggregations, for example, *Moving Average* and *Derivative*. Elasticsearch pipeline metrics require another metric to be based on. Use the eye icon next to the metric to hide metrics from appearing in the graph. This is useful for metrics you only have in the query for use in a pipeline metric.
**Moving Average** and **Derivative**. Elasticsearch pipeline metrics require another metric to be based on. Use the eye icon next to the metric
to hide metrics from appearing in the graph. This is useful for metrics you only have in the query to be used
The Elasticsearch datasource supports two types of queries you can use to fill template variables with values.
Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place.
Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns makes it easy to change the data
being displayed in your dashboard.
### Possible values for a field
Checkout the [Templating]({{<relref"reference/templating.md">}}) documentation for an introduction to the templating feature and the different
types of template variables.
```json
### Query variable
{"find":"terms","field":"@hostname"}
```
### Fields filtered by type
The Elasticsearch data source supports two types of queries you can use in the *Query* field of *Query* variables. The query is written using a custom JSON string.
```json
{"find":"fields","type":"string"}
Query | Description
```
------------ | -------------
*{"find": "fields", "type": "keyword"} | Returns a list of field names with the index type `keyword`.
*{"find": "terms", "field": "@hostname"}* | Returns a list of values for a field using term aggregation. Query will user current dashboard time range as time range for query.
*{"find": "terms", "field": "@hostname", "query": '<lucene query>'}* | Returns a list of values for a field using term aggregation & and a specified lucene query filter. Query will use current dashboard time range as time range for query.
### Fields filtered by type, with filter
You can use other variables inside the query. Example query definition for a variable named `$host`.
In the above example, we use another variable named `$source` inside the query definition. Whenever you change, via the dropdown, the current value of the ` $source` variable, it will trigger an update of the `$host` variable so it now only contains hostnames filtered by in this case the
`@source` document property.
### Using variables in queries
There are two syntaxes:
-`$<varname>` Example: @hostname:$hostname
-`[[varname]]` Example: @hostname:[[hostname]]
Why two ways? The first syntax is easier to read and write but does not allow you to use a variable in the middle of a word. When the *Multi-value* or *Include all value*
options are enabled, Grafana converts the labels from plain text to a lucene compatible condition.
In the above example, we have a lucene query that filters documents based on the `@hostname` property using a variable named `$hostname`. It is also using
a variable in the *Terms* group by field input box. This allows you to use a variable to quickly change how the data is grouped.
@@ -18,28 +18,22 @@ change function parameters and much more. The editor can handle all types of gra
...
@@ -18,28 +18,22 @@ change function parameters and much more. The editor can handle all types of gra
queries through the use of query references.
queries through the use of query references.
## Adding the data source
## Adding the data source
1. Open the side menu by clicking the the Grafana icon in the top header.
1. Open the side menu by clicking the Grafana icon in the top header.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
3. Click the `+ Add data source` button in the top header.
4. Select `Graphite` from the *Type* dropdown.
> NOTE: If this link is missing in the side menu it means that your current user does not have the `Admin` role for the current organization.
> NOTE: If you're not seeing the `Data Sources` link in your side menu it means that your current user does not have the `Admin` role for the current organization.
3. Click the `Add new` link in the top header.
4. Select `Graphite` from the dropdown.
Name | Description
Name | Description
------------ | -------------
------------ | -------------
Name | The data source name, important that this is the same as in Grafana v1.x if you plan to import old dashboards.
*Name* | The data source name. This is how you refer to the data source in panels & queries.
Default | Default data source means that it will be pre-selected for new panels.
*Default* | Default data source means that it will be pre-selected for new panels.
Url | The http protocol, ip and port of your graphite-web or graphite-api install.
*Url* | The HTTP protocol, IP, and port of your graphite-web or graphite-api install.
Access | Proxy = access via Grafana backend, Direct = access directly from browser.
*Access* | Proxy = access via Grafana backend, Direct = access directly from browser.
Proxy access means that the Grafana backend will proxy all requests from the browser, and send them on to the Data Source. This is useful because it can eliminate CORS (Cross Origin Site Resource) issues, as well as eliminate the need to disseminate authentication details to the Data Source to the browser.
Direct access is still supported because in some cases it may be useful to access a Data Source directly depending on the use case and topology of Grafana, the user, and the Data Source.
Proxy access means that the Grafana backend will proxy all requests from the browser, and send them on to the Data Source. This is useful because it can eliminate CORS (Cross Origin Site Resource) issues, as well as eliminate the need to disseminate authentication details to the browser.
## Metric editor
## Metric editor
...
@@ -50,6 +44,7 @@ or keyboard arrow keys. You can select a wildcard and still continue.
...
@@ -50,6 +44,7 @@ or keyboard arrow keys. You can select a wildcard and still continue.
### Functions
### Functions
Click the plus icon to the right to add a function. You can search for the function or select it from the menu. Once
Click the plus icon to the right to add a function. You can search for the function or select it from the menu. Once
a function is selected it will be added and your focus will be in the text box of the first parameter. To later change
a function is selected it will be added and your focus will be in the text box of the first parameter. To later change
a parameter just click on it and it will turn into a text box. To delete a function click the function name followed
a parameter just click on it and it will turn into a text box. To delete a function click the function name followed
...
@@ -57,42 +52,61 @@ by the x icon.
...
@@ -57,42 +52,61 @@ by the x icon.
### Optional parameters
### Optional parameters
Some functions like aliasByNode support an optional second argument. To add this parameter specify for example 3,-2 as the first parameter and the function editor will adapt and move the -2 to a second parameter. To remove the second optional parameter just click on it and leave it blank and the editor will remove it.
Some functions like aliasByNode support an optional second argument. To add this parameter specify for example 3,-2 as the first parameter and the function editor will adapt and move the -2 to a second parameter. To remove the second optional parameter just click on it and leave it blank and the editor will remove it.
You can reference queries by the row “letter” that they’re on (similar to Microsoft Excel). If you add a second query to graph, you can reference the first query simply by typing in #A. This provides an easy and convenient way to build compounded queries.
You can reference queries by the row “letter” that they’re on (similar to Microsoft Excel). If you add a second query to a graph, you can reference the first query simply by typing in #A. This provides an easy and convenient way to build compounded queries.
## Point consolidation
## Point consolidation
All Graphite metrics are consolidated so that Graphite doesn't return more data points than there are pixels in the graph. By default
All Graphite metrics are consolidated so that Graphite doesn't return more data points than there are pixels in the graph. By default,
this consolidation is done using `avg` function. You can how Graphite consolidates metrics by adding the Graphite consolidateBy function.
this consolidation is done using `avg` function. You can how Graphite consolidates metrics by adding the Graphite consolidateBy function.
> *Notice* This means that legend summary values (max, min, total) cannot be all correct at the same time. They are calculated
> *Notice* This means that legend summary values (max, min, total) cannot be all correct at the same time. They are calculated
> client side by Grafana. And depending on your consolidation function only one or two can be correct at the same time.
> client side by Grafana. And depending on your consolidation function only one or two can be correct at the same time.
## Templating
## Templating
You can create a template variable in Grafana and have that variable filled with values from any Graphite metric exploration query.
You can then use this variable in your Graphite queries, either as part of a metric path or as arguments to functions.
For example a query like `prod.servers.*` will fill the variable with all possible
Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place.
values that exists in the wildcard position.
Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns makes it easy to change the data
being displayed in your dashboard.
Checkout the [Templating]({{<relref"reference/templating.md">}}) documentation for an introduction to the templating feature and the different
types of template variables.
### Query variable
The query you specify in the query field should be a metric find type of query. For example, a query like `prod.servers.*` will fill the
variable with all possible values that exist in the wildcard position.
You can also create nested variables that use other variables in their definition. For example
You can also create nested variables that use other variables in their definition. For example
`apps.$app.servers.*` uses the variable `$app` in its query definition.
`apps.$app.servers.*` uses the variable `$app` in its query definition.
### Variable usage
You can use a variable in a metric node path or as a parameter to a function.
[Annotations]({{<relref"reference/annotations.md">}}) allows you to overlay rich event information on top of graphs. You add annotation
[Annotations]({{<relref"reference/annotations.md">}}) allows you to overlay rich event information on top of graphs. You add annotation
queries via the Dashboard menu / Annotations view.
queries via the Dashboard menu / Annotations view.
Graphite supports two ways to query annotations. A regular metric query, for this you use the `Graphite query` textbox. A Graphite events query, use the `Graphite event tags` textbox,
Graphite supports two ways to query annotations. A regular metric query, for this you use the `Graphite query` textbox. A Graphite events query, use the `Graphite event tags` textbox,
specify an tag or wildcard (leave empty should also work)
specify a tag or wildcard (leave empty should also work)
Grafana ships with very feature rich data source plugin for InfluxDB. Supporting a feature rich query editor, annotation and templating queries.
Grafana ships with very feature rich data source plugin for InfluxDB. Supporting a feature rich query editor, annotation and templating queries.
## Adding the data source
## Adding the data source
1. Open the side menu by clicking the the Grafana icon in the top header.
1. Open the side menu by clicking the Grafana icon in the top header.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
3. Click the `+ Add data source` button in the top header.
4. Select *InfluxDB* from the *Type* dropdown.
> NOTE: If this link is missing in the side menu it means that your current user does not have the `Admin` role for the current organization.
> NOTE: If you're not seeing the `Data Sources` link in your side menu it means that your current user does not have the `Admin` role for the current organization.
3. Click the `Add new` link in the top header.
Name | Description
Name | Description
------------ | -------------
------------ | -------------
Name | The data source name, important that this is the same as in Grafana v1.x if you plan to import old dashboards.
*Name* | The data source name. This is how you refer to the data source in panels & queries.
Default | Default data source means that it will be pre-selected for new panels.
*Default* | Default data source means that it will be pre-selected for new panels.
Url | The http protocol, ip and port of you influxdb api (influxdb api port is by default 8086)
*Url* | The http protocol, ip and port of you influxdb api (influxdb api port is by default 8086)
Access | Proxy = access via Grafana backend, Direct = access directly from browser.
*Access* | Proxy = access via Grafana backend, Direct = access directly from browser.
Database | Name of your influxdb database
*Database* | Name of your influxdb database
User | Name of your database user
*User* | Name of your database user
Password | Database user's password
*Password* | Database user's password
> Proxy access means that the Grafana backend will proxy all requests from the browser, and send them on to the Data Source. This is useful because it can eliminate CORS (Cross Origin Site Resource) issues, as well as eliminate the need to disseminate authentication details to the Data Source to the browser.
> Direct access is still supported because in some cases it may be useful to access a Data Source directly depending on the use case and topology of Grafana, the user, and the Data Source.
### Proxy vs Direct access
Proxy access means that the Grafana backend will proxy all requests from the browser. So requests to InfluxDB will be channeled through
`grafana-server`. This means that the URL you specify needs to be accessable from the server you are running Grafana on. Proxy access
mode is also more secure as the username & password will never reach the browser.
## Query Editor
## Query Editor
...
@@ -100,11 +100,21 @@ change the option `Format As` to `Table` if you want to show raw data in the `Ta
...
@@ -100,11 +100,21 @@ change the option `Format As` to `Table` if you want to show raw data in the `Ta
## Templating
## Templating
You can create a template variable in Grafana and have that variable filled with values from any InfluxDB metric exploration query.
You can then use this variable in your InfluxDB metric queries.
For example you can have a variable that contains all values for tag `hostname` if you specify a query like this
Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place.
in the templating edit view.
Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns makes it easy to change the data
being displayed in your dashboard.
Checkout the [Templating]({{<relref"reference/templating.md">}}) documentation for an introduction to the templating feature and the different
types of template variables.
### Query variable
If you add a template variable of the type `Query` you can write a InfluxDB exploration (meta data) query. These queries can
return things like measurement names, key names or key values.
For example you can have a variable that contains all values for tag `hostname` if you specify a query like this in the templating variable *Query* setting.
```sql
```sql
SHOWTAGVALUESWITHKEY="hostname"
SHOWTAGVALUESWITHKEY="hostname"
```
```
...
@@ -116,9 +126,41 @@ the hosts variable only show hosts from the current selected region with a query
...
@@ -116,9 +126,41 @@ the hosts variable only show hosts from the current selected region with a query
> Always use `regex values` or `regex wildcard` for All format or multi select format.
You can fetch key names for a given measurement.
```sql
SHOWTAGKEYS[FROM<measurement_name>]
```
If you have a variable with key names you can use this variable in a group by clause. This will allow you to change group by using the variable dropdown a the top
Why two ways? The first syntax is easier to read and write but does not allow you to use a variable in the middle of a word. When the *Multi-value* or *Include all value*
options are enabled, Grafana converts the labels from plain text to a regex compatible string. Which means you have to use `=~` instead of `=`.
InfluxDB supports the special `Ad hoc filters` variable type. This variable allows you to specify any number of key/value filters on the fly. These filters will automatically
Grafana includes support for Prometheus Datasources. While the process of adding the datasource is similar to adding a Graphite or OpenTSDB datasource type, Prometheus does have a few different options for building queries.
Grafana includes built-in support for Prometheus.
## Adding the data source to Grafana
## Adding the data source to Grafana
1. Open the side menu by clicking the Grafana icon in the top header.
1. Open the side menu by clicking the the Grafana icon in the top header.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
3. Click the `+ Add data source` button in the top header.
4. Select `Prometheus` from the *Type* dropdown.
> NOTE: If this link is missing in the side menu it means that your current user does not have the `Admin` role for the current organization.
> NOTE: If you're not seeing the `Data Sources` link in your side menu it means that your current user does not have the `Admin` role for the current organization.
3. Click the `Add new` link in the top header.
## Data source options
4. Select `Prometheus` from the dropdown.
Name | Description
Name | Description
------------ | -------------
------------ | -------------
Name | The data source name, important that this is the same as in Grafana v1.x if you plan to import old dashboards.
*Name* | The data source name. This is how you refer to the data source in panels & queries.
Default | Default data source means that it will be pre-selected for new panels.
*Default* | Default data source means that it will be pre-selected for new panels.
Url | The http protocol, ip and port of you Prometheus server (default port is usually 9090)
*Url* | The http protocol, ip and port of you Prometheus server (default port is usually 9090)
Access | Proxy = access via Grafana backend, Direct = access directly from browser.
*Access* | Proxy = access via Grafana backend, Direct = access directly from browser.
Basic Auth | Enable basic authentication to the Prometheus datasource.
*Basic Auth* | Enable basic authentication to the Prometheus data source.
User | Name of your Prometheus user
*User* | Name of your Prometheus user
Password | Database user's password
*Password* | Database user's password
> Proxy access means that the Grafana backend will proxy all requests from the browser, and send them on to the Data Source. This is useful because it can eliminate CORS (Cross Origin Site Resource) issues, as well as eliminate the need to disseminate authentication details to the Data Source to the browser.
> Direct access is still supported because in some cases it may be useful to access a Data Source directly depending on the use case and topology of Grafana, the user, and the Data Source.
## Query editor
## Query editor
Open a graph in edit mode by click the title.
For details on Prometheus metric queries check out the Prometheus documentation
Open a graph in edit mode by click the title > Edit (or by pressing `e` key while hovering over panel).
Prometheus Datasource Plugin provides the following functions in `Variables values query` field in Templating Editor to query `metric names` and `labels names` on the Prometheus server.
Name | Description
Name | Description
------- | --------
------- | --------
`label_values(label)` | Returns a list of label values for the `label` in every metric.
*Query expression* | Prometheus query expression, check out the [Prometheus documentation](http://prometheus.io/docs/querying/basics/).
`label_values(metric, label)` | Returns a list of label values for the `label` in the specified metric.
*Legend format* | Controls the name of the time series, using name or pattern. For example `{{hostname}}` will be replaced with label value for the label `hostname`.
`metrics(metric)` | Returns a list of metrics matching the specified `metric` regex.
*Min step* | Set a lower limit for the Prometheus step option. Step controls how big the jumps are when the Prometheus query engine performs range queries. Sadly there is no official prometheus documentation to link to for this very important option.
`query_result(query)` | Returns a list of Prometheus query result for the `query`.
*Resolution* | Controls the step option. Small steps create high-resolution graphs but can be slow over larger time ranges, lowering the resolution can speed things up. `1/2` will try to set step option to generate 1 data point for every other pixel. A value of `1/10` will try to set step option so there is a data point every 10 pixels.*Metric lookup* | Search for metric names in this input field.
*Format as* | **(New in v4.3)** Switch between Table & Time series. Table format will only work in the Table panel.
## Templating
For details of `metric names` & `label names`, and `label values`, please refer to the [Prometheus documentation](http://prometheus.io/docs/concepts/data_model/#metric-names-and-labels).
Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place.
Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns makes it easy to change the data
being displayed in your dashboard.
> Note: The part of queries is incompatible with the version before 2.6, if you specify like `foo.*`, please change like `metrics(foo.*)`.
Checkout the [Templating]({{<relref"reference/templating.md">}}) documentation for an introduction to the templating feature and the different
types of template variables.
You can create a template variable in Grafana and have that variable filled with values from any Prometheus metric exploration query.
### Query variable
You can then use this variable in your Prometheus metric queries.
For example you can have a variable that contains all values for label `hostname` if you specify a query like this in the templating edit view.
Variable of the type *Query* allows you to query Prometheus for a list of metrics, labels or label values. The Prometheus data source plugin
provides the following functions you can use in the `Query` input field.
Name | Description
---- | --------
*label_values(label)* | Returns a list of label values for the `label` in every metric.
*label_values(metric, label)* | Returns a list of label values for the `label` in the specified metric.
*metrics(metric)* | Returns a list of metrics matching the specified `metric` regex.
*query_result(query)* | Returns a list of Prometheus query result for the `query`.
```sql
For details of *metric names*, *label names* and *label values* are please refer to the [Prometheus documentation](http://prometheus.io/docs/concepts/data_model/#metric-names-and-labels).
label_values(hostname)
```
You can also use raw queries & regular expressions to extract anything you might need.
### Using variables in queries
### Using templated variables in queries
There are two syntaxes:
When the `Include All` option or `Multi-Value` option is enabled, Grafana converts the labels from plain text to a regex compatible string.
Which means you have to use `=~` instead of `=` in your Prometheus queries. For example `ALERTS{instance=~$instance}` instead of `ALERTS{instance=$instance}`.
Why two ways? The first syntax is easier to read and write but does not allow you to use a variable in the middle of a word. When the *Multi-value* or *Include all value*
options are enabled, Grafana converts the labels from plain text to a regex compatible string. Which means you have to use `=~` instead of `=`.
<selectclass="gf-form-input gf-size-auto"ng-model="ctrl.target.format"ng-options="f.value as f.text for f in ctrl.formats"ng-change="ctrl.refresh()"></select>
<selectclass="gf-form-input gf-size-auto"ng-model="ctrl.target.format"ng-options="f.value as f.text for f in ctrl.formats"ng-change="ctrl.refresh()"></select>
