Skip to content

N
nexpie-grafana-theme
Commit cafed8ce by Torkel Ödegaard
Options

Merge branch 'master' into develop

parents e98f5737 871b98c0
Showing with 1594 additions and 209 deletions
.gitignore
......@@ -39,8 +39,7 @@ conf/custom.ini
fig.yml
docker-compose.yml
docker-compose.yaml
/conf/dashboards/custom.yaml
/conf/datasources/custom.yaml
/conf/provisioning/**/custom.yaml
profile.cov
/grafana
.notouch
......
CHANGELOG.md
......@@ -9,6 +9,13 @@
# 4.7.0 (unreleased)
## Breaking changes
`[dashboard.json]` have been replaced with [dashboard provisioning](http://docs.grafana.org/administration/provisioning/).
Config files for provisioning datasources as configuration have changed from `/conf/datasources` to `/conf/provisioning/datasources`.
From `/etc/grafana/datasources` to `/etc/grafana/provisioning/datasources` when installed with deb/rpm packages.
## New Features
* **Data Source Proxy**: Add support for whitelisting specified cookies that will be passed through to the data source when proxying data source requests [#5457](https://github.com/grafana/grafana/issues/5457), thanks [@robingustafsson](https://github.com/robingustafsson)
* **Postgres/MySQL**: add __timeGroup macro for mysql [#9596](https://github.com/grafana/grafana/pull/9596), thanks [@svenklemm](https://github.com/svenklemm)
......@@ -17,6 +24,7 @@
* **Datasources**: Its now possible to configure datasources with config files [#1789](https://github.com/grafana/grafana/issues/1789)
* **Graphite**: Query editor updated to support new query by tag features [#9230](https://github.com/grafana/grafana/issues/9230)
* **Dashboard history**: New config file option versions_to_keep sets how many versions per dashboard to store, [#9671](https://github.com/grafana/grafana/issues/9671)
* **Dashboard as cfg**: Load dashboards from file into Grafana on startup/change [#9654](https://github.com/grafana/grafana/issues/9654) [#5269](https://github.com/grafana/grafana/issues/5269)
## Minor
* **Alert panel**: Adds placeholder text when no alerts are within the time range [#9624](https://github.com/grafana/grafana/issues/9624), thx [@straend](https://github.com/straend)
......@@ -25,9 +33,14 @@
* **Dashboard**: Make it possible to start dashboards from search and dashboard list panel [#1871](https://github.com/grafana/grafana/issues/1871)
* **Annotations**: Posting annotations now return the id of the annotation [#9798](https://github.com/grafana/grafana/issues/9798)
* **Systemd**: Use systemd notification ready flag [#10024](https://github.com/grafana/grafana/issues/10024), thx [@jgrassler](https://github.com/jgrassler)
* **Github**: Use organizations_url provided from github to verify user belongs in org. [#10111](https://github.com/grafana/grafana/issues/10111), thx
[@adiletmaratov](https://github.com/adiletmaratov)
* **Backend**: Fixed bug where Grafana exited before all sub routines where finished [#10131](https://github.com/grafana/grafana/issues/10131)
## Tech
* **RabbitMq**: Remove support for publishing events to RabbitMQ [#9645](https://github.com/grafana/grafana/issues/9645)
## Fixes
* **Sensu**: Send alert message to sensu output [#9551](https://github.com/grafana/grafana/issues/9551), thx [@cjchand](https://github.com/cjchand)
* **Singlestat**: suppress error when result contains no datapoints [#9636](https://github.com/grafana/grafana/issues/9636), thx [@utkarshcmu](https://github.com/utkarshcmu)
......
README.md
......@@ -19,7 +19,7 @@ If you have any problems please read the [troubleshooting guide](http://docs.gra
Be sure to read the [getting started guide](http://docs.grafana.org/guides/gettingstarted/) and the other feature guides.
## Run from master
If you want to build a package yourself, or contribute. Here is a guide for how to do that. You can always find
If you want to build a package yourself, or contribute - Here is a guide for how to do that. You can always find
the latest master builds [here](https://grafana.com/grafana/download)
### Dependencies
......@@ -97,7 +97,7 @@ Writing & watching frontend tests (we have two test runners)
## Contribute
If you have any idea for an improvement or found a bug do not hesitate to open an issue.
If you have any idea for an improvement or found a bug, do not hesitate to open an issue.
And if you have time clone this repo and submit a pull request and help me make Grafana
the kickass metrics & devops dashboard we all dream about!
......
conf/defaults.ini
......@@ -20,8 +20,8 @@ logs = data/log
# Directory where grafana will automatically scan and look for plugins
plugins = data/plugins
# Config files containing datasources that will be configured at startup
datasources = conf/datasources
# folder that contains provisioning config files that grafana will apply on startup and while running.
provisioning = conf/provisioning
#################################### Server ##############################
[server]
......@@ -391,11 +391,6 @@ facility =
# Syslog tag. By default, the process' argv[0] is used.
tag =
#################################### Dashboard JSON files ################
[dashboards.json]
enabled = false
path = /var/lib/grafana/dashboards
#################################### Usage Quotas ########################
[quota]
enabled = false
......
conf/provisioning/dashboards/sample.yaml 0 → 100644
# - name: 'default'
# org_id: 1
# folder: ''
# type: file
# options:
# folder: /var/lib/grafana/dashboards
\ No newline at end of file
conf/datasources/datasources.yaml → conf/provisioning/datasources/sample.yaml
# list of datasources that should be deleted from the database
delete_datasources:
# - name: Graphite
# org_id: 1
# # list of datasources that should be deleted from the database
#delete_datasources:
# - name: Graphite
# org_id: 1
# list of datasources to insert/update depending
# whats available in the datbase
datasources:
# # list of datasources to insert/update depending
# # whats available in the datbase
#datasources:
# # <string, required> name of the datasource. Required
# - name: Graphite
# # <string, required> datasource type. Required
......@@ -33,7 +33,7 @@ datasources:
# # <bool> mark as default datasource. Max one per org
# is_default:
# # <map> fields that will be converted to json and stored in json_data
# json_data:
# json_data:
# graphiteVersion: "1.1"
# tlsAuth: true
# tlsAuthWithCACert: true
......
conf/sample.ini
......@@ -20,8 +20,8 @@
# Directory where grafana will automatically scan and look for plugins
;plugins = /var/lib/grafana/plugins
# Config files containing datasources that will be configured at startup
;datasources = conf/datasources
# folder that contains provisioning config files that grafana will apply on startup and while running.
; provisioning = conf/provisioning
#################################### Server ####################################
[server]
......@@ -367,11 +367,6 @@ log_queries =
;tag =
;#################################### Dashboard JSON files ##########################
[dashboards.json]
;enabled = false
;path = /var/lib/grafana/dashboards
#################################### Alerting ############################
[alerting]
# Disable alerting engine & UI features
......
docs/sources/administration/provisioning.md
......@@ -74,7 +74,7 @@ Saltstack | [https://github.com/salt-formulas/salt-formula-grafana](https://gith
> This feature is available from v5.0
It's possible to manage datasources in Grafana by adding one or more yaml config files in the [`conf/datasources`](/installation/configuration/#datasources) directory. Each config file can contain a list of `datasources` that will be added or updated during start up. If the datasource already exists, Grafana will update it to match the configuration file. The config file can also contain a list of datasources that should be deleted. That list is called `delete_datasources`. Grafana will delete datasources listed in `delete_datasources` before inserting/updating those in the `datasource` list.
It's possible to manage datasources in Grafana by adding one or more yaml config files in the [`provisioning/datasources`](/installation/configuration/#provisioning) directory. Each config file can contain a list of `datasources` that will be added or updated during start up. If the datasource already exists, Grafana will update it to match the configuration file. The config file can also contain a list of datasources that should be deleted. That list is called `delete_datasources`. Grafana will delete datasources listed in `delete_datasources` before inserting/updating those in the `datasource` list.
### Running multiple grafana instances.
If you are running multiple instances of Grafana you might run into problems if they have different versions of the datasource.yaml configuration file. The best way to solve this problem is to add a version number to each datasource in the configuration and increase it when you update the config. Grafana will only update datasources with the same or lower version number than specified in the config. That way old configs cannot overwrite newer configs if they restart at the same time.
......@@ -165,3 +165,20 @@ Secure json data is a map of settings that will be encrypted with [secret key](/
| tlsClientKey | string | *All* |TLS Client key for outgoing requests |
| password | string | Postgre | password |
| user | string | Postgre | user |
### Dashboards
It's possible to manage dashboards in Grafana by adding one or more yaml config files in the [`provisioning/dashboards`](/installation/configuration/#provisioning) directory. Each config file can contain a list of `dashboards providers` that will load dashboards into grafana. Currently we only support reading dashboards from file but we will add more providers in the future.
The dashboard provider config file looks like this
```yaml
- name: 'default'
org_id: 1
folder: ''
type: file
options:
folder: /var/lib/grafana/dashboards
```
When grafana starts it will update/insert all dashboards available in the configured folders. If you modify the file the dashboard will also be updated.
\ No newline at end of file
docs/sources/features/datasources/cloudwatch.md
......@@ -78,11 +78,14 @@ CloudWatch Datasource Plugin provides the following queries you can specify in t
edit view. They allow you to fill a variable's options list with things like `region`, `namespaces`, `metric names`
and `dimension keys/values`.
In place of `region` you can specify `default` to use the default region configured in the datasource for the query,
e.g. `metrics(AWS/DynamoDB, default)` or `dimension_values(default, ..., ..., ...)`.
Name | Description
------- | --------
*regions()* | Returns a list of regions AWS provides their service.
*namespaces()* | Returns a list of namespaces CloudWatch support.
*metrics(namespace, [region])* | Returns a list of metrics in the namespace. (specify region for custom metrics)
*metrics(namespace, [region])* | Returns a list of metrics in the namespace. (specify region or use "default" for custom metrics)
*dimension_keys(namespace)* | Returns a list of dimension keys in the namespace.
*dimension_values(region, namespace, metric, dimension_key)* | Returns a list of dimension values matching the specified `region`, `namespace`, `metric` and `dimension_key`.
*ebs_volume_ids(region, instance_id)* | Returns a list of volume ids matching the specified `region`, `instance_id`.
......
docs/sources/features/datasources/postgres.md
......@@ -48,7 +48,7 @@ Macro example | Description
*$__timeFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name. For example, *extract(epoch from dateColumn) BETWEEN 1494410783 AND 1494497183*
*$__timeFrom()* | Will be replaced by the start of the currently active time selection. For example, *to_timestamp(1494410783)*
*$__timeTo()* | Will be replaced by the end of the currently active time selection. For example, *to_timestamp(1494497183)*
*$__timeGroup(dateColumn,'5m')* | Will be replaced by an expression usable in GROUP BY clause. For example, *(extract(epoch from "dateColumn")/300)::bigint*300*
*$__timeGroup(dateColumn,'5m')* | Will be replaced by an expression usable in GROUP BY clause. For example, *(extract(epoch from dateColumn)/300)::bigint*300 AS time*
*$__unixEpochFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name with times represented as unix timestamp. For example, *dateColumn > 1494410783 AND dateColumn < 1494497183*
*$__unixEpochFrom()* | Will be replaced by the start of the currently active time selection as unix timestamp. For example, *1494410783*
*$__unixEpochTo()* | Will be replaced by the end of the currently active time selection as unix timestamp. For example, *1494497183*
......@@ -94,7 +94,7 @@ Example with `metric` column
```sql
SELECT
$__timeGroup(time_date_time,'5m') as time,
$__timeGroup(time_date_time,'5m'),
min(value_double),
'min' as metric
FROM test_data
......@@ -107,7 +107,7 @@ Example with multiple columns:
```sql
SELECT
$__timeGroup(time_date_time,'5m') as time,
$__timeGroup(time_date_time,'5m'),
min(value_double) as min_value,
max(value_double) as max_value
FROM test_data
......
docs/sources/installation/configuration.md
......@@ -91,12 +91,11 @@ file.
Directory where grafana will automatically scan and look for plugins
### datasources
### provisioning
> This feature is available in 5.0+
Config files containing datasources that will be configured at startup.
You can read more about the config files at the [provisioning page](/administration/provisioning/#datasources).
Folder that contains [provisioning](/administration/provisioning) config files that grafana will apply on startup. Dashboards will be reloaded when the json files changes
## [server]
......@@ -635,8 +634,7 @@ Number dashboard versions to keep (per dashboard). Default: 20, Minimum: 1.
## [dashboards.json]
If you have a system that automatically builds dashboards as json files you can enable this feature to have the
Grafana backend index those json dashboards which will make them appear in regular dashboard search.
> This have been replaced with dashboards [provisioning](/administration/provisioning) in 5.0+
### enabled
`true` or `false`. Is disabled by default.
......
packaging/deb/control/postinst
......@@ -31,6 +31,12 @@ case "$1" in
cp /usr/share/grafana/conf/ldap.toml /etc/grafana/ldap.toml
fi
if [ ! -f $PROVISIONING_CFG_DIR ]; then
mkdir -p $PROVISIONING_CFG_DIR/dashboards $PROVISIONING_CFG_DIR/datasources
cp /usr/share/grafana/conf/provisioning/dashboards/sample.yaml $PROVISIONING_CFG_DIR/dashboards/sample.yaml
cp /usr/share/grafana/conf/provisioning/datasources/sample.yaml $PROVISIONING_CFG_DIR/datasources/sample.yaml
fi
# configuration files should not be modifiable by grafana user, as this can be a security issue
chown -Rh root:$GRAFANA_GROUP /etc/grafana/*
chmod 755 /etc/grafana
......
packaging/deb/default/grafana-server
......@@ -18,5 +18,7 @@ RESTART_ON_UPGRADE=true
PLUGINS_DIR=/var/lib/grafana/plugins
PROVISIONING_CFG_DIR=/etc/grafana/provisioning
# Only used on systemd systems
PID_FILE_DIR=/var/run/grafana
packaging/deb/init.d/grafana-server
......@@ -33,6 +33,7 @@ DATA_DIR=/var/lib/grafana
PLUGINS_DIR=/var/lib/grafana/plugins
LOG_DIR=/var/log/grafana
CONF_FILE=$CONF_DIR/grafana.ini
PROVISIONING_CFG_DIR=$CONF_DIR/provisioning
MAX_OPEN_FILES=10000
PID_FILE=/var/run/$NAME.pid
DAEMON=/usr/sbin/$NAME
......@@ -55,7 +56,7 @@ if [ -f "$DEFAULT" ]; then
. "$DEFAULT"
fi
DAEMON_OPTS="--pidfile=${PID_FILE} --config=${CONF_FILE} cfg:default.paths.data=${DATA_DIR} cfg:default.paths.logs=${LOG_DIR} cfg:default.paths.plugins=${PLUGINS_DIR}"
DAEMON_OPTS="--pidfile=${PID_FILE} --config=${CONF_FILE} cfg:default.paths.provisioning=$PROVISIONING_CFG_DIR cfg:default.paths.data=${DATA_DIR} cfg:default.paths.logs=${LOG_DIR} cfg:default.paths.plugins=${PLUGINS_DIR}"
function checkUser() {
if [ `id -u` -ne 0 ]; then
......
packaging/deb/systemd/grafana-server.service
......@@ -14,12 +14,15 @@ Restart=on-failure
WorkingDirectory=/usr/share/grafana
RuntimeDirectory=grafana
RuntimeDirectoryMode=0750
ExecStart=/usr/sbin/grafana-server \
--config=${CONF_FILE} \
--pidfile=${PID_FILE_DIR}/grafana-server.pid \
cfg:default.paths.logs=${LOG_DIR} \
cfg:default.paths.data=${DATA_DIR} \
cfg:default.paths.plugins=${PLUGINS_DIR}
ExecStart=/usr/sbin/grafana-server \
--config=${CONF_FILE} \
--pidfile=${PID_FILE_DIR}/grafana-server.pid \
cfg:default.paths.logs=${LOG_DIR} \
cfg:default.paths.data=${DATA_DIR} \
cfg:default.paths.plugins=${PLUGINS_DIR} \
cfg:default.paths.provisioning=${PROVISIONING_CFG_DIR}
LimitNOFILE=10000
TimeoutStopSec=20
UMask=0027
......
packaging/mac/bin/grafana
......@@ -6,10 +6,12 @@ HOMEPATH=/usr/local/share/grafana
LOGPATH=/usr/local/var/log/grafana
DATAPATH=/usr/local/var/lib/grafana
PLUGINPATH=/usr/local/var/lib/grafana/plugins
DATASOURCECFGPATH=/usr/local/etc/grafana/datasources
DASHBOARDSCFGPATH=/usr/local/etc/grafana/dashboards
case "$1" in
start)
$EXECUTABLE --config=$CONFIG --homepath=$HOMEPATH cfg:default.paths.logs=$LOGPATH cfg:default.paths.data=$DATAPATH cfg:default.paths.plugins=$PLUGINPATH 2> /dev/null &
$EXECUTABLE --config=$CONFIG --homepath=$HOMEPATH cfg:default.paths.datasources=$DATASOURCECFGPATH cfg:default.paths.dashboards=$DASHBOARDSCFGPATH cfg:default.paths.logs=$LOGPATH cfg:default.paths.data=$DATAPATH cfg:default.paths.plugins=$PLUGINPATH 2> /dev/null &
[ $? -eq 0 ] && echo "$DAEMON started"
;;
stop)
......
packaging/rpm/control/postinst
......@@ -45,6 +45,12 @@ if [ $1 -eq 1 ] ; then
cp /usr/share/grafana/conf/ldap.toml /etc/grafana/ldap.toml
fi
if [ ! -f $PROVISIONING_CFG_DIR ]; then
mkdir -p $PROVISIONING_CFG_DIR/dashboards $PROVISIONING_CFG_DIR/datasources
cp /usr/share/grafana/conf/provisioning/dashboards/sample.yaml $PROVISIONING_CFG_DIR/dashboards/sample.yaml
cp /usr/share/grafana/conf/provisioning/datasources/sample.yaml $PROVISIONING_CFG_DIR/datasources/sample.yaml
fi
# Set user permissions on /var/log/grafana, /var/lib/grafana
mkdir -p /var/log/grafana /var/lib/grafana
chown -R $GRAFANA_USER:$GRAFANA_GROUP /var/log/grafana /var/lib/grafana
......
packaging/rpm/init.d/grafana-server
......@@ -32,6 +32,7 @@ DATA_DIR=/var/lib/grafana
PLUGINS_DIR=/var/lib/grafana/plugins
LOG_DIR=/var/log/grafana
CONF_FILE=$CONF_DIR/grafana.ini
PROVISIONING_CFG_DIR=$CONF_DIR/provisioning
MAX_OPEN_FILES=10000
PID_FILE=/var/run/$NAME.pid
DAEMON=/usr/sbin/$NAME
......@@ -59,7 +60,7 @@ fi
# overwrite settings from default file
[ -e /etc/sysconfig/$NAME ] && . /etc/sysconfig/$NAME
DAEMON_OPTS="--pidfile=${PID_FILE} --config=${CONF_FILE} cfg:default.paths.data=${DATA_DIR} cfg:default.paths.logs=${LOG_DIR} cfg:default.paths.plugins=${PLUGINS_DIR}"
DAEMON_OPTS="--pidfile=${PID_FILE} --config=${CONF_FILE} cfg:default.paths.provisioning=$PROVISIONING_CFG_DIR cfg:default.paths.data=${DATA_DIR} cfg:default.paths.logs=${LOG_DIR} cfg:default.paths.plugins=${PLUGINS_DIR}"
function isRunning() {
status -p $PID_FILE $NAME > /dev/null 2>&1
......
packaging/rpm/sysconfig/grafana-server
......@@ -18,5 +18,7 @@ RESTART_ON_UPGRADE=true
PLUGINS_DIR=/var/lib/grafana/plugins
PROVISIONING_CFG_DIR=/etc/grafana/provisioning
# Only used on systemd systems
PID_FILE_DIR=/var/run/grafana
packaging/rpm/systemd/grafana-server.service
......@@ -14,12 +14,14 @@ Restart=on-failure
WorkingDirectory=/usr/share/grafana
RuntimeDirectory=grafana
RuntimeDirectoryMode=0750
ExecStart=/usr/sbin/grafana-server \
--config=${CONF_FILE} \
--pidfile=${PID_FILE_DIR}/grafana-server.pid \
cfg:default.paths.logs=${LOG_DIR} \
cfg:default.paths.data=${DATA_DIR} \
cfg:default.paths.plugins=${PLUGINS_DIR}
ExecStart=/usr/sbin/grafana-server \
--config=${CONF_FILE} \
--pidfile=${PID_FILE_DIR}/grafana-server.pid \
cfg:default.paths.logs=${LOG_DIR} \
cfg:default.paths.data=${DATA_DIR} \
cfg:default.paths.plugins=${PLUGINS_DIR} \
cfg:default.paths.provisioning=${PROVISIONING_CFG_DIR}
LimitNOFILE=10000
TimeoutStopSec=20
......
pkg/api/avatar/avatar.go
......@@ -25,6 +25,8 @@ import (
"github.com/grafana/grafana/pkg/log"
"github.com/grafana/grafana/pkg/setting"
"gopkg.in/macaron.v1"
gocache "github.com/patrickmn/go-cache"
)
var gravatarSource string
......@@ -92,7 +94,7 @@ func (this *Avatar) Update() (err error) {
type CacheServer struct {
notFound *Avatar
cache map[string]*Avatar
cache *gocache.Cache
}
func (this *CacheServer) mustInt(r *http.Request, defaultValue int, keys ...string) (v int) {
......@@ -110,7 +112,9 @@ func (this *CacheServer) Handler(ctx *macaron.Context) {
var avatar *Avatar
if avatar, _ = this.cache[hash]; avatar == nil {
if obj, exist := this.cache.Get(hash); exist {
avatar = obj.(*Avatar)
} else {
avatar = New(hash)
}
......@@ -124,7 +128,7 @@ func (this *CacheServer) Handler(ctx *macaron.Context) {
if avatar.notFound {
avatar = this.notFound
} else {
this.cache[hash] = avatar
this.cache.Add(hash, avatar, gocache.DefaultExpiration)
}
ctx.Resp.Header().Add("Content-Type", "image/jpeg")
......@@ -146,7 +150,7 @@ func NewCacheServer() *CacheServer {
return &CacheServer{
notFound: newNotFound(),
cache: make(map[string]*Avatar),
cache: gocache.New(time.Hour, time.Hour*2),
}
}
......
pkg/api/dashboard.go
......@@ -6,6 +6,8 @@ import (
"os"
"path"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/api/dtos"
"github.com/grafana/grafana/pkg/bus"
"github.com/grafana/grafana/pkg/components/dashdiffs"
......@@ -15,7 +17,6 @@ import (
"github.com/grafana/grafana/pkg/middleware"
m "github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/services/alerting"
"github.com/grafana/grafana/pkg/services/guardian"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/util"
......@@ -157,13 +158,6 @@ func PostDashboard(c *middleware.Context, cmd m.SaveDashboardCommand) Response {
dash := cmd.GetDashboardModel()
// look up existing dashboard
if dash.Id > 0 {
if existing, _ := getDashboardHelper(c.OrgId, "", dash.Id); existing != nil {
dash.HasAcl = existing.HasAcl
}
}
guardian := guardian.NewDashboardGuardian(dash.Id, c.OrgId, c.SignedInUser)
if canSave, err := guardian.CanSave(); err != nil || !canSave {
return dashboardGuardianResponse(err)
......@@ -188,17 +182,23 @@ func PostDashboard(c *middleware.Context, cmd m.SaveDashboardCommand) Response {
}
}
validateAlertsCmd := alerting.ValidateDashboardAlertsCommand{
dashItem := &dashboards.SaveDashboardItem{
Dashboard: dash,
Message: cmd.Message,
OrgId: c.OrgId,
UserId: c.UserId,
Dashboard: dash,
}
if err := bus.Dispatch(&validateAlertsCmd); err != nil {
dashboard, err := dashboards.GetRepository().SaveDashboard(dashItem)
if err == m.ErrDashboardTitleEmpty {
return ApiError(400, m.ErrDashboardTitleEmpty.Error(), nil)
}
if err == m.ErrDashboardContainsInvalidAlertData {
return ApiError(500, "Invalid alert data. Cannot save dashboard", err)
}
err := bus.Dispatch(&cmd)
if err != nil {
if err == m.ErrDashboardWithSameNameExists {
return Json(412, util.DynMap{"status": "name-exists", "message": err.Error()})
......@@ -220,18 +220,16 @@ func PostDashboard(c *middleware.Context, cmd m.SaveDashboardCommand) Response {
return ApiError(500, "Failed to save dashboard", err)
}
alertCmd := alerting.UpdateDashboardAlertsCommand{
OrgId: c.OrgId,
UserId: c.UserId,
Dashboard: cmd.Result,
}
if err := bus.Dispatch(&alertCmd); err != nil {
return ApiError(500, "Failed to save alerts", err)
if err == m.ErrDashboardFailedToUpdateAlertData {
return ApiError(500, "Invalid alert data. Cannot save dashboard", err)
}
c.TimeRequest(metrics.M_Api_Dashboard_Save)
return Json(200, util.DynMap{"status": "success", "slug": cmd.Result.Slug, "version": cmd.Result.Version, "id": cmd.Result.Id})
return Json(200, util.DynMap{"status": "success", "slug": dashboard.Slug, "version": dashboard.Version, "id": dashboard.Id})
}
func canEditDashboard(role m.RoleType) bool {
return role == m.ROLE_ADMIN || role == m.ROLE_EDITOR || role == m.ROLE_READ_ONLY_EDITOR
}
func GetHomeDashboard(c *middleware.Context) Response {
......
pkg/api/http_server.go
......@@ -95,7 +95,7 @@ func (hs *HttpServer) Start(ctx context.Context) error {
func (hs *HttpServer) Shutdown(ctx context.Context) error {
err := hs.httpSrv.Shutdown(ctx)
hs.log.Info("stopped http server")
hs.log.Info("Stopped HTTP server")
return err
}
......
pkg/api/render.go
......@@ -26,6 +26,7 @@ func RenderToPng(c *middleware.Context) {
UserId: c.UserId,
OrgRole: c.OrgRole,
Timezone: queryReader.Get("tz", ""),
Encoding: queryReader.Get("encoding", ""),
}
pngPath, err := renderer.RenderToPng(renderOpts)
......
pkg/cmd/grafana-server/main.go
......@@ -14,8 +14,8 @@ import (
"net/http"
_ "net/http/pprof"
"github.com/grafana/grafana/pkg/log"
"github.com/grafana/grafana/pkg/metrics"
"github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/setting"
_ "github.com/grafana/grafana/pkg/services/alerting/conditions"
......@@ -40,9 +40,6 @@ var homePath = flag.String("homepath", "", "path to grafana install/home path, d
var pidFile = flag.String("pidfile", "", "path to pid file")
var exitChan = make(chan int)
func init() {
}
func main() {
v := flag.Bool("v", false, "prints current version and exits")
profile := flag.Bool("profile", false, "Turn on pprof profiling")
......@@ -82,12 +79,28 @@ func main() {
setting.BuildStamp = buildstampInt64
metrics.M_Grafana_Version.WithLabelValues(version).Set(1)
shutdownCompleted := make(chan int)
server := NewGrafanaServer()
server.Start()
go listenToSystemSignals(server, shutdownCompleted)
go func() {
code := 0
if err := server.Start(); err != nil {
log.Error2("Startup failed", "error", err)
code = 1
}
exitChan <- code
}()
code := <-shutdownCompleted
log.Info2("Grafana shutdown completed.", "code", code)
log.Close()
os.Exit(code)
}
func listenToSystemSignals(server models.GrafanaServer) {
func listenToSystemSignals(server *GrafanaServerImpl, shutdownCompleted chan int) {
signalChan := make(chan os.Signal, 1)
ignoreChan := make(chan os.Signal, 1)
code := 0
......@@ -97,10 +110,12 @@ func listenToSystemSignals(server models.GrafanaServer) {
select {
case sig := <-signalChan:
// Stops trace if profiling has been enabled
trace.Stop()
trace.Stop() // Stops trace if profiling has been enabled
server.Shutdown(0, fmt.Sprintf("system signal: %s", sig))
shutdownCompleted <- 0
case code = <-exitChan:
trace.Stop() // Stops trace if profiling has been enabled
server.Shutdown(code, "startup error")
shutdownCompleted <- code
}
}
pkg/cmd/grafana-server/server.go
......@@ -11,7 +11,6 @@ import (
"strconv"
"time"
"github.com/grafana/grafana/pkg/cmd/grafana-cli/logger"
"github.com/grafana/grafana/pkg/services/provisioning"
"golang.org/x/sync/errgroup"
......@@ -20,7 +19,6 @@ import (
"github.com/grafana/grafana/pkg/log"
"github.com/grafana/grafana/pkg/login"
"github.com/grafana/grafana/pkg/metrics"
"github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/services/alerting"
"github.com/grafana/grafana/pkg/services/cleanup"
......@@ -33,7 +31,7 @@ import (
"github.com/grafana/grafana/pkg/tracing"
)
func NewGrafanaServer() models.GrafanaServer {
func NewGrafanaServer() *GrafanaServerImpl {
rootCtx, shutdownFn := context.WithCancel(context.Background())
childRoutines, childCtx := errgroup.WithContext(rootCtx)
......@@ -54,9 +52,7 @@ type GrafanaServerImpl struct {
httpServer *api.HttpServer
}
func (g *GrafanaServerImpl) Start() {
go listenToSystemSignals(g)
func (g *GrafanaServerImpl) Start() error {
g.initLogging()
g.writePIDFile()
......@@ -68,17 +64,13 @@ func (g *GrafanaServerImpl) Start() {
social.NewOAuthService()
plugins.Init()
if err := provisioning.StartUp(setting.DatasourcesPath); err != nil {
logger.Error("Failed to provision Grafana from config", "error", err)
g.Shutdown(1, "Startup failed")
return
if err := provisioning.Init(g.context, setting.HomePath, setting.Cfg); err != nil {
return fmt.Errorf("Failed to provision Grafana from config. error: %v", err)
}
closer, err := tracing.Init(setting.Cfg)
if err != nil {
g.log.Error("Tracing settings is not valid", "error", err)
g.Shutdown(1, "Startup failed")
return
return fmt.Errorf("Tracing settings is not valid. error: %v", err)
}
defer closer.Close()
......@@ -93,13 +85,12 @@ func (g *GrafanaServerImpl) Start() {
g.childRoutines.Go(func() error { return cleanUpService.Run(g.context) })
if err = notifications.Init(); err != nil {
g.log.Error("Notification service failed to initialize", "error", err)
g.Shutdown(1, "Startup failed")
return
return fmt.Errorf("Notification service failed to initialize. error: %v", err)
}
SendSystemdNotification("READY=1")
g.startHttpServer()
sendSystemdNotification("READY=1")
return g.startHttpServer()
}
func initSql() {
......@@ -123,16 +114,16 @@ func (g *GrafanaServerImpl) initLogging() {
setting.LogConfigurationInfo()
}
func (g *GrafanaServerImpl) startHttpServer() {
func (g *GrafanaServerImpl) startHttpServer() error {
g.httpServer = api.NewHttpServer()
err := g.httpServer.Start(g.context)
if err != nil {
g.log.Error("Fail to start server", "error", err)
g.Shutdown(1, "Startup failed")
return
return fmt.Errorf("Fail to start server. error: %v", err)
}
return nil
}
func (g *GrafanaServerImpl) Shutdown(code int, reason string) {
......@@ -145,10 +136,9 @@ func (g *GrafanaServerImpl) Shutdown(code int, reason string) {
g.shutdownFn()
err = g.childRoutines.Wait()
g.log.Info("Shutdown completed", "reason", err)
log.Close()
os.Exit(code)
if err != nil && err != context.Canceled {
g.log.Error("Server shutdown completed with an error", "error", err)
}
}
func (g *GrafanaServerImpl) writePIDFile() {
......@@ -173,7 +163,7 @@ func (g *GrafanaServerImpl) writePIDFile() {
g.log.Info("Writing PID file", "path", *pidFile, "pid", pid)
}
func SendSystemdNotification(state string) error {
func sendSystemdNotification(state string) error {
notifySocket := os.Getenv("NOTIFY_SOCKET")
if notifySocket == "" {
......
pkg/components/renderer/renderer.go
......@@ -31,6 +31,7 @@ type RenderOpts struct {
OrgRole models.RoleType
Timezone string
IsAlertContext bool
Encoding string
}
var ErrTimeout = errors.New("Timeout error. You can set timeout in seconds with &timeout url parameter")
......@@ -103,6 +104,10 @@ func RenderToPng(params *RenderOpts) (string, error) {
"renderKey=" + renderKey,
}
if params.Encoding != "" {
cmdArgs = append([]string{fmt.Sprintf("--output-encoding=%s", params.Encoding)}, cmdArgs...)
}
cmd := exec.Command(binPath, cmdArgs...)
stdout, err := cmd.StdoutPipe()
......
pkg/models/dashboards.go
......@@ -11,12 +11,14 @@ import (
// Typed errors
var (
ErrDashboardNotFound = errors.New("Dashboard not found")
ErrDashboardSnapshotNotFound = errors.New("Dashboard snapshot not found")
ErrDashboardWithSameNameExists = errors.New("A dashboard with the same name already exists")
ErrDashboardVersionMismatch = errors.New("The dashboard has been changed by someone else")
ErrDashboardTitleEmpty = errors.New("Dashboard title cannot be empty")
ErrDashboardFolderCannotHaveParent = errors.New("A Dashboard Folder cannot be added to another folder")
ErrDashboardNotFound = errors.New("Dashboard not found")
ErrDashboardSnapshotNotFound = errors.New("Dashboard snapshot not found")
ErrDashboardWithSameNameExists = errors.New("A dashboard with the same name already exists")
ErrDashboardVersionMismatch = errors.New("The dashboard has been changed by someone else")
ErrDashboardTitleEmpty = errors.New("Dashboard title cannot be empty")
ErrDashboardFolderCannotHaveParent = errors.New("A Dashboard Folder cannot be added to another folder")
ErrDashboardContainsInvalidAlertData = errors.New("Invalid alert data. Cannot save dashboard")
ErrDashboardFailedToUpdateAlertData = errors.New("Failed to save alert data")
)
type UpdatePluginDashboardError struct {
......@@ -156,6 +158,8 @@ type SaveDashboardCommand struct {
FolderId int64 `json:"folderId"`
IsFolder bool `json:"isFolder"`
UpdatedAt time.Time
Result *Dashboard
}
......
pkg/models/server.go deleted 100644 → 0
package models
type GrafanaServer interface {
Start()
Shutdown(code int, reason string)
}
pkg/services/dashboards/dashboards.go 0 → 100644
package dashboards
import (
"time"
"github.com/grafana/grafana/pkg/bus"
"github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/services/alerting"
)
type Repository interface {
SaveDashboard(*SaveDashboardItem) (*models.Dashboard, error)
}
var repositoryInstance Repository
func GetRepository() Repository {
return repositoryInstance
}
func SetRepository(rep Repository) {
repositoryInstance = rep
}
type SaveDashboardItem struct {
TitleLower string
OrgId int64
Folder string
UpdatedAt time.Time
UserId int64
Message string
Overwrite bool
Dashboard *models.Dashboard
}
type DashboardRepository struct{}
func (dr *DashboardRepository) SaveDashboard(json *SaveDashboardItem) (*models.Dashboard, error) {
dashboard := json.Dashboard
if dashboard.Title == "" {
return nil, models.ErrDashboardTitleEmpty
}
validateAlertsCmd := alerting.ValidateDashboardAlertsCommand{
OrgId: json.OrgId,
Dashboard: dashboard,
}
if err := bus.Dispatch(&validateAlertsCmd); err != nil {
return nil, models.ErrDashboardContainsInvalidAlertData
}
cmd := models.SaveDashboardCommand{
Dashboard: dashboard.Data,
Message: json.Message,
OrgId: json.OrgId,
Overwrite: json.Overwrite,
UserId: json.UserId,
}
if !json.UpdatedAt.IsZero() {
cmd.UpdatedAt = json.UpdatedAt
}
err := bus.Dispatch(&cmd)
if err != nil {
return nil, err
}
alertCmd := alerting.UpdateDashboardAlertsCommand{
OrgId: json.OrgId,
UserId: json.UserId,
Dashboard: cmd.Result,
}
if err := bus.Dispatch(&alertCmd); err != nil {
return nil, models.ErrDashboardFailedToUpdateAlertData
}
return cmd.Result, nil
}
pkg/services/provisioning/dashboards/config_reader.go 0 → 100644
package dashboards
import (
"io/ioutil"
"path/filepath"
"strings"
yaml "gopkg.in/yaml.v2"
)
type configReader struct {
path string
}
func (cr *configReader) readConfig() ([]*DashboardsAsConfig, error) {
files, err := ioutil.ReadDir(cr.path)
if err != nil {
return nil, err
}
var dashboards []*DashboardsAsConfig
for _, file := range files {
if !strings.HasSuffix(file.Name(), ".yaml") && !strings.HasSuffix(file.Name(), ".yml") {
continue
}
filename, _ := filepath.Abs(filepath.Join(cr.path, file.Name()))
yamlFile, err := ioutil.ReadFile(filename)
if err != nil {
return nil, err
}
var datasource []*DashboardsAsConfig
err = yaml.Unmarshal(yamlFile, &datasource)
if err != nil {
return nil, err
}
dashboards = append(dashboards, datasource...)
}
for i := range dashboards {
if dashboards[i].OrgId == 0 {
dashboards[i].OrgId = 1
}
}
return dashboards, nil
}
pkg/services/provisioning/dashboards/config_reader_test.go 0 → 100644
package dashboards
import (
"testing"
. "github.com/smartystreets/goconvey/convey"
)
var (
simpleDashboardConfig string = "./test-configs/dashboards-from-disk"
brokenConfigs string = "./test-configs/broken-configs"
)
func TestDashboardsAsConfig(t *testing.T) {
Convey("Dashboards as configuration", t, func() {
Convey("Can read config file", func() {
cfgProvifer := configReader{path: simpleDashboardConfig}
cfg, err := cfgProvifer.readConfig()
if err != nil {
t.Fatalf("readConfig return an error %v", err)
}
So(len(cfg), ShouldEqual, 2)
ds := cfg[0]
So(ds.Name, ShouldEqual, "general dashboards")
So(ds.Type, ShouldEqual, "file")
So(ds.OrgId, ShouldEqual, 2)
So(ds.Folder, ShouldEqual, "developers")
So(ds.Editable, ShouldBeTrue)
So(len(ds.Options), ShouldEqual, 1)
So(ds.Options["folder"], ShouldEqual, "/var/lib/grafana/dashboards")
ds2 := cfg[1]
So(ds2.Name, ShouldEqual, "default")
So(ds2.Type, ShouldEqual, "file")
So(ds2.OrgId, ShouldEqual, 1)
So(ds2.Folder, ShouldEqual, "")
So(ds2.Editable, ShouldBeFalse)
So(len(ds2.Options), ShouldEqual, 1)
So(ds2.Options["folder"], ShouldEqual, "/var/lib/grafana/dashboards")
})
Convey("Should skip broken config files", func() {
cfgProvifer := configReader{path: brokenConfigs}
cfg, err := cfgProvifer.readConfig()
if err != nil {
t.Fatalf("readConfig return an error %v", err)
}
So(len(cfg), ShouldEqual, 0)
})
})
}
pkg/services/provisioning/dashboards/dashboard.go 0 → 100644
package dashboards
import (
"context"
"fmt"
"github.com/grafana/grafana/pkg/log"
)
type DashboardProvisioner struct {
cfgReader *configReader
log log.Logger
ctx context.Context
}
func Provision(ctx context.Context, configDirectory string) (*DashboardProvisioner, error) {
d := &DashboardProvisioner{
cfgReader: &configReader{path: configDirectory},
log: log.New("provisioning.dashboard"),
ctx: ctx,
}
err := d.Provision(ctx)
return d, err
}
func (provider *DashboardProvisioner) Provision(ctx context.Context) error {
cfgs, err := provider.cfgReader.readConfig()
if err != nil {
return err
}
for _, cfg := range cfgs {
switch cfg.Type {
case "file":
fileReader, err := NewDashboardFileReader(cfg, provider.log.New("type", cfg.Type, "name", cfg.Name))
if err != nil {
return err
}
go fileReader.ReadAndListen(ctx)
default:
return fmt.Errorf("type %s is not supported", cfg.Type)
}
}
return nil
}
pkg/services/provisioning/dashboards/file_reader.go 0 → 100644
package dashboards
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/bus"
"github.com/grafana/grafana/pkg/components/simplejson"
"github.com/grafana/grafana/pkg/log"
"github.com/grafana/grafana/pkg/models"
gocache "github.com/patrickmn/go-cache"
)
type fileReader struct {
Cfg *DashboardsAsConfig
Path string
log log.Logger
dashboardRepo dashboards.Repository
cache *gocache.Cache
}
func NewDashboardFileReader(cfg *DashboardsAsConfig, log log.Logger) (*fileReader, error) {
path, ok := cfg.Options["folder"].(string)
if !ok {
return nil, fmt.Errorf("Failed to load dashboards. folder param is not a string")
}
if _, err := os.Stat(path); os.IsNotExist(err) {
log.Error("Cannot read directory", "error", err)
}
return &fileReader{
Cfg: cfg,
Path: path,
log: log,
dashboardRepo: dashboards.GetRepository(),
cache: gocache.New(5*time.Minute, 30*time.Minute),
}, nil
}
func (fr *fileReader) addCache(key string, json *dashboards.SaveDashboardItem) {
fr.cache.Add(key, json, time.Minute*10)
}
func (fr *fileReader) getCache(key string) (*dashboards.SaveDashboardItem, bool) {
obj, exist := fr.cache.Get(key)
if !exist {
return nil, exist
}
dash, ok := obj.(*dashboards.SaveDashboardItem)
if !ok {
return nil, ok
}
return dash, ok
}
func (fr *fileReader) ReadAndListen(ctx context.Context) error {
ticker := time.NewTicker(time.Second * 3)
if err := fr.walkFolder(); err != nil {
fr.log.Error("failed to search for dashboards", "error", err)
}
running := false
for {
select {
case <-ticker.C:
if !running { // avoid walking the filesystem in parallel. incase fs is very slow.
running = true
go func() {
fr.walkFolder()
running = false
}()
}
case <-ctx.Done():
return nil
}
}
}
func (fr *fileReader) walkFolder() error {
if _, err := os.Stat(fr.Path); err != nil {
if os.IsNotExist(err) {
return err
}
}
return filepath.Walk(fr.Path, func(path string, fileInfo os.FileInfo, err error) error {
if err != nil {
return err
}
if fileInfo.IsDir() {
if strings.HasPrefix(fileInfo.Name(), ".") {
return filepath.SkipDir
}
return nil
}
if !strings.HasSuffix(fileInfo.Name(), ".json") {
return nil
}
cachedDashboard, exist := fr.getCache(path)
if exist && cachedDashboard.UpdatedAt == fileInfo.ModTime() {
return nil
}
dash, err := fr.readDashboardFromFile(path)
if err != nil {
fr.log.Error("failed to load dashboard from ", "file", path, "error", err)
return nil
}
cmd := &models.GetDashboardQuery{Slug: dash.Dashboard.Slug}
err = bus.Dispatch(cmd)
// if we dont have the dashboard in the db, save it!
if err == models.ErrDashboardNotFound {
fr.log.Debug("saving new dashboard", "file", path)
_, err = fr.dashboardRepo.SaveDashboard(dash)
return err
}
if err != nil {
fr.log.Error("failed to query for dashboard", "slug", dash.Dashboard.Slug, "error", err)
return nil
}
// break if db version is newer then fil version
if cmd.Result.Updated.Unix() >= fileInfo.ModTime().Unix() {
return nil
}
fr.log.Debug("loading dashboard from disk into database.", "file", path)
_, err = fr.dashboardRepo.SaveDashboard(dash)
return err
})
}
func (fr *fileReader) readDashboardFromFile(path string) (*dashboards.SaveDashboardItem, error) {
reader, err := os.Open(path)
if err != nil {
return nil, err
}
defer reader.Close()
data, err := simplejson.NewFromReader(reader)
if err != nil {
return nil, err
}
stat, err := os.Stat(path)
if err != nil {
return nil, err
}
dash, err := createDashboardJson(data, stat.ModTime(), fr.Cfg)
if err != nil {
return nil, err
}
fr.addCache(path, dash)
return dash, nil
}
pkg/services/provisioning/dashboards/file_reader_test.go 0 → 100644
package dashboards
import (
"os"
"testing"
"time"
"github.com/grafana/grafana/pkg/bus"
"github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/log"
. "github.com/smartystreets/goconvey/convey"
)
var (
defaultDashboards string = "./test-dashboards/folder-one"
brokenDashboards string = "./test-dashboards/broken-dashboards"
oneDashboard string = "./test-dashboards/one-dashboard"
fakeRepo *fakeDashboardRepo
)
func TestDashboardFileReader(t *testing.T) {
Convey("Reading dashboards from disk", t, func() {
bus.ClearBusHandlers()
fakeRepo = &fakeDashboardRepo{}
bus.AddHandler("test", mockGetDashboardQuery)
dashboards.SetRepository(fakeRepo)
logger := log.New("test.logger")
cfg := &DashboardsAsConfig{
Name: "Default",
Type: "file",
OrgId: 1,
Folder: "",
Options: map[string]interface{}{},
}
Convey("Can read default dashboard", func() {
cfg.Options["folder"] = defaultDashboards
reader, err := NewDashboardFileReader(cfg, logger)
So(err, ShouldBeNil)
err = reader.walkFolder()
So(err, ShouldBeNil)
So(len(fakeRepo.inserted), ShouldEqual, 2)
})
Convey("Should not update dashboards when db is newer", func() {
cfg.Options["folder"] = oneDashboard
fakeRepo.getDashboard = append(fakeRepo.getDashboard, &models.Dashboard{
Updated: time.Now().Add(time.Hour),
Slug: "grafana",
})
reader, err := NewDashboardFileReader(cfg, logger)
So(err, ShouldBeNil)
err = reader.walkFolder()
So(err, ShouldBeNil)
So(len(fakeRepo.inserted), ShouldEqual, 0)
})
Convey("Can read default dashboard and replace old version in database", func() {
cfg.Options["folder"] = oneDashboard
stat, _ := os.Stat(oneDashboard + "/dashboard1.json")
fakeRepo.getDashboard = append(fakeRepo.getDashboard, &models.Dashboard{
Updated: stat.ModTime().AddDate(0, 0, -1),
Slug: "grafana",
})
reader, err := NewDashboardFileReader(cfg, logger)
So(err, ShouldBeNil)
err = reader.walkFolder()
So(err, ShouldBeNil)
So(len(fakeRepo.inserted), ShouldEqual, 1)
})
Convey("Invalid configuration should return error", func() {
cfg := &DashboardsAsConfig{
Name: "Default",
Type: "file",
OrgId: 1,
Folder: "",
}
_, err := NewDashboardFileReader(cfg, logger)
So(err, ShouldNotBeNil)
})
Convey("Broken dashboards should not cause error", func() {
cfg := &DashboardsAsConfig{
Name: "Default",
Type: "file",
OrgId: 1,
Folder: "",
Options: map[string]interface{}{
"folder": brokenDashboards,
},
}
_, err := NewDashboardFileReader(cfg, logger)
So(err, ShouldBeNil)
})
})
}
type fakeDashboardRepo struct {
inserted []*dashboards.SaveDashboardItem
getDashboard []*models.Dashboard
}
func (repo *fakeDashboardRepo) SaveDashboard(json *dashboards.SaveDashboardItem) (*models.Dashboard, error) {
repo.inserted = append(repo.inserted, json)
return json.Dashboard, nil
}
func mockGetDashboardQuery(cmd *models.GetDashboardQuery) error {
for _, d := range fakeRepo.getDashboard {
if d.Slug == cmd.Slug {
cmd.Result = d
return nil
}
}
return models.ErrDashboardNotFound
}
pkg/services/provisioning/dashboards/test-configs/broken-configs/commented.yaml 0 → 100644
# - name: 'default'
# org_id: 1
# folder: ''
# type: file
# options:
# folder: /var/lib/grafana/dashboards
pkg/services/provisioning/dashboards/test-configs/dashboards-from-disk/dev-dashboards.yaml 0 → 100644
- name: 'general dashboards'
org_id: 2
folder: 'developers'
editable: true
type: file
options:
folder: /var/lib/grafana/dashboards
- name: 'default'
type: file
options:
folder: /var/lib/grafana/dashboards
pkg/services/provisioning/dashboards/test-dashboards/folder-one/dashboard1.json 0 → 100644
{
"title": "Grafana",
"tags": [],
"style": "dark",
"timezone": "browser",
"editable": true,
"rows": [
{
"title": "New row",
"height": "150px",
"collapse": false,
"editable": true,
"panels": [
{
"id": 1,
"span": 12,
"editable": true,
"type": "text",
"mode": "html",
"content": "<div class=\"text-center\" style=\"padding-top: 15px\">\n<img src=\"img/logo_transparent_200x.png\"> \n</div>",
"style": {},
"title": "Welcome to"
}
]
},
{
"title": "Welcome to Grafana",
"height": "210px",
"collapse": false,
"editable": true,
"panels": [
{
"id": 2,
"span": 6,
"type": "text",
"mode": "html",
"content": "<br/>\n\n<div class=\"row-fluid\">\n <div class=\"span6\">\n <ul>\n <li>\n <a href=\"http://grafana.org/docs#configuration\" target=\"_blank\">Configuration</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/troubleshooting\" target=\"_blank\">Troubleshooting</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/support\" target=\"_blank\">Support</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/intro\" target=\"_blank\">Getting started</a> (Must read!)\n </li>\n </ul>\n </div>\n <div class=\"span6\">\n <ul>\n <li>\n <a href=\"http://grafana.org/docs/features/graphing\" target=\"_blank\">Graphing</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/annotations\" target=\"_blank\">Annotations</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/graphite\" target=\"_blank\">Graphite</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/influxdb\" target=\"_blank\">InfluxDB</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/opentsdb\" target=\"_blank\">OpenTSDB</a>\n </li>\n </ul>\n </div>\n</div>",
"style": {},
"title": "Documentation Links"
},
{
"id": 3,
"span": 6,
"type": "text",
"mode": "html",
"content": "<br/>\n\n<div class=\"row-fluid\">\n <div class=\"span12\">\n <ul>\n <li>Ctrl+S saves the current dashboard</li>\n <li>Ctrl+F Opens the dashboard finder</li>\n <li>Ctrl+H Hide/show row controls</li>\n <li>Click and drag graph title to move panel</li>\n <li>Hit Escape to exit graph when in fullscreen or edit mode</li>\n <li>Click the colored icon in the legend to change series color</li>\n <li>Ctrl or Shift + Click legend name to hide other series</li>\n </ul>\n </div>\n</div>\n",
"style": {},
"title": "Tips & Shortcuts"
}
]
},
{
"title": "test",
"height": "250px",
"editable": true,
"collapse": false,
"panels": [
{
"id": 4,
"span": 12,
"type": "graph",
"x-axis": true,
"y-axis": true,
"scale": 1,
"y_formats": [
"short",
"short"
],
"grid": {
"max": null,
"min": null,
"leftMax": null,
"rightMax": null,
"leftMin": null,
"rightMin": null,
"threshold1": null,
"threshold2": null,
"threshold1Color": "rgba(216, 200, 27, 0.27)",
"threshold2Color": "rgba(234, 112, 112, 0.22)"
},
"resolution": 100,
"lines": true,
"fill": 1,
"linewidth": 2,
"dashes": false,
"dashLength": 10,
"spaceLength": 10,
"points": false,
"pointradius": 5,
"bars": false,
"stack": true,
"spyable": true,
"options": false,
"legend": {
"show": true,
"values": false,
"min": false,
"max": false,
"current": false,
"total": false,
"avg": false
},
"interactive": true,
"legend_counts": true,
"timezone": "browser",
"percentage": false,
"nullPointMode": "connected",
"steppedLine": false,
"tooltip": {
"value_type": "cumulative",
"query_as_alias": true
},
"targets": [
{
"target": "randomWalk('random walk')",
"function": "mean",
"column": "value"
}
],
"aliasColors": {},
"aliasYAxis": {},
"title": "First Graph (click title to edit)",
"datasource": "graphite",
"renderer": "flot",
"annotate": {
"enable": false
}
}
]
}
],
"nav": [
{
"type": "timepicker",
"collapse": false,
"enable": true,
"status": "Stable",
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
],
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"now": true
}
],
"time": {
"from": "now-6h",
"to": "now"
},
"templating": {
"list": []
},
"version": 5
}
\ No newline at end of file
pkg/services/provisioning/dashboards/test-dashboards/folder-one/dashboard2.json 0 → 100644
{
"title": "Grafana",
"tags": [],
"style": "dark",
"timezone": "browser",
"editable": true,
"rows": [
{
"title": "New row",
"height": "150px",
"collapse": false,
"editable": true,
"panels": [
{
"id": 1,
"span": 12,
"editable": true,
"type": "text",
"mode": "html",
"content": "<div class=\"text-center\" style=\"padding-top: 15px\">\n<img src=\"img/logo_transparent_200x.png\"> \n</div>",
"style": {},
"title": "Welcome to"
}
]
},
{
"title": "Welcome to Grafana",
"height": "210px",
"collapse": false,
"editable": true,
"panels": [
{
"id": 2,
"span": 6,
"type": "text",
"mode": "html",
"content": "<br/>\n\n<div class=\"row-fluid\">\n <div class=\"span6\">\n <ul>\n <li>\n <a href=\"http://grafana.org/docs#configuration\" target=\"_blank\">Configuration</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/troubleshooting\" target=\"_blank\">Troubleshooting</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/support\" target=\"_blank\">Support</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/intro\" target=\"_blank\">Getting started</a> (Must read!)\n </li>\n </ul>\n </div>\n <div class=\"span6\">\n <ul>\n <li>\n <a href=\"http://grafana.org/docs/features/graphing\" target=\"_blank\">Graphing</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/annotations\" target=\"_blank\">Annotations</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/graphite\" target=\"_blank\">Graphite</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/influxdb\" target=\"_blank\">InfluxDB</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/opentsdb\" target=\"_blank\">OpenTSDB</a>\n </li>\n </ul>\n </div>\n</div>",
"style": {},
"title": "Documentation Links"
},
{
"id": 3,
"span": 6,
"type": "text",
"mode": "html",
"content": "<br/>\n\n<div class=\"row-fluid\">\n <div class=\"span12\">\n <ul>\n <li>Ctrl+S saves the current dashboard</li>\n <li>Ctrl+F Opens the dashboard finder</li>\n <li>Ctrl+H Hide/show row controls</li>\n <li>Click and drag graph title to move panel</li>\n <li>Hit Escape to exit graph when in fullscreen or edit mode</li>\n <li>Click the colored icon in the legend to change series color</li>\n <li>Ctrl or Shift + Click legend name to hide other series</li>\n </ul>\n </div>\n</div>\n",
"style": {},
"title": "Tips & Shortcuts"
}
]
},
{
"title": "test",
"height": "250px",
"editable": true,
"collapse": false,
"panels": [
{
"id": 4,
"span": 12,
"type": "graph",
"x-axis": true,
"y-axis": true,
"scale": 1,
"y_formats": [
"short",
"short"
],
"grid": {
"max": null,
"min": null,
"leftMax": null,
"rightMax": null,
"leftMin": null,
"rightMin": null,
"threshold1": null,
"threshold2": null,
"threshold1Color": "rgba(216, 200, 27, 0.27)",
"threshold2Color": "rgba(234, 112, 112, 0.22)"
},
"resolution": 100,
"lines": true,
"fill": 1,
"linewidth": 2,
"dashes": false,
"dashLength": 10,
"spaceLength": 10,
"points": false,
"pointradius": 5,
"bars": false,
"stack": true,
"spyable": true,
"options": false,
"legend": {
"show": true,
"values": false,
"min": false,
"max": false,
"current": false,
"total": false,
"avg": false
},
"interactive": true,
"legend_counts": true,
"timezone": "browser",
"percentage": false,
"nullPointMode": "connected",
"steppedLine": false,
"tooltip": {
"value_type": "cumulative",
"query_as_alias": true
},
"targets": [
{
"target": "randomWalk('random walk')",
"function": "mean",
"column": "value"
}
],
"aliasColors": {},
"aliasYAxis": {},
"title": "First Graph (click title to edit)",
"datasource": "graphite",
"renderer": "flot",
"annotate": {
"enable": false
}
}
]
}
],
"nav": [
{
"type": "timepicker",
"collapse": false,
"enable": true,
"status": "Stable",
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
],
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"now": true
}
],
"time": {
"from": "now-6h",
"to": "now"
},
"templating": {
"list": []
},
"version": 5
}
\ No newline at end of file
pkg/services/provisioning/dashboards/test-dashboards/one-dashboard/dashboard1.json 0 → 100644
{
"title": "Grafana",
"tags": [],
"style": "dark",
"timezone": "browser",
"editable": true,
"rows": [
{
"title": "New row",
"height": "150px",
"collapse": false,
"editable": true,
"panels": [
{
"id": 1,
"span": 12,
"editable": true,
"type": "text",
"mode": "html",
"content": "<div class=\"text-center\" style=\"padding-top: 15px\">\n<img src=\"img/logo_transparent_200x.png\"> \n</div>",
"style": {},
"title": "Welcome to"
}
]
},
{
"title": "Welcome to Grafana",
"height": "210px",
"collapse": false,
"editable": true,
"panels": [
{
"id": 2,
"span": 6,
"type": "text",
"mode": "html",
"content": "<br/>\n\n<div class=\"row-fluid\">\n <div class=\"span6\">\n <ul>\n <li>\n <a href=\"http://grafana.org/docs#configuration\" target=\"_blank\">Configuration</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/troubleshooting\" target=\"_blank\">Troubleshooting</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/support\" target=\"_blank\">Support</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/intro\" target=\"_blank\">Getting started</a> (Must read!)\n </li>\n </ul>\n </div>\n <div class=\"span6\">\n <ul>\n <li>\n <a href=\"http://grafana.org/docs/features/graphing\" target=\"_blank\">Graphing</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/annotations\" target=\"_blank\">Annotations</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/graphite\" target=\"_blank\">Graphite</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/influxdb\" target=\"_blank\">InfluxDB</a>\n </li>\n <li>\n <a href=\"http://grafana.org/docs/features/opentsdb\" target=\"_blank\">OpenTSDB</a>\n </li>\n </ul>\n </div>\n</div>",
"style": {},
"title": "Documentation Links"
},
{
"id": 3,
"span": 6,
"type": "text",
"mode": "html",
"content": "<br/>\n\n<div class=\"row-fluid\">\n <div class=\"span12\">\n <ul>\n <li>Ctrl+S saves the current dashboard</li>\n <li>Ctrl+F Opens the dashboard finder</li>\n <li>Ctrl+H Hide/show row controls</li>\n <li>Click and drag graph title to move panel</li>\n <li>Hit Escape to exit graph when in fullscreen or edit mode</li>\n <li>Click the colored icon in the legend to change series color</li>\n <li>Ctrl or Shift + Click legend name to hide other series</li>\n </ul>\n </div>\n</div>\n",
"style": {},
"title": "Tips & Shortcuts"
}
]
},
{
"title": "test",
"height": "250px",
"editable": true,
"collapse": false,
"panels": [
{
"id": 4,
"span": 12,
"type": "graph",
"x-axis": true,
"y-axis": true,
"scale": 1,
"y_formats": [
"short",
"short"
],
"grid": {
"max": null,
"min": null,
"leftMax": null,
"rightMax": null,
"leftMin": null,
"rightMin": null,
"threshold1": null,
"threshold2": null,
"threshold1Color": "rgba(216, 200, 27, 0.27)",
"threshold2Color": "rgba(234, 112, 112, 0.22)"
},
"resolution": 100,
"lines": true,
"fill": 1,
"linewidth": 2,
"dashes": false,
"dashLength": 10,
"spaceLength": 10,
"points": false,
"pointradius": 5,
"bars": false,
"stack": true,
"spyable": true,
"options": false,
"legend": {
"show": true,
"values": false,
"min": false,
"max": false,
"current": false,
"total": false,
"avg": false
},
"interactive": true,
"legend_counts": true,
"timezone": "browser",
"percentage": false,
"nullPointMode": "connected",
"steppedLine": false,
"tooltip": {
"value_type": "cumulative",
"query_as_alias": true
},
"targets": [
{
"target": "randomWalk('random walk')",
"function": "mean",
"column": "value"
}
],
"aliasColors": {},
"aliasYAxis": {},
"title": "First Graph (click title to edit)",
"datasource": "graphite",
"renderer": "flot",
"annotate": {
"enable": false
}
}
]
}
],
"nav": [
{
"type": "timepicker",
"collapse": false,
"enable": true,
"status": "Stable",
"time_options": [
"5m",
"15m",
"1h",
"6h",
"12h",
"24h",
"2d",
"7d",
"30d"
],
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
],
"now": true
}
],
"time": {
"from": "now-6h",
"to": "now"
},
"templating": {
"list": []
},
"version": 5
}
\ No newline at end of file
pkg/services/provisioning/dashboards/types.go 0 → 100644
package dashboards
import (
"strings"
"time"
"github.com/grafana/grafana/pkg/components/simplejson"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/models"
)
type DashboardsAsConfig struct {
Name string `json:"name" yaml:"name"`
Type string `json:"type" yaml:"type"`
OrgId int64 `json:"org_id" yaml:"org_id"`
Folder string `json:"folder" yaml:"folder"`
Editable bool `json:"editable" yaml:"editable"`
Options map[string]interface{} `json:"options" yaml:"options"`
}
func createDashboardJson(data *simplejson.Json, lastModified time.Time, cfg *DashboardsAsConfig) (*dashboards.SaveDashboardItem, error) {
dash := &dashboards.SaveDashboardItem{}
dash.Dashboard = models.NewDashboardFromJson(data)
dash.TitleLower = strings.ToLower(dash.Dashboard.Title)
dash.UpdatedAt = lastModified
dash.Overwrite = true
dash.OrgId = cfg.OrgId
dash.Folder = cfg.Folder
dash.Dashboard.Data.Set("editable", cfg.Editable)
if dash.Dashboard.Title == "" {
return nil, models.ErrDashboardTitleEmpty
}
return dash, nil
}
pkg/services/provisioning/datasources/datasources.go
......@@ -118,13 +118,19 @@ func (configReader) readConfig(path string) ([]*DatasourcesAsConfig, error) {
return nil, err
}
datasources = append(datasources, datasource)
if datasource != nil {
datasources = append(datasources, datasource)
}
}
}
defaultCount := 0
for _, cfg := range datasources {
for _, ds := range cfg.Datasources {
for i := range datasources {
if datasources[i].Datasources == nil {
continue
}
for _, ds := range datasources[i].Datasources {
if ds.OrgId == 0 {
ds.OrgId = 1
}
......@@ -137,7 +143,7 @@ func (configReader) readConfig(path string) ([]*DatasourcesAsConfig, error) {
}
}
for _, ds := range cfg.DeleteDatasources {
for _, ds := range datasources[i].DeleteDatasources {
if ds.OrgId == 0 {
ds.OrgId = 1
}
......
pkg/services/provisioning/datasources/test-configs/broken-yaml/commented.yaml 0 → 100644
# # list of datasources that should be deleted from the database
#delete_datasources:
# - name: Graphite
# org_id: 1
# # list of datasources to insert/update depending
# # whats available in the datbase
#datasources:
# # <string, required> name of the datasource. Required
# - name: Graphite
# # <string, required> datasource type. Required
# type: graphite
# # <string, required> access mode. direct or proxy. Required
# access: proxy
# # <int> org id. will default to org_id 1 if not specified
# org_id: 1
# # <string> url
# url: http://localhost:8080
# # <string> database password, if used
# password:
# # <string> database user, if used
# user:
# # <string> database name, if used
# database:
# # <bool> enable/disable basic auth
# basic_auth:
# # <string> basic auth username
# basic_auth_user:
# # <string> basic auth password
# basic_auth_password:
# # <bool> enable/disable with credentials headers
# with_credentials:
# # <bool> mark as default datasource. Max one per org
# is_default:
# # <map> fields that will be converted to json and stored in json_data
# json_data:
# graphiteVersion: "1.1"
# tlsAuth: true
# tlsAuthWithCACert: true
# # <string> json object of data that will be encrypted.
# secure_json_data:
# tlsCACert: "..."
# tlsClientCert: "..."
# tlsClientKey: "..."
# version: 1
# # <bool> allow users to edit datasources from the UI.
# editable: false
pkg/services/provisioning/provisioning.go
package provisioning
import (
"github.com/grafana/grafana/pkg/log"
"context"
"path"
"path/filepath"
"github.com/grafana/grafana/pkg/services/provisioning/dashboards"
"github.com/grafana/grafana/pkg/services/provisioning/datasources"
ini "gopkg.in/ini.v1"
)
var (
logger log.Logger = log.New("services.provisioning")
)
func Init(ctx context.Context, homePath string, cfg *ini.File) error {
provisioningPath := makeAbsolute(cfg.Section("paths").Key("provisioning").String(), homePath)
datasourcePath := path.Join(provisioningPath, "datasources")
if err := datasources.Provision(datasourcePath); err != nil {
return err
}
dashboardPath := path.Join(provisioningPath, "dashboards")
_, err := dashboards.Provision(ctx, dashboardPath)
if err != nil {
return err
}
return nil
}
func StartUp(datasourcePath string) error {
return datasources.Provision(datasourcePath)
func makeAbsolute(path string, root string) string {
if filepath.IsAbs(path) {
return path
}
return filepath.Join(root, path)
}
pkg/services/search/handlers_test.go
......@@ -12,6 +12,7 @@ func TestSearch(t *testing.T) {
Convey("Given search query", t, func() {
query := Query{Limit: 2000, SignedInUser: &m.SignedInUser{IsGrafanaAdmin: true}}
bus.AddHandler("test", func(query *FindPersistedDashboardsQuery) error {
query.Result = HitList{
&Hit{Id: 16, Title: "CCAA", Type: "dash-db", Tags: []string{"BB", "AA"}},
......
pkg/services/sqlstore/dashboard.go
......@@ -84,6 +84,11 @@ func SaveDashboard(cmd *m.SaveDashboardCommand) error {
} else {
dash.Version++
dash.Data.Set("version", dash.Version)
if !cmd.UpdatedAt.IsZero() {
dash.Updated = cmd.UpdatedAt
}
affectedRows, err = sess.MustCols("folder_id", "has_acl").Id(dash.Id).Update(dash)
}
......
pkg/services/sqlstore/sqlstore.go
......@@ -12,6 +12,7 @@ import (
"github.com/grafana/grafana/pkg/log"
m "github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/services/annotations"
"github.com/grafana/grafana/pkg/services/dashboards"
"github.com/grafana/grafana/pkg/services/sqlstore/migrations"
"github.com/grafana/grafana/pkg/services/sqlstore/migrator"
"github.com/grafana/grafana/pkg/setting"
......@@ -100,6 +101,7 @@ func SetEngine(engine *xorm.Engine) (err error) {
// Init repo instances
annotations.SetRepository(&SqlAnnotationRepo{})
dashboards.SetRepository(&dashboards.DashboardRepository{})
return nil
}
......
pkg/setting/setting.go
......@@ -50,12 +50,12 @@ var (
BuildStamp int64
// Paths
LogsPath string
HomePath string
DataPath string
PluginsPath string
DatasourcesPath string
CustomInitPath = "conf/custom.ini"
LogsPath string
HomePath string
DataPath string
PluginsPath string
ProvisioningPath string
CustomInitPath = "conf/custom.ini"
// Log settings.
LogModes []string
......@@ -474,8 +474,7 @@ func NewConfigContext(args *CommandLineArgs) error {
Env = Cfg.Section("").Key("app_mode").MustString("development")
InstanceName = Cfg.Section("").Key("instance_name").MustString("unknown_instance_name")
PluginsPath = makeAbsolute(Cfg.Section("paths").Key("plugins").String(), HomePath)
DatasourcesPath = makeAbsolute(Cfg.Section("paths").Key("datasources").String(), HomePath)
ProvisioningPath = makeAbsolute(Cfg.Section("paths").Key("provisioning").String(), HomePath)
server := Cfg.Section("server")
AppUrl, AppSubUrl = parseAppUrlAndSubUrl(server)
......@@ -670,6 +669,6 @@ func LogConfigurationInfo() {
logger.Info("Path Data", "path", DataPath)
logger.Info("Path Logs", "path", LogsPath)
logger.Info("Path Plugins", "path", PluginsPath)
logger.Info("Path Datasources", "path", DatasourcesPath)
logger.Info("Path Provisioning", "path", ProvisioningPath)
logger.Info("App mode " + Env)
}
pkg/social/github_oauth.go
......@@ -58,12 +58,12 @@ func (s *SocialGithub) IsTeamMember(client *http.Client) bool {
return false
}
func (s *SocialGithub) IsOrganizationMember(client *http.Client) bool {
func (s *SocialGithub) IsOrganizationMember(client *http.Client, organizationsUrl string) bool {
if len(s.allowedOrganizations) == 0 {
return true
}
organizations, err := s.FetchOrganizations(client)
organizations, err := s.FetchOrganizations(client, organizationsUrl)
if err != nil {
return false
}
......@@ -167,12 +167,12 @@ func (s *SocialGithub) HasMoreRecords(headers http.Header) (string, bool) {
}
func (s *SocialGithub) FetchOrganizations(client *http.Client) ([]string, error) {
func (s *SocialGithub) FetchOrganizations(client *http.Client, organizationsUrl string) ([]string, error) {
type Record struct {
Login string `json:"login"`
}
response, err := HttpGet(client, fmt.Sprintf(s.apiUrl+"/orgs"))
response, err := HttpGet(client, organizationsUrl)
if err != nil {
return nil, fmt.Errorf("Error getting organizations: %s", err)
}
......@@ -193,10 +193,12 @@ func (s *SocialGithub) FetchOrganizations(client *http.Client) ([]string, error)
}
func (s *SocialGithub) UserInfo(client *http.Client) (*BasicUserInfo, error) {
var data struct {
Id int `json:"id"`
Login string `json:"login"`
Email string `json:"email"`
Id int `json:"id"`
Login string `json:"login"`
Email string `json:"email"`
OrganizationsUrl string `json:"organizations_url"`
}
response, err := HttpGet(client, s.apiUrl)
......@@ -219,7 +221,7 @@ func (s *SocialGithub) UserInfo(client *http.Client) (*BasicUserInfo, error) {
return nil, ErrMissingTeamMembership
}
if !s.IsOrganizationMember(client) {
if !s.IsOrganizationMember(client, data.OrganizationsUrl) {
return nil, ErrMissingOrganizationMembership
}
......
pkg/tsdb/cloudwatch/credentials.go
......@@ -11,6 +11,7 @@ import (
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
"github.com/aws/aws-sdk-go/aws/credentials/endpointcreds"
"github.com/aws/aws-sdk-go/aws/defaults"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/cloudwatch"
......@@ -128,10 +129,10 @@ func remoteCredProvider(sess *session.Session) credentials.Provider {
func ecsCredProvider(sess *session.Session, uri string) credentials.Provider {
const host = `169.254.170.2`
c := ec2metadata.New(sess)
d := defaults.Get()
return endpointcreds.NewProviderClient(
c.Client.Config,
c.Client.Handlers,
*d.Config,
d.Handlers,
fmt.Sprintf("http://%s%s", host, uri),
func(p *endpointcreds.Provider) { p.ExpiryWindow = 5 * time.Minute })
}
......@@ -141,6 +142,11 @@ func ec2RoleProvider(sess *session.Session) credentials.Provider {
}
func (e *CloudWatchExecutor) getDsInfo(region string) *DatasourceInfo {
defaultRegion := e.DataSource.JsonData.Get("defaultRegion").MustString()
if region == "default" {
region = defaultRegion
}
authType := e.DataSource.JsonData.Get("authType").MustString()
assumeRoleArn := e.DataSource.JsonData.Get("assumeRoleArn").MustString()
accessKey := ""
......
pkg/tsdb/postgres/macros.go
......@@ -89,7 +89,7 @@ func (m *PostgresMacroEngine) evaluateMacro(name string, args []string) (string,
if err != nil {
return "", fmt.Errorf("error parsing interval %v", args[1])
}
return fmt.Sprintf("(extract(epoch from \"%s\")/%v)::bigint*%v", args[0], interval.Seconds(), interval.Seconds()), nil
return fmt.Sprintf("(extract(epoch from %s)/%v)::bigint*%v AS time", args[0], interval.Seconds(), interval.Seconds()), nil
case "__unixEpochFilter":
if len(args) == 0 {
return "", fmt.Errorf("missing time column argument for macro %v", name)
......
pkg/tsdb/postgres/macros_test.go
......@@ -45,7 +45,7 @@ func TestMacroEngine(t *testing.T) {
sql, err := engine.Interpolate(timeRange, "GROUP BY $__timeGroup(time_column,'5m')")
So(err, ShouldBeNil)
So(sql, ShouldEqual, "GROUP BY (extract(epoch from \"time_column\")/300)::bigint*300")
So(sql, ShouldEqual, "GROUP BY (extract(epoch from time_column)/300)::bigint*300 AS time")
})
Convey("interpolate __timeTo function", func() {
......
public/app/plugins/datasource/cloudwatch/datasource.js
......@@ -39,7 +39,7 @@ function (angular, _, moment, dateMath, kbn, templatingVariable) {
!!item.metricName &&
!_.isEmpty(item.statistics);
}).map(function (item) {
item.region = templateSrv.replace(item.region, options.scopedVars);
item.region = templateSrv.replace(self.getActualRegion(item.region), options.scopedVars);
item.namespace = templateSrv.replace(item.namespace, options.scopedVars);
item.metricName = templateSrv.replace(item.metricName, options.scopedVars);
item.dimensions = self.convertDimensionFormat(item.dimensions, options.scopeVars);
......@@ -165,21 +165,21 @@ function (angular, _, moment, dateMath, kbn, templatingVariable) {
this.getMetrics = function (namespace, region) {
return this.doMetricQueryRequest('metrics', {
region: templateSrv.replace(region),
region: templateSrv.replace(this.getActualRegion(region)),
namespace: templateSrv.replace(namespace)
});
};
this.getDimensionKeys = function(namespace, region) {
return this.doMetricQueryRequest('dimension_keys', {
region: templateSrv.replace(region),
region: templateSrv.replace(this.getActualRegion(region)),
namespace: templateSrv.replace(namespace)
});
};
this.getDimensionValues = function(region, namespace, metricName, dimensionKey, filterDimensions) {
return this.doMetricQueryRequest('dimension_values', {
region: templateSrv.replace(region),
region: templateSrv.replace(this.getActualRegion(region)),
namespace: templateSrv.replace(namespace),
metricName: templateSrv.replace(metricName),
dimensionKey: templateSrv.replace(dimensionKey),
......@@ -189,14 +189,14 @@ function (angular, _, moment, dateMath, kbn, templatingVariable) {
this.getEbsVolumeIds = function(region, instanceId) {
return this.doMetricQueryRequest('ebs_volume_ids', {
region: templateSrv.replace(region),
region: templateSrv.replace(this.getActualRegion(region)),
instanceId: templateSrv.replace(instanceId)
});
};
this.getEc2InstanceAttribute = function(region, attributeName, filters) {
return this.doMetricQueryRequest('ec2_instance_attribute', {
region: templateSrv.replace(region),
region: templateSrv.replace(this.getActualRegion(region)),
attributeName: templateSrv.replace(attributeName),
filters: filters
});
......@@ -267,7 +267,7 @@ function (angular, _, moment, dateMath, kbn, templatingVariable) {
period = parseInt(period, 10);
var parameters = {
prefixMatching: annotation.prefixMatching,
region: templateSrv.replace(annotation.region),
region: templateSrv.replace(this.getActualRegion(annotation.region)),
namespace: templateSrv.replace(annotation.namespace),
metricName: templateSrv.replace(annotation.metricName),
dimensions: this.convertDimensionFormat(annotation.dimensions, {}),
......@@ -341,6 +341,13 @@ function (angular, _, moment, dateMath, kbn, templatingVariable) {
return this.defaultRegion;
};
this.getActualRegion = function(region) {
if (region === 'default' || _.isEmpty(region)) {
return this.getDefaultRegion();
}
return region;
};
this.getExpandedVariables = function(target, dimensionKey, variable, templateSrv) {
/* if the all checkbox is marked we should add all values to the targets */
var allSelected = _.find(variable.options, {'selected': true, 'text': 'All'});
......
public/app/plugins/datasource/cloudwatch/query_parameter_ctrl.ts
......@@ -28,7 +28,7 @@ export class CloudWatchQueryParameterCtrl {
target.statistics = target.statistics || ['Average'];
target.dimensions = target.dimensions || {};
target.period = target.period || '';
target.region = target.region || '';
target.region = target.region || 'default';
$scope.regionSegment = uiSegmentSrv.getSegmentForValue($scope.target.region, 'select region');
$scope.namespaceSegment = uiSegmentSrv.getSegmentForValue($scope.target.namespace, 'select namespace');
......@@ -51,7 +51,7 @@ export class CloudWatchQueryParameterCtrl {
$scope.removeStatSegment = uiSegmentSrv.newSegment({fake: true, value: '-- remove stat --'});
if (_.isEmpty($scope.target.region)) {
$scope.target.region = $scope.datasource.getDefaultRegion();
$scope.target.region = 'default';
}
if (!$scope.onChange) {
......@@ -148,6 +148,10 @@ export class CloudWatchQueryParameterCtrl {
$scope.getRegions = function() {
return $scope.datasource.metricFindQuery('regions()')
.then(function(results) {
results.unshift({ text: 'default'});
return results;
})
.then($scope.transformToSegments(true));
};
......
public/app/plugins/datasource/cloudwatch/specs/datasource_specs.ts
......@@ -165,6 +165,55 @@ describe('CloudWatchDatasource', function() {
});
});
describe('When query region is "default"', function () {
it('should return the datasource region if empty or "default"', function() {
var defaultRegion = instanceSettings.jsonData.defaultRegion;
expect(ctx.ds.getActualRegion()).to.be(defaultRegion);
expect(ctx.ds.getActualRegion('')).to.be(defaultRegion);
expect(ctx.ds.getActualRegion("default")).to.be(defaultRegion);
});
it('should return the specified region if specified', function() {
expect(ctx.ds.getActualRegion('some-fake-region-1')).to.be('some-fake-region-1');
});
var requestParams;
beforeEach(function() {
ctx.ds.performTimeSeriesQuery = function(request) {
requestParams = request;
return ctx.$q.when({data: {}});
};
});
it('should query for the datasource region if empty or "default"', function(done) {
var query = {
range: { from: 'now-1h', to: 'now' },
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
region: 'default',
namespace: 'AWS/EC2',
metricName: 'CPUUtilization',
dimensions: {
InstanceId: 'i-12345678'
},
statistics: ['Average'],
period: 300
}
]
};
ctx.ds.query(query).then(function(result) {
expect(requestParams.queries[0].region).to.be(instanceSettings.jsonData.defaultRegion);
done();
});
ctx.$rootScope.$apply();
});
});
describe('When performing CloudWatch query for extended statistics', function() {
var query = {
range: { from: 'now-1h', to: 'now' },
......@@ -345,6 +394,26 @@ describe('CloudWatchDatasource', function() {
});
});
describeMetricFindQuery('dimension_values(default,AWS/EC2,CPUUtilization,InstanceId)', scenario => {
scenario.setup(() => {
scenario.requestResponse = {
results: {
metricFindQuery: {
tables: [
{ rows: [['i-12345678', 'i-12345678']] }
]
}
}
};
});
it('should call __ListMetrics and return result', () => {
expect(scenario.result[0].text).to.contain('i-12345678');
expect(scenario.request.queries[0].type).to.be('metricFindQuery');
expect(scenario.request.queries[0].subtype).to.be('dimension_values');
});
});
it('should caclculate the correct period', function () {
var hourSec = 60 * 60;
var daySec = hourSec * 24;
......
public/app/plugins/datasource/graphite/func_editor.js
......@@ -208,7 +208,7 @@ function (angular, _, $) {
if ($target.hasClass('fa-arrow-left')) {
$scope.$apply(function() {
_.move(ctrl.functions, $scope.$index, $scope.$index - 1);
_.move(ctrl.queryModel.functions, $scope.$index, $scope.$index - 1);
ctrl.targetChanged();
});
return;
......@@ -216,7 +216,7 @@ function (angular, _, $) {
if ($target.hasClass('fa-arrow-right')) {
$scope.$apply(function() {
_.move(ctrl.functions, $scope.$index, $scope.$index + 1);
_.move(ctrl.queryModel.functions, $scope.$index, $scope.$index + 1);
ctrl.targetChanged();
});
return;
......
public/app/plugins/datasource/graphite/graphite_query.ts
......@@ -46,7 +46,7 @@ export default class GraphiteQuery {
}
try {
this.parseTargetRecursive(astNode, null, 0);
this.parseTargetRecursive(astNode, null);
} catch (err) {
console.log('error parsing target:', err.message);
this.error = err.message;
......@@ -75,7 +75,7 @@ export default class GraphiteQuery {
}, "");
}
parseTargetRecursive(astNode, func, index) {
parseTargetRecursive(astNode, func) {
if (astNode === null) {
return null;
}
......@@ -83,42 +83,35 @@ export default class GraphiteQuery {
switch (astNode.type) {
case 'function':
var innerFunc = gfunc.createFuncInstance(astNode.name, { withDefaultParams: false });
_.each(astNode.params, (param, index) => {
this.parseTargetRecursive(param, innerFunc, index);
_.each(astNode.params, param => {
this.parseTargetRecursive(param, innerFunc);
});
innerFunc.updateText();
this.functions.push(innerFunc);
break;
case 'series-ref':
this.addFunctionParameter(func, astNode.value, index, this.segments.length > 0);
if (this.segments.length > 0) {
this.addFunctionParameter(func, astNode.value);
} else {
this.segments.push(astNode);
}
break;
case 'bool':
case 'string':
case 'number':
if ((index-1) >= func.def.params.length) {
throw { message: 'invalid number of parameters to method ' + func.def.name };
}
var shiftBack = this.isShiftParamsBack(func);
this.addFunctionParameter(func, astNode.value, index, shiftBack);
break;
this.addFunctionParameter(func, astNode.value);
break;
case 'metric':
if (this.segments.length > 0) {
if (astNode.segments.length !== 1) {
throw { message: 'Multiple metric params not supported, use text editor.' };
this.addFunctionParameter(func, _.join(_.map(astNode.segments, 'value'), '.'));
} else {
this.segments = astNode.segments;
}
this.addFunctionParameter(func, astNode.segments[0].value, index, true);
break;
}
this.segments = astNode.segments;
}
}
isShiftParamsBack(func) {
return func.def.name !== 'seriesByTag';
}
updateSegmentValue(segment, index) {
this.segments[index].value = segment.value;
}
......@@ -127,6 +120,10 @@ export default class GraphiteQuery {
this.segments.push({value: "select metric"});
}
hasSelectMetric() {
return this.segments[this.segments.length - 1].value === 'select metric';
}
addFunction(newFunc) {
this.functions.push(newFunc);
this.moveAliasFuncLast();
......@@ -145,11 +142,11 @@ export default class GraphiteQuery {
}
}
addFunctionParameter(func, value, index, shiftBack) {
if (shiftBack) {
index = Math.max(index - 1, 0);
addFunctionParameter(func, value) {
if (func.params.length >= func.def.params.length) {
throw { message: 'too many parameters for function ' + func.def.name };
}
func.params[index] = value;
func.params.push(value);
}
removeFunction(func) {
......@@ -159,7 +156,7 @@ export default class GraphiteQuery {
updateModelTarget(targets) {
// render query
if (!this.target.textEditor) {
var metricPath = this.getSegmentPathUpTo(this.segments.length);
var metricPath = this.getSegmentPathUpTo(this.segments.length).replace(/\.select metric$/, '');
this.target.target = _.reduce(this.functions, wrapFunction, metricPath);
}
......
public/app/plugins/datasource/graphite/query_ctrl.ts
......@@ -62,6 +62,10 @@ export class GraphiteQueryCtrl extends QueryCtrl {
}
checkOtherSegments(fromIndex) {
if (this.queryModel.segments.length === 1 && this.queryModel.segments[0].type === 'series-ref') {
return;
}
if (fromIndex === 0) {
this.addSelectMetricSegment();
return;
......@@ -108,8 +112,23 @@ export class GraphiteQueryCtrl extends QueryCtrl {
if (altSegments.length === 0) { return altSegments; }
// add query references
if (index === 0) {
_.eachRight(this.panelCtrl.panel.targets, target => {
if (target.refId === this.queryModel.target.refId) {
return;
}
altSegments.unshift(this.uiSegmentSrv.newSegment({
type: 'series-ref',
value: '#' + target.refId,
expandable: false,
}));
});
}
// add template variables
_.each(this.templateSrv.variables, variable => {
_.eachRight(this.templateSrv.variables, variable => {
altSegments.unshift(this.uiSegmentSrv.newSegment({
type: 'template',
value: '$' + variable.name,
......@@ -199,11 +218,8 @@ export class GraphiteQueryCtrl extends QueryCtrl {
var oldTarget = this.queryModel.target.target;
this.updateModelTarget();
if (this.queryModel.target !== oldTarget) {
var lastSegment = this.segments.length > 0 ? this.segments[this.segments.length - 1] : {};
if (lastSegment.value !== 'select metric') {
this.panelCtrl.refresh();
}
if (this.queryModel.target !== oldTarget && !this.queryModel.hasSelectMetric()) {
this.panelCtrl.refresh();
}
}
......
public/app/plugins/datasource/graphite/specs/query_ctrl_specs.ts
......@@ -95,11 +95,11 @@ describe('GraphiteQueryCtrl', function() {
});
it('should not add select metric segment', function() {
expect(ctx.ctrl.segments.length).to.be(0);
expect(ctx.ctrl.segments.length).to.be(1);
});
it('should add both series refs as params', function() {
expect(ctx.ctrl.queryModel.functions[0].params.length).to.be(2);
it('should add second series ref as param', function() {
expect(ctx.ctrl.queryModel.functions[0].params.length).to.be(1);
});
});
......@@ -170,7 +170,7 @@ describe('GraphiteQueryCtrl', function() {
describe('when updating targets with nested query', function() {
beforeEach(function() {
ctx.ctrl.target.target = 'scaleToSeconds(#A)';
ctx.ctrl.target.target = 'scaleToSeconds(#A, 60)';
ctx.ctrl.datasource.metricFindQuery = sinon.stub().returns(ctx.$q.when([{expandable: false}]));
ctx.ctrl.parseTarget();
......@@ -183,11 +183,11 @@ describe('GraphiteQueryCtrl', function() {
});
it('target should remain the same', function() {
expect(ctx.ctrl.target.target).to.be('scaleToSeconds(#A)');
expect(ctx.ctrl.target.target).to.be('scaleToSeconds(#A, 60)');
});
it('targetFull should include nexted queries', function() {
expect(ctx.ctrl.target.targetFull).to.be('scaleToSeconds(nested.query.count)');
expect(ctx.ctrl.target.targetFull).to.be('scaleToSeconds(nested.query.count, 60)');
});
});
......
public/app/plugins/datasource/postgres/partials/query.editor.html
......@@ -50,11 +50,11 @@ Macros:
- $__timeEpoch -&gt; extract(epoch from column) as "time"
- $__timeFilter(column) -&gt; extract(epoch from column) BETWEEN 1492750877 AND 1492750877
- $__unixEpochFilter(column) -&gt; column &gt; 1492750877 AND column &lt; 1492750877
- $__timeGroup(column,'5m') -&gt; (extract(epoch from "dateColumn")/300)::bigint*300
- $__timeGroup(column,'5m') -&gt; (extract(epoch from column)/300)::bigint*300 AS time
Example of group by and order by with $__timeGroup:
SELECT
$__timeGroup(date_time_col, '1h') AS time,
$__timeGroup(date_time_col, '1h'),
sum(value) as value
FROM yourtable
GROUP BY time
......
scripts/grunt/release_task.js
......@@ -26,7 +26,7 @@ module.exports = function(grunt) {
});
grunt.config('copy.backend_files', {
expand: true,
src: ['conf/*', 'vendor/phantomjs/*', 'scripts/*'],
src: ['conf/**', 'vendor/phantomjs/*', 'scripts/*'],
options: { mode: true},
dest: '<%= tempDir %>'
});
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment