Skip to content

N
nexpie-grafana-theme
Commit 957c88ea by Arve Knudsen Committed by GitHub
Options

CloudWatch: Re-implement authentication (#25548) 

* CloudWatch: Revisit authentication

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Simplify auth code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Use ARN

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Add Drone configuration

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove unused code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove .drone.yml

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix external ID usage

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Fix issues after merge

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove stale code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove stale code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Use auth type enum

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test snapshot

* Coordinate frontend and backend option names

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove old comments

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix front-end tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Introduce session cache

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Use constants

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix field alignment

* CloudWatch: Fix log message

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Tidy go.mod

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Handle arn auth type

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Fix role assumption duration

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Inline unnecessary constants

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Use serial comma in UI

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Inline unnecessary constants

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Fail if missing region

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Handle unconfigured region

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Log when using cached session

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Include region in cache key

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Add UI warnings for lecagy support

* Do not clear ARN fields whenging change authentication provider

* Graph NG: annotations display (#27972)

* Annotations support POC

* Fix markers memoization

* dev dashboard update

* Update public/app/plugins/panel/graph3/plugins/AnnotationsPlugin.tsx

* CloudWatch: Remove errors.BadRequest

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Undo unintentional change

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove log line

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix cache key computation

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Add region to cache key

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve log messages

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CloudWatch: Add documentation

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve tooltip

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve docs

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve docs

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve docs

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve tooltip

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Add role assumption provisioning example

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Add upgrade notes

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve docs

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Apply suggestions from code review

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* backend: use latest sdk (#28147)

fixes #27713 via https://github.com/grafana/grafana-plugin-sdk-go/pull/227

* Docs: Update Permissions documentation (#28144)

* removed overview.md

* content updates

* Update datasource_permissions.md

* update content

* content updates

* Update organization_roles.md

* Update docs/sources/enterprise/saml.md

Co-authored-by: Kyle Brandt <kyle@grafana.com>

* Update dashboard_folder_permissions.md

Co-authored-by: Kyle Brandt <kyle@grafana.com>

* area/grafana/toolkit: ci-package needs to use synchronous writes (#28148)

* ci needs to use synchronous writes or the file ends up with zero length

* <Enterprise Docs> Add instructions to upload license via UI (#28067)

* Add UI license upload option, reformat Enterprise license activation section

Added the option to upload a license file through the Server Admin UI, and did a little reformatting to make license activation look more like a process.

* Headers not bold, hyphens not asterisks

* Github: run metrics collector workflow every 10min (#28153)

* GithubActions: Updated cron schedule

* Updated

* Docs: Update explore docs: remove dot at the end of line (#28151)

HI - Removed Dot(.) at the end of line to make it consistent with other 2 points.

Thanks,
Ashish

* Fix frontend tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix frontend tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Docs: Update upgrade notes

Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
Co-authored-by: Brian Gann <briangann@users.noreply.github.com>
Co-authored-by: Mitch Seaman <mjseaman@users.noreply.github.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
Co-authored-by: ashishagarwal06 <34888589+ashishagarwal06@users.noreply.github.com>
parent 519ec93c
Showing with 428 additions and 361 deletions
CHANGELOG.md
# 7.3.0-beta1 (2020-10-14)
### Breaking changes
- **CloudWatch**: The AWS CloudWatch data source's authentication scheme has changed. See the [upgrade notes](https://grafana.com/docs/grafana/latest/installation/upgrading/#upgrading-to-v73) for details and how this may affect you.
# 7.2.1 (2020-10-08)
### Features / Enhancements
......
docs/sources/administration/provisioning.md
......@@ -140,7 +140,7 @@ Please refer to each datasource documentation for specific provisioning examples
| ------------- | ---------------------------------------------------------------------------------- |
| Elasticsearch | Elasticsearch uses the `database` property to configure the index for a datasource |
#### Json Data
#### JSON Data
Since not all datasources have the same configuration settings we only have the most common ones as fields. The rest should be stored as a json blob in the `jsonData` field. Here are the most common settings that the core datasources use.
......@@ -157,11 +157,12 @@ Since not all datasources have the same configuration settings we only have the
| interval | string | Elasticsearch | Index date time format. nil(No Pattern), 'Hourly', 'Daily', 'Weekly', 'Monthly' or 'Yearly' |
| logMessageField | string | Elasticsearch | Which field should be used as the log message |
| logLevelField | string | Elasticsearch | Which field should be used to indicate the priority of the log message |
| authType | string | Cloudwatch | Auth provider. keys/credentials/arn |
| assumeRoleArn | string | Cloudwatch | ARN of Assume Role |
| defaultRegion | string | Cloudwatch | AWS region |
| authType | string | Cloudwatch | Auth provider. default/credentials/keys |
| externalId | string | Cloudwatch | Optional External ID |
| assumeRoleArn | string | Cloudwatch | Optional ARN role to assume |
| defaultRegion | string | Cloudwatch | Optional default AWS region |
| customMetricsNamespaces | string | Cloudwatch | Namespaces of Custom Metrics |
| profile | string | Cloudwatch | Custom credentials profile |
| profile | string | Cloudwatch | Optional credentials profile |
| tsdbVersion | string | OpenTSDB | Version |
| tsdbResolution | string | OpenTSDB | Resolution |
| sslmode | string | PostgreSQL | SSLmode. 'disable', 'require', 'verify-ca' or 'verify-full' |
......
docs/sources/datasources/cloudwatch.md
......@@ -31,23 +31,28 @@ build dashboards or use Explore with CloudWatch metrics and CloudWatch Logs.
| _Default_ | Default data source means that it will be pre-selected for new panels. |
| _Default Region_ | Used in query editor to set region (can be changed on per query basis) |
| _Custom Metrics namespace_ | Specify the CloudWatch namespace of Custom metrics |
| _Auth Provider_ | Specify the provider to get credentials. |
| _Credentials_ profile name | Specify the name of the profile to use (if you use `~/.aws/credentials` file), leave blank for default. |
| _Assume Role Arn_ | Specify the ARN of the role to assume |
| _Authentication Provider_ | Specify the authentication method. |
| _Credentials Profile Name_ | If you use "Credentials file" for _Authentication Provider_, optionally specify a non-default profile. |
| _Assume Role ARN_ | Optionally specify the ARN of a role to assume. |
| _External ID_ | If you are assuming a role in another account, that has been created with an external ID, specify the external ID here. |
## Authentication
### IAM Roles
### AWS credentials
Currently all access to CloudWatch is done server side by the Grafana backend using the official AWS SDK. If your Grafana
server is running on AWS you can use IAM Roles and authentication will be handled automatically.
There are three different authentication methods available. `AWS SDK Default` performs no custom configuration at all and instead uses the [default provider](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html) as specified by the AWS SDK for Go. This requires you to configure your AWS credentials separately, such as if you've [configured the CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html), if you're [running on an EC2 instance](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), [in an ECS task](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html) or for a [Service Account in a Kubernetes cluster](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html).
See the AWS documentation on [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
`Credentials file` corresponds directly to the [SharedCredentialsProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#SharedCredentialsProvider) provider in the Go SDK. In short, it will read the AWS shared credentials file and find the given profile. While `AWS SDK Default` will also find the shared credentials file, this option allows you to specify which profile to use without using environment variables. It doesn't have any implicit fallbacks to other credential providers, and will fail if using credentials from the credentials file doesn't work.
> **Note:** [AWS Role Switching](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-cli.html) is not supported at the moment.
`Access & secret key` corresponds to the [StaticProvider](https://docs.aws.amazon.com/sdk-for-go/api/aws/credentials/#StaticProvider) and uses the given access key ID and secret key to authenticate. This method doesn't have any fallbacks, and will fail if the provided key pair doesn't work.
## IAM Policies
### IAM roles
Currently all access to CloudWatch is done server side by the Grafana backend using the official AWS SDK. Providing you have chosen the _AWS SDK Default_ authentication method, and your Grafana server is running on AWS, you can use IAM Roles to handle authentication automically.
See the AWS documentation on [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
### IAM policies
Grafana needs permissions granted via IAM to be able to read CloudWatch metrics
and EC2 tags/instances/regions. You can attach these permissions to IAM roles and
......@@ -101,22 +106,26 @@ Here is a minimal policy example:
}
```
### AWS credentials
### Assuming a role
If Auth Provider is `Credentials file`, Grafana tries to get credentials in the following order.
The `Assume Role ARN` field allows you to specify which IAM role to assume, if any. When left blank, the provided credentials are used directly and the associated role or user should have the required permissions. If this field is non-blank, on the other hand, the provided credentials are used to perform an [sts:AssumeRole](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) call.
- Environment variables. (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`)
- Hard-code credentials.
- Shared credentials file.
- IAM role for Amazon EC2.
### EKS IAM roles for service accounts
See the AWS documentation on [Configuring the AWS SDK for Go](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html)
The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. In order to allow user 472 to access the credentials (and avoid it falling back to the IAM role attached to the EC2 instance), you will need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod.
```
securityContext:
fsGroup: 472
runAsUser: 472
runAsGroup: 472
```
### AWS credentials file
Create a file at `~/.aws/credentials`. That is the `HOME` path for user running grafana-server.
> **Note:** If you think you have the credentials file in the right place but it is still not working then you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions.
> **Note:** If you think you have the credentials file in the right place and it is still not working, you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions.
Example content:
......@@ -372,13 +381,25 @@ It's now possible to configure data sources using config files with Grafana's pr
Here are some provisioning examples for this data source.
### Using AWS SDK Default
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
defaultRegion: eu-west-2
```
### Using credentials profile name (non-default)
```yaml
apiVersion: 1
datasources:
- name: Cloudwatch
- name: CloudWatch
type: cloudwatch
jsonData:
authType: credentials
......@@ -393,7 +414,7 @@ datasources:
apiVersion: 1
datasources:
- name: Cloudwatch
- name: CloudWatch
type: cloudwatch
jsonData:
authType: keys
......@@ -402,3 +423,16 @@ datasources:
accessKey: '<your access key>'
secretKey: '<your secret key>'
```
### Using AWS SDK Default and ARN of IAM Role to Assume
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
assumeRoleArn: arn:aws:iam::123456789012:root
defaultRegion: eu-west-2
```
docs/sources/installation/upgrading.md
......@@ -277,3 +277,22 @@ For existing alert notification channels, there is no automatic migration of sto
> Please note that when migrating a notification channel and later downgrading Grafana to an earlier version, the notification channel will not be able to read stored sensitive settings and, as a result, not function as expected.
For provisioning of alert notification channels, refer to [Alert notification channels]({{< relref "../administration/provisioning.md#alert-notification-channels" >}}).
## Upgrading to v7.3
### AWS CloudWatch data source
The AWS CloudWatch data source's authentication scheme has changed in Grafana 7.3. Most importantly the authentication method _ARN_ has been removed, and a new one has been added: _AWS SDK Default_. Existing data source configurations using the former will fallback to the latter. Assuming an IAM role will still work though, and the old _ARN_ method would use the default AWS SDK authentication method under the hood anyway.
Since _ARN_ has been removed as an authentication method, we have instead made it into an option for providing the ARN of an IAM role to assume. This works independently of the authentication method you choose.
The new authentication method, _AWS SDK Default_, uses the default AWS Go SDK credential chain, which at the time of writing looks for credentials in the following order:
1. Environment variables.
1. Shared credentials file.
1. If your application uses an ECS task definition or RunTask API operation, IAM role for tasks.
1. If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2.
The other authentication methods, _Access & secret key_ and _Credentials file_, have changed in regards to fallbacks. If these methods fail, they no longer fallback to other methods. e.g. environment variables. If you want fallbacks, you should use _AWS SDK Default_ instead.
For more information and details, please refer to [Using AWS CloudWatch in Grafana]({{< relref "../datasources/cloudwatch.md#authentication" >}}).
go.mod
......@@ -36,7 +36,6 @@ require (
github.com/go-sql-driver/mysql v1.5.0
github.com/go-stack/stack v1.8.0
github.com/gobwas/glob v0.2.3
github.com/golang/mock v1.4.3
github.com/golang/protobuf v1.4.2
github.com/google/go-cmp v0.5.0
github.com/gosimple/slug v1.4.2
......
pkg/tsdb/cloudwatch/cloudwatch.go
......@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"regexp"
"strings"
"sync"
"time"
......@@ -11,6 +12,8 @@ import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/client"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/cloudwatch"
......@@ -31,8 +34,8 @@ import (
type datasourceInfo struct {
Profile string
Region string
AuthType string
AssumeRoleArn string
AuthType authType
AssumeRoleARN string
ExternalID string
Namespace string
......@@ -74,16 +77,105 @@ type cloudWatchExecutor struct {
func (e *cloudWatchExecutor) newSession(region string) (*session.Session, error) {
dsInfo := e.getDSInfo(region)
creds, err := getCredentials(dsInfo)
bldr := strings.Builder{}
for i, s := range []string{
dsInfo.AuthType.String(), dsInfo.AccessKey, dsInfo.Profile, dsInfo.AssumeRoleARN, region,
} {
if i != 0 {
bldr.WriteString(":")
}
bldr.WriteString(strings.ReplaceAll(s, ":", `\:`))
}
cacheKey := bldr.String()
sessCacheLock.RLock()
if env, ok := sessCache[cacheKey]; ok {
if env.expiration.After(time.Now().UTC()) {
sessCacheLock.RUnlock()
return env.session, nil
}
}
sessCacheLock.RUnlock()
cfgs := []*aws.Config{
{
CredentialsChainVerboseErrors: aws.Bool(true),
},
}
var regionCfg *aws.Config
if dsInfo.Region == defaultRegion {
plog.Warn("Region is set to \"default\", which is unsupported")
dsInfo.Region = ""
}
if dsInfo.Region != "" {
regionCfg = &aws.Config{Region: aws.String(dsInfo.Region)}
cfgs = append(cfgs, regionCfg)
}
switch dsInfo.AuthType {
case authTypeSharedCreds:
plog.Debug("Authenticating towards AWS with shared credentials", "profile", dsInfo.Profile,
"region", dsInfo.Region)
cfgs = append(cfgs, &aws.Config{
Credentials: credentials.NewSharedCredentials("", dsInfo.Profile),
})
case authTypeKeys:
plog.Debug("Authenticating towards AWS with an access key pair", "region", dsInfo.Region)
cfgs = append(cfgs, &aws.Config{
Credentials: credentials.NewStaticCredentials(dsInfo.AccessKey, dsInfo.SecretKey, ""),
})
case authTypeDefault:
plog.Debug("Authenticating towards AWS with default SDK method", "region", dsInfo.Region)
default:
panic(fmt.Sprintf("Unrecognized authType: %d", dsInfo.AuthType))
}
sess, err := newSession(cfgs...)
if err != nil {
return nil, err
}
cfg := &aws.Config{
Region: aws.String(dsInfo.Region),
Credentials: creds,
duration := stscreds.DefaultDuration
expiration := time.Now().Add(duration)
if dsInfo.AssumeRoleARN != "" {
// We should assume a role in AWS
plog.Debug("Trying to assume role in AWS", "arn", dsInfo.AssumeRoleARN)
cfgs := []*aws.Config{
{
CredentialsChainVerboseErrors: aws.Bool(true),
},
{
Credentials: newSTSCredentials(sess, dsInfo.AssumeRoleARN, func(p *stscreds.AssumeRoleProvider) {
// Not sure if this is necessary, overlaps with p.Duration and is undocumented
p.Expiry.SetExpiration(expiration, 0)
p.Duration = duration
if dsInfo.ExternalID != "" {
p.ExternalID = aws.String(dsInfo.ExternalID)
}
}),
},
}
if regionCfg != nil {
cfgs = append(cfgs, regionCfg)
}
sess, err = newSession(cfgs...)
if err != nil {
return nil, err
}
}
return newSession(cfg)
plog.Debug("Successfully created AWS session")
sessCacheLock.Lock()
sessCache[cacheKey] = envelope{
session: sess,
expiration: expiration,
}
sessCacheLock.Unlock()
return sess, nil
}
func (e *cloudWatchExecutor) getCWClient(region string) (cloudwatchiface.CloudWatchAPI, error) {
......@@ -282,18 +374,54 @@ func (e *cloudWatchExecutor) executeLogAlertQuery(ctx context.Context, queryCont
return response, nil
}
type authType int
const (
authTypeDefault authType = iota
authTypeSharedCreds
authTypeKeys
)
func (at authType) String() string {
switch at {
case authTypeDefault:
return "default"
case authTypeSharedCreds:
return "sharedCreds"
case authTypeKeys:
return "keys"
default:
panic(fmt.Sprintf("Unrecognized auth type %d", at))
}
}
func (e *cloudWatchExecutor) getDSInfo(region string) *datasourceInfo {
if region == defaultRegion {
region = e.DataSource.JsonData.Get("defaultRegion").MustString()
}
authType := e.DataSource.JsonData.Get("authType").MustString()
assumeRoleArn := e.DataSource.JsonData.Get("assumeRoleArn").MustString()
atStr := e.DataSource.JsonData.Get("authType").MustString()
assumeRoleARN := e.DataSource.JsonData.Get("assumeRoleArn").MustString()
externalID := e.DataSource.JsonData.Get("externalId").MustString()
decrypted := e.DataSource.DecryptedValues()
accessKey := decrypted["accessKey"]
secretKey := decrypted["secretKey"]
at := authTypeDefault
switch atStr {
case "credentials":
at = authTypeSharedCreds
case "keys":
at = authTypeKeys
case "default":
at = authTypeDefault
case "arn":
at = authTypeDefault
plog.Warn("Authentication type \"arn\" is deprecated, falling back to default")
default:
plog.Warn("Unrecognized AWS authentication type", "type", atStr)
}
profile := e.DataSource.JsonData.Get("profile").MustString()
if profile == "" {
profile = e.DataSource.Database // legacy support
......@@ -302,8 +430,8 @@ func (e *cloudWatchExecutor) getDSInfo(region string) *datasourceInfo {
return &datasourceInfo{
Region: region,
Profile: profile,
AuthType: authType,
AssumeRoleArn: assumeRoleArn,
AuthType: at,
AssumeRoleARN: assumeRoleARN,
ExternalID: externalID,
AccessKey: accessKey,
SecretKey: secretKey,
......
pkg/tsdb/cloudwatch/credentials.go deleted 100644 → 0
package cloudwatch
import (
"fmt"
"os"
"sync"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/client"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
"github.com/aws/aws-sdk-go/aws/credentials/endpointcreds"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/defaults"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/sts"
"github.com/aws/aws-sdk-go/service/sts/stsiface"
)
type envelope struct {
credentials *credentials.Credentials
expiration *time.Time
}
var awsCredsCache = map[string]envelope{}
var credsCacheLock sync.RWMutex
// Session factory.
// Stubbable by tests.
//nolint:gocritic
var newSession = func(cfgs ...*aws.Config) (*session.Session, error) {
return session.NewSession(cfgs...)
}
// STS service factory.
// Stubbable by tests.
//nolint:gocritic
var newSTSService = func(p client.ConfigProvider, cfgs ...*aws.Config) stsiface.STSAPI {
return sts.New(p, cfgs...)
}
// EC2Metadata service factory.
// Stubbable by tests.
//nolint:gocritic
var newEC2Metadata = func(p client.ConfigProvider, cfgs ...*aws.Config) *ec2metadata.EC2Metadata {
return ec2metadata.New(p, cfgs...)
}
func getCredentials(dsInfo *datasourceInfo) (*credentials.Credentials, error) {
cacheKey := fmt.Sprintf("%s:%s:%s:%s", dsInfo.AuthType, dsInfo.AccessKey, dsInfo.Profile, dsInfo.AssumeRoleArn)
credsCacheLock.RLock()
if env, ok := awsCredsCache[cacheKey]; ok {
if env.expiration != nil && env.expiration.After(time.Now().UTC()) {
result := env.credentials
credsCacheLock.RUnlock()
return result, nil
}
}
credsCacheLock.RUnlock()
accessKeyID := ""
secretAccessKey := ""
sessionToken := ""
var expiration *time.Time = nil
if dsInfo.AuthType == "arn" {
params := &sts.AssumeRoleInput{
RoleArn: aws.String(dsInfo.AssumeRoleArn),
RoleSessionName: aws.String("GrafanaSession"),
DurationSeconds: aws.Int64(900),
}
if dsInfo.ExternalID != "" {
params.ExternalId = aws.String(dsInfo.ExternalID)
}
stsSess, err := newSession()
if err != nil {
return nil, err
}
stsCreds := credentials.NewChainCredentials(
[]credentials.Provider{
&credentials.EnvProvider{},
&credentials.SharedCredentialsProvider{Filename: "", Profile: dsInfo.Profile},
webIdentityProvider(stsSess),
remoteCredProvider(stsSess),
})
stsConfig := &aws.Config{
Region: aws.String(dsInfo.Region),
Credentials: stsCreds,
}
sess, err := newSession(stsConfig)
if err != nil {
return nil, err
}
svc := newSTSService(sess, stsConfig)
resp, err := svc.AssumeRole(params)
if err != nil {
return nil, err
}
if resp.Credentials != nil {
accessKeyID = *resp.Credentials.AccessKeyId
secretAccessKey = *resp.Credentials.SecretAccessKey
sessionToken = *resp.Credentials.SessionToken
expiration = resp.Credentials.Expiration
}
} else {
now := time.Now()
e := now.Add(5 * time.Minute)
expiration = &e
}
sess, err := newSession()
if err != nil {
return nil, err
}
creds := credentials.NewChainCredentials(
[]credentials.Provider{
&credentials.StaticProvider{Value: credentials.Value{
AccessKeyID: accessKeyID,
SecretAccessKey: secretAccessKey,
SessionToken: sessionToken,
}},
&credentials.EnvProvider{},
&credentials.StaticProvider{Value: credentials.Value{
AccessKeyID: dsInfo.AccessKey,
SecretAccessKey: dsInfo.SecretKey,
}},
&credentials.SharedCredentialsProvider{Filename: "", Profile: dsInfo.Profile},
webIdentityProvider(sess),
remoteCredProvider(sess),
})
credsCacheLock.Lock()
awsCredsCache[cacheKey] = envelope{
credentials: creds,
expiration: expiration,
}
credsCacheLock.Unlock()
return creds, nil
}
func webIdentityProvider(sess client.ConfigProvider) credentials.Provider {
svc := newSTSService(sess)
roleARN := os.Getenv("AWS_ROLE_ARN")
tokenFilepath := os.Getenv("AWS_WEB_IDENTITY_TOKEN_FILE")
roleSessionName := os.Getenv("AWS_ROLE_SESSION_NAME")
return stscreds.NewWebIdentityRoleProvider(svc, roleARN, roleSessionName, tokenFilepath)
}
func remoteCredProvider(sess *session.Session) credentials.Provider {
ecsCredURI := os.Getenv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI")
if len(ecsCredURI) > 0 {
return ecsCredProvider(sess, ecsCredURI)
}
return ec2RoleProvider(sess)
}
func ecsCredProvider(sess *session.Session, uri string) credentials.Provider {
const host = `169.254.170.2`
d := defaults.Get()
return endpointcreds.NewProviderClient(
*d.Config,
d.Handlers,
fmt.Sprintf("http://%s%s", host, uri),
func(p *endpointcreds.Provider) { p.ExpiryWindow = 5 * time.Minute })
}
func ec2RoleProvider(sess client.ConfigProvider) credentials.Provider {
return &ec2rolecreds.EC2RoleProvider{Client: newEC2Metadata(sess), ExpiryWindow: 5 * time.Minute}
}
pkg/tsdb/cloudwatch/credentials_test.go deleted 100644 → 0
package cloudwatch
import (
"os"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/client"
"github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
"github.com/aws/aws-sdk-go/aws/credentials/endpointcreds"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/sts"
"github.com/aws/aws-sdk-go/service/sts/stsiface"
"github.com/golang/mock/gomock"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mock_stsiface"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestECSCredProvider(t *testing.T) {
os.Setenv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI", "/abc/123")
t.Cleanup(func() {
os.Unsetenv("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI")
})
sess, err := session.NewSession()
require.NoError(t, err)
provider := remoteCredProvider(sess)
require.NotNil(t, provider)
ecsProvider, ok := provider.(*endpointcreds.Provider)
require.NotNil(t, ecsProvider)
require.True(t, ok)
assert.Equal(t, "http://169.254.170.2/abc/123", ecsProvider.Client.Endpoint)
}
func TestDefaultEC2RoleProvider(t *testing.T) {
sess, err := session.NewSession()
require.NoError(t, err)
provider := remoteCredProvider(sess)
require.NotNil(t, provider)
ec2Provider, ok := provider.(*ec2rolecreds.EC2RoleProvider)
require.NotNil(t, ec2Provider)
require.True(t, ok)
}
func TestGetCredentials_ARNAuthType(t *testing.T) {
ctrl := gomock.NewController(t)
var stsMock *mock_stsiface.MockSTSAPI
origNewSession := newSession
origNewSTSService := newSTSService
origNewEC2Metadata := newEC2Metadata
t.Cleanup(func() {
newSession = origNewSession
newSTSService = origNewSTSService
newEC2Metadata = origNewEC2Metadata
})
newSession = func(cfgs ...*aws.Config) (*session.Session, error) {
return &session.Session{}, nil
}
newSTSService = func(p client.ConfigProvider, cfgs ...*aws.Config) stsiface.STSAPI {
return stsMock
}
newEC2Metadata = func(p client.ConfigProvider, cfgs ...*aws.Config) *ec2metadata.EC2Metadata {
return nil
}
t.Run("Without external ID", func(t *testing.T) {
stsMock = mock_stsiface.NewMockSTSAPI(ctrl)
stsMock.
EXPECT().
AssumeRole(gomock.Eq(&sts.AssumeRoleInput{
RoleArn: aws.String(""),
DurationSeconds: aws.Int64(900),
RoleSessionName: aws.String("GrafanaSession"),
})).
Return(&sts.AssumeRoleOutput{
Credentials: &sts.Credentials{
AccessKeyId: aws.String("id"),
SecretAccessKey: aws.String("secret"),
SessionToken: aws.String("token"),
},
}, nil).
Times(1)
creds, err := getCredentials(&datasourceInfo{
AuthType: "arn",
})
require.NoError(t, err)
require.NotNil(t, creds)
})
t.Run("With external ID", func(t *testing.T) {
stsMock = mock_stsiface.NewMockSTSAPI(ctrl)
stsMock.
EXPECT().
AssumeRole(gomock.Eq(&sts.AssumeRoleInput{
RoleArn: aws.String(""),
DurationSeconds: aws.Int64(900),
RoleSessionName: aws.String("GrafanaSession"),
ExternalId: aws.String("external-id"),
})).
Return(&sts.AssumeRoleOutput{
Credentials: &sts.Credentials{
AccessKeyId: aws.String("id"),
SecretAccessKey: aws.String("secret"),
SessionToken: aws.String("token"),
},
}, nil).
Times(1)
creds, err := getCredentials(&datasourceInfo{
AuthType: "arn",
ExternalID: "external-id",
})
require.NoError(t, err)
require.NotNil(t, creds)
})
}
pkg/tsdb/cloudwatch/session.go 0 → 100644
package cloudwatch
import (
"sync"
"time"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
)
type envelope struct {
session *session.Session
expiration time.Time
}
var sessCache = map[string]envelope{}
var sessCacheLock sync.RWMutex
// Session factory.
// Stubbable by tests.
//nolint:gocritic
var newSession = func(cfgs ...*aws.Config) (*session.Session, error) {
return session.NewSession(cfgs...)
}
// STS credentials factory.
// Stubbable by tests.
//nolint:gocritic
var newSTSCredentials = stscreds.NewCredentials
// EC2Metadata service factory.
// Stubbable by tests.
//nolint:gocritic
var newEC2Metadata = ec2metadata.New
pkg/tsdb/cloudwatch/session_test.go 0 → 100644
package cloudwatch
import (
"reflect"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/client"
"github.com/aws/aws-sdk-go/aws/credentials"
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
"github.com/aws/aws-sdk-go/aws/ec2metadata"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// Test cloudWatchExecutor.newSession with assumption of IAM role.
func TestNewSession_AssumeRole(t *testing.T) {
origNewSession := newSession
origNewSTSCredentials := newSTSCredentials
origNewEC2Metadata := newEC2Metadata
t.Cleanup(func() {
newSession = origNewSession
newSTSCredentials = origNewSTSCredentials
newEC2Metadata = origNewEC2Metadata
})
newSession = func(cfgs ...*aws.Config) (*session.Session, error) {
cfg := aws.Config{}
cfg.MergeIn(cfgs...)
return &session.Session{
Config: &cfg,
}, nil
}
newSTSCredentials = func(c client.ConfigProvider, roleARN string,
options ...func(*stscreds.AssumeRoleProvider)) *credentials.Credentials {
p := &stscreds.AssumeRoleProvider{
RoleARN: roleARN,
}
for _, o := range options {
o(p)
}
return credentials.NewCredentials(p)
}
newEC2Metadata = func(p client.ConfigProvider, cfgs ...*aws.Config) *ec2metadata.EC2Metadata {
return nil
}
duration := stscreds.DefaultDuration
t.Run("Without external ID", func(t *testing.T) {
t.Cleanup(func() {
sessCache = map[string]envelope{}
})
const roleARN = "test"
e := newExecutor()
e.DataSource = fakeDataSource(fakeDataSourceCfg{
assumeRoleARN: roleARN,
})
sess, err := e.newSession(defaultRegion)
require.NoError(t, err)
require.NotNil(t, sess)
expCreds := credentials.NewCredentials(&stscreds.AssumeRoleProvider{
RoleARN: roleARN,
Duration: duration,
})
diff := cmp.Diff(expCreds, sess.Config.Credentials, cmp.Exporter(func(_ reflect.Type) bool {
return true
}), cmpopts.IgnoreFields(stscreds.AssumeRoleProvider{}, "Expiry"))
assert.Empty(t, diff)
})
t.Run("With external ID", func(t *testing.T) {
t.Cleanup(func() {
sessCache = map[string]envelope{}
})
const roleARN = "test"
const externalID = "external"
e := newExecutor()
e.DataSource = fakeDataSource(fakeDataSourceCfg{
assumeRoleARN: roleARN,
externalID: externalID,
})
sess, err := e.newSession(defaultRegion)
require.NoError(t, err)
require.NotNil(t, sess)
expCreds := credentials.NewCredentials(&stscreds.AssumeRoleProvider{
RoleARN: roleARN,
ExternalID: aws.String(externalID),
Duration: duration,
})
diff := cmp.Diff(expCreds, sess.Config.Credentials, cmp.Exporter(func(_ reflect.Type) bool {
return true
}), cmpopts.IgnoreFields(stscreds.AssumeRoleProvider{}, "Expiry"))
assert.Empty(t, diff)
})
}
pkg/tsdb/cloudwatch/test_utils.go
......@@ -18,9 +18,23 @@ import (
"github.com/grafana/grafana/pkg/models"
)
func fakeDataSource() *models.DataSource {
type fakeDataSourceCfg struct {
assumeRoleARN string
externalID string
}
func fakeDataSource(cfgs ...fakeDataSourceCfg) *models.DataSource {
jsonData := simplejson.New()
jsonData.Set("defaultRegion", "default")
jsonData.Set("defaultRegion", defaultRegion)
jsonData.Set("authType", "default")
for _, cfg := range cfgs {
if cfg.assumeRoleARN != "" {
jsonData.Set("assumeRoleArn", cfg.assumeRoleARN)
}
if cfg.externalID != "" {
jsonData.Set("externalId", cfg.externalID)
}
}
return &models.DataSource{
Id: 1,
Database: "default",
......
public/app/plugins/datasource/cloudwatch/components/ConfigEditor.test.tsx
......@@ -78,7 +78,7 @@ describe('Render', () => {
expect(wrapper).toMatchSnapshot();
});
it('should should show credentials profile name field', () => {
it('should show credentials profile name field', () => {
const wrapper = setup({
jsonData: {
authType: 'credentials',
......@@ -87,7 +87,7 @@ describe('Render', () => {
expect(wrapper).toMatchSnapshot();
});
it('should should show access key and secret access key fields', () => {
it('should show access key and secret access key fields', () => {
const wrapper = setup({
jsonData: {
authType: 'keys',
......@@ -96,7 +96,7 @@ describe('Render', () => {
expect(wrapper).toMatchSnapshot();
});
it('should should show arn role field', () => {
it('should show arn role field', () => {
const wrapper = setup({
jsonData: {
authType: 'arn',
......
public/app/plugins/datasource/cloudwatch/components/ConfigEditor.tsx
......@@ -2,6 +2,7 @@ import React, { PureComponent } from 'react';
import { InlineFormLabel, LegacyForms, Button } from '@grafana/ui';
const { Select, Input } = LegacyForms;
import {
AppEvents,
DataSourcePluginOptionsEditorProps,
onUpdateDatasourceJsonDataOptionSelect,
onUpdateDatasourceResetOption,
......@@ -13,11 +14,12 @@ import { getDatasourceSrv } from 'app/features/plugins/datasource_srv';
import { CloudWatchDatasource } from '../datasource';
import { CloudWatchJsonData, CloudWatchSecureJsonData } from '../types';
import { CancelablePromise, makePromiseCancelable } from 'app/core/utils/CancelablePromise';
import { appEvents } from 'app/core/core';
const authProviderOptions = [
{ label: 'AWS SDK Default', value: 'default' },
{ label: 'Access & secret key', value: 'keys' },
{ label: 'Credentials file', value: 'credentials' },
{ label: 'ARN', value: 'arn' },
] as SelectableValue[];
export type Props = DataSourcePluginOptionsEditorProps<CloudWatchJsonData, CloudWatchSecureJsonData>;
......@@ -44,6 +46,22 @@ export class ConfigEditor extends PureComponent<Props, State> {
console.warn('Cloud Watch ConfigEditor has unmounted, initialization was canceled');
}
});
if (this.props.options.jsonData.authType === 'arn') {
appEvents.emit(AppEvents.alertWarning, [
'Since grafana 7.3 authentication type "arn" is deprecated, falling back to default SDK provider',
]);
} else if (
this.props.options.jsonData.authType === 'credentials' &&
!this.props.options.jsonData.profile &&
!this.props.options.jsonData.database
) {
appEvents.emit(AppEvents.alertWarning, [
'As of grafana 7.3 authentication type "credentials" should be used only for shared file credentials. \
If you don\'t have a credentials file, switch to the default SDK provider for extracting credentials \
from environment variables or IAM roles',
]);
}
}
componentWillUnmount() {
......@@ -125,17 +143,18 @@ export class ConfigEditor extends PureComponent<Props, State> {
<div className="gf-form-group">
<div className="gf-form-inline">
<div className="gf-form">
<InlineFormLabel className="width-14">Auth Provider</InlineFormLabel>
<InlineFormLabel
className="width-14"
tooltip="Specify which AWS credentials chain to use. AWS SDK Default is the recommended option for EKS, ECS, or if you've attached an IAM role to your EC2 instance."
>
Authentication Provider
</InlineFormLabel>
<Select
className="width-30"
value={authProviderOptions.find(authProvider => authProvider.value === options.jsonData.authType)}
options={authProviderOptions}
defaultValue={options.jsonData.authType}
onChange={option => {
if (options.jsonData.authType === 'arn' && option.value !== 'arn') {
delete this.props.options.jsonData.assumeRoleArn;
delete this.props.options.jsonData.externalId;
}
onUpdateDatasourceJsonDataOptionSelect(this.props, 'authType')(option);
}}
/>
......@@ -229,21 +248,24 @@ export class ConfigEditor extends PureComponent<Props, State> {
)}
</div>
)}
{options.jsonData.authType === 'arn' && (
<div className="gf-form-inline">
<div className="gf-form">
<InlineFormLabel className="width-14" tooltip="ARN of Assume Role">
Assume Role ARN
</InlineFormLabel>
<div className="width-30">
<Input
className="width-30"
placeholder="arn:aws:iam:*"
value={options.jsonData.assumeRoleArn || ''}
onChange={onUpdateDatasourceJsonDataOption(this.props, 'assumeRoleArn')}
/>
</div>
<div className="gf-form-inline">
<div className="gf-form">
<InlineFormLabel
className="width-14"
tooltip="Optionally, specify the ARN of a role to assume. Specifying a role here will ensure that the selected authentication provider is used to assume the specified role rather than using the credentials directly. Leave blank if you don't need to assume a role at all"
>
Assume Role ARN
</InlineFormLabel>
<div className="width-30">
<Input
className="width-30"
placeholder="arn:aws:iam:*"
value={options.jsonData.assumeRoleArn || ''}
onChange={onUpdateDatasourceJsonDataOption(this.props, 'assumeRoleArn')}
/>
</div>
</div>
<div className="gf-form-inline">
<div className="gf-form">
<InlineFormLabel
className="width-14"
......@@ -261,7 +283,7 @@ export class ConfigEditor extends PureComponent<Props, State> {
</div>
</div>
</div>
)}
</div>
<div className="gf-form-inline">
<div className="gf-form">
<InlineFormLabel
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment