Skip to content

Commit 00dc441

Browse files
authored
Merge pull request #6 from dbsystel/feature-redshift-data-api
Feature redshift data api
2 parents 7525ede + 74e8e7c commit 00dc441

27 files changed

Lines changed: 1009 additions & 411 deletions

README.md

Lines changed: 36 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,40 @@
11
# Terraform Provider for AWS Redshift
22

3-
This provider allows to manage with Terraform [AWS Redshift](https://aws.amazon.com/redshift/) objects like users, groups, schemas, etc..
3+
This provider allows to manage with Terraform [AWS Redshift](https://aws.amazon.com/redshift/) objects like users, groups, schemas, etc...
44

55
It's published on the [Terraform registry](https://registry.terraform.io/providers/dbsystel/redshift/latest/docs).
66

77
## Requirements
88

9-
- [Terraform](https://www.terraform.io/downloads.html) >= 1.0
10-
- [Go](https://golang.org/doc/install) 1.21 (to build the provider plugin)
9+
- [Terraform](https://www.terraform.io/downloads.html) >= 1.0
10+
- [Go](https://golang.org/doc/install) 1.24 (to build the provider plugin)
11+
12+
## Limitations
13+
14+
### Untested features
15+
16+
Due to limited testing capacities, the following features are not tested/stable yet:
17+
18+
* External Schemas
19+
* Hive Database
20+
* RDS Postgres Database
21+
* RDS MySQL Database
22+
* Redshift Database
23+
* Temporary Credentials Cluster Identifier
24+
* Temporary Credentials Assume Role
25+
* Datashares
26+
27+
### Using the AWS Redshift Data API
28+
29+
This provider *does* support connecting to the Redshift instance using the AWS Redshift Data API. However, this is not
30+
the default behavior, requires some additional configuration and comes along with some caveats:
31+
32+
* Transactions are not run as real DB-level transactions, but rather as a sequence of individual statements (`BatchExecuteStatement` executes all statements at once and does not support queries while being in transaction mode).
33+
* Due to the unsupported state of transactions, interfering DB interactions might lead to unexpected results.
34+
* In order to
35+
prevent [errors due to conflicts with concurrent transactions](https://stackoverflow.com/questions/37344942/redshift-could-not-complete-because-of-conflict-with-concurrent-transaction),
36+
all statements depend on one lock across resources. This may lead to longer execution times, especially when multiple
37+
resources are created or updated at the same time.
1138

1239
## Building The Provider
1340

@@ -21,14 +48,15 @@ Enter the provider directory and build the provider
2148
$ cd terraform-provider-redshift
2249
$ make build
2350
```
51+
2452
## Development
2553

2654
If you're new to provider development, a good place to start is the [Extending
2755
Terraform](https://www.terraform.io/docs/extend/index.html) docs.
2856

2957
### Running Tests
3058

31-
Acceptance tests require a running real AWS Redshift cluster.
59+
Acceptance tests require a running real AWS Redshift cluster.
3260

3361
```sh
3462
REDSHIFT_HOST=<cluster ip or DNS>
@@ -39,6 +67,7 @@ make testacc
3967
```
4068

4169
If your cluster is only accessible from within the VPC, you can connect via a socks proxy:
70+
4271
```sh
4372
ALL_PROXY=socks5[h]://[<socks-user>:<socks-password>@]<socks-host>[:<socks-port>]
4473
NO_PROXY=127.0.0.1,192.168.0.0/24,*.example.com,localhost
@@ -58,8 +87,9 @@ Use `go generate` to update generated docs.
5887

5988
## Releasing
6089

61-
Builds and releases are automated with GitHub Actions and [GoReleaser](https://github.com/goreleaser/goreleaser/).
62-
The changelog is managed with [github-changelog-generator](https://github.com/github-changelog-generator/github-changelog-generator).
90+
Builds and releases are automated with GitHub Actions and [GoReleaser](https://github.com/goreleaser/goreleaser/).
91+
The changelog is managed
92+
with [github-changelog-generator](https://github.com/github-changelog-generator/github-changelog-generator).
6393

6494
Currently there are a few manual steps to this:
6595

docs/index.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,8 @@ The Redshift provider provides configuration management resources for
1212

1313
## Example Usage
1414

15+
Please note that only one authentication method can be used at a time. There is no logic to fall back to another method if the first one fails.
16+
1517
### Authentication using fixed password
1618

1719
```terraform
@@ -22,6 +24,18 @@ provider "redshift" {
2224
}
2325
```
2426

27+
### Authentication using Redshift Data API
28+
29+
```terraform
30+
provider "redshift" {
31+
database = var.redshift_database
32+
data_api {
33+
workgroup_name = var.redshift_workgroup
34+
region = var.aws_region
35+
}
36+
}
37+
```
38+
2539
### Authentication using temporary credentials
2640

2741
```terraform
@@ -54,6 +68,7 @@ provider "redshift" {
5468

5569
### Optional
5670

71+
- `data_api` (Block List, Max: 1) Configuration for using the Redshift Data API. This can only be used for serverless Redshift clusters. (see [below for nested schema](#nestedblock--data_api))
5772
- `database` (String) The name of the database to connect to. The default is `redshift`.
5873
- `host` (String) Name of Redshift server address to connect to.
5974
- `max_connections` (Number) Maximum number of connections to establish to the database. Zero means unlimited.
@@ -63,6 +78,15 @@ provider "redshift" {
6378
- `temporary_credentials` (Block List, Max: 1) Configuration for obtaining a temporary password using redshift:GetClusterCredentials (see [below for nested schema](#nestedblock--temporary_credentials))
6479
- `username` (String) Redshift user name to connect as.
6580

81+
<a id="nestedblock--data_api"></a>
82+
### Nested Schema for `data_api`
83+
84+
Required:
85+
86+
- `region` (String) The AWS region where the Redshift Serverless workgroup is located. If not specified, the region will be determined from the AWS SDK configuration.
87+
- `workgroup_name` (String) The name of the Redshift Serverless workgroup to connect to.
88+
89+
6690
<a id="nestedblock--temporary_credentials"></a>
6791
### Nested Schema for `temporary_credentials`
6892

docs/resources/group.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,8 @@ resource "redshift_group" "staff" {
4141

4242
Import is supported using the following syntax:
4343

44+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
45+
4446
```shell
4547
# Import group with grosysid: SELECT grosysid FROM pg_group WHERE groname = 'mygroup'
4648

docs/resources/schema.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,8 @@ Optional:
279279

280280
Import is supported using the following syntax:
281281

282+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
283+
282284
```shell
283285
# Import schema with oid: SELECT oid FROM pg_catalog.pg_namespace WHERE nspname = 'myschema';
284286

docs/resources/user.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ resource "redshift_user" "user_with_unrestricted_syslog" {
5050

5151
Import is supported using the following syntax:
5252

53+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
54+
5355
```shell
5456
# Import user with usesysid: SELECT usesysid FROM pg_user_info WHERE usename = 'mememe'
5557

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
provider "redshift" {
2+
database = "exampledb"
3+
data_api {
4+
workgroup_name = "example-workgroup"
5+
region = "us-west-2"
6+
}
7+
}

go.mod

Lines changed: 32 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,20 @@
11
module github.com/dbsystel/terraform-provider-redshift
22

3-
go 1.23.7
3+
go 1.24
44

55
toolchain go1.24.5
66

77
require (
8-
github.com/aws/aws-sdk-go-v2 v1.37.0
9-
github.com/aws/aws-sdk-go-v2/config v1.30.1
10-
github.com/aws/aws-sdk-go-v2/credentials v1.18.1
11-
github.com/aws/aws-sdk-go-v2/service/redshift v1.55.0
12-
github.com/aws/aws-sdk-go-v2/service/sts v1.35.0
8+
github.com/aws/aws-sdk-go-v2 v1.38.0
9+
github.com/aws/aws-sdk-go-v2/config v1.31.0
10+
github.com/aws/aws-sdk-go-v2/credentials v1.18.4
11+
github.com/aws/aws-sdk-go-v2/service/redshift v1.57.0
12+
github.com/aws/aws-sdk-go-v2/service/sts v1.37.0
1313
github.com/hashicorp/terraform-plugin-docs v0.22.0
1414
github.com/hashicorp/terraform-plugin-sdk/v2 v2.37.0
1515
github.com/lib/pq v1.10.9
16-
golang.org/x/net v0.42.0
16+
github.com/mmichaelb/redshift-data-sql-driver v0.4.0
17+
golang.org/x/net v0.43.0
1718
)
1819

1920
require (
@@ -23,22 +24,23 @@ require (
2324
github.com/Masterminds/semver/v3 v3.2.0 // indirect
2425
github.com/Masterminds/sprig/v3 v3.2.3 // indirect
2526
github.com/ProtonMail/go-crypto v1.1.6 // indirect
26-
github.com/agext/levenshtein v1.2.2 // indirect
27+
github.com/agext/levenshtein v1.2.3 // indirect
2728
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
2829
github.com/armon/go-radix v1.0.0 // indirect
29-
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.0 // indirect
30-
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.0 // indirect
31-
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.0 // indirect
30+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.3 // indirect
31+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.3 // indirect
32+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.3 // indirect
3233
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.3 // indirect
3334
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.0 // indirect
34-
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.0 // indirect
35-
github.com/aws/aws-sdk-go-v2/service/sso v1.26.0 // indirect
36-
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.31.0 // indirect
35+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.3 // indirect
36+
github.com/aws/aws-sdk-go-v2/service/redshiftdata v1.36.0 // indirect
37+
github.com/aws/aws-sdk-go-v2/service/sso v1.28.0 // indirect
38+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.33.0 // indirect
3739
github.com/aws/smithy-go v1.22.5 // indirect
3840
github.com/bgentry/speakeasy v0.1.0 // indirect
3941
github.com/bmatcuk/doublestar/v4 v4.8.1 // indirect
4042
github.com/cloudflare/circl v1.6.1 // indirect
41-
github.com/fatih/color v1.16.0 // indirect
43+
github.com/fatih/color v1.18.0 // indirect
4244
github.com/golang/protobuf v1.5.4 // indirect
4345
github.com/google/go-cmp v0.7.0 // indirect
4446
github.com/google/uuid v1.6.0 // indirect
@@ -49,31 +51,31 @@ require (
4951
github.com/hashicorp/go-cty v1.5.0 // indirect
5052
github.com/hashicorp/go-hclog v1.6.3 // indirect
5153
github.com/hashicorp/go-multierror v1.1.1 // indirect
52-
github.com/hashicorp/go-plugin v1.6.3 // indirect
54+
github.com/hashicorp/go-plugin v1.7.0 // indirect
5355
github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
5456
github.com/hashicorp/go-uuid v1.0.3 // indirect
5557
github.com/hashicorp/go-version v1.7.0 // indirect
5658
github.com/hashicorp/hc-install v0.9.2 // indirect
57-
github.com/hashicorp/hcl/v2 v2.23.0 // indirect
59+
github.com/hashicorp/hcl/v2 v2.24.0 // indirect
5860
github.com/hashicorp/logutils v1.0.0 // indirect
5961
github.com/hashicorp/terraform-exec v0.23.0 // indirect
6062
github.com/hashicorp/terraform-json v0.25.0 // indirect
61-
github.com/hashicorp/terraform-plugin-go v0.27.0 // indirect
63+
github.com/hashicorp/terraform-plugin-go v0.28.0 // indirect
6264
github.com/hashicorp/terraform-plugin-log v0.9.0 // indirect
63-
github.com/hashicorp/terraform-registry-address v0.2.5 // indirect
65+
github.com/hashicorp/terraform-registry-address v0.3.0 // indirect
6466
github.com/hashicorp/terraform-svchost v0.1.1 // indirect
65-
github.com/hashicorp/yamux v0.1.1 // indirect
67+
github.com/hashicorp/yamux v0.1.2 // indirect
6668
github.com/huandu/xstrings v1.3.3 // indirect
6769
github.com/imdario/mergo v0.3.15 // indirect
6870
github.com/mattn/go-colorable v0.1.14 // indirect
6971
github.com/mattn/go-isatty v0.0.20 // indirect
7072
github.com/mattn/go-runewidth v0.0.9 // indirect
7173
github.com/mitchellh/copystructure v1.2.0 // indirect
7274
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
73-
github.com/mitchellh/go-wordwrap v1.0.0 // indirect
75+
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
7476
github.com/mitchellh/mapstructure v1.5.0 // indirect
7577
github.com/mitchellh/reflectwalk v1.0.2 // indirect
76-
github.com/oklog/run v1.0.0 // indirect
78+
github.com/oklog/run v1.2.0 // indirect
7779
github.com/posener/complete v1.2.3 // indirect
7880
github.com/shopspring/decimal v1.3.1 // indirect
7981
github.com/spf13/cast v1.5.0 // indirect
@@ -84,17 +86,17 @@ require (
8486
github.com/yuin/goldmark-meta v1.1.0 // indirect
8587
github.com/zclconf/go-cty v1.16.3 // indirect
8688
go.abhg.dev/goldmark/frontmatter v0.2.0 // indirect
87-
golang.org/x/crypto v0.40.0 // indirect
89+
golang.org/x/crypto v0.41.0 // indirect
8890
golang.org/x/exp v0.0.0-20230626212559-97b1e661b5df // indirect
89-
golang.org/x/mod v0.25.0 // indirect
91+
golang.org/x/mod v0.27.0 // indirect
9092
golang.org/x/sync v0.16.0 // indirect
91-
golang.org/x/sys v0.34.0 // indirect
92-
golang.org/x/text v0.27.0 // indirect
93-
golang.org/x/tools v0.34.0 // indirect
93+
golang.org/x/sys v0.35.0 // indirect
94+
golang.org/x/text v0.28.0 // indirect
95+
golang.org/x/tools v0.36.0 // indirect
9496
google.golang.org/appengine v1.6.8 // indirect
95-
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a // indirect
96-
google.golang.org/grpc v1.72.1 // indirect
97-
google.golang.org/protobuf v1.36.6 // indirect
97+
google.golang.org/genproto/googleapis/rpc v0.0.0-20250811230008-5f3141c8851a // indirect
98+
google.golang.org/grpc v1.74.2 // indirect
99+
google.golang.org/protobuf v1.36.7 // indirect
98100
gopkg.in/yaml.v2 v2.3.0 // indirect
99101
gopkg.in/yaml.v3 v3.0.1 // indirect
100102
)

0 commit comments

Comments
 (0)