Skip to content

Commit 5a6f783

Browse files
support Dolt (testcontainers#2177)
* /modules/dolt: wip, kinda working * /modules/dolt: get tests passing * /{.github,.vscode,docs,mkdocs,modules,sonar-project}: use modulegen tool * /modules/dolt/{dolt.go,examples_test.go}: run linter * /modules/dolt/{dolt.go,examples_test.go}: add methods for cloning * /{docs, modules}: add with creds file * /{docs,modules}: pr feedback, cleanup * /modules/dolt/examples_test.go: remove panics, lint * chore: run mod tidy * chore: include MustConnectionString method * chore: do not use named returns * chore: perform initialisation before the container has started --------- Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
1 parent ca2ea86 commit 5a6f783

16 files changed

+980
-2
lines changed

.github/workflows/ci.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ jobs:
9494
matrix:
9595
go-version: [1.21.x, 1.x]
9696
platform: [ubuntu-latest]
97-
module: [artemis, cassandra, chroma, clickhouse, cockroachdb, compose, consul, couchbase, elasticsearch, gcloud, inbucket, influxdb, k3s, k6, kafka, localstack, mariadb, milvus, minio, mockserver, mongodb, mssql, mysql, nats, neo4j, ollama, openfga, openldap, opensearch, postgres, pulsar, qdrant, rabbitmq, redis, redpanda, registry, surrealdb, vault, weaviate]
97+
module: [artemis, cassandra, chroma, clickhouse, cockroachdb, compose, consul, couchbase, dolt, elasticsearch, gcloud, inbucket, influxdb, k3s, k6, kafka, localstack, mariadb, milvus, minio, mockserver, mongodb, mssql, mysql, nats, neo4j, ollama, openfga, openldap, opensearch, postgres, pulsar, qdrant, rabbitmq, redis, redpanda, registry, surrealdb, vault, weaviate]
9898
uses: ./.github/workflows/ci-test-go.yml
9999
with:
100100
go-version: ${{ matrix.go-version }}

.vscode/.testcontainers-go.code-workspace

+4
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,10 @@
4545
"name": "module / couchbase",
4646
"path": "../modules/couchbase"
4747
},
48+
{
49+
"name": "module / dolt",
50+
"path": "../modules/dolt"
51+
},
4852
{
4953
"name": "module / elasticsearch",
5054
"path": "../modules/elasticsearch"

docs/modules/dolt.md

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Dolt
2+
3+
Not available until the next release of testcontainers-go <a href="https://github.com/testcontainers/testcontainers-go"><span class="tc-version">:material-tag: main</span></a>
4+
5+
## Introduction
6+
7+
The Testcontainers module for Dolt.
8+
9+
## Adding this module to your project dependencies
10+
11+
Please run the following command to add the Dolt module to your Go dependencies:
12+
13+
```
14+
go get github.com/testcontainers/testcontainers-go/modules/dolt
15+
```
16+
17+
## Usage example
18+
19+
<!--codeinclude-->
20+
[Creating a Dolt container](../../modules/dolt/examples_test.go) inside_block:runDoltContainer
21+
<!--/codeinclude-->
22+
23+
## Module reference
24+
25+
The Dolt module exposes one entrypoint function to create the Dolt container, and this function receives two parameters:
26+
27+
```golang
28+
func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*DoltContainer, error)
29+
```
30+
31+
- `context.Context`, the Go context.
32+
- `testcontainers.ContainerCustomizer`, a variadic argument for passing options.
33+
34+
### Container Options
35+
36+
When starting the Dolt container, you can pass options in a variadic way to configure it.
37+
38+
#### Image
39+
40+
If you need to set a different Dolt Docker image, you can use `testcontainers.WithImage` with a valid Docker image
41+
for Dolt. E.g. `testcontainers.WithImage("dolthub/dolt-sql-server:1.32.4")`.
42+
43+
{% include "../features/common_functional_options.md" %}
44+
45+
#### Set username, password and database name
46+
47+
If you need to set a different database, and its credentials, you can use `WithUsername`, `WithPassword`, `WithDatabase`
48+
options.
49+
50+
!!!info
51+
The default values for the username is `root`, for password is `test` and for the default database name is `test`.
52+
53+
#### Init Scripts
54+
55+
If you would like to perform DDL or DML operations in the Dolt container, add one or more `*.sql`, `*.sql.gz`, or `*.sh`
56+
scripts to the container request, using the `WithScripts(scriptPaths ...string)`. Those files will be copied under `/docker-entrypoint-initdb.d`.
57+
58+
#### Clone from remotes
59+
60+
If you would like to clone data from a remote into the Dolt container, add an `*.sh`
61+
scripts to the container request, using the `WithScripts(scriptPaths ...string)`. Additionally, use `WithDoltCloneRemoteUrl(url string)` to specify
62+
the remote to clone, and use `WithDoltCredsPublicKey(key string)` along with `WithCredsFile(credsFile string)` to authorize the Dolt container to clone from the remote.
63+
64+
<!--codeinclude-->
65+
[Example of Clone script](../../modules/dolt/testdata/clone-db.sh)
66+
<!--/codeinclude-->
67+
68+
#### Custom configuration
69+
70+
If you need to set a custom configuration, you can use `WithConfigFile` option to pass the path to a custom configuration file.
71+
72+
### Container Methods
73+
74+
#### ConnectionString
75+
76+
This method returns the connection string to connect to the Dolt container, using the default `3306` port.
77+
It's possible to pass extra parameters to the connection string, e.g. `tls=skip-verify` or `application_name=myapp`, in a variadic way.
78+
79+
<!--codeinclude-->
80+
[Get connection string](../../modules/dolt/dolt_test.go) inside_block:connectionString
81+
<!--/codeinclude-->

mkdocs.yml

+1
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ nav:
7171
- modules/cockroachdb.md
7272
- modules/consul.md
7373
- modules/couchbase.md
74+
- modules/dolt.md
7475
- modules/elasticsearch.md
7576
- modules/gcloud.md
7677
- modules/inbucket.md

modules/dolt/Makefile

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
include ../../commons-test.mk
2+
3+
.PHONY: test
4+
test:
5+
$(MAKE) test-dolt

modules/dolt/dolt.go

+251
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
package dolt
2+
3+
import (
4+
"context"
5+
"database/sql"
6+
"fmt"
7+
"path/filepath"
8+
"strings"
9+
10+
"github.com/testcontainers/testcontainers-go"
11+
"github.com/testcontainers/testcontainers-go/wait"
12+
)
13+
14+
const (
15+
rootUser = "root"
16+
defaultUser = "test"
17+
defaultPassword = "test"
18+
defaultDatabaseName = "test"
19+
)
20+
21+
const defaultImage = "dolthub/dolt-sql-server:1.32.4"
22+
23+
// DoltContainer represents the Dolt container type used in the module
24+
type DoltContainer struct {
25+
testcontainers.Container
26+
username string
27+
password string
28+
database string
29+
}
30+
31+
func WithDefaultCredentials() testcontainers.CustomizeRequestOption {
32+
return func(req *testcontainers.GenericContainerRequest) {
33+
username := req.Env["DOLT_USER"]
34+
if strings.EqualFold(rootUser, username) {
35+
delete(req.Env, "DOLT_USER")
36+
delete(req.Env, "DOLT_PASSWORD")
37+
}
38+
}
39+
}
40+
41+
// RunContainer creates an instance of the Dolt container type
42+
func RunContainer(ctx context.Context, opts ...testcontainers.ContainerCustomizer) (*DoltContainer, error) {
43+
req := testcontainers.ContainerRequest{
44+
Image: defaultImage,
45+
ExposedPorts: []string{"3306/tcp", "33060/tcp"},
46+
Env: map[string]string{
47+
"DOLT_USER": defaultUser,
48+
"DOLT_PASSWORD": defaultPassword,
49+
"DOLT_DATABASE": defaultDatabaseName,
50+
},
51+
WaitingFor: wait.ForLog("Server ready. Accepting connections."),
52+
}
53+
54+
genericContainerReq := testcontainers.GenericContainerRequest{
55+
ContainerRequest: req,
56+
Started: true,
57+
}
58+
59+
opts = append(opts, WithDefaultCredentials())
60+
61+
for _, opt := range opts {
62+
opt.Customize(&genericContainerReq)
63+
}
64+
65+
createUser := true
66+
username, ok := req.Env["DOLT_USER"]
67+
if !ok {
68+
username = rootUser
69+
createUser = false
70+
}
71+
password := req.Env["DOLT_PASSWORD"]
72+
73+
database := req.Env["DOLT_DATABASE"]
74+
if database == "" {
75+
database = defaultDatabaseName
76+
}
77+
78+
if len(password) == 0 && password == "" && !strings.EqualFold(rootUser, username) {
79+
return nil, fmt.Errorf("empty password can be used only with the root user")
80+
}
81+
82+
container, err := testcontainers.GenericContainer(ctx, genericContainerReq)
83+
if err != nil {
84+
return nil, err
85+
}
86+
87+
dc := &DoltContainer{container, username, password, database}
88+
89+
// dolthub/dolt-sql-server does not create user or database, so we do so here
90+
err = dc.initialize(ctx, createUser)
91+
return dc, err
92+
}
93+
94+
func (c *DoltContainer) initialize(ctx context.Context, createUser bool) error {
95+
connectionString, err := c.initialConnectionString(ctx)
96+
if err != nil {
97+
return err
98+
}
99+
100+
var db *sql.DB
101+
db, err = sql.Open("mysql", connectionString)
102+
if err != nil {
103+
return err
104+
}
105+
defer func() {
106+
rerr := db.Close()
107+
if err == nil {
108+
err = rerr
109+
}
110+
}()
111+
112+
if err = db.Ping(); err != nil {
113+
return fmt.Errorf("error pinging db: %w", err)
114+
}
115+
116+
// create database
117+
_, err = db.Exec(fmt.Sprintf("CREATE DATABASE IF NOT EXISTS %s;", c.database))
118+
if err != nil {
119+
return fmt.Errorf("error creating database %s: %w", c.database, err)
120+
}
121+
122+
if createUser {
123+
// create user
124+
_, err = db.Exec(fmt.Sprintf("CREATE USER IF NOT EXISTS '%s' IDENTIFIED BY '%s';", c.username, c.password))
125+
if err != nil {
126+
return fmt.Errorf("error creating user %s: %w", c.username, err)
127+
}
128+
129+
q := fmt.Sprintf("GRANT ALL ON %s.* TO '%s';", c.database, c.username)
130+
// grant user privileges
131+
_, err = db.Exec(q)
132+
if err != nil {
133+
return fmt.Errorf("error creating user %s: %w", c.username, err)
134+
}
135+
}
136+
137+
return nil
138+
}
139+
140+
func (c *DoltContainer) initialConnectionString(ctx context.Context) (string, error) {
141+
containerPort, err := c.MappedPort(ctx, "3306/tcp")
142+
if err != nil {
143+
return "", err
144+
}
145+
146+
host, err := c.Host(ctx)
147+
if err != nil {
148+
return "", err
149+
}
150+
151+
connectionString := fmt.Sprintf("root:@tcp(%s:%s)/", host, containerPort.Port())
152+
return connectionString, nil
153+
}
154+
155+
func (c *DoltContainer) MustConnectionString(ctx context.Context, args ...string) string {
156+
addr, err := c.ConnectionString(ctx, args...)
157+
if err != nil {
158+
panic(err)
159+
}
160+
return addr
161+
}
162+
163+
func (c *DoltContainer) ConnectionString(ctx context.Context, args ...string) (string, error) {
164+
containerPort, err := c.MappedPort(ctx, "3306/tcp")
165+
if err != nil {
166+
return "", err
167+
}
168+
169+
host, err := c.Host(ctx)
170+
if err != nil {
171+
return "", err
172+
}
173+
174+
extraArgs := ""
175+
if len(args) > 0 {
176+
extraArgs = strings.Join(args, "&")
177+
}
178+
if extraArgs != "" {
179+
extraArgs = "?" + extraArgs
180+
}
181+
182+
connectionString := fmt.Sprintf("%s:%s@tcp(%s:%s)/%s%s", c.username, c.password, host, containerPort.Port(), c.database, extraArgs)
183+
return connectionString, nil
184+
}
185+
186+
func WithUsername(username string) testcontainers.CustomizeRequestOption {
187+
return func(req *testcontainers.GenericContainerRequest) {
188+
req.Env["DOLT_USER"] = username
189+
}
190+
}
191+
192+
func WithPassword(password string) testcontainers.CustomizeRequestOption {
193+
return func(req *testcontainers.GenericContainerRequest) {
194+
req.Env["DOLT_PASSWORD"] = password
195+
}
196+
}
197+
198+
func WithDoltCredsPublicKey(key string) testcontainers.CustomizeRequestOption {
199+
return func(req *testcontainers.GenericContainerRequest) {
200+
req.Env["DOLT_CREDS_PUB_KEY"] = key
201+
}
202+
}
203+
204+
func WithDoltCloneRemoteUrl(url string) testcontainers.CustomizeRequestOption {
205+
return func(req *testcontainers.GenericContainerRequest) {
206+
req.Env["DOLT_REMOTE_CLONE_URL"] = url
207+
}
208+
}
209+
210+
func WithDatabase(database string) testcontainers.CustomizeRequestOption {
211+
return func(req *testcontainers.GenericContainerRequest) {
212+
req.Env["DOLT_DATABASE"] = database
213+
}
214+
}
215+
216+
func WithConfigFile(configFile string) testcontainers.CustomizeRequestOption {
217+
return func(req *testcontainers.GenericContainerRequest) {
218+
cf := testcontainers.ContainerFile{
219+
HostFilePath: configFile,
220+
ContainerFilePath: "/etc/dolt/servercfg.d/server.cnf",
221+
FileMode: 0o755,
222+
}
223+
req.Files = append(req.Files, cf)
224+
}
225+
}
226+
227+
func WithCredsFile(credsFile string) testcontainers.CustomizeRequestOption {
228+
return func(req *testcontainers.GenericContainerRequest) {
229+
cf := testcontainers.ContainerFile{
230+
HostFilePath: credsFile,
231+
ContainerFilePath: "/root/.dolt/creds/" + filepath.Base(credsFile),
232+
FileMode: 0o755,
233+
}
234+
req.Files = append(req.Files, cf)
235+
}
236+
}
237+
238+
func WithScripts(scripts ...string) testcontainers.CustomizeRequestOption {
239+
return func(req *testcontainers.GenericContainerRequest) {
240+
var initScripts []testcontainers.ContainerFile
241+
for _, script := range scripts {
242+
cf := testcontainers.ContainerFile{
243+
HostFilePath: script,
244+
ContainerFilePath: "/docker-entrypoint-initdb.d/" + filepath.Base(script),
245+
FileMode: 0o755,
246+
}
247+
initScripts = append(initScripts, cf)
248+
}
249+
req.Files = append(req.Files, initScripts...)
250+
}
251+
}

0 commit comments

Comments
 (0)