feat: Add support to seed data when using RunBigQueryContainer #2521
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for testcontainers-go ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
mdelapenya
reviewed
Apr 30, 2024
modules/gcloud/gcloud.go
Outdated
| @@ -68,6 +70,20 @@ func WithProjectID(projectID string) Option { | |||
| } | |||
| } | |||
|
|
|||
| // WithDataYamlFile seeds the Bigquery project for the GCloud container. | |||
| func WithDataYamlFile(dataYamlFile string) testcontainers.CustomizeRequestOption { | |||
| return func(req *testcontainers.GenericContainerRequest) { | |||
There was a problem hiding this comment.
Yeah I can update that 👍 Thank you for pointing this out! I may not get to it today but definitely by tomorrow
…e, compose, qdrant, couchbase, k3s, milvus, mockserver, pulsar, kafka) (#2505) * chore(deps): bump golang.org/x/net in /modules/kafka Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/pulsar Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/mockserver Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/milvus Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net from 0.19.0 to 0.23.0 in /modules/k3s Bumps [golang.org/x/net](https://github.com/golang/net) from 0.19.0 to 0.23.0. - [Commits](golang/net@v0.19.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/couchbase Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/qdrant Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/compose Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/weaviate Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/gcloud Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0. - [Commits](golang/net@v0.21.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/minio Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0. - [Commits](golang/net@v0.21.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* feat: add testcontainers labels to compose containers * feat: support reaper for compose * chore: increase ryuk reconnection timeout on CI * chore: cache containers on UP * chore: more tuning for compose * chore: more consistent assertion * chore: the compose stack asks for the reaper, but each container then connects to it * chore: use different error groups the first time wait is called, the context is cancelled * chore: the lookup method include cache checks * chore: update tests to make them deterministic * chore: rename local compose testss * chore: support returning the dynamic port in the helper function * chore: try with default reconnection timeout * feat: support removing networks from compose * chore: support naming test services with local and api It will allow the tests to be more deterministic, as there could be service containers started from the local test suite with the same name as in the API test suite. * Revert "chore: try with default reconnection timeout" This reverts commit 336760c. * fix: typo
…pose instance (#2509) * feat: support passing io.Reader when creating a compose instance * docs: change title
* fix: don't retry on permanent APIClient errors * fix: add more tests for un-retryable scenarios
* chore: start a foundational package for interacting with Docker networks * feat: add an SSH tunnel forwarding a host port to a container * fix: rename struct * chore: pass the original context to the exposeHostPorts function * chore: start tunnel using context * chore: push goroutines to the method where they are used * fix: proper eval of first network * fix: handle dockerignore exclusions properly (#2476) * chore: only include the dockerignore if it contains ignore files * fix: the inclusions must be relative to the context * docs: document the dockerignore feature * chore: only include the dockerignore file if it exists * Elasticsearch disable CA retrieval when ssl is disabled (#2475) * skip search for CACert if ssl has been turned off * add tests with and without ssl enabled * add all config keys that disable CA gen, restrict check to version 8 * rename test to match content * chore(deps): bump idna from 3.6 to 3.7 (#2480) Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7. - [Release notes](https://github.com/kjd/idna/releases) - [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst) - [Commits](kjd/idna@v3.6...v3.7) --- updated-dependencies: - dependency-name: idna dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: create TLS certs in a consistent manner (#2478) * fix: remove suspicious filepath.Join * chore: fix lint * fix: handle error * chore: reverse assertion for lint * feat: support generating TLS certificates on the fly * chore: apply to cockroachdb * chore: support saving the cert and priv key files to disk * chore: apply to rabbitmq * chore: simplify * chore: use in redpanda module * chore: lint * chore: set validFrom internally * fix: properly use the new API in redpanda * docs: document the TLS helpers * chore: simplify WithParent to accept the struct directly * chore: use tlscert package instead * fix: use non-deprecated API * docs: update * docs: fix examples * chore: use released version of tlscert * fix: add common name for the node cert * support Dolt (#2177) * /modules/dolt: wip, kinda working * /modules/dolt: get tests passing * /{.github,.vscode,docs,mkdocs,modules,sonar-project}: use modulegen tool * /modules/dolt/{dolt.go,examples_test.go}: run linter * /modules/dolt/{dolt.go,examples_test.go}: add methods for cloning * /{docs, modules}: add with creds file * /{docs,modules}: pr feedback, cleanup * /modules/dolt/examples_test.go: remove panics, lint * chore: run mod tidy * chore: include MustConnectionString method * chore: do not use named returns * chore: perform initialisation before the container has started --------- Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com> * feat: Bump default postgres version (#2481) * Bump default postgres version * Bump to use latest pg * Bump version from non-ancient version --------- Co-authored-by: bstrausser <bstrausser@locusrobotics.com> * fix(postgres): Fix the non-default dbname error (#2489) * Fix the non-default dbname error The linked issue described in great detail an issue where we assumed everyone would use the default database user, whose home DB defaults to the postgres database. When that was not the case, the snapshots would fail silently as the user would not connect to the right database to take the commands. This PR fixes the issue by adding the dbname by default in the command, and adds a test to validate this works as intended. In addition, it also adds some logic to handle any error that does not cause the exec command to fail, such as database access failures. Run the added test to test this works as intended. Closes #2474 * Document the postgres dbname issue in the docs * fix: fallback to URL-path when parsing auth config URL without scheme (#2488) * chore(deps): bump golang.org/x/net in modules (minio, gcloud, weaviate, compose, qdrant, couchbase, k3s, milvus, mockserver, pulsar, kafka) (#2505) * chore(deps): bump golang.org/x/net in /modules/kafka Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/pulsar Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/mockserver Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/milvus Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0. - [Commits](golang/net@v0.17.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net from 0.19.0 to 0.23.0 in /modules/k3s Bumps [golang.org/x/net](https://github.com/golang/net) from 0.19.0 to 0.23.0. - [Commits](golang/net@v0.19.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/couchbase Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/qdrant Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/compose Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/weaviate Bumps [golang.org/x/net](https://github.com/golang/net) from 0.20.0 to 0.23.0. - [Commits](golang/net@v0.20.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/gcloud Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0. - [Commits](golang/net@v0.21.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump golang.org/x/net in /modules/minio Bumps [golang.org/x/net](https://github.com/golang/net) from 0.21.0 to 0.23.0. - [Commits](golang/net@v0.21.0...v0.23.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat: support Ryuk for the compose module (#2485) * feat: add testcontainers labels to compose containers * feat: support reaper for compose * chore: increase ryuk reconnection timeout on CI * chore: cache containers on UP * chore: more tuning for compose * chore: more consistent assertion * chore: the compose stack asks for the reaper, but each container then connects to it * chore: use different error groups the first time wait is called, the context is cancelled * chore: the lookup method include cache checks * chore: update tests to make them deterministic * chore: rename local compose testss * chore: support returning the dynamic port in the helper function * chore: try with default reconnection timeout * feat: support removing networks from compose * chore: support naming test services with local and api It will allow the tests to be more deterministic, as there could be service containers started from the local test suite with the same name as in the API test suite. * Revert "chore: try with default reconnection timeout" This reverts commit 336760c. * fix: typo * chore: add funding button for testcontainers (#2510) * feat: support passing io.Reader for compose files when creating a compose instance (#2509) * feat: support passing io.Reader when creating a compose instance * docs: change title * feat: support overriding the default recreate options for compose (#2511) * feat: support overriding the default recreate options for compose * chore: validate recreation values * fix: don't retry on permanent APIClient errors (#2506) * fix: don't retry on permanent APIClient errors * fix: add more tests for un-retryable scenarios * chore: run mod tidy * chore: implement the port-forwarding correctly * chore: use new sshd image * chore: simplify channel creation to avoid allocations * fix: do not leak goroutines Detected with go.uber.org/goleak * chore: expose host internal constant * fix: update variables --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Laurent Saint-Félix <laurent.saintfelix@elastic.co> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Dustin Brown <dustin@dolthub.com> Co-authored-by: Barrett Strausser <bearrito@users.noreply.github.com> Co-authored-by: bstrausser <bstrausser@locusrobotics.com> Co-authored-by: Guillaume St-Pierre <guillaume.stpierre@docker.com> Co-authored-by: Patrick Jahn <33724206+p-jahn@users.noreply.github.com>
* fix: enhance host configuration port binding This commit refactors the mergePortBinding function to accommodate exposed ports with their respective protocols. * dep: run go mod tidy for all modules --------- Co-authored-by: Willian S. de Souza <willian.souza@ifood.com.br>
* docs: document the SSHd tunnel * chore: add functional option for customising modules and host ports
Adds documentation for the Exec method in the DockerContainer struct.
Used that hardcoded string for testing locally
* chore!: return error from Customize Change ContainerCustomizer.Customize method to return an error so that options can handle errors gracefully instead of relying on panic or just a log entry, neither of which are user friendly. Enable errcheck linter to ensure that errors that aren't handled are reported. Run go mod tidy on k3s and weaviate to allow tests to be run using go 1.22. Run gofumpt on a few files to satisfy golangci-lint. Fix direct comparison with http.ErrServerClosed flagged by errcheck. Fixes #2266 BREAKING CHANGE: `ContainerCustomizer.Customize` now returns an error. * fix(mongodb): captured loop variable Fix captured loop variable in mongodb test reported by govet. * fix(k3s): test formatting Fix formatting in test file reported by gci during linting. * chore: add missing Customize error returns Add missing error returns for implementations of CustomizeRequestOption. * chore: update modules with the new error API * fix: use logger * fix: update modulegen tests * chore: fix lint * chore: handle customise errors * chore: update new host port access option * docs: move new error API to the right doc * docs: move customise request to the bottom * docs: sync functional options * chore: avoid panics in localstack * chore: require no error in test * chore: ignore error in copy API, using nolint:errcheck * chore: convert log into error --------- Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>
….15 (#2517) * chore(deps): bump github.com/containerd/containerd from 1.7.12 to 1.7.14 * `make tidy-all` * chore: bump containerd to 1.7.15 --------- Co-authored-by: Andrii Dema <a.dema@jazzserve.com> Co-authored-by: Andrii Dema <demaandrii@gmail.com>
…c8 to v2.1.0 (#2519) * chore(deps): bump compose to v2.1.0 * chore: mod tidy dolt module
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Updates the
RunBigQueryContainerto optionally use the emulator'sdata-from-yamloption, allowing users to seed the container with data.Why is it important?
This give the user of the
RunBigQueryContainerthe option to seed test data on the container creation/startup.Related issues