Allow DockerClientProviderStrategy to provide remote docker host uri

[darl]

After testcontainers-cloud announcement, I prototyped support for docker over ssh:
https://gist.github.com/darl/7215ec147558d371d9c34430fde2add9

The ugly part is unix socket path mapping.
ryuk uses local dockerHost path, but it runs on a remote machine, where it's not accessible.

With this change, DockerClientProviderStrategy can provide different path to unix socket when needed.

[kiview]

Hi @darl, thanks a lot for raising this PR and the interesting idea.

Don't you think that the custom Docker host configuration capabilities in Testcontainers (which will be used by EnvironmentAndSystemPropertyClientProviderStrategy) are sufficient enough? Indeed, in the last days, we have seen many users utilize them successfully for different more advanced Docker setups and Docker Desktop substitutes.

[bsideup]

Since what you need can currently be achieved with TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE and just DOCKER_HOST, I don't think we should extend our API. As you mentioned Testcontainers Cloud, it is an example that what you are trying to achieve can be done with the current API.

[darl]

I can use custom config to set up my local env.
But I want an easier & automated way for my team.

1. There is no way to provide TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE via properties file. So I end up with hardcoding it in .bashrc
2. I use bazel as a build tool. Bazel cleans most env variables to increase reproducibility and to cache test results. So I need to explicitly pass TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE via command line args when running tests.

My plan was to write custom DockerClientProviderStrategy that parses docker context inspect output and:

• set ups ssh tunnel for ssh://* docker host
• overrides remoteDockerHost for known docker runtimes (like colima)

[bsideup]

@darl have you considered tuning your colima setup so that it listens on the well known socket location inside the VM?

[darl]

It already listens on "well known" socket location inside VM (//var/run/docker.sock)
But outside VM it located in different place (unix:///Users/darl/.colima/docker.sock in my case)

[darl]

As you mentioned Testcontainers Cloud, it is an example that what you are trying to achieve can be done with the current API.

Are there any details how testcontainers-cloud works (on client side)?

[bsideup]

It already listens on "well known" socket location inside VM (//var/run/docker.sock)
But outside VM it located in different place (unix:///Users/darl/.colima/docker.sock in my case)

then setting DOCKER_HOST/docker.host would just work.

Are there any details how testcontainers-cloud works (on client side)?

No. It is a proprietary technology :)

[darl]

then setting DOCKER_HOST/docker.host would just work.

I don't understand how it works.
When docker.host=unix:/$HOME/.colima/docker.sock, ryuk cannot bind this path (because it not exists in VM)
When docker.host=unix:///var/run/docker.sock, it cannot be found on host machine and testcountainers wouldn't start.

[kiview]

Does this config work for you?

DOCKER_HOST=unix:///Users/darl/.colima/docker.sock
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock

If you would set docker.host to a tcp scheme, it should work as proposed by @bsideup (since Testcontainers will fallback to /var/run/docker.sock for Ryuk in this case).

[darl]

Does this config work for you?

Yes. It works. I thought single docker.host is enough from Sergey's comment.

Again. I can make it work "on my PC". I can write a comprehensive guide to "setup local PC/mac with docker".
But a also want to make this guide as simple as possible.

So, what do you think about this changes in library:

1. reading TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE from ~/.testcontainers.properties too? Now it can be passed only with ENV.
2. Making new DockerClientProviderStrategy that uses docker context call to discover docker socket.
3. some hack for mac runtimes like colima.
4. setup ssh tunnel for ssh://... paths. (transport module supporting remote connections though ssh docker-java/docker-java#1440 looks stale)

[kiview]

Thanks for your proposal @darl, I will go through each item separately.

1. reading TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE from ~/.testcontainers.properties too? Now it can be passed only with ENV.

This sounds like a good idea, it can be a distinct and small PR.

2. Making new DockerClientProviderStrategy that uses docker context call to discover docker socket.

Docker Context is currently a Docker CLI feature and the format is not straightforward to work with. A clean integration would mean a call to Docker CLI and this is something we want to avoid.

3. some hack for mac runtimes like colima.

As seen in the other issues such as #5034, there should be no need for a hack.

4. setup ssh tunnel for ssh://... paths. (transport module supporting remote connections though ssh docker-java/docker-java#1440 looks stale)

docker-java feels that this change is too big to be incorporated (see also this comment about closing the PR: docker-java/docker-java#1440 (comment)). However, both testcontainers-java and docker-java provide abstractions that allow adding custom transports and users are free to experiment with alternatives. We can always reconsider integrating other transports in the future, once more experience has been gained with them and feedback collected. But for the time being, we don't want to maintain and support such a strategy in testcontainers-java.

[darl]

Docker Context is currently a Docker CLI feature and the format is not straightforward to work with. A clean integration would mean a call to Docker CLI and this is something we want to avoid.

I still think it pretty useful, so I want implement it internally.

However, both testcontainers-java and docker-java provide abstractions that allow adding custom transports

I see only DockerClientProviderStrategy as such abstraction accessible from code.
Is there any others?

Its true that with DockerClientProviderStrategy I can provide custom DockerClient with whatever transport I want, but I still have issue with remoteDockerHost.

Btw, I found workaround for this issue by explicitly providing DockerClient with different TransportConfig:
https://gist.github.com/darl/4ae284b47723b6d690d59ae612c6b9c4

I found it pretty hacky, but I can live with it.
Thank you for your patience ;-)

[OscarPalafox]

Since what you need can currently be achieved with TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE and just DOCKER_HOST, I don't think we should extend our API. As you mentioned Testcontainers Cloud, it is an example that what you are trying to achieve can be done with the current API.

Hey @kiview @bsideup.

I've been trying to set up Testcontainers to use my SSH docker endpoint to no avail. Using the docker context works fine though. Would it be possible to please get some guidance on how to achieve this? I have tried following the custom Docker host configuration guide, but it doesn't seem to be super explicit on what the procedure should be in this case. Testcontainers keeps throwing an error saying IllegalArgumentException (Unsupported protocol scheme: ssh://<docker-daemon-machine> when DOCKER_HOST is set to ssh://<docker-daemon-machine>, and the docker-java/docker-java#1440 (comment) says that the SSH protocol will not be supported, hence my confusion since you mentioned it is doable.

Any help would be greatly appreciated! Thanks!