[Feature]: Support a reuseWindow for cleaning up reusable containers if no tests run within a certain duration

[seanlafferty-ibm]

Problem

I run tests that require a mysql. While I'm iterating on my tests, I'd like to leverage reuse to only pay that boot price on the first test, and then subsequent tests can reuse the same mysql (which is close to a 6x improvement in speed).

However, I also want the reaper to clean up this container once I'm done with my testing "session", so I don't have a mysql running that I forgot to manually clean up.

I'm imagining a config like reuse: true, reuseTimeout: 5m. Which makes it so as long as I run a test within 5m of the last test, the reaper leaves the mysql container alone, but once 5m elapses without triggering a test, the reaper cleans everything up.

I thought this might be possible today with a combination of reuse: true and TESTCONTAINERS_RYUK_RECONNECTION_TIMEOUT=5m, but that still cleans up my mysql after 5m from the first test (and also spawns a new reaper for every run, which quickly pile up):

CONTAINER ID  IMAGE                                                  COMMAND     CREATED             STATUS             PORTS                                                                   NAMES
b551cb564516  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   2 minutes ago       Up 2 minutes       0.0.0.0:39033->8080/tcp, 8080/tcp                                       reaper_42d84025974814469f367f10224b4657f7ad888e2a5d827e76703f06292c39ec
5a7876b38a26  docker.io/mysql:8.0.36                                 mysqld      2 minutes ago       Up 2 minutes       0.0.0.0:42559->3306/tcp, 0.0.0.0:44633->33060/tcp, 3306/tcp, 33060/tcp  tests-mysql
c93fdf315582  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   2 minutes ago       Up 2 minutes       0.0.0.0:34995->8080/tcp, 8080/tcp                                       reaper_3ccf995c484a246a36990a92f85c3fc17ffb3912f867ba70b029aaf323f48547
e63820a62ce3  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   2 minutes ago       Up 2 minutes       0.0.0.0:44359->8080/tcp, 8080/tcp                                       reaper_fe49fa57fcc7031f3d1e9dbb6f305a2f4b392d1b10921d42b6a0df56e4970667
56d082f1fbbd  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   About a minute ago  Up About a minute  0.0.0.0:38461->8080/tcp, 8080/tcp                                       reaper_669ec11b7226f1c446fddd1a0e3f1aadfd1627687ed6adf3b5c846e66ecbe276
9922c527850c  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   About a minute ago  Up About a minute  0.0.0.0:34041->8080/tcp, 8080/tcp                                       reaper_74976de06285397df7d0edffbdd2e5fd284cf833f5b3de025705fbcad5e3608b
c76bcd09a892  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   52 seconds ago      Up 52 seconds      0.0.0.0:32785->8080/tcp, 8080/tcp                                       reaper_6e46bea7477b04a07464880892df435bdbd55753551da0d80b51d8917f4ab094
df476617ebdf  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   42 seconds ago      Up 42 seconds      0.0.0.0:39403->8080/tcp, 8080/tcp                                       reaper_24fdf990fc0d0ce6e34d030f619611233f7b7e714e623e1b6c635f7a85fdee4a
fc398aaabe82  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   27 seconds ago      Up 27 seconds      0.0.0.0:34295->8080/tcp, 8080/tcp                                       reaper_f9596c3bfefde5404c227e99286798894109b8d3b1de2fe207580a1baf340ac6
8ac0c88b5298  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   18 seconds ago      Up 18 seconds      0.0.0.0:42809->8080/tcp, 8080/tcp                                       reaper_8a5edab68adc986d01ff49f0774b013c3933ee8854bda319df02d26dfa233ded
64edfcfa3ec0  docker.io/testcontainers/ryuk:0.8.1                    /bin/ryuk   8 seconds ago       Up 9 seconds       0.0.0.0:35387->8080/tcp, 8080/tcp                                       reaper_3d76e1e0b630828fad795b4fcc603edaf11179da4d56148248f89862e92139d1

Solution

Add a rolling reaper cleanup window

Benefit

Users can reuse containers, but not have to worry about manually cleaning them up

Alternatives

Manually delete reusable containers

Would you like to help contributing this feature?

Yes

[jcmfernandes]

So, first, weirdly, it seems like the env var is named RYUK_RECONNECTION_TIMEOUT and not TESTCONTAINERS_RYUK_RECONNECTION_TIMEOUT, according to https://github.com/testcontainers/testcontainers-go/blob/main/internal/config/config.go#L71

I think you can achieve your goal by setting both RYUK_RECONNECTION_TIMEOUT and RYUK_CONNECTION_TIMEOUT.

[seanlafferty-ibm]

At the time of writing, those env vars were prefaced with TESTCONTAINERS_

2cec4a1

I haven't tried recently, but I imagine the behavior is still the same (eg the timeout is the duration since the first test invocation, not the most recent invocation)

[jcmfernandes]

At the time of writing, those env vars were prefaced with TESTCONTAINERS_

2cec4a1

I had missed that. :)

I haven't tried recently, but I imagine the behavior is still the same (eg the timeout is the duration since the first test invocation, not the most recent invocation)

I also wanted my testcontainers to be long-running, as I'm using Apache Pulsar and it takes a long time to boot. Therefore, I modified testcontainers-go to allow reusing the reaper across test runs. You can find my changes at https://github.com/bckground/testcontainers-go/tree/reuse-reaper which I will attempt to get upstream. Simply set a session ID for your project, e.g., TESTCONTAINERS_SESSION_ID=myproject, and a fairly large value for both RYUK_CONNECTION_TIMEOUT and RYUK_RECONNECTION_TIMEOUT (I'm going for 1 hour). This should give you the rolling-window behavior you're looking for.

[seanlafferty-ibm]

Ah cool, do you still see the reaper containers piling up with your changes? Right now If I ran the tests 20 times, I'd have 20 reaper containers spinning for an hour each 😅

Edit: I see your changes regarding the session ID: Setting this value will preclude runs from creating more than one reaper. That would be a welcome change!