[Bug]: Regression for Kafka

[mixnblend]

Testcontainers version

0.33.0

Using the latest Testcontainers version?

Yes

Host OS

Mac

Host arch

ARM

Go version

1.22

Docker version

docker version
Client:
 Version:           25.0.4-rd
 API version:       1.44
 Go version:        go1.21.8
 Git commit:        c4cd0a9
 Built:             Fri Mar  8 09:09:46 2024
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:47 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

docker info
Client: Docker Engine - Community
 Version:    27.1.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2
    Path:     /Users/<username>/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.29.2
    Path:     /Users/<username>/.docker/cli-plugins/docker-compose

Server:
 Containers: 5
  Running: 5
  Paused: 0
  Stopped: 0
 Images: 57
 Server Version: 26.1.1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-31-generic
 Operating System: Ubuntu 24.04 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 7.738GiB
 Name: colima
 ID: be95d926-6cd1-4ed5-8556-ac01c0037ced
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

In version 0.32.0 integration tests were passing with a KafkaContainer. In 0.33.0 the following lines in the Kafka module at the top of copyStarterScript result in the context deadline being exceeded and the test failing. If I remove them my tests pass again.

if err := wait.ForListeningPort(publicPort).
		SkipInternalCheck().
		WaitUntilReady(ctx, c); err != nil {
		return fmt.Errorf("wait for exposed port: %w", err)
	}

The Kafka container I am creating has the following properties:

kafkaContainer, err := kafka.Run(ctx,
		cfg.image,
		kafka.WithClusterID(cfg.clusterId),
		testcontainers.WithLogger(testcontainers.TestLogger(t)),
	)

The image for the container is confluentinc/confluent-local:7.7.0.

I suspect this may have to do with me using Colima on a Mac and host port mapping, what is confusing to me is that if I comment those lines out and pass the following in my container params then the tests still pass.

kafkaContainer, err := kafka.Run(ctx,
		cfg.image,
		kafka.WithClusterID(cfg.clusterId),
		testcontainers.WithLogger(testcontainers.TestLogger(t)),
		testcontainers.WithWaitStrategy(
		 	wait.ForListeningPort("9093/tcp"),
		 	wait.ForLog(".*Transitioning from RECOVERY to RUNNING.*").AsRegexp(),
		),
	)

Relevant log output

lifecycle.go:62: 🐳 Creating container for image postgres:15.3-alpine
    lifecycle.go:68: ✅ Container created: c6eaaf53bec4
    lifecycle.go:74: 🐳 Starting container: c6eaaf53bec4
    lifecycle.go:80: ✅ Container started: c6eaaf53bec4
    lifecycle.go:271: ⏳ Waiting for container id c6eaaf53bec4 image: postgres:15.3-alpine. Waiting for: &{timeout:<nil> deadline:0x1400059fc68 Strategies:[0x14000345410 0x14000345440]}
    lifecycle.go:86: 🔔 Container is ready: c6eaaf53bec4
    lifecycle.go:62: 🐳 Creating container for image confluentinc/confluent-local:7.7.0
    lifecycle.go:68: ✅ Container created: f49cdd632ddd
    lifecycle.go:74: 🐳 Starting container: f49cdd632ddd
    lifecycle.go:80: ✅ Container started: f49cdd632ddd
    lifecycle.go:341: container logs (copy starter script: wait for exposed port: get state: Get "http://%2FUsers%2F*******%2F.colima%2Fdocker.sock/v1.45/containers/f49cdd632ddd469cbc51a3f2c820a72dd44ffece473c5f5a8e5a38cbd9c04e99/json": context deadline exceeded):
        
2024/08/26 11:42:21 failed to start container: start container: started hook: copy starter script: wait for exposed port: get state: Get "http://%2FUsers%2F******%2F.colima%2Fdocker.sock/v1.45/containers/f49cdd632ddd469cbc51a3f2c820a72dd44ffece473c5f5a8e5a38cbd9c04e99/json": context deadline exceeded.

Additional information

No response

[VRmnv]

Having a same issue(

[harsh1231]

facing same issue when ran on gitlab ci (ubuntu) , issue is for both kafka and redpanda modules .

Could not start kafka container start container: started hook: copy starter script: wait for exposed port: get state: Get "http://docker:2375/v1.44/containers/69aa9d728f56052c537a2e23a125171ff9a5ac26421b27f1156d73de063cb235/json": context deadline exceeded

after reverting to

github.com/testcontainers/testcontainers-go/modules/kafka v0.32.0

issue resolved for me .

[kiview]

@VRmnv Are you also on Colima, or is this on Docker?
@harsh1231 Reporting it for on Ubuntu suggest the usage of Docker CE (but possibly DinD?), so I don't think this issue is specifically Colima related.

[VRmnv]

@kiview this is docker

[kannancmohan]

facing the same issue in v0.35.0. The aforementioned snippet "wait.ForListeningPort(publicPort).SkipInternalCheck().WaitUntilReady(ctx, c) " is causing this issue

Environment:

Server Version: 27.0.3 API Version: 1.46 Operating System: Ubuntu 22.04.4 LTS Total Memory: 5921 MB Testcontainers for Go Version: v0.35.0 Resolved Docker Host: unix:///tmp/remote-docker-gotest.sock Resolved Docker Socket Path: /tmp/remote-docker-gotest.sock

[rituraj-junglee]

Removing TESTCONTAINERS_HOST_OVERRIDE=$(colima ls -j | jq -r '.address') from my rc file worked for me.