Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Azure Event Hubs module #1342

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
8fe0070
Support Azure EventHubs
WakaToa May 22, 2024
3d5c226
Adjust EventHubsConfiguration
WakaToa May 22, 2024
ebccc8c
Add CPM for Azure.Messaging.EventHubs
WakaToa May 22, 2024
b264e42
better ordering of package version Azure.Messaging.EventHubs
WakaToa May 22, 2024
2503ceb
fix: file-scope namespace
rafek1241 Jan 18, 2025
b411706
fix: global using
rafek1241 Jan 18, 2025
459dae9
Merge branch 'develop' into feature/add-azure-event-hubs
rafek1241 Jan 18, 2025
0ba4f84
feat: use same pattern for EULA license
rafek1241 Jan 18, 2025
79d4b40
chore: remove redundant reference to config builder
rafek1241 Jan 18, 2025
da84fea
fix: target frameworks
rafek1241 Jan 18, 2025
4a1383a
fix: target framework of the test project
rafek1241 Jan 18, 2025
87f4d32
chore: small refactor + add docs for better understanding
rafek1241 Jan 18, 2025
2c7d413
Revert "chore: remove redundant reference to config builder"
rafek1241 Jan 18, 2025
8ca479d
refactor: add validation of configuration builder
rafek1241 Jan 18, 2025
6b27965
chore: move waiting strategy to init()
rafek1241 Jan 18, 2025
a5116e2
fix: validation
rafek1241 Jan 18, 2025
de5d54b
fix: test name and network hardcoded name
rafek1241 Jan 18, 2025
99efb2c
feat: implement built-in and configurable azurite docker container in…
rafek1241 Jan 18, 2025
2d28534
fix: default namespace name according to documentation
rafek1241 Jan 18, 2025
f650ea2
build: workflow
rafek1241 Jan 18, 2025
40bd09a
refactor: azurite dependency
rafek1241 Jan 18, 2025
0758827
feat: tests to cover custom azurite case
rafek1241 Jan 18, 2025
f7121d1
revert: changes copied from PR !1335
rafek1241 Jan 25, 2025
a9fe768
refactor: move creation of dependent stuff in build() process
rafek1241 Jan 25, 2025
7eabf4e
Merge remote-tracking branch 'upstream/develop' into feature/add-azur…
rafek1241 Jan 25, 2025
21637c5
docs: azure eventhubs documentation page
rafek1241 Jan 25, 2025
72c6b52
fix: mkdocs
rafek1241 Jan 25, 2025
5cfa3d8
fix: docs
rafek1241 Jan 25, 2025
58a9dd9
docs: fix module name
rafek1241 Jan 25, 2025
c3bf5e0
Merge remote-tracking branch 'origin/develop' into fork/rafek1241/fea…
HofmeisterAn Feb 5, 2025
53150dc
Merge remote-tracking branch 'origin/develop' into fork/rafek1241/fea…
HofmeisterAn Feb 5, 2025
0b03f27
chore: Align EventHubs module implementation
HofmeisterAn Feb 5, 2025
cfb21b8
Merge branch 'develop' into feature/add-azure-event-hubs
HofmeisterAn Feb 5, 2025
bfe1996
Merge remote-tracking branch 'rafek1241/feature/add-azure-event-hubs'…
HofmeisterAn Feb 5, 2025
34cc45d
chore: Remove WithNetworkAliases
HofmeisterAn Feb 5, 2025
d9f4bdc
Merge pull request #1 from testcontainers/fork/rafek1241/feature/add-…
rafek1241 Feb 6, 2025
4f932be
fix: format
rafek1241 Feb 6, 2025
73e6af7
fix: docs
rafek1241 Feb 6, 2025
e9df035
refactor: logic to have service configuration instead of Configuratio…
rafek1241 Feb 6, 2025
aee785d
tests(WIP): eventhubs kafka usage example
rafek1241 Feb 6, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/cicd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ jobs:
{ name: "Testcontainers.DynamoDb", runs-on: "ubuntu-22.04" },
{ name: "Testcontainers.Elasticsearch", runs-on: "ubuntu-22.04" },
{ name: "Testcontainers.EventStoreDb", runs-on: "ubuntu-22.04" },
{ name: "Testcontainers.EventHubs", runs-on: "ubuntu-22.04" },
{ name: "Testcontainers.FakeGcsServer", runs-on: "ubuntu-22.04" },
{ name: "Testcontainers.FirebirdSql", runs-on: "ubuntu-22.04" },
{ name: "Testcontainers.Firestore", runs-on: "ubuntu-22.04" },
Expand Down
1 change: 1 addition & 0 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<PackageVersion Include="AWSSDK.SQS" Version="3.7.100.71"/>
<PackageVersion Include="Azure.Data.Tables" Version="12.8.0"/>
<PackageVersion Include="Azure.Messaging.ServiceBus" Version="7.18.2"/>
<PackageVersion Include="Azure.Messaging.EventHubs" Version="5.11.3"/>
<PackageVersion Include="Azure.Storage.Blobs" Version="12.17.0"/>
<PackageVersion Include="Azure.Storage.Queues" Version="12.15.0"/>
<PackageVersion Include="ClickHouse.Client" Version="7.9.1"/>
Expand Down
14 changes: 14 additions & 0 deletions Testcontainers.sln
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.DynamoDb", "
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.Elasticsearch", "src\Testcontainers.Elasticsearch\Testcontainers.Elasticsearch.csproj", "{641DDEA5-B6E0-41E6-BA11-7A28C0913127}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.EventHubs", "src\Testcontainers.EventHubs\Testcontainers.EventHubs.csproj", "{0EF885E9-E973-47DC-AA9C-3A5E9175B0F3}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.EventStoreDb", "src\Testcontainers.EventStoreDb\Testcontainers.EventStoreDb.csproj", "{84D707E0-C9FA-4327-85DC-0AFEBEA73572}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.FakeGcsServer", "src\Testcontainers.FakeGcsServer\Testcontainers.FakeGcsServer.csproj", "{FF86B509-2F9E-4269-ABC2-912B3339DE29}"
Expand Down Expand Up @@ -137,6 +139,8 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.DynamoDb.Tes
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.Elasticsearch.Tests", "tests\Testcontainers.Elasticsearch.Tests\Testcontainers.Elasticsearch.Tests.csproj", "{DD5B3678-468F-4D73-AECE-705E3D66CD43}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.EventHubs.Tests", "tests\Testcontainers.EventHubs.Tests\Testcontainers.EventHubs.Tests.csproj", "{4A0C5523-CEB2-49C9-AE62-9187A01B016B}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.EventStoreDb.Tests", "tests\Testcontainers.EventStoreDb.Tests\Testcontainers.EventStoreDb.Tests.csproj", "{64F8E9B9-78FD-4E13-BDDF-0340E2D4E1D0}"
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Testcontainers.FakeGcsServer.Tests", "tests\Testcontainers.FakeGcsServer.Tests\Testcontainers.FakeGcsServer.Tests.csproj", "{9F27AA1B-C25D-400C-BCB0-6B0BF1A1DCEA}"
Expand Down Expand Up @@ -270,6 +274,10 @@ Global
{641DDEA5-B6E0-41E6-BA11-7A28C0913127}.Debug|Any CPU.Build.0 = Debug|Any CPU
{641DDEA5-B6E0-41E6-BA11-7A28C0913127}.Release|Any CPU.ActiveCfg = Release|Any CPU
{641DDEA5-B6E0-41E6-BA11-7A28C0913127}.Release|Any CPU.Build.0 = Release|Any CPU
{0EF885E9-E973-47DC-AA9C-3A5E9175B0F3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{0EF885E9-E973-47DC-AA9C-3A5E9175B0F3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{0EF885E9-E973-47DC-AA9C-3A5E9175B0F3}.Release|Any CPU.ActiveCfg = Release|Any CPU
{0EF885E9-E973-47DC-AA9C-3A5E9175B0F3}.Release|Any CPU.Build.0 = Release|Any CPU
{84D707E0-C9FA-4327-85DC-0AFEBEA73572}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{84D707E0-C9FA-4327-85DC-0AFEBEA73572}.Debug|Any CPU.Build.0 = Debug|Any CPU
{84D707E0-C9FA-4327-85DC-0AFEBEA73572}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down Expand Up @@ -466,6 +474,10 @@ Global
{DD5B3678-468F-4D73-AECE-705E3D66CD43}.Debug|Any CPU.Build.0 = Debug|Any CPU
{DD5B3678-468F-4D73-AECE-705E3D66CD43}.Release|Any CPU.ActiveCfg = Release|Any CPU
{DD5B3678-468F-4D73-AECE-705E3D66CD43}.Release|Any CPU.Build.0 = Release|Any CPU
{4A0C5523-CEB2-49C9-AE62-9187A01B016B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{4A0C5523-CEB2-49C9-AE62-9187A01B016B}.Debug|Any CPU.Build.0 = Debug|Any CPU
{4A0C5523-CEB2-49C9-AE62-9187A01B016B}.Release|Any CPU.ActiveCfg = Release|Any CPU
{4A0C5523-CEB2-49C9-AE62-9187A01B016B}.Release|Any CPU.Build.0 = Release|Any CPU
{64F8E9B9-78FD-4E13-BDDF-0340E2D4E1D0}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{64F8E9B9-78FD-4E13-BDDF-0340E2D4E1D0}.Debug|Any CPU.Build.0 = Debug|Any CPU
{64F8E9B9-78FD-4E13-BDDF-0340E2D4E1D0}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down Expand Up @@ -625,6 +637,7 @@ Global
{DCECB1F6-D9AA-431F-AE42-25D56B9E7DFC} = {673F23AE-7694-4BB9-ABD4-136D6C13634E}
{2EAFA567-9F68-4C52-9DBC-8F3EC11BB2CE} = {673F23AE-7694-4BB9-ABD4-136D6C13634E}
{641DDEA5-B6E0-41E6-BA11-7A28C0913127} = {673F23AE-7694-4BB9-ABD4-136D6C13634E}
{0EF885E9-E973-47DC-AA9C-3A5E9175B0F3} = {673F23AE-7694-4BB9-ABD4-136D6C13634E}
{84D707E0-C9FA-4327-85DC-0AFEBEA73572} = {673F23AE-7694-4BB9-ABD4-136D6C13634E}
{FF86B509-2F9E-4269-ABC2-912B3339DE29} = {673F23AE-7694-4BB9-ABD4-136D6C13634E}
{31BAF2C4-0608-4C0F-845A-14FE7C0A1670} = {673F23AE-7694-4BB9-ABD4-136D6C13634E}
Expand Down Expand Up @@ -674,6 +687,7 @@ Global
{DA54916E-1128-4200-B6AE-9F5BF02D832D} = {7164F1FB-7F24-444A-ACD2-2C329C2B3CCF}
{101515E6-74C1-40F9-85C8-871F742A378D} = {7164F1FB-7F24-444A-ACD2-2C329C2B3CCF}
{DD5B3678-468F-4D73-AECE-705E3D66CD43} = {7164F1FB-7F24-444A-ACD2-2C329C2B3CCF}
{4A0C5523-CEB2-49C9-AE62-9187A01B016B} = {7164F1FB-7F24-444A-ACD2-2C329C2B3CCF}
{64F8E9B9-78FD-4E13-BDDF-0340E2D4E1D0} = {7164F1FB-7F24-444A-ACD2-2C329C2B3CCF}
{9F27AA1B-C25D-400C-BCB0-6B0BF1A1DCEA} = {7164F1FB-7F24-444A-ACD2-2C329C2B3CCF}
{E39095AC-9B34-4178-A486-04C902B6FD33} = {7164F1FB-7F24-444A-ACD2-2C329C2B3CCF}
Expand Down
45 changes: 45 additions & 0 deletions docs/modules/eventhubs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Azure EventHubs

[Azure EventHubs](https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-about) is a native data-streaming service in the cloud that can stream millions of events per second, with low latency, from any source to any destination. Event Hubs is compatible with Apache Kafka. It enables you to run existing Kafka workloads without any code changes.
In this module, you will learn how to use Testcontainers to start an [Azure EventHubs emulator](https://learn.microsoft.com/en-us/azure/event-hubs/overview-emulator) container for testing. To be able to use the Azure EventHubs emulator, you need to accept the [Microsoft Event Hubs Emulator License](https://github.com/Azure/azure-event-hubs-emulator-installer/blob/main/EMULATOR_EULA.md).

!!!Warning

In the official documentation, there are **known limitations** to the Azure EventHubs emulator. You can find it [here](https://learn.microsoft.com/en-us/azure/event-hubs/overview-emulator#known-limitations).

Add the following dependency to your project file:

```shell title="NuGet"
dotnet add package Testcontainers.EventHubs
```

You can start a Azure EventHubs emulator instance from any .NET application. Here, we create different container instances and pass them to the base test class. This allows us to test different configurations.

To create a container instance with minimal configuration, use the following code:

=== "Create initial configuration JSON"
```csharp
--8<-- "tests/Testcontainers.EventHubs.Tests/EventHubsContainerTest.cs:MinimalConfigurationJson"
```

=== "Create Container Instance"
```csharp
--8<-- "tests/Testcontainers.EventHubs.Tests/EventHubsContainerTest.cs:MinimalConfigurationEventHubs"
```

This example uses xUnit.net's `IAsyncLifetime` interface to manage the lifecycle of the container. The container is started in the `InitializeAsync` method before the test method runs, ensuring that the environment is ready for testing. After the test completes, the container is removed in the `DisposeAsync` method.

=== "Usage Example"
```csharp
--8<-- "tests/Testcontainers.EventHubs.Tests/EventHubsContainerTest.cs:EventHubsUsage"
```

=== "Kafka support"

Azure Event Hubs is compatible with Apache Kafka. You can use the Azure Event Hubs Kafka endpoint to connect to the Event Hubs instance.
The following example demonstrates how to use the Azure Event Hubs Kafka endpoint with the Testcontainers library. Please,
keep in mind that only consumer and producer API is supported. More information about known limitations can be found [here](https://learn.microsoft.com/en-us/azure/event-hubs/overview-emulator#known-limitations).

```csharp
--8<-- "tests/Testcontainers.EventHubs.Tests/EventHubsContainerTest.cs:EventHubsKafkaUsage"
```
1 change: 1 addition & 0 deletions docs/modules/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ await moduleNameContainer.StartAsync();
| ActiveMQ Artemis | `apache/activemq-artemis:2.31.2` | [NuGet](https://www.nuget.org/packages/Testcontainers.ActiveMq) | [Source](https://github.com/testcontainers/testcontainers-dotnet/tree/develop/src/Testcontainers.ActiveMq) |
| ArangoDB | `arangodb:3.11.5` | [NuGet](https://www.nuget.org/packages/Testcontainers.ArangoDb) | [Source](https://github.com/testcontainers/testcontainers-dotnet/tree/develop/src/Testcontainers.ArangoDb) |
| Azure Cosmos DB | `mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest` | [NuGet](https://www.nuget.org/packages/Testcontainers.CosmosDb) | [Source](https://github.com/testcontainers/testcontainers-dotnet/tree/develop/src/Testcontainers.CosmosDb) |
| Azure Event Hubs | `mcr.microsoft.com/azure-messaging/eventhubs-emulator:latest` | [NuGet](https://www.nuget.org/packages/Testcontainers.EventHubs) | [Source](https://github.com/testcontainers/testcontainers-dotnet/tree/develop/src/Testcontainers.EventHubs) |
| Azure Service Bus | `mcr.microsoft.com/azure-messaging/servicebus-emulator:latest` | [NuGet](https://www.nuget.org/packages/Testcontainers.ServiceBus) | [Source](https://github.com/testcontainers/testcontainers-dotnet/tree/develop/src/Testcontainers.ServiceBus) |
| Azurite | `mcr.microsoft.com/azure-storage/azurite:3.24.0` | [NuGet](https://www.nuget.org/packages/Testcontainers.Azurite) | [Source](https://github.com/testcontainers/testcontainers-dotnet/tree/develop/src/Testcontainers.Azurite) |
| BigQuery | `ghcr.io/goccy/bigquery-emulator:0.4` | [NuGet](https://www.nuget.org/packages/Testcontainers.BigQuery) | [Source](https://github.com/testcontainers/testcontainers-dotnet/tree/develop/src/Testcontainers.BigQuery) |
Expand Down
3 changes: 2 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ nav:
- examples/aspnet.md
- Modules:
- modules/index.md
- modules/eventhubs.md
- modules/elasticsearch.md
- modules/mongodb.md
- modules/mssql.md
Expand All @@ -55,4 +56,4 @@ nav:
- modules/pulsar.md
- modules/rabbitmq.md
- contributing.md
- contributing_docs.md
- contributing_docs.md
15 changes: 6 additions & 9 deletions src/Testcontainers.Azurite/AzuriteContainer.cs
Original file line number Diff line number Diff line change
Expand Up @@ -32,30 +32,27 @@ public string GetConnectionString()
/// <summary>
/// Gets the blob endpoint
/// </summary>
/// <returns>The azurite blob endpoint</returns>
/// <returns>The Azurite blob endpoint</returns>
public string GetBlobEndpoint()
{
return new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(AzuriteBuilder.BlobPort),
AzuriteBuilder.AccountName).ToString();
return new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(AzuriteBuilder.BlobPort), AzuriteBuilder.AccountName).ToString();
}

/// <summary>
/// Gets the queue endpoint
/// </summary>
/// <returns>The azurite queue endpoint</returns>
/// <returns>The Azurite queue endpoint</returns>
public string GetQueueEndpoint()
{
return new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(AzuriteBuilder.QueuePort),
AzuriteBuilder.AccountName).ToString();
return new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(AzuriteBuilder.QueuePort), AzuriteBuilder.AccountName).ToString();
}

/// <summary>
/// Gets the table endpoint
/// </summary>
/// <returns>The azurite table endpoint</returns>
/// <returns>The Azurite table endpoint</returns>
public string GetTableEndpoint()
{
return new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(AzuriteBuilder.TablePort),
AzuriteBuilder.AccountName).ToString();
return new UriBuilder(Uri.UriSchemeHttp, Hostname, GetMappedPublicPort(AzuriteBuilder.TablePort), AzuriteBuilder.AccountName).ToString();
}
}
1 change: 1 addition & 0 deletions src/Testcontainers.EventHubs/.editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
root = true
Loading
Loading