Fix broken `DataStreamsMonitoringRabbitMQTests` #6629

andrewlock · 2025-02-05T13:12:55Z

Summary of changes

Fix broken DataStreamsMonitoringRabbitMQTests by removing TFM limit

Reason for change

#6479 added support for RabbitMQ v7, but also accidentally restricted the Samples.DataStreams.RabbitMQ project to only building for .NET 8, which was picked up in the version bump PR: #6558.

I'm surprised we didn't notice it before... I need to look into why

Implementation details

Remove the TFM restriction

Test coverage

Will do a full run to be sure

bouwkast

😨 Thanks for catching and fixing

datadog-ddstaging · 2025-02-05T13:48:14Z

Datadog Report

Branch report: andrew/ci/fix-rabbitmq-datastreams
Commit report: 7e443e7
Test service: dd-trace-dotnet

✅ 0 Failed, 245401 Passed, 2384 Skipped, 18h 44m 28.57s Total Time

andrewlock · 2025-02-05T13:48:22Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6629) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (70ms)  : 65, 74
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6629) - mean (995ms)  : 969, 1020
     .   : milestone, 995,
    master - mean (991ms)  : 973, 1009
     .   : milestone, 991,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6629) - mean (103ms)  : 101, 105
     .   : milestone, 103,
    master - mean (102ms)  : 100, 104
     .   : milestone, 102,

    section CallTarget+Inlining+NGEN
    This PR (6629) - mean (674ms)  : 660, 689
     .   : milestone, 674,
    master - mean (670ms)  : 654, 686
     .   : milestone, 670,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6629) - mean (90ms)  : 87, 92
     .   : milestone, 90,
    master - mean (90ms)  : 88, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (6629) - mean (632ms)  : 616, 648
     .   : milestone, 632,
    master - mean (633ms)  : 614, 651
     .   : milestone, 633,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6629) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (190ms)  : 185, 195
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6629) - mean (1,105ms)  : 1066, 1144
     .   : milestone, 1105,
    master - mean (1,100ms)  : 1071, 1130
     .   : milestone, 1100,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6629) - mean (270ms)  : 266, 274
     .   : milestone, 270,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (6629) - mean (867ms)  : 831, 903
     .   : milestone, 867,
    master - mean (862ms)  : 829, 894
     .   : milestone, 862,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6629) - mean (262ms)  : 257, 266
     .   : milestone, 262,
    master - mean (261ms)  : 257, 266
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (6629) - mean (848ms)  : 813, 883
     .   : milestone, 848,
    master - mean (846ms)  : 817, 875
     .   : milestone, 846,

andrewlock · 2025-02-05T14:23:42Z

Benchmarks Report for tracer 🐌

Benchmarks for #6629 compared to master:

1 benchmarks are faster, with geometric mean 1.127
1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.85μs	45.1ns	337ns	0.0163	0.00815	0	5.62 KB
master	`StartStopWithChild`	netcoreapp3.1	10μs	54.8ns	310ns	0.0303	0.0151	0.00505	5.8 KB
master	`StartStopWithChild`	net472	16.5μs	38.3ns	148ns	1.04	0.308	0.0999	6.21 KB
#6629	`StartStopWithChild`	net6.0	7.95μs	44.7ns	296ns	0.0158	0.00788	0	5.61 KB
#6629	`StartStopWithChild`	netcoreapp3.1	10.2μs	52.9ns	259ns	0.0191	0.00477	0	5.8 KB
#6629	`StartStopWithChild`	net472	16.2μs	60.5ns	234ns	1.04	0.294	0.103	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	475μs	245ns	916ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	661μs	287ns	1.11μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	856μs	484ns	1.87μs	0.428	3.3 KB
#6629	`WriteAndFlushEnrichedTraces`	net6.0	491μs	316ns	1.18μs	0	2.7 KB
#6629	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	653μs	253ns	981ns	0	2.7 KB
#6629	`WriteAndFlushEnrichedTraces`	net472	860μs	426ns	1.54μs	0.428	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	130μs	319ns	1.24μs	0.192	14.47 KB
master	`SendRequest`	netcoreapp3.1	145μs	377ns	1.46μs	0.219	17.27 KB
master	`SendRequest`	net472	0.0105ns	0.00278ns	0.0108ns	0	0 b
#6629	`SendRequest`	net6.0	132μs	411ns	1.59μs	0.199	14.48 KB
#6629	`SendRequest`	netcoreapp3.1	138μs	420ns	1.57μs	0.21	17.27 KB
#6629	`SendRequest`	net472	0.0229ns	0.00212ns	0.0082ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6629

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.81 KB	41.48 KB	-330 B	-0.79%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	600μs	3.28μs	24.4μs	0.563	0	0	41.81 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	651μs	2.85μs	10.3μs	0.319	0	0	41.79 KB
master	`WriteAndFlushEnrichedTraces`	net472	834μs	3.56μs	13.8μs	8.12	2.44	0.406	53.31 KB
#6629	`WriteAndFlushEnrichedTraces`	net6.0	553μs	2.96μs	15.1μs	0.536	0	0	41.48 KB
#6629	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	676μs	3.47μs	17.4μs	0.327	0	0	41.76 KB
#6629	`WriteAndFlushEnrichedTraces`	net472	829μs	2.92μs	10.9μs	8.39	2.52	0.419	53.29 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.37μs	0.953ns	3.69ns	0.0145	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.78μs	1.22ns	4.58ns	0.0141	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.11μs	1.41ns	5.47ns	0.157	0.00105	987 B
#6629	`ExecuteNonQuery`	net6.0	1.26μs	1.01ns	3.93ns	0.014	0	1.02 KB
#6629	`ExecuteNonQuery`	netcoreapp3.1	1.74μs	2.34ns	9.06ns	0.0139	0	1.02 KB
#6629	`ExecuteNonQuery`	net472	2.06μs	1.33ns	4.97ns	0.156	0.00103	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.31μs	0.643ns	2.49ns	0.0136	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.6μs	0.568ns	1.97ns	0.0137	976 B
master	`CallElasticsearch`	net472	2.54μs	1.46ns	5.45ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.28μs	1.1ns	4.12ns	0.0135	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.57μs	2.41ns	9.32ns	0.0143	1.02 KB
master	`CallElasticsearchAsync`	net472	2.58μs	1.05ns	4.08ns	0.166	1.05 KB
#6629	`CallElasticsearch`	net6.0	1.21μs	0.475ns	1.78ns	0.0133	976 B
#6629	`CallElasticsearch`	netcoreapp3.1	1.61μs	0.741ns	2.87ns	0.0128	976 B
#6629	`CallElasticsearch`	net472	2.65μs	1.91ns	7.14ns	0.157	995 B
#6629	`CallElasticsearchAsync`	net6.0	1.33μs	0.762ns	2.95ns	0.0134	952 B
#6629	`CallElasticsearchAsync`	netcoreapp3.1	1.63μs	2.71ns	10.5ns	0.014	1.02 KB
#6629	`CallElasticsearchAsync`	net472	2.75μs	1.5ns	5.41ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.31μs	0.505ns	1.96ns	0.0131	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.62μs	2.77ns	10.7ns	0.0129	952 B
master	`ExecuteAsync`	net472	1.84μs	1.14ns	4.41ns	0.145	915 B
#6629	`ExecuteAsync`	net6.0	1.39μs	0.61ns	2.28ns	0.0131	952 B
#6629	`ExecuteAsync`	netcoreapp3.1	1.64μs	1.32ns	4.93ns	0.0129	952 B
#6629	`ExecuteAsync`	net472	1.81μs	0.47ns	1.69ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.49μs	1.54ns	5.95ns	0.0313	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.28μs	2.26ns	8.74ns	0.0368	2.85 KB
master	`SendAsync`	net472	7.46μs	2.72ns	10.5ns	0.495	3.12 KB
#6629	`SendAsync`	net6.0	4.5μs	1.21ns	4.53ns	0.0314	2.31 KB
#6629	`SendAsync`	netcoreapp3.1	5.28μs	1.82ns	6.57ns	0.037	2.85 KB
#6629	`SendAsync`	net472	7.36μs	2.92ns	10.9ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.46μs	0.439ns	1.58ns	0.0231	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.21μs	1.26ns	4.87ns	0.022	1.64 KB
master	`EnrichedLog`	net472	2.57μs	0.817ns	2.95ns	0.249	1.57 KB
#6629	`EnrichedLog`	net6.0	1.5μs	0.974ns	3.77ns	0.0226	1.64 KB
#6629	`EnrichedLog`	netcoreapp3.1	2.29μs	1.52ns	5.89ns	0.022	1.64 KB
#6629	`EnrichedLog`	net472	2.61μs	1.37ns	5.29ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	111μs	85.4ns	331ns	0.0554	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	117μs	69.9ns	252ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	98.1ns	380ns	0.678	0.226	4.46 KB
#6629	`EnrichedLog`	net6.0	113μs	137ns	511ns	0.0562	0	4.28 KB
#6629	`EnrichedLog`	netcoreapp3.1	116μs	218ns	846ns	0	0	4.28 KB
#6629	`EnrichedLog`	net472	151μs	90.4ns	350ns	0.676	0.225	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.98μs	1.34ns	5.19ns	0.0314	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.22μs	1.64ns	5.91ns	0.0296	2.2 KB
master	`EnrichedLog`	net472	4.95μs	1.28ns	4.96ns	0.32	2.02 KB
#6629	`EnrichedLog`	net6.0	3.06μs	0.918ns	3.44ns	0.0313	2.2 KB
#6629	`EnrichedLog`	netcoreapp3.1	4.4μs	3.69ns	14.3ns	0.0285	2.2 KB
#6629	`EnrichedLog`	net472	5.02μs	1.74ns	6.49ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.39μs	0.88ns	3.41ns	0.0159	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.7μs	1.02ns	3.94ns	0.0153	1.14 KB
master	`SendReceive`	net472	2.13μs	1.8ns	6.97ns	0.183	1.16 KB
#6629	`SendReceive`	net6.0	1.4μs	0.95ns	3.68ns	0.0162	1.14 KB
#6629	`SendReceive`	netcoreapp3.1	1.72μs	0.867ns	3.36ns	0.0155	1.14 KB
#6629	`SendReceive`	net472	2.05μs	0.774ns	3ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.81μs	0.689ns	2.67ns	0.0211	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.81μs	1.78ns	6.65ns	0.0209	1.65 KB
master	`EnrichedLog`	net472	4.5μs	3.1ns	12ns	0.323	2.04 KB
#6629	`EnrichedLog`	net6.0	2.89μs	1ns	3.89ns	0.0229	1.6 KB
#6629	`EnrichedLog`	netcoreapp3.1	3.92μs	2.17ns	8.4ns	0.0215	1.65 KB
#6629	`EnrichedLog`	net472	4.45μs	2.73ns	10.6ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6629

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.127	786.35	697.54

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	404ns	0.446ns	1.73ns	0.00813	576 B
master	`StartFinishSpan`	netcoreapp3.1	553ns	0.558ns	2.16ns	0.00771	576 B
master	`StartFinishSpan`	net472	583ns	0.701ns	2.71ns	0.0918	578 B
master	`StartFinishScope`	net6.0	481ns	0.471ns	1.82ns	0.0098	696 B
master	`StartFinishScope`	netcoreapp3.1	786ns	0.929ns	3.6ns	0.00952	696 B
master	`StartFinishScope`	net472	776ns	1.2ns	4.63ns	0.104	658 B
#6629	`StartFinishSpan`	net6.0	405ns	0.446ns	1.73ns	0.00813	576 B
#6629	`StartFinishSpan`	netcoreapp3.1	563ns	0.672ns	2.6ns	0.00784	576 B
#6629	`StartFinishSpan`	net472	618ns	0.895ns	3.47ns	0.0915	578 B
#6629	`StartFinishScope`	net6.0	474ns	0.628ns	2.43ns	0.00972	696 B
#6629	`StartFinishScope`	netcoreapp3.1	698ns	0.827ns	3.09ns	0.00932	696 B
#6629	`StartFinishScope`	net472	838ns	2.25ns	8.7ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	592ns	0.54ns	2.09ns	0.00974	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	901ns	1.51ns	5.86ns	0.00942	696 B
master	`RunOnMethodBegin`	net472	1.11μs	1.36ns	5.28ns	0.104	658 B
#6629	`RunOnMethodBegin`	net6.0	597ns	0.694ns	2.69ns	0.00974	696 B
#6629	`RunOnMethodBegin`	netcoreapp3.1	955ns	1.65ns	6.38ns	0.009	696 B
#6629	`RunOnMethodBegin`	net472	1.1μs	2.03ns	7.87ns	0.104	658 B

Remove accidental limiting of TFMs

7e443e7

andrewlock added area:builds project files, build scripts, pipelines, versioning, releases, packages area:test-apps apps used to test integrations labels Feb 5, 2025

andrewlock requested review from a team as code owners February 5, 2025 13:12

github-actions bot added the area:tests unit tests, integration tests label Feb 5, 2025

andrewlock mentioned this pull request Feb 5, 2025

[Test Package Versions Bump] Updating package versions #6558

Merged

bouwkast approved these changes Feb 5, 2025

View reviewed changes

andrewlock merged commit 5bb0aa8 into master Feb 5, 2025
138 of 139 checks passed

andrewlock deleted the andrew/ci/fix-rabbitmq-datastreams branch February 5, 2025 14:51

github-actions bot added this to the vNext-v3 milestone Feb 5, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix broken `DataStreamsMonitoringRabbitMQTests` #6629

Fix broken `DataStreamsMonitoringRabbitMQTests` #6629

andrewlock commented Feb 5, 2025 •

edited

Loading

bouwkast left a comment

datadog-ddstaging bot commented Feb 5, 2025 •

edited

Loading

andrewlock commented Feb 5, 2025

andrewlock commented Feb 5, 2025

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #6629

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6629

Raw results

Raw results

Fix broken DataStreamsMonitoringRabbitMQTests #6629

Fix broken DataStreamsMonitoringRabbitMQTests #6629

Conversation

andrewlock commented Feb 5, 2025 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

bouwkast left a comment

Choose a reason for hiding this comment

datadog-ddstaging bot commented Feb 5, 2025 • edited Loading

Datadog Report

andrewlock commented Feb 5, 2025

Execution-Time Benchmarks Report ⏱️

andrewlock commented Feb 5, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Fewer allocations 🎉 in #6629

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6629

Raw results

Raw results

Fix broken `DataStreamsMonitoringRabbitMQTests` #6629

Fix broken `DataStreamsMonitoringRabbitMQTests` #6629

andrewlock commented Feb 5, 2025 •

edited

Loading

datadog-ddstaging bot commented Feb 5, 2025 •

edited

Loading