Pin the lang version in the Nuke build project #6411

andrewlock · 2024-12-09T14:06:36Z

Summary of changes

Pin the language version to C#11 (i.e. .NET 7)

Reason for change

Without this, Rider/VS prompts you to use C#12/13 constructs, which then cause the build to fail.

Implementation details

Just pin the version

Test coverage

It works

datadog-ddstaging · 2024-12-09T14:57:39Z

Datadog Report

Branch report: andrew/ci/fix-lang-version-in-build-project
Commit report: b52fb44
Test service: dd-trace-dotnet

✅ 0 Failed, 457915 Passed, 2838 Skipped, 19h 55m 47.98s Total Time
❄️ 1 New Flaky

New Flaky Tests (1)

TestInstrumentedUnitTests - Datadog.Trace.Security.IntegrationTests.Iast.IastInstrumentationUnitTests - Last Failure
Expand for error
```
 Expected exit code: 0, actual exit code: 1.
```

andrewlock · 2024-12-09T16:17:53Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6411) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (70ms)  : 67, 72
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6411) - mean (979ms)  : 955, 1002
     .   : milestone, 979,
    master - mean (988ms)  : 963, 1014
     .   : milestone, 988,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6411) - mean (109ms)  : 106, 111
     .   : milestone, 109,
    master - mean (110ms)  : 107, 114
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (6411) - mean (680ms)  : 662, 698
     .   : milestone, 680,
    master - mean (687ms)  : 669, 704
     .   : milestone, 687,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6411) - mean (92ms)  : 89, 94
     .   : milestone, 92,
    master - mean (93ms)  : 91, 96
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (6411) - mean (630ms)  : 612, 647
     .   : milestone, 630,
    master - mean (643ms)  : 626, 660
     .   : milestone, 643,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6411) - mean (190ms)  : 186, 195
     .   : milestone, 190,
    master - mean (190ms)  : 186, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6411) - mean (1,096ms)  : 1067, 1124
     .   : milestone, 1096,
    master - mean (1,091ms)  : 1063, 1119
     .   : milestone, 1091,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6411) - mean (277ms)  : 272, 282
     .   : milestone, 277,
    master - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6411) - mean (868ms)  : 839, 897
     .   : milestone, 868,
    master - mean (869ms)  : 839, 900
     .   : milestone, 869,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6411) - mean (266ms)  : 263, 270
     .   : milestone, 266,
    master - mean (266ms)  : 262, 269
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6411) - mean (851ms)  : 813, 889
     .   : milestone, 851,
    master - mean (850ms)  : 818, 882
     .   : milestone, 850,

andrewlock · 2024-12-09T17:18:56Z

Benchmarks Report for tracer 🐌

Benchmarks for #6411 compared to master:

4 benchmarks are slower, with geometric mean 1.176
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.93μs	39.7ns	186ns	0.0159	0.00797	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	56.5ns	339ns	0.0148	0.00493	0	5.8 KB
master	`StartStopWithChild`	net472	16.3μs	61.9ns	240ns	1.04	0.307	0.105	6.23 KB
#6411	`StartStopWithChild`	net6.0	8.14μs	42.6ns	221ns	0.0165	0.00826	0	5.61 KB
#6411	`StartStopWithChild`	netcoreapp3.1	9.94μs	53ns	300ns	0.023	0.00921	0	5.8 KB
#6411	`StartStopWithChild`	net472	16.3μs	47.1ns	182ns	1.05	0.323	0.0969	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	477μs	2.24μs	8.67μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	665μs	436ns	1.63μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	841μs	349ns	1.3μs	0.422	3.3 KB
#6411	`WriteAndFlushEnrichedTraces`	net6.0	491μs	632ns	2.45μs	0	2.7 KB
#6411	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	658μs	280ns	1.08μs	0	2.7 KB
#6411	`WriteAndFlushEnrichedTraces`	net472	844μs	543ns	2.1μs	0.419	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	150μs	877ns	7.8μs	0.138	14.47 KB
master	`SendRequest`	netcoreapp3.1	172μs	998ns	8.41μs	0.158	17.27 KB
master	`SendRequest`	net472	0.00094ns	0.00047ns	0.00169ns	0	0 b
#6411	`SendRequest`	net6.0	149μs	828ns	5.49μs	0.16	14.47 KB
#6411	`SendRequest`	netcoreapp3.1	171μs	1.02μs	10.1μs	0.161	17.27 KB
#6411	`SendRequest`	net472	0.000559ns	0.000229ns	0.000826ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	559μs	2.78μs	13.6μs	0.601	0	0	41.65 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	667μs	3.6μs	20.1μs	0.319	0	0	41.79 KB
master	`WriteAndFlushEnrichedTraces`	net472	838μs	3.19μs	12.3μs	8.87	2.82	0.403	53.33 KB
#6411	`WriteAndFlushEnrichedTraces`	net6.0	578μs	2.74μs	15μs	0.551	0	0	41.74 KB
#6411	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	696μs	4.01μs	34.2μs	0.322	0	0	41.9 KB
#6411	`WriteAndFlushEnrichedTraces`	net472	854μs	4.25μs	17.5μs	8.33	2.5	0.417	53.29 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.4μs	1.05ns	4.07ns	0.014	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.77μs	2.31ns	8.93ns	0.0142	0	1.02 KB
master	`ExecuteNonQuery`	net472	1.99μs	1.6ns	6.21ns	0.156	0.000999	987 B
#6411	`ExecuteNonQuery`	net6.0	1.33μs	0.899ns	3.36ns	0.0142	0	1.02 KB
#6411	`ExecuteNonQuery`	netcoreapp3.1	1.79μs	3.58ns	13.9ns	0.0133	0	1.02 KB
#6411	`ExecuteNonQuery`	net472	2.08μs	1ns	3.47ns	0.156	0.00104	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0	1.140	1,165.56	1,328.16

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.17μs	0.913ns	3.42ns	0.0135	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.52μs	0.919ns	3.56ns	0.013	976 B
master	`CallElasticsearch`	net472	2.53μs	1.08ns	4.06ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.25μs	0.451ns	1.69ns	0.013	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.61μs	0.657ns	2.46ns	0.0137	1.02 KB
master	`CallElasticsearchAsync`	net472	2.57μs	0.822ns	3.07ns	0.167	1.05 KB
#6411	`CallElasticsearch`	net6.0	1.33μs	2ns	7.77ns	0.0133	976 B
#6411	`CallElasticsearch`	netcoreapp3.1	1.52μs	0.748ns	2.7ns	0.013	976 B
#6411	`CallElasticsearch`	net472	2.6μs	2.31ns	8.96ns	0.157	995 B
#6411	`CallElasticsearchAsync`	net6.0	1.23μs	0.413ns	1.55ns	0.0132	952 B
#6411	`CallElasticsearchAsync`	netcoreapp3.1	1.56μs	1.02ns	3.8ns	0.014	1.02 KB
#6411	`CallElasticsearchAsync`	net472	2.64μs	1.28ns	4.94ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.27μs	0.882ns	3.42ns	0.0133	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.62μs	1.41ns	5.26ns	0.0123	952 B
master	`ExecuteAsync`	net472	1.85μs	0.582ns	2.26ns	0.145	915 B
#6411	`ExecuteAsync`	net6.0	1.3μs	0.423ns	1.58ns	0.0137	952 B
#6411	`ExecuteAsync`	netcoreapp3.1	1.71μs	1.16ns	4.33ns	0.0129	952 B
#6411	`ExecuteAsync`	net472	1.89μs	0.401ns	1.55ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.41μs	2.2ns	8.22ns	0.0329	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.33μs	4.73ns	18.3ns	0.0378	2.85 KB
master	`SendAsync`	net472	7.28μs	1.79ns	6.71ns	0.495	3.12 KB
#6411	`SendAsync`	net6.0	4.57μs	1.56ns	5.84ns	0.032	2.31 KB
#6411	`SendAsync`	netcoreapp3.1	5.3μs	4.42ns	15.9ns	0.037	2.85 KB
#6411	`SendAsync`	net472	7.37μs	1.65ns	6.37ns	0.493	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.56μs	0.648ns	2.43ns	0.0227	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.33μs	1.04ns	4.04ns	0.0224	1.64 KB
master	`EnrichedLog`	net472	2.66μs	0.633ns	2.37ns	0.249	1.57 KB
#6411	`EnrichedLog`	net6.0	1.48μs	0.858ns	3.21ns	0.023	1.64 KB
#6411	`EnrichedLog`	netcoreapp3.1	2.23μs	2.98ns	11.6ns	0.0222	1.64 KB
#6411	`EnrichedLog`	net472	2.51μs	1.79ns	6.94ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	120μs	108ns	419ns	0.0598	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	122μs	78ns	270ns	0	0	4.28 KB
master	`EnrichedLog`	net472	153μs	86.2ns	334ns	0.689	0.23	4.46 KB
#6411	`EnrichedLog`	net6.0	121μs	119ns	444ns	0	0	4.28 KB
#6411	`EnrichedLog`	netcoreapp3.1	122μs	129ns	484ns	0	0	4.28 KB
#6411	`EnrichedLog`	net472	152μs	89.1ns	345ns	0.681	0.227	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.18μs	1.5ns	5.42ns	0.0303	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.29μs	2.02ns	7.83ns	0.0299	2.2 KB
master	`EnrichedLog`	net472	5μs	1.43ns	5.53ns	0.319	2.02 KB
#6411	`EnrichedLog`	net6.0	3.01μs	0.967ns	3.74ns	0.0301	2.2 KB
#6411	`EnrichedLog`	netcoreapp3.1	4.23μs	1.8ns	6.96ns	0.0297	2.2 KB
#6411	`EnrichedLog`	net472	4.99μs	1.24ns	4.81ns	0.319	2.02 KB

Benchmarks.Trace.RedisBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.RedisBenchmark.SendReceive‑netcoreapp3.1	1.149	1,643.81	1,888.30

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.36μs	0.419ns	1.57ns	0.0163	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.64μs	0.723ns	2.71ns	0.0148	1.14 KB
master	`SendReceive`	net472	2.09μs	1.14ns	4.42ns	0.183	1.16 KB
#6411	`SendReceive`	net6.0	1.42μs	0.62ns	2.4ns	0.0157	1.14 KB
#6411	`SendReceive`	netcoreapp3.1	1.88μs	6.14ns	23.8ns	0.0157	1.14 KB
#6411	`SendReceive`	net472	1.98μs	1.29ns	4.81ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.79μs	1.08ns	4.17ns	0.0223	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.88μs	1.36ns	5.1ns	0.0217	1.65 KB
master	`EnrichedLog`	net472	4.41μs	1.34ns	5ns	0.322	2.04 KB
#6411	`EnrichedLog`	net6.0	2.75μs	1.19ns	4.62ns	0.0233	1.6 KB
#6411	`EnrichedLog`	netcoreapp3.1	3.82μs	1.9ns	7.35ns	0.0211	1.65 KB
#6411	`EnrichedLog`	net472	4.5μs	4.04ns	15.6ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.175	481.09	565.05

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	465ns	0.701ns	2.72ns	0.00818	576 B
master	`StartFinishSpan`	netcoreapp3.1	554ns	0.866ns	3.35ns	0.00776	576 B
master	`StartFinishSpan`	net472	710ns	1.07ns	4.13ns	0.0915	578 B
master	`StartFinishScope`	net6.0	481ns	0.636ns	2.46ns	0.00985	696 B
master	`StartFinishScope`	netcoreapp3.1	776ns	0.843ns	3.26ns	0.00927	696 B
master	`StartFinishScope`	net472	823ns	2.35ns	9.09ns	0.104	658 B
#6411	`StartFinishSpan`	net6.0	453ns	0.663ns	2.57ns	0.00808	576 B
#6411	`StartFinishSpan`	netcoreapp3.1	553ns	0.785ns	2.83ns	0.00775	576 B
#6411	`StartFinishSpan`	net472	666ns	1.08ns	4.05ns	0.0916	578 B
#6411	`StartFinishScope`	net6.0	564ns	0.635ns	2.46ns	0.00969	696 B
#6411	`StartFinishScope`	netcoreapp3.1	769ns	0.979ns	3.79ns	0.00922	696 B
#6411	`StartFinishScope`	net472	889ns	1.22ns	4.72ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6411

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.245	602.96	750.45

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	602ns	0.69ns	2.67ns	0.0097	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	956ns	1.25ns	4.86ns	0.00942	696 B
master	`RunOnMethodBegin`	net472	1.16μs	1.96ns	7.58ns	0.104	658 B
#6411	`RunOnMethodBegin`	net6.0	750ns	1.26ns	4.87ns	0.00979	696 B
#6411	`RunOnMethodBegin`	netcoreapp3.1	912ns	0.905ns	3.39ns	0.00935	696 B
#6411	`RunOnMethodBegin`	net472	1.08μs	1.95ns	7.55ns	0.104	658 B

## Summary of changes Pin the language version to C#11 (i.e. .NET 7) ## Reason for change Without this, Rider/VS prompts you to use C#12/13 constructs, which then cause the build to fail. ## Implementation details Just pin the version ## Test coverage It works

Pin the lang version in the build

b52fb44

andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Dec 9, 2024

andrewlock requested a review from a team as a code owner December 9, 2024 14:06

kevingosse approved these changes Dec 9, 2024

View reviewed changes

andrewlock merged commit 2157b3e into master Dec 9, 2024
61 of 65 checks passed

andrewlock deleted the andrew/ci/fix-lang-version-in-build-project branch December 9, 2024 18:43

github-actions bot added this to the vNext-v3 milestone Dec 9, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pin the lang version in the Nuke build project #6411

Pin the lang version in the Nuke build project #6411

andrewlock commented Dec 9, 2024

datadog-ddstaging bot commented Dec 9, 2024 •

edited

Loading

andrewlock commented Dec 9, 2024

andrewlock commented Dec 9, 2024

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6411

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6411

Raw results

Raw results

Slower ⚠️ in #6411

Raw results

Slower ⚠️ in #6411

Raw results

Pin the lang version in the Nuke build project #6411

Pin the lang version in the Nuke build project #6411

Conversation

andrewlock commented Dec 9, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

datadog-ddstaging bot commented Dec 9, 2024 • edited Loading

Datadog Report

New Flaky Tests (1)

andrewlock commented Dec 9, 2024

Execution-Time Benchmarks Report ⏱️

andrewlock commented Dec 9, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6411

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6411

Raw results

Raw results

Slower ⚠️ in #6411

Raw results

Slower ⚠️ in #6411

Raw results

datadog-ddstaging bot commented Dec 9, 2024 •

edited

Loading