Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move the Nuke arguments into the gitlab file #6611

Merged
merged 2 commits into from
Jan 31, 2025

Conversation

andrewlock
Copy link
Member

Summary of changes

Allow specifying the Nuke targets to run in the gitlab file

Reason for change

Currently they're baked into the build image. That means if we need to change the args, we need to build a whole new image

Implementation details

Pass the arguments as commands in the docker run call. These get passed as arguments to the entrypoint.bat script. The script then reads the args and splats them into the Nuke call.

Test coverage

I've tested the splatting, but an end-to-end test requires rebuilding the image. The good news is that this change works with the old image (it's a no-op) so it's safe to merge anyway.

Other details

That docker line was getting a bit hefty, so split it for "readability"

@andrewlock andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jan 30, 2025
@andrewlock andrewlock requested a review from a team as a code owner January 30, 2025 17:58
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Jan 30, 2025

Datadog Report

Branch report: andrew/gitlab-improvements
Commit report: 8eb9d66
Test service: dd-trace-dotnet

✅ 0 Failed, 240749 Passed, 2083 Skipped, 18h 50m 24.18s Total Time

@andrewlock
Copy link
Member Author

andrewlock commented Jan 30, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6611) - mean (69ms)  : 67, 71
     .   : milestone, 69,
    master - mean (69ms)  : 66, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6611) - mean (980ms)  : 957, 1003
     .   : milestone, 980,
    master - mean (981ms)  : 962, 1000
     .   : milestone, 981,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6611) - mean (108ms)  : 105, 110
     .   : milestone, 108,
    master - mean (108ms)  : 106, 111
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6611) - mean (675ms)  : 654, 696
     .   : milestone, 675,
    master - mean (676ms)  : 662, 691
     .   : milestone, 676,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6611) - mean (91ms)  : 90, 93
     .   : milestone, 91,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6611) - mean (630ms)  : 613, 647
     .   : milestone, 630,
    master - mean (632ms)  : 616, 648
     .   : milestone, 632,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6611) - mean (189ms)  : 185, 194
     .   : milestone, 189,
    master - mean (190ms)  : 186, 193
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6611) - mean (1,085ms)  : 1056, 1113
     .   : milestone, 1085,
    master - mean (1,089ms)  : 1061, 1117
     .   : milestone, 1089,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6611) - mean (276ms)  : 270, 282
     .   : milestone, 276,
    master - mean (277ms)  : 270, 283
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (6611) - mean (863ms)  : 828, 897
     .   : milestone, 863,
    master - mean (869ms)  : 841, 898
     .   : milestone, 869,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6611) - mean (264ms)  : 260, 269
     .   : milestone, 264,
    master - mean (264ms)  : 259, 268
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (6611) - mean (842ms)  : 811, 872
     .   : milestone, 842,
    master - mean (847ms)  : 809, 884
     .   : milestone, 847,

Loading

@lucaspimentel lucaspimentel changed the title Move the definition of the ags to pass to Nuke into the gitlab file Move the Nuke arguments into the gitlab file Jan 30, 2025
@lucaspimentel lucaspimentel requested a review from a team January 30, 2025 18:48
@andrewlock
Copy link
Member Author

Benchmarks Report for tracer 🐌

Benchmarks for #6611 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.155
  • 2 benchmarks are slower, with geometric mean 1.152
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.06μs 45.9ns 331ns 0.017 0.00851 0 5.61 KB
master StartStopWithChild netcoreapp3.1 10.3μs 56.2ns 351ns 0.0151 0.00502 0 5.81 KB
master StartStopWithChild net472 16.1μs 47.7ns 185ns 1.05 0.327 0.0958 6.21 KB
#6611 StartStopWithChild net6.0 8μs 44.6ns 296ns 0.0153 0.00383 0 5.61 KB
#6611 StartStopWithChild netcoreapp3.1 9.81μs 51.2ns 246ns 0.0231 0.00924 0 5.8 KB
#6611 StartStopWithChild net472 16μs 49.2ns 191ns 1.05 0.324 0.108 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 475μs 163ns 609ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 673μs 222ns 830ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 859μs 393ns 1.52μs 0.422 0 0 3.3 KB
#6611 WriteAndFlushEnrichedTraces net6.0 490μs 197ns 765ns 0 0 0 2.7 KB
#6611 WriteAndFlushEnrichedTraces netcoreapp3.1 651μs 247ns 926ns 0 0 0 2.7 KB
#6611 WriteAndFlushEnrichedTraces net472 854μs 652ns 2.52μs 0.422 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 127μs 449ns 1.74μs 0.19 0 0 14.48 KB
master SendRequest netcoreapp3.1 144μs 316ns 1.22μs 0.213 0 0 17.27 KB
master SendRequest net472 0.000593ns 0.00042ns 0.00151ns 0 0 0 0 b
#6611 SendRequest net6.0 129μs 406ns 1.57μs 0.194 0 0 14.47 KB
#6611 SendRequest netcoreapp3.1 146μs 361ns 1.35μs 0.22 0 0 17.27 KB
#6611 SendRequest net472 0.000599ns 0.000242ns 0.000936ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 545μs 2.51μs 10.1μs 0.536 0 0 41.65 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 648μs 3.52μs 20.2μs 0.322 0 0 41.66 KB
master WriteAndFlushEnrichedTraces net472 832μs 3.07μs 11.9μs 8.28 2.48 0.414 53.3 KB
#6611 WriteAndFlushEnrichedTraces net6.0 562μs 2.69μs 11.1μs 0.551 0 0 41.59 KB
#6611 WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 2.84μs 10.6μs 0.345 0 0 41.84 KB
#6611 WriteAndFlushEnrichedTraces net472 845μs 3.14μs 12.2μs 8.33 2.5 0.417 53.27 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.29μs 0.889ns 3.32ns 0.0141 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.77μs 1.19ns 4.59ns 0.0139 0 0 1.02 KB
master ExecuteNonQuery net472 2.08μs 1.29ns 4.84ns 0.156 0.00104 0 987 B
#6611 ExecuteNonQuery net6.0 1.18μs 0.903ns 3.5ns 0.0142 0 0 1.02 KB
#6611 ExecuteNonQuery netcoreapp3.1 1.8μs 1.91ns 7.41ns 0.0135 0 0 1.02 KB
#6611 ExecuteNonQuery net472 2.04μs 1.52ns 5.89ns 0.156 0.00102 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6611

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.157 1,277.64 1,104.35

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.28μs 0.583ns 2.18ns 0.0134 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.65μs 0.807ns 3.02ns 0.0129 0 0 976 B
master CallElasticsearch net472 2.64μs 1.92ns 7.43ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.32μs 1.29ns 4.45ns 0.0132 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.64μs 1.04ns 3.77ns 0.0138 0 0 1.02 KB
master CallElasticsearchAsync net472 2.65μs 1.23ns 4.77ns 0.166 0 0 1.05 KB
#6611 CallElasticsearch net6.0 1.1μs 0.359ns 1.34ns 0.0139 0 0 976 B
#6611 CallElasticsearch netcoreapp3.1 1.53μs 0.46ns 1.59ns 0.013 0 0 976 B
#6611 CallElasticsearch net472 2.61μs 1.65ns 6.4ns 0.157 0 0 995 B
#6611 CallElasticsearchAsync net6.0 1.26μs 0.443ns 1.72ns 0.0132 0 0 952 B
#6611 CallElasticsearchAsync netcoreapp3.1 1.74μs 0.585ns 2.19ns 0.0138 0 0 1.02 KB
#6611 CallElasticsearchAsync net472 2.6μs 1.47ns 5.7ns 0.167 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.26μs 0.447ns 1.67ns 0.0132 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.55μs 1.59ns 6.15ns 0.0132 0 0 952 B
master ExecuteAsync net472 1.84μs 0.444ns 1.72ns 0.145 0 0 915 B
#6611 ExecuteAsync net6.0 1.33μs 2.12ns 8.22ns 0.0132 0 0 952 B
#6611 ExecuteAsync netcoreapp3.1 1.57μs 1.13ns 4.09ns 0.0126 0 0 952 B
#6611 ExecuteAsync net472 1.8μs 0.491ns 1.84ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.39μs 1.68ns 6.28ns 0.0328 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.41μs 1.16ns 4.16ns 0.038 0 0 2.85 KB
master SendAsync net472 7.35μs 1.98ns 7.66ns 0.493 0 0 3.12 KB
#6611 SendAsync net6.0 4.35μs 1.69ns 6.32ns 0.0307 0 0 2.31 KB
#6611 SendAsync netcoreapp3.1 5.35μs 4.2ns 16.3ns 0.0376 0 0 2.85 KB
#6611 SendAsync net472 7.4μs 1.93ns 7.23ns 0.493 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.54μs 0.83ns 3.1ns 0.0231 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.23μs 1.07ns 3.99ns 0.0215 0 0 1.64 KB
master EnrichedLog net472 2.7μs 7.06ns 27.4ns 0.249 0 0 1.57 KB
#6611 EnrichedLog net6.0 1.43μs 0.632ns 2.45ns 0.0229 0 0 1.64 KB
#6611 EnrichedLog netcoreapp3.1 2.1μs 0.815ns 2.82ns 0.0222 0 0 1.64 KB
#6611 EnrichedLog net472 2.58μs 0.868ns 3.36ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 117μs 87ns 337ns 0.0582 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 120μs 174ns 673ns 0 0 0 4.28 KB
master EnrichedLog net472 150μs 152ns 588ns 0.676 0.225 0 4.46 KB
#6611 EnrichedLog net6.0 114μs 118ns 457ns 0.0571 0 0 4.28 KB
#6611 EnrichedLog netcoreapp3.1 120μs 120ns 464ns 0.0596 0 0 4.28 KB
#6611 EnrichedLog net472 150μs 84.2ns 315ns 0.672 0.224 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.05μs 1.08ns 4.19ns 0.0305 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.17μs 1.55ns 6.01ns 0.0292 0 0 2.2 KB
master EnrichedLog net472 4.85μs 1.27ns 4.75ns 0.319 0 0 2.02 KB
#6611 EnrichedLog net6.0 3.03μs 1ns 3.76ns 0.0304 0 0 2.2 KB
#6611 EnrichedLog netcoreapp3.1 4.24μs 1.11ns 4.14ns 0.0297 0 0 2.2 KB
#6611 EnrichedLog net472 4.85μs 2.2ns 8.21ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.42μs 0.581ns 2.25ns 0.0163 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.7μs 1.01ns 3.79ns 0.0154 0 0 1.14 KB
master SendReceive net472 2.14μs 0.993ns 3.44ns 0.183 0 0 1.16 KB
#6611 SendReceive net6.0 1.4μs 1.1ns 4.25ns 0.0161 0 0 1.14 KB
#6611 SendReceive netcoreapp3.1 1.74μs 1.54ns 5.95ns 0.0157 0 0 1.14 KB
#6611 SendReceive net472 2.09μs 0.721ns 2.7ns 0.183 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.58μs 2.23ns 8.64ns 0.0218 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 4.1μs 1.09ns 4.22ns 0.0224 0 0 1.65 KB
master EnrichedLog net472 4.42μs 3.37ns 12.6ns 0.323 0 0 2.04 KB
#6611 EnrichedLog net6.0 2.8μs 1.43ns 5.34ns 0.0223 0 0 1.6 KB
#6611 EnrichedLog netcoreapp3.1 3.86μs 2.7ns 10.5ns 0.0214 0 0 1.65 KB
#6611 EnrichedLog net472 4.44μs 2.24ns 8.39ns 0.323 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6611

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.172 562.96 659.58
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.132 407.13 461.01

Faster 🎉 in #6611

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.154 826.13 716.15

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 407ns 0.213ns 0.824ns 0.00816 0 0 576 B
master StartFinishSpan netcoreapp3.1 563ns 0.495ns 1.78ns 0.0076 0 0 576 B
master StartFinishSpan net472 613ns 1.02ns 3.96ns 0.0916 0 0 578 B
master StartFinishScope net6.0 509ns 0.783ns 3.03ns 0.00977 0 0 696 B
master StartFinishScope netcoreapp3.1 744ns 0.896ns 3.47ns 0.00928 0 0 696 B
master StartFinishScope net472 827ns 1.9ns 7.37ns 0.105 0 0 658 B
#6611 StartFinishSpan net6.0 461ns 0.571ns 2.21ns 0.00807 0 0 576 B
#6611 StartFinishSpan netcoreapp3.1 660ns 0.741ns 2.77ns 0.00761 0 0 576 B
#6611 StartFinishSpan net472 654ns 0.892ns 3.46ns 0.0918 0 0 578 B
#6611 StartFinishScope net6.0 528ns 0.852ns 3.19ns 0.00986 0 0 696 B
#6611 StartFinishScope netcoreapp3.1 692ns 1.4ns 5.44ns 0.00939 0 0 696 B
#6611 StartFinishScope net472 716ns 1.22ns 4.72ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 622ns 3.33ns 16ns 0.00974 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 942ns 4.19ns 16.2ns 0.00948 0 0 696 B
master RunOnMethodBegin net472 1.08μs 2.04ns 7.35ns 0.104 0 0 658 B
#6611 RunOnMethodBegin net6.0 599ns 1.03ns 3.98ns 0.0096 0 0 696 B
#6611 RunOnMethodBegin netcoreapp3.1 934ns 1.3ns 4.88ns 0.00942 0 0 696 B
#6611 RunOnMethodBegin net472 1.07μs 1.8ns 6.5ns 0.104 0 0 658 B

@andrewlock andrewlock force-pushed the andrew/gitlab-improvements branch from 5ae7214 to 45cf01f Compare January 31, 2025 07:11
Comment on lines +34 to +35
DOTNET_DOWNLOAD_URL="https://download.visualstudio.microsoft.com/download/pr/5f46239c-783c-4d49-a4a2-cd5b0a47ec51/9b72af54efd90a3874b63e4dd43855e7/dotnet-sdk-9.0.102-win-x64.exe" \
DOTNET_SHA512="91505782b13937392bd73d1531c01807275ef476f9e37f8ef22c2cee4b19be8282207149b4eb958668dee0c05cef02b0a6bc375b71e8e94864c3d89dea7ba534"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The URLs seem to have changed at some point, maybe related to #6484?

@andrewlock andrewlock merged commit 642f1cb into master Jan 31, 2025
132 of 138 checks passed
@andrewlock andrewlock deleted the andrew/gitlab-improvements branch January 31, 2025 08:57
@github-actions github-actions bot added this to the vNext-v3 milestone Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants