Increase GraphQL benchmark stability

[watson]

What does this PR do?

• Increase GraphQL benchmark iterations from 2 to 30.
• Increase the speed of the GraphQL benchmarks

Motivation

This is done to try and increase stability of the benchmark as the numbers collected currently fluctuate a lot, which results in a lot of warnings on PRs saying that the performance of the GraphQL instrumentation has either significantly improved or gotten worse, even though the PR shouldn't affect it.

Plugin Checklist

• Unit tests.
• TypeScript definitions.
• TypeScript tests.
• API documentation.
• CircleCI jobs/workflows.
• Plugin is exported.

Additional Notes

[watson]

• Increase GraphQL benchmark stability #5284 👈 (View in Graphite)
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.91%. Comparing base (14a3795) to head (e4b24ef).
Report is 3 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5284      +/-   ##
==========================================
+ Coverage   80.87%   80.91%   +0.04%     
==========================================
  Files         484      488       +4     
  Lines       21783    21817      +34     
==========================================
+ Hits        17616    17654      +38     
+ Misses       4167     4163       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Overall package size

Self size: 8.75 MB
Deduped: 94.95 MB
No deduping: 95.47 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.4.0 | 29.44 MB | 29.44 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.3.0 | 13.77 MB | 13.78 MB | | @datadog/pprof | 5.5.1 | 9.79 MB | 10.17 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.8.0 | 2.6 MB | 2.74 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 835.4 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[datadog-datadog-prod-us1]

Datadog Report

Branch report: watson/bench-graphql-stability
Commit report: 54940ac
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 673 Passed, 0 Skipped, 16m 17.89s Total Time

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-02-18 14:29:16

Comparing candidate commit e4b24ef in PR branch watson/bench-graphql-stability with baseline commit 14a3795 in branch master.

Found 68 performance improvements and 0 performance regressions! Performance is the same for 848 metrics, 17 unstable metrics.

scenario:plugin-graphql-control-18

• 🟩 cpu_user_time [-2.083s; -2.063s] or [-87.757%; -86.908%]
• 🟩 execution_time [-2.392s; -2.380s] or [-87.465%; -87.018%]
• 🟩 instructions [-4.7G instructions; -4.6G instructions] or [-86.980%; -85.381%]
• 🟩 max_rss_usage [-401.406MB; -400.589MB] or [-83.757%; -83.586%]

scenario:plugin-graphql-control-20

• 🟩 cpu_user_time [-1.933s; -1.873s] or [-87.417%; -84.732%]
• 🟩 execution_time [-2.234s; -2.203s] or [-86.714%; -85.530%]
• 🟩 instructions [-4.3G instructions; -4.3G instructions] or [-87.075%; -85.772%]
• 🟩 max_rss_usage [-412.376MB; -399.819MB] or [-85.610%; -83.003%]

scenario:plugin-graphql-control-22

• 🟩 cpu_user_time [-1.994s; -1.987s] or [-88.073%; -87.754%]
• 🟩 execution_time [-2.301s; -2.283s] or [-88.066%; -87.371%]
• 🟩 instructions [-4.6G instructions; -4.4G instructions] or [-89.237%; -84.944%]
• 🟩 max_rss_usage [-406.967MB; -406.638MB] or [-83.357%; -83.290%]

scenario:plugin-graphql-with-async-hooks-18

• 🟩 cpu_user_time [-4.071s; -4.030s] or [-91.941%; -91.023%]
• 🟩 execution_time [-4.428s; -4.386s] or [-91.557%; -90.695%]
• 🟩 instructions [-7.6G instructions; -7.5G instructions] or [-90.140%; -88.854%]
• 🟩 max_rss_usage [-445.196MB; -443.679MB] or [-84.211%; -83.924%]

scenario:plugin-graphql-with-async-hooks-20

• 🟩 cpu_user_time [-3.640s; -3.631s] or [-90.726%; -90.482%]
• 🟩 execution_time [-4.022s; -3.991s] or [-90.639%; -89.928%]
• 🟩 instructions [-7.0G instructions; -6.9G instructions] or [-89.957%; -89.238%]
• 🟩 max_rss_usage [-452.419MB; -451.838MB] or [-85.032%; -84.923%]

scenario:plugin-graphql-with-async-hooks-22

• 🟩 cpu_user_time [-3.822s; -3.704s] or [-93.322%; -90.442%]
• 🟩 execution_time [-4.225s; -4.077s] or [-93.148%; -89.888%]
• 🟩 instructions [-7.3G instructions; -7.0G instructions] or [-92.078%; -88.210%]
• 🟩 max_rss_usage [-468.484MB; -436.641MB] or [-87.209%; -81.282%]

scenario:plugin-graphql-with-depth-and-collapse-off-18

• 🟩 cpu_user_time [-16.651s; -15.358s] or [-94.305%; -86.981%]
• 🟩 execution_time [-17.636s; -16.401s] or [-93.746%; -87.181%]
• 🟩 instructions [-38.2G instructions; -38.1G instructions] or [-91.360%; -91.152%]
• 🟩 max_rss_usage [-1.083GB; -1.068GB] or [-88.442%; -87.249%]

scenario:plugin-graphql-with-depth-and-collapse-off-20

• 🟩 instructions [-59.2G instructions; -57.4G instructions] or [-94.624%; -91.702%]
• 🟩 max_rss_usage [-2.305GB; -2.189GB] or [-94.129%; -89.399%]

scenario:plugin-graphql-with-depth-and-collapse-off-22

• 🟩 instructions [-56.9G instructions; -56.5G instructions] or [-93.300%; -92.573%]
• 🟩 max_rss_usage [-2.336GB; -2.324GB] or [-92.005%; -91.514%]

scenario:plugin-graphql-with-depth-and-collapse-on-18

• 🟩 cpu_user_time [-9.936s; -9.539s] or [-92.547%; -88.848%]
• 🟩 execution_time [-10.572s; -10.243s] or [-91.836%; -88.976%]
• 🟩 instructions [-21.7G instructions; -21.2G instructions] or [-91.298%; -89.459%]
• 🟩 max_rss_usage [-744.993MB; -734.030MB] or [-86.201%; -84.932%]

scenario:plugin-graphql-with-depth-and-collapse-on-20

• 🟩 cpu_user_time [-16.371s; -16.010s] or [-92.945%; -90.897%]
• 🟩 execution_time [-18.109s; -17.483s] or [-93.545%; -90.311%]
• 🟩 instructions [-43.5G instructions; -43.3G instructions] or [-93.452%; -93.015%]
• 🟩 max_rss_usage [-1.991GB; -1.986GB] or [-92.169%; -91.926%]

scenario:plugin-graphql-with-depth-and-collapse-on-22

• 🟩 cpu_user_time [-16.244s; -15.517s] or [-94.437%; -90.209%]
• 🟩 execution_time [-17.894s; -17.076s] or [-94.413%; -90.095%]
• 🟩 instructions [-43.1G instructions; -42.8G instructions] or [-93.314%; -92.801%]
• 🟩 max_rss_usage [-1.990GB; -1.989GB] or [-91.689%; -91.661%]

scenario:plugin-graphql-with-depth-off-18

• 🟩 cpu_user_time [-10.209s; -9.862s] or [-92.313%; -89.171%]
• 🟩 execution_time [-10.875s; -10.534s] or [-91.925%; -89.047%]
• 🟩 instructions [-22.4G instructions; -22.3G instructions] or [-90.587%; -90.376%]
• 🟩 max_rss_usage [-731.016MB; -724.655MB] or [-85.515%; -84.771%]

scenario:plugin-graphql-with-depth-off-20

• 🟩 cpu_user_time [-17.682s; -16.075s] or [-96.502%; -87.732%]
• 🟩 execution_time [-19.228s; -17.669s] or [-95.990%; -88.208%]
• 🟩 instructions [-44.9G instructions; -44.2G instructions] or [-94.019%; -92.583%]
• 🟩 max_rss_usage [-2.001GB; -1.996GB] or [-92.149%; -91.933%]

scenario:plugin-graphql-with-depth-off-22

• 🟩 cpu_user_time [-17.010s; -15.634s] or [-96.307%; -88.516%]
• 🟩 execution_time [-18.557s; -17.215s] or [-95.783%; -88.857%]
• 🟩 instructions [-44.1G instructions; -43.9G instructions] or [-93.284%; -92.914%]
• 🟩 max_rss_usage [-2.001GB; -2.000GB] or [-91.746%; -91.661%]

scenario:plugin-graphql-with-depth-on-max-18

• 🟩 cpu_user_time [-9.728s; -9.687s] or [-90.822%; -90.442%]
• 🟩 execution_time [-10.440s; -10.306s] or [-90.942%; -89.775%]
• 🟩 instructions [-21.5G instructions; -21.5G instructions] or [-90.453%; -90.326%]
• 🟩 max_rss_usage [-743.365MB; -734.363MB] or [-86.065%; -85.022%]

scenario:plugin-graphql-with-depth-on-max-20

• 🟩 cpu_user_time [-16.631s; -16.026s] or [-93.674%; -90.265%]
• 🟩 execution_time [-18.295s; -17.599s] or [-93.775%; -90.206%]
• 🟩 instructions [-43.6G instructions; -43.3G instructions] or [-93.491%; -92.968%]
• 🟩 max_rss_usage [-1.991GB; -1.987GB] or [-92.133%; -91.955%]

scenario:plugin-graphql-with-depth-on-max-22

• 🟩 cpu_user_time [-16.526s; -15.296s] or [-95.893%; -88.753%]
• 🟩 execution_time [-18.067s; -16.907s] or [-95.309%; -89.190%]
• 🟩 instructions [-42.8G instructions; -42.8G instructions] or [-93.021%; -92.963%]
• 🟩 max_rss_usage [-1.990GB; -1.989GB] or [-91.711%; -91.652%]

[ddyurchenko]

Looks way better in BP UI.