PROF-9374: Move timeline events serialization from the hot path

[szegedi]

What does this PR do?

Moves the conversion of performance events into pprof samples from being performed in bulk at the profile call to being performed in event handler callback.

Motivation

Profile serialization and encoding are CPU intensive and contribute to latency spikes when we prepare pprof files for upload. Moving as much processing out of that code path reduces those latency spikes. The cost will now instead be amortized across PerformanceObserver callbacks.

Additional Notes

I added some of the relevant commits from the approved #4149. I don't want to merge all of #4149 until we are happy with the latencies.

Security

Datadog employees:

• If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
• This PR doesn't touch any of that.

[github-actions]

Overall package size

Self size: 6.24 MB
Deduped: 61.19 MB
No deduping: 61.94 MB

Dependency sizes

name	version	self size	total size
@datadog/native-iast-taint-tracking	1.7.0	16.71 MB	16.72 MB
@datadog/native-appsec	7.1.0	14.37 MB	14.38 MB
@datadog/pprof	5.1.0	8.83 MB	9.68 MB
protobufjs	7.2.5	2.77 MB	6.56 MB
@datadog/native-iast-rewriter	2.3.0	2.15 MB	2.24 MB
@opentelemetry/core	1.14.0	872.87 kB	1.47 MB
@datadog/native-metrics	2.0.0	898.77 kB	1.3 MB
@opentelemetry/api	1.4.1	780.32 kB	780.32 kB
import-in-the-middle	1.7.3	67.62 kB	731.01 kB
pprof-format	2.0.7	588.12 kB	588.12 kB
msgpack-lite	0.1.26	201.16 kB	281.59 kB
opentracing	0.14.7	194.81 kB	194.81 kB
semver	7.5.4	93.4 kB	123.8 kB
@datadog/sketches-js	2.1.0	109.9 kB	109.9 kB
lodash.sortby	4.7.0	75.76 kB	75.76 kB
lru-cache	7.14.0	74.95 kB	74.95 kB
ipaddr.js	2.1.0	60.23 kB	60.23 kB
ignore	5.2.4	51.22 kB	51.22 kB
int64-buffer	0.1.10	49.18 kB	49.18 kB
shell-quote	1.8.1	44.96 kB	44.96 kB
istanbul-lib-coverage	3.2.0	29.34 kB	29.34 kB
tlhunter-sorted-set	0.1.0	24.94 kB	24.94 kB
limiter	1.1.5	23.17 kB	23.17 kB
dc-polyfill	0.1.4	23.1 kB	23.1 kB
retry	0.13.1	18.85 kB	18.85 kB
node-abort-controller	3.1.1	16.89 kB	16.89 kB
jest-docblock	29.7.0	8.99 kB	12.76 kB
crypto-randomuuid	1.0.0	11.18 kB	11.18 kB
path-to-regexp	0.1.7	6.78 kB	6.78 kB
koalas	1.0.2	6.47 kB	6.47 kB
methods	1.1.2	5.29 kB	5.29 kB
module-details-from-path	1.0.3	4.47 kB	4.47 kB

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[codecov]

Codecov Report

Attention: Patch coverage is 41.17647% with 20 lines in your changes are missing coverage. Please review.

Project coverage is 85.22%. Comparing base (76d76e0) to head (586c2e8).

Files	Patch %	Lines
...ackages/dd-trace/src/profiling/profilers/events.js	41.17%	20 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4173      +/-   ##
==========================================
+ Coverage   85.06%   85.22%   +0.15%     
==========================================
  Files         247      247              
  Lines       10953    10948       -5     
  Branches       33       33              
==========================================
+ Hits         9317     9330      +13     
+ Misses       1636     1618      -18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-03-21 10:35:32

Comparing candidate commit 586c2e8 in PR branch szegedi/serialize-timeline with baseline commit 76d76e0 in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 258 metrics, 7 unstable metrics.

scenario:plugin-graphql-with-depth-off-18

• 🟩 max_rss_usage [-138.105MB; -76.999MB] or [-14.609%; -8.145%]

[nsavoire]

LGTM !

[szegedi]

No real changes, just resolved conflicts with #4146