feat(llmobs): improved LLM Observability data submission experience

[sabrenner]

What does this PR do?

need to flush out description more, but:

• eval metrics can be submitted through agent proxy
• adds new feature to detect if agent is running if agentlessEnabled is not explicitly set by checking the /info endpoint (now defaults to undefined)
  • while this might be a finicky approach for older agent versions, i think LLMObs is OK to take this approach. need to verify with team
  • if agentlessEnabled is either set to true or false explicitly, we will use old logic of either trying to force proxy the agent and logging if not possible, or if set to true, checking if an API key is available, throwing if not, and if is available then using agentless data submission

a side effect of this is that most of our tests need to specify agentlessEnabled: false, as our test agent does not have an /info endpoint and we do not set an api key value (even though we stub our writer/requests for the most part). this should be OK, as we might be fine with this failing for customers, as this is meant as an onboarding improvement. if for some reason they have an agent running without an /info endpoint, and don't have an API key, we want to alert them to this in the development/onboarding phase. A better onboarding experience in the DD UI will assist this.

TODO:

• add some error parsing to log for possible site + api key mismatch
• add some checking on the info endpoint to see if the /proxy/v2 endpoint exists

Motivation

Plugin Checklist

• Unit tests.
• TypeScript definitions.
• TypeScript tests.
• API documentation.
• CircleCI jobs/workflows.
• Plugin is exported.

Additional Notes

[github-actions]

Overall package size

Self size: 9 MB
Deduped: 101.53 MB
No deduping: 102.05 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.5.0 | 29.83 MB | 29.83 MB | | @datadog/native-appsec | 8.5.1 | 19.26 MB | 19.27 MB | | @datadog/native-iast-taint-tracking | 3.3.0 | 13.77 MB | 13.78 MB | | @datadog/pprof | 5.6.0 | 9.79 MB | 10.16 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.4.0 | 2.77 MB | 5.42 MB | | @datadog/native-iast-rewriter | 2.8.0 | 2.6 MB | 2.74 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.13.1 | 117.64 kB | 839.26 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | dc-polyfill | 0.1.6 | 24.56 kB | 24.56 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

[datadog-datadog-prod-us1]

Datadog Report

Branch report: sabrenner/llmobs-writers-changes
Commit report: 1151fd6
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 798 Passed, 0 Skipped, 10m 57.43s Total Time

[codecov]

Codecov Report

Attention: Patch coverage is 57.37705% with 26 lines in your changes missing coverage. Please review.

Project coverage is 79.52%. Comparing base (450abff) to head (2fd14dc).
Report is 17 commits behind head on master.

Files with missing lines	Patch %	Lines
packages/dd-trace/src/llmobs/index.js	38.09%	13 Missing ⚠️
packages/dd-trace/src/llmobs/writers/base.js	67.85%	9 Missing ⚠️
packages/dd-trace/src/llmobs/util.js	25.00%	3 Missing ⚠️
packages/dd-trace/src/llmobs/writers/spans.js	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5466      +/-   ##
==========================================
+ Coverage   79.49%   79.52%   +0.02%     
==========================================
  Files         509      508       -1     
  Lines       22811    22951     +140     
==========================================
+ Hits        18134    18251     +117     
- Misses       4677     4700      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-03-21 18:35:24

Comparing candidate commit 2fd14dc in PR branch sabrenner/llmobs-writers-changes with baseline commit 450abff in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 919 metrics, 14 unstable metrics.