Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(llmobs): additional options like for manual span instrumentation are properly recognized #5424

Merged
merged 2 commits into from
Mar 18, 2025

Conversation

sabrenner
Copy link
Collaborator

@sabrenner sabrenner commented Mar 17, 2025

What does this PR do?

Fixes a bug where specifying additional LLM Observability span options other than name and kind on manually instrumented spans were not passed on appropriately to the tagger. This was due to a bad refactor in #4960 for _activate in which a parameter name was changed without spreading it in correctly.

There should have been a test for this (there was on the tagger side, but not on the SDK side to assert proper usage) - tests are now in place.

We would previously try and pass this object shape into the tagger.registerLLMObsSpan options object:

{
  kind: ...
  options: {
    mlApp: ...,
    sessionId: ...,
    ...
  }
}

instead of

{
  kind: ...
  mlApp: ...,
  sessionId: ...
  ...
}

which leads to those options not being correctly read in the tagger, as the tagger assumes they're all just properties on the same object.

Motivation

MLOB-2419

Copy link

github-actions bot commented Mar 17, 2025

Overall package size

Self size: 8.96 MB
Deduped: 101.5 MB
No deduping: 102.01 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.5.0 | 29.83 MB | 29.83 MB | | @datadog/native-appsec | 8.5.0 | 19.26 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.3.0 | 13.77 MB | 13.78 MB | | @datadog/pprof | 5.5.1 | 9.79 MB | 10.17 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.4.0 | 2.77 MB | 5.42 MB | | @datadog/native-iast-rewriter | 2.8.0 | 2.6 MB | 2.74 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.13.1 | 117.64 kB | 839.26 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.2 | 53.63 kB | 53.63 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | dc-polyfill | 0.1.6 | 24.56 kB | 24.56 kB | | shell-quote | 1.8.2 | 23.54 kB | 23.54 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

Copy link

codecov bot commented Mar 17, 2025

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 79.47%. Comparing base (4762662) to head (665ac47).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
packages/dd-trace/src/llmobs/sdk.js 33.33% 2 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #5424   +/-   ##
=======================================
  Coverage   79.47%   79.47%           
=======================================
  Files         509      509           
  Lines       22811    22811           
=======================================
  Hits        18128    18128           
  Misses       4683     4683           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter
Copy link

pr-commenter bot commented Mar 17, 2025

Benchmarks

Benchmark execution time: 2025-03-18 13:37:58

Comparing candidate commit 665ac47 in PR branch sabrenner/fix-llmobs-additional-options-manual-inst with baseline commit 4762662 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 912 metrics, 21 unstable metrics.

@sabrenner sabrenner marked this pull request as ready for review March 17, 2025 14:44
@sabrenner sabrenner requested a review from a team as a code owner March 17, 2025 14:44
@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Mar 17, 2025

Datadog Report

Branch report: sabrenner/fix-llmobs-additional-options-manual-inst
Commit report: 9dfbd48
Test service: dd-trace-js-integration-tests

✅ 0 Failed, 801 Passed, 0 Skipped, 17m 17.74s Total Time

@sabrenner sabrenner enabled auto-merge (squash) March 17, 2025 14:59
@sabrenner sabrenner merged commit 28eaa41 into master Mar 18, 2025
414 of 420 checks passed
@sabrenner sabrenner deleted the sabrenner/fix-llmobs-additional-options-manual-inst branch March 18, 2025 13:41
juan-fernandez pushed a commit that referenced this pull request Mar 18, 2025
…are properly recognized (#5424)

* fix approach to passing additional llm observability span options to the tagger
@juan-fernandez juan-fernandez mentioned this pull request Mar 18, 2025
juan-fernandez pushed a commit that referenced this pull request Mar 19, 2025
…are properly recognized (#5424)

* fix approach to passing additional llm observability span options to the tagger
@juan-fernandez juan-fernandez mentioned this pull request Mar 19, 2025
juan-fernandez pushed a commit that referenced this pull request Mar 19, 2025
…are properly recognized (#5424)

* fix approach to passing additional llm observability span options to the tagger
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants