lib: add built-in OpenTelemetry tracing by bengl · Pull Request #66587 · nodejs/node

bengl · 2026-10-08T05:08:31Z

Adds experimental OpenTelemetry tracing support, as described in the included docs. Tracing is activated via environment variables only and requires the --experimental-otel flag.

There is no programmatic API yet. Custom instrumentation is not supported.

Assisted-by: pi:glm-5.3

This is a re-hash of #61907 restricting it to the OOTB behaviour only, leaving any API surface as an exercise for future PRs (and decisions around them). All configuration is currently through env vars. Most if not all still-relevant review comments from the previous PR are addressed here. In addition, a benchmark is added to measure future improvements.

Metrics and logging signals are also left to future PRs.

nodejs-github-bot · 2026-10-08T05:08:37Z

Review requested:

@nodejs/config
@nodejs/performance

codecov · 2026-10-08T06:52:04Z

Codecov Report

❌ Patch coverage is 92.51185% with 79 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.43%. Comparing base (7514ef8) to head (b3d5999).
⚠️ Report is 11 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/otel/flush.js	89.62%	35 Missing and 1 partial ⚠️
lib/internal/otel/instrumentations.js	90.87%	20 Missing and 4 partials ⚠️
lib/internal/otel/span.js	95.19%	8 Missing and 2 partials ⚠️
lib/internal/otel/core.js	93.38%	8 Missing ⚠️
lib/internal/process/pre_execution.js	98.24%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #66587      +/-   ##
==========================================
- Coverage   92.79%   90.43%   -2.36%     
==========================================
  Files         422      796     +374     
  Lines      193669   277648   +83979     
  Branches    29886    53308   +23422     
==========================================
+ Hits       179712   251102   +71390     
- Misses      13630    16948    +3318     
- Partials      327     9598    +9271

Files with missing lines	Coverage Δ
lib/internal/otel/id.js	`100.00% <100.00%> (ø)`
src/node_options.cc	`81.81% <100.00%> (ø)`
src/node_options.h	`95.67% <100.00%> (ø)`
lib/internal/process/pre_execution.js	`96.61% <98.24%> (+19.02%)`	⬆️
lib/internal/otel/core.js	`93.38% <93.38%> (ø)`
lib/internal/otel/span.js	`95.19% <95.19%> (ø)`
lib/internal/otel/instrumentations.js	`90.87% <90.87%> (ø)`
lib/internal/otel/flush.js	`89.62% <89.62%> (ø)`

... and 497 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

jasnell

First pass... Generally looks reasonable. Main concern is on instrumentation cost. Left a couple comments.

jasnell · 2026-10-08T07:02:29Z

+  }
+
+  setAttribute(key, value) {
+    this.#attributes[key] = value;


#attributes is going to be put into a slow dictionary mode with a internal shape transition every time an attribute is set... Meaning this is going to have a non-trivial cost.

Could this use a SafeMap instead?

Obviously that makes getAttributes trickier below but I would assume that setAttribute is the hotter path?

Fair point. The set of attribute keys is closed in this PR (no public API, only the internal instrumentations write attributes), so I went the other way: the attributes object is preallocated with the full key set, one identical shape for every span, and writes only ever hit existing properties. Unused keys are skipped at export time.

This is something that'll need to be revisited once there's a public API, of course.

jasnell · 2026-10-08T07:05:30Z

+  }
+
+  addEvent(name, attributes) {
+    ArrayPrototypePush(this.#events, {


How many events might we expect? If it's a lot, or completely unbounded, this becomes quadratic after a while.

Right now, without a public API, there's only ever one, and that's for errors.

I left a TODO to cap this (as is typically done in OTel-esque SDKs) once there's a public API.

jsumners-nr · 2026-10-08T11:40:26Z

+The subsystem is independent of the OpenTelemetry JavaScript packages. It
+is not interoperable with `@opentelemetry/api`, and spans created by one
+are not visible to the other. Running both in the same process produces
+duplicate trace and span IDs. The built-in instrumentation also
+overwrites the `traceparent` (and, when present, `tracestate`) header on
+outgoing HTTP requests, discarding whatever a userland propagator may
+have set. Users should run either the built-in subsystem or a userland
+OpenTelemetry SDK, not both.


Then I wonder what the purpose of this is. From my perspective, this feature should work in one of two ways:

Provides an implementation of the API such that compliant instrumentations can register with it and it delivers the appropriate signals data to a collector.

Registers with whatever OTEL implementation is present such that the signals data is inlined correctly.

With the caveats highlighted in this paragraph, I'm just not clear what this accomplishes.

I think we need to start somewhere. I can see this eventually working with userland sdk's before it comes out of experimental but in order to make progress at all we should work incrementally here.

Indeed. This is a step 1 (or maybe a step 0). It serves two purposes:

Provide OOTB (limited) tracing with no dependencies or code changes.

Provide a baseline implementation of the data model and egress pipeline in core. This can be followed up in future PRs with all kinds of ideas, like:

A TracerProvider implementation, so as to support @opentelemetry/api

Prototyping a much more minimal interface, like tracing in Rust, which could then be adopted as official by OTel (like .NET's prior art).

Providing a default core for OTel SDK.

All kinds of other options!

Basically this PR exists to eliminate the more contentious parts of my previous one, so that we can avoid blocking this core functionality while we deliberate on that.

I might miss some contexts here about the goal of OpenTelemetry support. OpenTelemetry specification defines a baseline data model and egress pipeline. Would the goal of this built-in support comply with the https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md ?

Would the goal of this built-in support comply with the open-telemetry/opentelemetry-specification@main/specification/trace/sdk.md ?

Eventually, yes, that's an option for future PRs, as indicated by the first bullet point in the purpose number two in my previous comment. This PR only creates OTLP tracing data based on inbound and outbound HTTP calls, and ships it off to a collector. All other concerns/discussions are left for future PRs, and the feature should probably stay experimental until those are figured out.

Thank you for clarifying! I'm trying to understand the goal of this PR and the future path on a built-in Observability support. So I'd like to make sure the goal is clear even if this is an initial experimental PR.

The PR might be good on its own, but a future work that may create incompat or create preferential/minimal API towards particular observability data backend vendors must be a non-starter. I'd be wary of future paths like "I don't use xxx so let's not implement xxx in OpenTelemetry", or "I need xxx but it's not in OpenTelemetry, let's skip OpenTelemetry and do xxx".

OpenTelemetry has been the de-facto standard and a collaborative project between observability backend vendors, so I'd appreciate that a vendor neutral path is the goal of Node.js built-in observability support.

Qard

Overall LGTM, though is there any particular reason you chose to limit this to only the main thread? Seems to me like it'd be useful to have this get set up on every worker thread.

bengl · 2026-10-09T17:21:01Z

@jsumners-nr The vendor I work for is also not likely to use this at this stage. It does provide some utility for situations where users can't or won't inject new code into a service, but are happy to change environment variables.

As mentioned elsewhere, future PRs can make this more useful to o11y vendors. In the meantime, can you clarify what you're looking for with DC here?

jsumners-nr · 2026-10-09T18:17:38Z

As mentioned elsewhere, future PRs can make this more useful to o11y vendors. In the meantime, can you clarify what you're looking for with DC here?

We chatted privately and concluded with a mutual understanding.

Adds experimental OpenTelemetry tracing support, as described in the included docs. Tracing is activated via environment variables only and requires the --experimental-otel flag. There is no programmatic API yet. Custom instrumentation is not supported. Assisted-by: pi:glm-5.3 Signed-off-by: Bryan English <bryan@bryanenglish.com>

legendecas · 2026-10-09T19:32:45Z

Adding tsc-agenda label given that this means Node.js adopting new standards (OpenTelemetry and W3C Trace Context).

nodejs-github-bot added lib / src Issues and PRs involving general changes in the lib/ or src/ directories. needs-ci PRs that need a full CI run. labels Oct 8, 2026

bengl requested review from Flarna and Qard October 8, 2026 05:09

bengl mentioned this pull request Oct 8, 2026

otel: add new module #61907

Closed

atlowChemi reviewed Oct 8, 2026

View reviewed changes

Comment thread doc/api/cli.md Outdated

Comment thread lib/internal/otel/core.js Outdated

Comment thread doc/api/otel.md

jasnell reviewed Oct 8, 2026

View reviewed changes

jsumners-nr reviewed Oct 8, 2026

View reviewed changes

bengl force-pushed the bengl/otel-2 branch 2 times, most recently from 0128458 to ac2de76 Compare October 8, 2026 16:00

Qard approved these changes Oct 8, 2026

View reviewed changes

legendecas reviewed Oct 8, 2026

View reviewed changes

Comment thread doc/api/otel.md Outdated

Comment thread doc/api/otel.md

Comment thread lib/internal/otel/span.js Outdated

Comment thread lib/internal/otel/flush.js Outdated

Comment thread doc/api/otel.md Outdated

Comment thread lib/internal/otel/instrumentations.js Outdated

bengl force-pushed the bengl/otel-2 branch from ac2de76 to 1bb965a Compare October 9, 2026 12:55

This comment was marked as resolved.

Sign in to view

jsumners-nr approved these changes Oct 9, 2026

View reviewed changes

legendecas reviewed Oct 9, 2026

View reviewed changes

Comment thread lib/internal/otel/span.js Outdated

bengl force-pushed the bengl/otel-2 branch from 1bb965a to b3d5999 Compare October 9, 2026 18:23

legendecas added the tsc-agenda Issues and PRs to discuss during Technical Steering Committee meetings. label Oct 9, 2026

legendecas mentioned this pull request Oct 9, 2026

Node.js Technical Steering Committee (TSC) Meeting 2026-10-14 nodejs/TSC#1902

Open

Conversation

bengl commented Oct 8, 2026

Uh oh!

nodejs-github-bot commented Oct 8, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Oct 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jasnell left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Qard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

bengl commented Oct 9, 2026

Uh oh!

jsumners-nr commented Oct 9, 2026

Uh oh!

Uh oh!

legendecas commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

codecov Bot commented Oct 8, 2026 •

edited

Loading

legendecas commented Oct 9, 2026 •

edited

Loading