Skip to content

feat(memtrack): support disabling allocator tracking - #469

Open
not-matthias wants to merge 10 commits into
mainfrom
cod-3231-support-disabling-allocator-tracking
Open

feat(memtrack): support disabling allocator tracking#469
not-matthias wants to merge 10 commits into
mainfrom
cod-3231-support-disabling-allocator-tracking

Conversation

@not-matthias

@not-matthias not-matthias commented Jul 24, 2026

Copy link
Copy Markdown
Member

What

memtrack track --track-allocators (default on, env CODSPEED_TRACK_ALLOCATORS). When disabled, memtrack skips the allocator uprobe machinery and only emits coarse mmap/munmap/brk events.

Why

Allocation-heavy programs (e.g. a Rust build) generate an overwhelming number of malloc/free events, and the per-allocation uprobes slow the target significantly. This trades allocation granularity for lower overhead while still collecting RSS-relevant memory events.

How

  • mmap/munmap/brk syscall tracepoints are now always attached in a memory run.
  • With the flag off, the exec watcher and allocator attach worker are never started.
  • No runner CLI flag: the env var is inherited by the memtrack subprocess, keeping the runner decoupled from the installed memtrack version.

@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown

Greptile Summary

Adds configurable allocator tracking alongside expanded RSS accounting.

  • Adds --track-allocators support and conditionally disables allocator uprobes while retaining coarse memory events.
  • Adds eBPF RSS, reverse-mapping, and process-lifecycle event collection.
  • Extends shared memtrack event schemas and reconstruction logic for the new event types.
  • Adds architecture-matrix CI coverage and RSS integration fixtures and snapshots.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failures remain within the follow-up review scope.

No blocking failures remain.

Important Files Changed

Filename Overview
crates/memtrack/src/ebpf/tracker.rs Makes allocator attachment optional while retaining the common tracker lifecycle and tracepoint setup.
crates/memtrack/src/ebpf/c/rss.bpf.h Introduces RSS, reverse-mapping, fork, exec, and exit event collection with process ownership tracking.
crates/memtrack/src/ebpf/memtrack/tracking.rs Attaches syscall, RSS, lifecycle, and supported reverse-mapping probes.
crates/memtrack/src/main.rs Exposes the allocator-tracking toggle and passes it into tracker construction.
crates/runner-shared/src/artifacts/memtrack/mod.rs Extends the shared memtrack artifact schema for RSS and process-lifecycle events.
.github/workflows/ci.yml Expands eBPF integration coverage across x86-64 and ARM runners and adds RSS tests.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    CLI[run / exec / memtrack CLI] --> CFG[track_allocators configuration]
    CFG --> TRACKER[Tracker]
    TRACKER --> TRACE[Always-attached syscall and lifecycle tracepoints]
    CFG -->|true| UPROBES[Allocator watcher and uprobes]
    CFG -->|false| SKIP[Skip allocator uprobes]
    TRACE --> EVENTS[Memtrack events]
    UPROBES --> EVENTS
    EVENTS --> ARTIFACT[runner-shared artifact encoding]
Loading

Reviews (3): Last reviewed commit: "feat: add --track-allocators toggle for ..." | Re-trigger Greptile

@codspeed-hq

codspeed-hq Bot commented Jul 24, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 17 untouched benchmarks


Comparing cod-3231-support-disabling-allocator-tracking (bbff5ff) with cod-3089-collect-rss-in-memtrack (7704c29)

Open in CodSpeed

@not-matthias
not-matthias force-pushed the cod-3231-support-disabling-allocator-tracking branch 2 times, most recently from d983261 to 131f8a3 Compare July 24, 2026 10:40
@not-matthias
not-matthias changed the base branch from main to cod-3089-collect-rss-in-memtrack July 24, 2026 12:04

@GuillaumeLagrange GuillaumeLagrange left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what's really the usecase of this? olgtm but I'm not sure we've discussed

Comment thread crates/memtrack/src/main.rs Outdated
@not-matthias
not-matthias force-pushed the cod-3231-support-disabling-allocator-tracking branch 2 times, most recently from f1dbc8a to efc66c8 Compare July 30, 2026 10:48
@not-matthias
not-matthias force-pushed the cod-3231-support-disabling-allocator-tracking branch from 7e96222 to bdd0455 Compare July 30, 2026 12:40
@not-matthias
not-matthias force-pushed the cod-3231-support-disabling-allocator-tracking branch from bdd0455 to e158870 Compare July 30, 2026 15:35
@not-matthias not-matthias changed the title feat: add --track-allocators toggle for memory mode feat(memtrack): support disabling allocator tracking Aug 18, 2026
Set AllowShortFunctionsOnASingleLine: None in .clang-format and apply it, reformatting the allocator uprobe macros accordingly.
A forked child's inherited RSS is invisible to rss_stat: the fork-time
counter copies fire outside the child's context, and anon COW faults
are counter-neutral, so a child that only touches inherited memory
never reports anything on its own. A fork event carrying the parent
pid lets consumers seed the child from the parent's last absolutes;
exec and exit mark where the address space is replaced or torn down.
Sample the kernel's per-mm resident counter through the kmem:rss_stat tracepoint, emitting absolute byte values per mm member. Adds the EVENT_TYPE_RSS contract, MemtrackEventKind::Rss, the parser arm, and a writer bench case.

An rss_stat update from reclaim or another process's madvise fires in the actor's context; track (mm_id, member) -> owning pid so those updates reach the owner. External events may only lower a counter, so stale reads and mm_id collisions cannot invent peaks.
Attach fentry hooks on the folio-rmap add/remove functions, emitting
signed page-count deltas per MM_* bucket so anon, file, and shmem RSS
can be reconstructed over time. Gated behind
CODSPEED_MEMTRACK_TRACK_RMAP; the programs stay autoload-off by default
so the skeleton loads on any kernel, with the PUD pair (only present
since v6.15) gated separately from the core set so rmap still works on
older kernels. Adds the EVENT_TYPE_RMAP contract, MemtrackEventKind::Rmap,
parser arm, and bench case.

Recover the owning pid for rmap events run by another task (reclaim,
process_madvise, khugepaged, KSM) from the mm_struct pointer, and
maintain the ownership maps across exec and thread-group exit. The same
ownership binding also validates external (curr==0) rss_stat updates,
so a stale mm can no longer attribute a counter to the wrong pid.
Add the rss_tests integration suite: per-workload RSS/rmap reconstruction
snapshots against /proc ground truth, fork-seeded child RSS, exec/exit
resets, foreign-actor rmap attribution (reclaim, external madvise), and
mm-ownership across CLONE_VM and exec. Extend tests/shared.rs with the
tracker/fixture helpers these tests need and move compile_c_source into
it for reuse.

The suite needs two surfaces the production paths don't: a tracker mode
that skips the allocator probes and exec-mapping watcher, and readers for
the mm-ownership maps.
Add an aarch64 lane to the bpf-tests matrix and run the rss integration tests alongside the existing test binaries.
Add a --track-allocators flag (default on, env CODSPEED_TRACK_ALLOCATORS)
to the memtrack track subcommand. When disabled, memtrack skips the
allocator uprobe machinery (exec watcher + attach worker) and only emits
coarse mmap/munmap/brk events, reducing overhead on allocation-heavy
programs. The mmap/munmap/brk syscall tracepoints are now always attached
in every memory run.

The runner does not add a CLI flag for this: it relies on the
CODSPEED_TRACK_ALLOCATORS environment variable being inherited by the
memtrack subprocess, keeping the runner decoupled from the installed
memtrack version. Standalone memtrack can still use the CLI flag.
@not-matthias
not-matthias force-pushed the cod-3231-support-disabling-allocator-tracking branch from e158870 to bbff5ff Compare August 20, 2026 08:25
Base automatically changed from cod-3089-collect-rss-in-memtrack to main August 24, 2026 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants