feat(plugin): support pending marks from LLM intercepts#327
feat(plugin): support pending marks from LLM intercepts#327bbednarski9 wants to merge 4 commits into
Conversation
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Enterprise Run ID: 📒 Files selected for processing (5)
📜 Recent review details⏰ Context from checks skipped due to timeout. (2)
🧰 Additional context used📓 Path-based instructions (15)**/*.rs📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Files:
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Files:
**/{Cargo.toml,**/*.rs}📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Files:
**/*.{h,hpp,c,cpp,rs}📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Files:
**/*.{rs,toml}📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Files:
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}📄 CodeRabbit inference engine (AGENTS.md)
Files:
**/*.{rs,py,go,js,ts,tsx}📄 CodeRabbit inference engine (AGENTS.md)
Files:
crates/**/*.rs📄 CodeRabbit inference engine (AGENTS.md)
Files:
**⚙️ CodeRabbit configuration file
Files:
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}⚙️ CodeRabbit configuration file
Files:
crates/core/src/api/**/*.rs📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Files:
{crates/core,crates/adaptive}/**/*📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Files:
crates/core/**/*.rs📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
Files:
crates/{core,adaptive}/**📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Files:
crates/{core,adaptive}/**/*.rs⚙️ CodeRabbit configuration file
Files:
🔇 Additional comments (7)
WalkthroughIntroduces LLM Request Intercept Pending Marks
ChangesLLM Request Intercept Pending Marks
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
github-actions
Bot
added
size:L
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/core/tests/unit/shared_tests.rs (1)
196-221: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winAdd a positive pending-mark case for the codec branch.
This only checks that the new third return value is empty. The codec path is the branch that changed, and a regression that drops
pending_marksaftercodec.encodewould still pass here. Add one intercept that returns a realPendingMarkSpecand assert it survives theSome(codec)path.Suggested test shape
- register_llm_request_intercept( + register_llm_request_intercept_with_marks( "shared-codec", 1, false, Arc::new(|_name, mut request, annotated| { let mut annotated = annotated.expect("codec should provide annotated request"); annotated.model = Some("intercepted-model".into()); request.headers.insert("x-codec".into(), json!(true)); - Ok((request, Some(annotated))) + Ok( + LlmRequestInterceptOutcome::new(request, Some(annotated)) + .with_pending_mark(PendingMarkSpec::builder().name("codec.mark").build()), + ) }), ) .unwrap(); @@ - assert!(pending_marks_with_codec.is_empty()); + assert_eq!(pending_marks_with_codec.len(), 1); + assert_eq!(pending_marks_with_codec[0].name, "codec.mark");As per path instructions, "Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant. Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/core/tests/unit/shared_tests.rs` around lines 196 - 221, Add a positive assertion for the codec branch in shared_tests by exercising run_request_intercepts_with_codec with an interceptor that returns a real PendingMarkSpec, then verify pending_marks_with_codec still contains that mark after codec.encode runs. Keep the existing assertions on request_with_codec and annotated_with_codec, but replace the shallow is_empty check with a check that the pending mark survives the Some(codec) path so the regression around dropping pending_marks is covered.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/core/src/api/shared.rs`:
- Around line 107-113: The fallback branch in the request encoding flow drops
LlmRequestInterceptOutcome.annotated_request whenever no codec is configured,
which loses interceptor-produced annotations. Update the matching logic around
the codec/annotated_request handling so the branch in shared response building
preserves and returns the annotated_request even when codec is None, using the
existing codec/encode path and outcome.request tuple assembly as the place to
adjust.
In `@crates/core/src/plugin/dynamic/native.rs`:
- Around line 1788-1794: The NativeLlmRequestInterceptOutcome deserialization is
too strict for pending_marks, causing empty marked outcomes from
LlmRequestInterceptOutcome to fail. Update NativeLlmRequestInterceptOutcome to
default pending_marks when omitted, alongside the existing annotated_request
handling, so the native intercept outcome can deserialize the empty form safely.
In `@crates/core/tests/integration/native_plugin_tests.rs`:
- Around line 421-427: The stream test currently checks only that
fixture.native.llm_request.mark is parented to native-fixture-llm-stream, which
can miss incorrect emission order. In
native_plugin_tests::llm_stream_call_execute, add the same timestamp comparison
used in the non-stream path by capturing the ScopeCategory::Start event for
native-fixture-llm-stream and asserting the mark event occurs after that start
event, not just that it shares the start UUID. Keep the existing parent_uuid
assertion and extend it with the ordering check so the test covers lifecycle
sequencing as well as attachment.
In `@crates/plugin/tests/typed_callbacks.rs`:
- Around line 4396-4446: The trampoline test for
register_llm_request_intercept_with_marks only covers a null annotated request,
so it misses the envelope path where an existing AnnotatedLlmRequest could be
dropped. Update the test to pass a non-null annotation through the registration
callback and verify the resulting metadata from the LlmRequestInterceptOutcome
path preserves it under metadata["annotated_request"], while still checking the
pending mark and rewritten request behavior.
In `@crates/types/tests/serialization_tests.rs`:
- Around line 82-113: The serialization test for LlmRequestInterceptOutcome only
covers the case where pending_marks is present; add a backward-compatibility
assertion in llm_request_intercept_outcome_round_trips_pending_marks that
deserializing JSON without pending_marks still succeeds and yields an empty
pending_marks vector. Reuse the existing LlmRequestInterceptOutcome serde
round-trip setup in crates/types/tests/serialization_tests.rs to verify the
omitted-field decode path explicitly.
---
Outside diff comments:
In `@crates/core/tests/unit/shared_tests.rs`:
- Around line 196-221: Add a positive assertion for the codec branch in
shared_tests by exercising run_request_intercepts_with_codec with an interceptor
that returns a real PendingMarkSpec, then verify pending_marks_with_codec still
contains that mark after codec.encode runs. Keep the existing assertions on
request_with_codec and annotated_with_codec, but replace the shallow is_empty
check with a check that the pending mark survives the Some(codec) path so the
regression around dropping pending_marks is covered.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 182915b6-f580-4ff4-b702-8f61bf5d9a0a
📒 Files selected for processing (20)
crates/core/src/api/llm.rscrates/core/src/api/registry.rscrates/core/src/api/runtime.rscrates/core/src/api/runtime/callbacks.rscrates/core/src/api/runtime/state.rscrates/core/src/api/shared.rscrates/core/src/context/registries.rscrates/core/src/plugin.rscrates/core/src/plugin/dynamic/native.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/unit/shared_tests.rscrates/plugin/src/lib.rscrates/plugin/tests/typed_callbacks.rscrates/types/src/api/event.rscrates/types/src/api/llm.rscrates/types/tests/serialization_tests.rsexamples/rust-native-plugin/README.mdexamples/rust-native-plugin/src/lib.rs
📜 Review details
⏰ Context from checks skipped due to timeout. (2)
- GitHub Check: Check / Run
- GitHub Check: Preview docs
🧰 Additional context used
📓 Path-based instructions (31)
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Rust identifiers (e.g.,nemo_relay_tool_call)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node work
**/*.rs: Runcargo fmt --allto format all Rust code
Runcargo clippy --workspace --all-targets -- -D warningsto enforce all clippy lints as errors
**/*.rs: Runcargo fmt --allwhen Rust files changed as part of WebAssembly work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files changed as part of WebAssembly work
**/*.rs: If any Rust code changed, always runjust test-rust
If any Rust code changed, also runcargo fmt --all
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting withcargo fmt --all
Run Rust linting withcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Usecargo fmtfor Rust code formatting
Runcargo clippy -- -D warningsto lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code withuv run pre-commit run --all-filesto enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rsexamples/rust-native-plugin/src/lib.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Maintain documented and tested validation and report behavior for adaptive surfaces
Files:
crates/core/tests/unit/shared_tests.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rs
**/{Cargo.toml,**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency between Rust package names in
Cargo.tomland their actual usage across the codebase
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rsexamples/rust-native-plugin/src/lib.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
**/*.{h,hpp,c,cpp,rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Ensure FFI header and library naming follows consistent conventions across platform-specific builds
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rsexamples/rust-native-plugin/src/lib.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
**/*.{rs,toml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update Rust crate names and module prefixes during coordinated rename operations
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rsexamples/rust-native-plugin/src/lib.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
crates/core/**/*.rs: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
crates/{core,adaptive}/**
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full matrix across Rust, Python, Go, Node.js, and WebAssembly
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}
📄 CodeRabbit inference engine (AGENTS.md)
Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rsexamples/rust-native-plugin/src/lib.rscrates/core/src/api/shared.rsexamples/rust-native-plugin/README.mdcrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
**/*.{rs,py,go,js,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions: Rust and Python use
snake_case, C FFI exports prefixednemo_relay_, Go usesPascalCasefor public APIs, Node.js usescamelCase.
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rsexamples/rust-native-plugin/src/lib.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
crates/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
crates/**/*.rs: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
UseJson = serde_json::Valuein Rust-facing runtime APIs for JSON payload handling.
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
**
⚙️ CodeRabbit configuration file
**:AGENTS.md
This file provides guidance to agents, including Claude Code and OpenAI Codex, when working in this repository.
Project Overview
NeMo Relay is a multi-language agent runtime framework for execution scopes, lifecycle events, middleware, plugins, and observability around tool and LLM calls. The core runtime is Rust. Primary supported bindings are Rust, Python, and Node.js. Go, WebAssembly, and the raw C FFI are experimental and source-first.
The shared runtime model is:
- Scope stacks decide where work belongs and which scope-local behavior is visible.
- Middleware registries decide what guardrails and intercepts run around managed calls.
- Plugins install reusable runtime behavior from configuration.
- Events record runtime behavior in ATOF form.
- Subscribers and exporters consume events in-process or export them to ATIF, OpenTelemetry, OpenInference, or other backends.
Repository Structure
The repository layout separates the Rust runtime, language bindings, documentation,
integration patches, and agent-facing skills.crates/ core/ # Rust core runtime crate, published as nemo-relay adaptive/ # Adaptive runtime primitives and plugin components python/ # PyO3 native extension for the Python package ffi/ # Raw C ABI layer used by downstream bindings such as Go node/ # NAPI Node.js binding and JavaScript/TypeScript entry points wasm/ # wasm-bindgen WebAssembly binding and JS wrappers python/ nemo_relay/ # Python wrapper package: scopes, tools, LLM, middleware, typed helpers, plugins, adaptive helpers tests/ # Python tests go/ nemo_relay/ # Experimental Go CGo binding and tests fern/ # Fern documentation site scripts/ # Stable wrappers and helper scripts; build/test/docs entry points live in justfile third_party/ # P...
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rsexamples/rust-native-plugin/src/lib.rscrates/core/src/api/shared.rsexamples/rust-native-plugin/README.mdcrates/core/src/plugin.rscrates/types/src/api/event.rscrates/core/src/api/runtime/callbacks.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/plugin/src/lib.rscrates/core/tests/integration/middleware_tests.rscrates/types/src/api/llm.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/tests/unit/shared_tests.rscrates/core/src/context/registries.rscrates/core/src/api/shared.rscrates/core/src/plugin.rscrates/core/src/api/runtime/callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/core/src/api/runtime.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rscrates/core/src/plugin/dynamic/native.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/core/tests/unit/shared_tests.rscrates/plugin/tests/typed_callbacks.rscrates/core/tests/fixtures/native_plugin/src/lib.rscrates/types/tests/serialization_tests.rscrates/core/tests/integration/native_plugin_tests.rscrates/core/tests/integration/middleware_tests.rs
{docs/**,examples/**,README.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Verify examples still run with documented commands for large or public-facing changes
Files:
examples/rust-native-plugin/src/lib.rsexamples/rust-native-plugin/README.md
crates/core/src/api/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Implement behavior first in Rust core API modules:
crates/core/src/api/and related core modules such ascrates/core/src/api/runtime/,crates/core/src/codec/, orcrates/core/src/json.rs
Files:
crates/core/src/api/shared.rscrates/core/src/api/runtime/callbacks.rscrates/core/src/api/runtime.rscrates/core/src/api/runtime/state.rscrates/core/src/api/registry.rscrates/core/src/api/llm.rs
**/*.{md,rst,html,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
**/*.{md,rst,html,txt}: Always spellNVIDIAin all caps. Do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names withNVIDIAon first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms withs, not an apostrophe, such asGPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such asCPU,GPU,PC,API, andUIusually do not need to be spelled out for developer audiences.
Files:
examples/rust-native-plugin/README.md
**/*.{md,rst,html}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
Link the first mention of a product name when the destination helps the reader.
Files:
examples/rust-native-plugin/README.md
**/*.md
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
Documentation must be updated if activation or usage changed
**/*.md: Use title case consistently in technical documentation headings
Avoid quotation marks, ampersands, and exclamation marks in headings
Keep product, event, research, and whitepaper names in their official title case
Use title case for table headers
Do not force social-media sentence case into technical docs
Format code elements, commands, parameters, package names, and expressions in monospace
Format directories, file names, and paths in monospace using backticks
Use angle brackets inside monospace for variables inside paths, such as/home/<username>/.login
Format error messages and strings in quotation marks, keeping literal code strings in code formatting when clearer
Format UI buttons, menus, fields, and labels in bold
Use angle brackets between UI labels for menu paths, such as File > Save As
Use italics for new terms on first use, sparingly and only when introducing the term
Use italics for publication titles
Format keyboard shortcuts in plain text, such as Press Ctrl+Alt+Delete
Use owner/repo link text for GitHub repositories, preferring[NVIDIA/NeMo](link)over prose references like 'the GitHub repo'
Introduce every code block with a complete sentence
Do not make a code block complete the grammar of the previous sentence
Do not continue a sentence after a code block
Use syntax highlighting when the format supports it for code blocks
Avoid the word 'snippet' unless the surrounding docs already use it as a term of art
Keep inline method, function, and class references consistent with nearby docs, omitting empty parentheses for prose readability when no call is shown
Use descriptive anchor text that matches the destination title when possible for links
Avoid raw URLs in running text
Avoid generic anchor text such as 'here,' 'this page,' and 'read more'
Include acronyms in link text when a linked term includes an acronym
Do not link long sentences or multiple sentences
Avoid links ...
Files:
examples/rust-native-plugin/README.md
**/{docs,examples,**/*.md,*.patch,*.diff,.github,*.sh,*.yaml,*.yml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update documentation, examples, CI configuration, and patch artifacts when performing rename operations
Files:
examples/rust-native-plugin/README.md
**/*.{md,rst,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
Spell
NVIDIAin all caps. Do not useNvidia,nvidia, orNV.
Files:
examples/rust-native-plugin/README.md
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,rst}: Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as "here" or "read more."
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce.
Preferrefer tooverseewhen the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical docs.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented docs unless the source, platform, or legal guidance explicitly requires them.
Files:
examples/rust-native-plugin/README.md
{docs/**,README.md,CONTRIBUTING.md,**/*.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Run docs link validation with
just docs-linkcheckwhen links change
Files:
examples/rust-native-plugin/README.md
{docs/**,README.md,**/Cargo.toml,**/package.json,**/*.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
Ensure renamed public surfaces are reflected consistently in manifests and docs for large or public-facing changes
Files:
examples/rust-native-plugin/README.md
**/*.{md,mdx,py,sh,yaml,yml,toml,json}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repo references, and build commands current
Files:
examples/rust-native-plugin/README.md
**/*.{html,md,mdx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license header in HTML and Markdown files using HTML comment syntax
Files:
examples/rust-native-plugin/README.md
**/README.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant crate or package README when that surface changed
Files:
examples/rust-native-plugin/README.md
crates/core/src/api/runtime/callbacks.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Define or reuse callback type aliases in
crates/core/src/api/runtime/callbacks.rsusingBox<dyn Fn(...)>pattern for new middleware types
Files:
crates/core/src/api/runtime/callbacks.rs
crates/core/src/api/runtime/state.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
crates/core/src/api/runtime/state.rs: Add registry fields asSortedRegistry<GuardrailEntry<T>>orSortedRegistry<Intercept<T>>toNemoRelayContextStateincrates/core/src/api/runtime/state.rs
Add chain execution helpers toNemoRelayContextStatefollowing the pattern of existing methods liketool_sanitize_request_chainortool_request_intercepts_chain
Files:
crates/core/src/api/runtime/state.rs
crates/core/src/api/registry.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Use existing
global_*_registry_api!andscope_*_registry_api!macro patterns incrates/core/src/api/registry.rsfor both global and scope-local registration APIs unless design explicitly rules one out
Files:
crates/core/src/api/registry.rs
crates/core/src/api/{tool,llm}.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Wire the new middleware chain into the execute path in
crates/core/src/api/tool.rsorcrates/core/src/api/llm.rsat the appropriate pipeline stage
Files:
crates/core/src/api/llm.rs
🔇 Additional comments (19)
crates/core/tests/fixtures/native_plugin/src/lib.rs (1)
8-11: LGTM!Also applies to: 117-139
examples/rust-native-plugin/src/lib.rs (1)
5-7: LGTM!Also applies to: 236-255
examples/rust-native-plugin/README.md (1)
87-101: LGTM!crates/core/src/plugin.rs (2)
30-39: LGTM!
354-384: 📐 Maintainability & Code QualityVerify required Rust and core-runtime validation before merge.
This PR touches Rust and
crates/core, so please confirmcargo fmt --all,cargo clippy --workspace --all-targets -- -D warnings,just test-rust, and the affected binding matrix passed. As per coding guidelines, "**/*.rs: Any Rust change must runjust test-rust,cargo fmt --all,cargo clippy --workspace --all-targets -- -D warnings" and "{crates/core,crates/adaptive}/**/*: Changes tocrates/coreorcrates/adaptivemust run the full language matrix."Source: Coding guidelines
crates/core/src/plugin/dynamic/native.rs (1)
29-41: LGTM!Also applies to: 1336-1341, 1689-1786
crates/plugin/src/lib.rs (1)
19-22: LGTM!Also applies to: 1495-1527, 2227-2294
crates/core/tests/integration/middleware_tests.rs (1)
17-24: LGTM!Also applies to: 33-40, 2594-2782
crates/core/tests/integration/native_plugin_tests.rs (1)
345-362: LGTM!crates/types/src/api/event.rs (1)
367-389: LGTM!crates/types/src/api/llm.rs (1)
10-18: LGTM!Also applies to: 32-75
crates/core/src/api/runtime/callbacks.rs (1)
18-18: LGTM!Also applies to: 180-189
crates/core/src/api/registry.rs (1)
550-573: 📐 Maintainability & Code QualityConfirm the required core validation matrix ran.
This changes
crates/corerequest-intercept registration and shared runtime behavior, so please verify the broader repo checks ran, not just local Rust compilation:validate-change,just test-rust,cargo fmt --all,cargo clippy --workspace --all-targets -- -D warnings, and the affected binding tests.As per coding guidelines, "If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation" and "Changes tocrates/coreorcrates/adaptivemust run the full language matrix."Also applies to: 672-697
Source: Coding guidelines
crates/core/src/api/runtime.rs (1)
14-16: LGTM!crates/core/src/api/llm.rs (2)
6-12: LGTM!Also applies to: 32-32, 302-330, 620-639, 777-796, 852-876
620-639: 🩺 Stability & AvailabilityRun the Rust and binding validation matrix for this LLM lifecycle change. This touches
crates/coreLLM start/end emission on sync and streaming paths, so includecargo fmt --all,cargo clippy --workspace --all-targets -- -D warnings,just test-rust, and the affected binding suites (just test-python,just test-go,just test-node,just test-wasm).crates/core/src/api/runtime/state.rs (1)
23-26: LGTM!Also applies to: 66-66, 1014-1015, 1044-1062
crates/core/src/context/registries.rs (1)
14-14: LGTM!Also applies to: 44-44
crates/core/src/api/shared.rs (1)
77-81: LGTM!
Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>
1 participant
Overview
Allow LLM request interceptors to schedule marks that Relay emits under the LLM scope created after interception.
Details
PendingMarkSpecandLlmRequestInterceptOutcomeas typed contracts for request rewrites, annotations, and deferred marks.break_chain, and discard pending marks if the interceptor chain fails.Request interceptors execute before Relay creates the managed LLM scope. Emitting a mark directly from an interceptor therefore cannot reliably attach it to that future scope. Returning a pending mark lets the lifecycle owner materialize it at the correct boundary.
Where should the reviewer start?
Start with
crates/types/src/api/llm.rsandcrates/core/src/api/llm.rs. The native compatibility adapter is incrates/core/src/plugin/dynamic/native.rs, and the lifecycle contract tests are incrates/core/tests/integration/middleware_tests.rs.Testing
cargo test -p nemo-relay-typescargo test -p nemo-relay-plugincargo test -p nemo-relaycargo fmt --all -- --checkcargo clippy -p nemo-relay-types -p nemo-relay-plugin -p nemo-relay --all-targets -- -D warningspre-commit run --from-ref github/main --to-ref HEADThe repository-wide
pre-commit run --all-filesalso completed all checks except the existingattributions-rustdrift, which rewrites the unrelatedmd-5license entry. That generated change is not included here.Breaking changes
None. Existing request-interceptor APIs and the native ABI v1 layout remain supported.
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit
New Features
PendingMarkSpec/LlmRequestInterceptOutcomesupport in native plugin APIs and examples.Bug Fixes