NeMo Relay Call Instrumentation
Wrap tool and LLM calls with Relay lifecycle events, middleware, and guardrails.
The skill explicitly recommends managed execution at existing call boundaries and transparently describes request rewriting, sanitization, rejection behavior, and preservation of raw callback results. The skill card also advises against secret leakage and recommends least-privilege credentials. However, it lacks user-confirmation requirements, rollback guidance, concrete credential scoping, dependency-security review, and detailed external-effect controls, so points are deducted.
SKILL.md covers multi-language APIs, manual lifecycle pairing, failure-end events, early stream drops, and middleware ordering, giving useful failure semantics. However, there are no executable tests for this selected skill, no version-pinned dependency validation, installation verification, or key-path error examples. The benchmark is only a statically visible evaluation report, so the score remains below the static ceiling.
The activation scenario, call boundaries, Rust/Python/Node.js/Go APIs, and neighboring-skill routing are reasonably clear. However, input/output contracts, non-fit boundaries, environment requirements, and Chinese-language usage guidance are not systematically specified, leaving limited evidence for reliable environment fit.
The name, description, license, author, skill card, version SHA, evaluation date, related skills, and repository-level publication/maintenance signals are documented. However, SKILL.md lacks recommended Instructions and Examples sections, the author format failed the reported schema check, and installation/dependency notes, limitations, changelog, and explicit maintenance ownership are incomplete.
The skill directly addresses the core task of instrumenting existing tool and LLM calls through Relay, and the evaluation files describe positive and negative routing tasks. However, it provides no representative code examples or statically verifiable complete outputs for this selected skill, so users still need to fill in concrete types, configuration, and invocation details.
Committed BENCHMARK.md, evals/evals.json, and version information provide some auditability through stated tasks, metrics, and results. However, there is no reproducible execution evidence or independent corroboration attached to the skill, and the visible repository tests do not cover its key paths, so only limited credit is warranted.
- The skill gives API shapes and concepts but no complete copy-ready Rust, Python, Node.js, or Go examples; verify the actual NeMo Relay version and signatures before integration.
- Request and response sanitization affects emitted event payloads only, not the raw values returned to application code; do not treat it as a business-data redaction boundary.
- Manual lifecycle mode requires every start call to have a matching end or error path; dropping a streaming call early may leave final events incomplete.
- The skill card does not specify API-key handling, external-service behavior, data retention, or regional compliance requirements; production users must supply that review.
What does this skill do, and when should you use it?
This skill targets applications that already own tool functions or model/provider call sites and need to run them through NeMo Relay correctly. It recommends defining a scope at the natural agent, request, workflow, or graph boundary and using managed execution APIs first. The guidance covers tool calls, LLM requests, request and execution intercepts, conditional guardrails, event-payload sanitization, streaming, and context propagation. It is a fit when a team wants Relay lifecycle capture without changing the original callable’s business behavior.
It guides developers in wrapping existing tool functions with NeMo Relay managed tool execution APIs and existing LLM/provider calls with the corresponding managed LLM APIs. It names the Rust, Python, Node.js, and Go entry points and explains how conditional guardrails, request intercepts, execution intercepts, and request/response sanitization affect inputs, outputs, and emitted events. It also documents end-event behavior after failures, pairing requirements for manual lifecycle APIs, finalization behavior for streaming LLM wrappers, and context propagation across threads or asynchronous tasks.
- An AI-agent engineer needs to add Relay lifecycle events and metadata around existing tool functions.
- A platform engineer maintains model or provider wrappers and needs to place LLM interceptors and guardrails at the right abstraction layer.
- A team building agent workflows in Rust, Python, Node.js, or Go wants to use Relay managed execution APIs.
- A framework adapter already owns execution and needs explicit start, end, and error lifecycle handling.
- A team implementing streaming LLM calls needs complete response collection and final event delivery when the stream ends.
What are this skill's strengths and limitations?
- Covers managed execution, lifecycle events, middleware, and guardrail semantics for both tools and LLM calls.
- Names API entry points for Rust, Python, Node.js, and Go.
- Clearly distinguishes raw input, effective execution input, event payloads, and application-visible return values.
- Includes guidance for failures, streaming, manual lifecycle paths, and context propagation.
- This is integration guidance, not a package installation guide; it omits SDK versions and complete code samples.
- It provides no test suite, performance measurements, or verified platform matrix.
- Manual lifecycle and partial-intercept APIs require application developers to implement correct pairing, error handling, and framework adaptation.
How do you install this skill?
Install the individual skill with the skills CLI flow documented in the repository README:
npx skills add nvidia/skills --skill nemo-relay-instrument-calls --yes
The source does not document the required NeMo Relay SDK installation command, version, or platform matrix.
How do you use this skill?
After placing the skill where the agent can load it, trigger it with a relevant request such as: “Instrument my existing Python tool and LLM calls with NeMo Relay managed execution APIs while preserving their current arguments and return values.” Choose the scope boundary first, then use the language-specific managed execution API; use manual lifecycle APIs only when the host framework cannot be wrapped. The source does not include a complete project code example.
How does this skill compare with similar options?
The source does not name a directly comparable alternative. It only points to neighboring NeMo Relay skills for observability export, runtime debugging, context isolation, and reusable plugin behavior.