Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Call Instrumentation

Wrap tool and LLM calls with Relay lifecycle events, middleware, and guardrails.

55/ 100
Use with care

Useful, but reliability, evidence or controls still have material gaps.

See how it was scored ↓
Works as-is in
Codex · Claude Code
Stars
★ 3.8k
Last updated
today
License
Apache-2.0
nemo-relaytool-instrumentationllm-observabilitymiddleware
+2guardrailslifecycle-events

What does this skill do, and when should you use it?

This skill targets applications that already own tool functions or model/provider call sites and need to run them through NeMo Relay correctly. It recommends defining a scope at the natural agent, request, workflow, or graph boundary and using managed execution APIs first. The guidance covers tool calls, LLM requests, request and execution intercepts, conditional guardrails, event-payload sanitization, streaming, and context propagation. It is a fit when a team wants Relay lifecycle capture without changing the original callable’s business behavior.

It guides developers in wrapping existing tool functions with NeMo Relay managed tool execution APIs and existing LLM/provider calls with the corresponding managed LLM APIs. It names the Rust, Python, Node.js, and Go entry points and explains how conditional guardrails, request intercepts, execution intercepts, and request/response sanitization affect inputs, outputs, and emitted events. It also documents end-event behavior after failures, pairing requirements for manual lifecycle APIs, finalization behavior for streaming LLM wrappers, and context propagation across threads or asynchronous tasks.

Good fit
  • An AI-agent engineer needs to add Relay lifecycle events and metadata around existing tool functions.
  • A platform engineer maintains model or provider wrappers and needs to place LLM interceptors and guardrails at the right abstraction layer.
  • A team building agent workflows in Rust, Python, Node.js, or Go wants to use Relay managed execution APIs.
  • A framework adapter already owns execution and needs explicit start, end, and error lifecycle handling.
  • A team implementing streaming LLM calls needs complete response collection and final event delivery when the stream ends.

How do you install this skill?

Before you use it
  • The skill gives API shapes and concepts but no complete copy-ready Rust, Python, Node.js, or Go examples; verify the actual NeMo Relay version and signatures before integration.
  • Request and response sanitization affects emitted event payloads only, not the raw values returned to application code; do not treat it as a business-data redaction boundary.
  • Manual lifecycle mode requires every start call to have a matching end or error path; dropping a streaming call early may leave final events incomplete.
  • The skill card does not specify API-key handling, external-service behavior, data retention, or regional compliance requirements; production users must supply that review.
Before you start
Install first
  • NeMo Relay runtime or SDK

Install the individual skill with the skills CLI flow documented in the repository README:

npx skills add nvidia/skills --skill nemo-relay-instrument-calls --yes

The source does not document the required NeMo Relay SDK installation command, version, or platform matrix.

Generic route: install into Claude Code manually (macOS / Linux)
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/skills/nemo-relay-instrument-calls" ~/.claude/skills/
rm -rf "$tmp"

Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.

How do you use this skill?

Try saying

Once installed, send your agent any of these to trigger it:

  • Instrument my existing Python tool and LLM calls with NeMo Relay managed execution APIs while preserving their current arguments and return values.

After placing the skill where the agent can load it, trigger it with a relevant request such as: “Instrument my existing Python tool and LLM calls with NeMo Relay managed execution APIs while preserving their current arguments and return values.” Choose the scope boundary first, then use the language-specific managed execution API; use manual lifecycle APIs only when the host framework cannot be wrapped. The source does not include a complete project code example.

What are this skill's strengths and limitations?

Pros
  • Covers managed execution, lifecycle events, middleware, and guardrail semantics for both tools and LLM calls.
  • Names API entry points for Rust, Python, Node.js, and Go.
  • Clearly distinguishes raw input, effective execution input, event payloads, and application-visible return values.
  • Includes guidance for failures, streaming, manual lifecycle paths, and context propagation.
Limitations
  • This is integration guidance, not a package installation guide; it omits SDK versions and complete code samples.
  • It provides no test suite, performance measurements, or verified platform matrix.
  • Manual lifecycle and partial-intercept APIs require application developers to implement correct pairing, error handling, and framework adaptation.

How does this skill compare with similar options?

Side by side with related skills; every score comes from the same FSRS standard.

Skill FS score Stars Last updated License
NeMo Relay Call Instrumentation this page ✓ NVIDIA · Official 55 · Use with care ★ 3.8k today Apache-2.0
NeMo Relay Runtime Integration Debugger ✓ NVIDIA · Official 56 · Use with care ★ 3.8k today Apache-2.0
NeMo Relay Typed Wrappers & Codecs ✓ NVIDIA · Official 55 · Use with care ★ 3.8k today Apache-2.0
NeMo Relay Quick Start ✓ NVIDIA · Official 61 · Recommended ★ 3.8k today Apache-2.0
NeMo Relay Adaptive Tuning ✓ NVIDIA · Official 57 · Use with care ★ 3.8k today Apache-2.0

The source does not name a directly comparable alternative. It only points to neighboring NeMo Relay skills for observability export, runtime debugging, context isolation, and reusable plugin behavior.

How did FollowSkills review this skill?

FollowSkills review · FSRS-2.0
Use with care
55/ 100 5-point scale 2.8 / 5
The upstream repository has new commits since this review. The score still applies to the reviewed revision shown and may not cover the latest changes.
1Trust18 / 25 · 3.6/5

The skill explicitly recommends managed execution at existing call boundaries and transparently describes request rewriting, sanitization, rejection behavior, and preservation of raw callback results. The skill card also advises against secret leakage and recommends least-privilege credentials. However, it lacks user-confirmation requirements, rollback guidance, concrete credential scoping, dependency-security review, and detailed external-effect controls, so points are deducted.

2Reliability8 / 20 · 2.0/5

SKILL.md covers multi-language APIs, manual lifecycle pairing, failure-end events, early stream drops, and middleware ordering, giving useful failure semantics. However, there are no executable tests for this selected skill, no version-pinned dependency validation, installation verification, or key-path error examples. The benchmark is only a statically visible evaluation report, so the score remains below the static ceiling.

3Adaptability10 / 15 · 3.3/5

The activation scenario, call boundaries, Rust/Python/Node.js/Go APIs, and neighboring-skill routing are reasonably clear. However, input/output contracts, non-fit boundaries, environment requirements, and Chinese-language usage guidance are not systematically specified, leaving limited evidence for reliable environment fit.

4Convention9 / 15 · 3.0/5

The name, description, license, author, skill card, version SHA, evaluation date, related skills, and repository-level publication/maintenance signals are documented. However, SKILL.md lacks recommended Instructions and Examples sections, the author format failed the reported schema check, and installation/dependency notes, limitations, changelog, and explicit maintenance ownership are incomplete.

5Effectiveness6 / 15 · 2.0/5

The skill directly addresses the core task of instrumenting existing tool and LLM calls through Relay, and the evaluation files describe positive and negative routing tasks. However, it provides no representative code examples or statically verifiable complete outputs for this selected skill, so users still need to fill in concrete types, configuration, and invocation details.

6Verifiability4 / 10 · 2.0/5

Committed BENCHMARK.md, evals/evals.json, and version information provide some auditability through stated tasks, metrics, and results. However, there is no reproducible execution evidence or independent corroboration attached to the skill, and the visible repository tests do not cover its key paths, so only limited credit is warranted.

1 2 3 4 5 6

Open a dimension to read why it scored that way

Reviewed Jul 20, 2026 Reviewed revision 55f18499943e Review evidence[1][2][3][4][5][6][7][8][9][10][11][12][13]

Evidence confidence:Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.

See the full review method →

FAQ

Will this skill change values returned to the application?
According to the source, response-sanitization guardrails affect only end-event payloads; application code still receives the raw callback or execution-intercept result. An execution intercept may, however, short-circuit and return its own result.
When should manual lifecycle APIs be used?
Use them when the host framework cannot be wrapped by the managed execution helpers and already owns execution. Every start call must have a matching end or error path.
What is the main streaming caveat?
Streaming wrappers collect chunks and finalize the response at stream end. Dropping a stream early can prevent finalizers and subscribers from receiving a complete output.
Does this skill configure observability export?
No. The source directs traces, ATIF, and export setup to the separate nemo-relay-plugin-observability skill.

More skills from this repository

All from NVIDIA/skills

Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Runtime Integration Debugger

Diagnose application-side runtime failures after NeMo Relay is installed or imported.

★ 3.8k FS 56 Use with care today
Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Typed Wrappers & Codecs

Add typed boundaries to NeMo Relay integrations while preserving predictable JSON middleware semantics and caller-visible behavior.

★ 3.8k FS 55 Use with care today
Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Quick Start

Helps first-time NeMo Relay users prove observable execution value through the smallest suitable trial.

★ 3.8k FS 61 Recommended today
Automation & Ops ✓ NVIDIA · Official

NeMo Relay Observability Plugin

Configure NeMo Relay event capture and telemetry exports.

★ 3.8k FS 60 Recommended today
Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Adaptive Tuning

Tune NeMo Relay plugin behavior from runtime signals with measured, reversible rollouts.

★ 3.8k FS 57 Use with care today
Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Installation Guide

Choose and verify the right NeMo Relay path for CLIs, language packages, and maintained frameworks.

★ 3.8k FS 62 Recommended today
Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Plugin Builder

Guides reusable NeMo Relay runtime behavior into configurable plugins.

★ 3.8k FS 60 Recommended today
Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Migration Assistant

Safely migrate NeMo Flow projects to NeMo Relay.

★ 3.8k FS 57 Use with care today
Dev & Engineering ✓ NVIDIA · Official

NeMo Relay Context Isolation

Keeps NeMo Relay scope stacks independent across concurrent requests and async workflows while preserving ancestry propagation.

★ 3.8k FS 55 Use with care today
Automation & Ops ✓ NVIDIA · Official

Nemotron Safety Policy Generator

Generate deployable custom safety policies for NVIDIA Nemotron content-safety models.

★ 3.8k FS 53 Use with care today
Dev & Engineering ✓ NVIDIA · Official

NeMo Evaluator Plugin Skill

Run and manage NeMo Platform model evaluations through the Evaluator CLI and Python SDK.

★ 3.8k FS 58 Recommended today
Data & Analysis ✓ NVIDIA · Official

NeMo Data Designer Synthetic Data Skill

Build synthetic datasets and declarative data-generation pipelines from a natural-language description.

★ 3.8k FS 53 Use with care today
Dev & Engineering ✓ NVIDIA · Official

NeMo AutoModel Model Onboarding

A structured guide for adding new LLM, MoE, and VLM architectures to NeMo AutoModel.

★ 3.8k FS 60 Recommended today
Dev & Engineering ✓ NVIDIA · Official

NeMo AutoModel Distributed Training

Choose and configure multi-GPU and multi-node training strategies for NeMo AutoModel.

★ 3.8k FS 58 Recommended today
Writing & Content ✓ NVIDIA · Official

NeMo-RL Documentation Conventions

Keeps NeMo-RL documentation, indexes, and docstrings consistent.

★ 3.8k FS 58 Recommended today
Data & Analysis ✓ NVIDIA · Official

Nemotron Speech ASR Customization Orchestrator

Chooses and coordinates the lowest-cost path for adapting speech recognition to a domain or language.

★ 3.8k FS 56 Use with care today
Data & Analysis ✓ NVIDIA · Official

Clinical ASR Evaluation and KER Leaderboard

Evaluate clinical ASR manifests and isolate terminology-recognition failures.

★ 3.8k FS 55 Use with care today
Dev & Engineering ✓ NVIDIA · Official

DOCA Telemetry Exporter Development

Guides DOCA applications in defining, emitting, and debugging structured telemetry for external consumers.

★ 3.8k FS 55 Use with care today
Dev & Engineering ✓ NVIDIA · Official

NeMo AutoModel Recipe Development

Build, modify, and validate NeMo AutoModel training and evaluation recipes.

★ 3.8k FS 55 Use with care today
Dev & Engineering ✓ NVIDIA · Official

DOCA Flow Tune

Guides engineers through snapshotting, analyzing, and optimizing live or captured DOCA Flow pipelines.

★ 3.8k FS 54 Use with care today

Related skills