Vertex AI Agent Alerting
Configure dynamic reliability and quality alerts for Agent Runtime agents.
The evidence requires confirmation for billable online monitors and resource creation and defaults to no notification channels. However, it permits Terraform, gcloud, and Python cloud changes, broadly targets all active agents in a project, and does not adequately document least privilege, sensitive trace-data handling, complete data flows, general rollback, or deployment isolation, so points are deducted.
The document specifies telemetry checks, dependency installation, configuration validation, self-correction, and failure-feedback procedures. However, the referenced scripts, requirements, references, and configuration templates are not supplied, and static review cannot reproduce the key paths. Several rules are also rigid, including mandatory telemetry checks and exactly six policies, so the score remains below the static ceiling.
The skill clearly targets Agent Runtime and distinguishes reliability, quality monitoring, historical-traffic cases, and traffic patterns, giving it reasonably precise triggers. However, non-fit boundaries, input/output contracts, and Chinese-language support are limited. Core operation depends on Google Cloud, gcloud, Cloud Trace, and Online Monitor, with no mainland-China reachability discussion, so points are deducted.
The document is well organized and includes safety tiers, algorithm mapping, caveats, dependency notes, command examples, license metadata, and official links. It does not establish a version, changelog, maintainer, update path, FAQ, or the referenced scripts and supporting files, so maintainability points are deducted.
The stated design offers dynamic baselines, PromQL reliability policies, and online quality evaluation, theoretically covering six alert types. However, no committed Terraform output, execution result, or test evidence is provided, and quality monitoring requires additional billable resources. Static evidence therefore supports only limited direct usability, within the static effectiveness ceiling.
Concrete commands, metric names, filter structures, and Google Cloud official documentation links provide limited auditability. There is no test suite, CI evidence, script source, or third-party execution evidence, so the conclusions are only partially reproducible and points are deducted.
- Before deployment, verify that the referenced scripts, dependencies, references, and Terraform templates exist, and review least-privilege permissions for gcloud, Terraform, and service accounts.
- Quality monitoring incurs additional charges and processes Cloud Trace data; retention, access scope, sampling, and compliance requirements should be made explicit.
- Dynamic coverage of every active agent in a project may be overly broad; confirm resource scope, notification behavior, deduplication, and rollback procedures before applying changes.
- Confirm that Google Cloud and Online Monitor services are reachable in the target network environment and provide Chinese-language operational guidance.
What does this skill do, and when should you use it?
This skill targets reasoning-engine agents deployed on Google Cloud Vertex AI Agent Runtime. It configures six alert policies covering latency, error rate, final response quality, tool use quality, and hallucination. Reliability signals use PromQL and dynamic historical baselines, while quality signals use scores exported by Online Monitors. It produces Terraform configuration and provides Python-based telemetry, duplicate-policy, and validation workflows.
Checks telemetry readiness for an Agent Runtime reasoning engine; selects a latency algorithm based on traffic patterns; generates PromQL policies for latency and error rate; generates standard threshold policies for quality scores; scans for existing policies; validates Terraform, PromQL, and HCL; provisions an Online Monitor through a Python script; and writes observability configuration as Terraform files.
- A Google Cloud operations team monitoring latency anomalies for a production Agent Runtime agent.
- A reliability engineer using one-hour and three-day multi-window burn-rate policies for agent error rates.
- An Agent Platform team monitoring final response quality, tool use quality, and hallucination scores.
- A platform engineer enabling telemetry and an Online Monitor for a newly deployed agent.
What are this skill's strengths and limitations?
- Covers both reliability and quality monitoring with six explicitly defined signals.
- Uses dynamic baselines suited to changing AI-agent traffic instead of relying only on fixed thresholds.
- Uses Terraform for alert configuration and includes duplicate detection and validation workflows.
- Supports dynamic multi-agent aggregation without pinning policies to one agent ID.
- Works only for Agent Runtime, not general GCP alerting or non-agent GCP policies.
- Online Monitors and telemetry incur additional charges, and quality alerts depend on their exported evaluation data.
- Missing project, resource-path, or traffic-history details require user input or a default traffic-pattern choice.
- The source does not document a test suite, supported Terraform version, or runtime environments beyond the named tools.
How do you install this skill?
In an Agent Skills-compatible client, run: npx skills add google/skills. Select agent-platform-alert-configuration during installation. The source does not document additional client-specific installation steps.
How do you use this skill?
Give an Agent Skills-compatible client a request such as: “Configure reliability and quality alerts for my Vertex AI Agent Runtime reasoning engine.” Provide the project ID, the full agent resource path (projects/<project_id>/locations/<location>/reasoningEngines/<agent_id>), and the relevant Terraform workspace details. Before creating an Online Monitor or enabling telemetry, confirm approval for the additional charges. The source does not provide a complete no-argument execution command.
How does this skill compare with similar options?
The skill explicitly contrasts dynamic PromQL baselines with static thresholds and requires PromQL for reliability signals but standard threshold filters for quality signals. It is not a general-purpose GCP alerting tool.