Automation & Ops

SageMaker Production Defaults

Deploy SageMaker real-time or async endpoints with autoscaling, CloudWatch alarms, and consistent tagging built in.

46/ 100
Use with care

Useful, but reliability, evidence or controls still have material gaps.

See how it was scored ↓
Works as-is in
Codex · Claude Code
Stars
★ 11k
Last updated
18d ago
License
Apache-2.0
aws-sagemakerautoscalingcloudwatch-alarmsasync-inference
+3vllmteiaws-deployment

What does this skill do, and when should you use it?

This skill from the Hugging Face Skills repository handles the final deployment step after the image URI and IAM role are known. It creates the SageMaker model, endpoint configuration, and endpoint, then enables autoscaling, CloudWatch alarms, and consistent tags by default. It supports real-time inference and asynchronous inference with genuine scale-to-zero behavior. A deployment is not considered complete until a real invocation succeeds and endpoint logs have been checked for worker crashes.

Runs the bundled scripts/deploy.py to create real-time SageMaker models, endpoint configurations, and endpoints, register invocation-based target tracking, and create latency, error, and platform-overhead alarms. Runs scripts/deploy_async.py to create asynchronous endpoints, backlog alarms, and both target-tracking and step-scaling policies needed to wake an endpoint from zero capacity. The scripts accept environment variables, instance settings, S3 model artifacts, data-capture options, SNS notifications, and capacity limits; wait for InService; and emit JSON containing resource names and teardown information. invoke_endpoint.py validates real-time requests while handling JSON and UTF-8 BOM issues. Async invocation uploads input to S3, calls invoke-endpoint-async, and polls the output location.

Good fit
  • A team has selected a serving image and IAM role and needs a production-oriented real-time LLM endpoint with scaling and alarms.
  • An engineer is deploying a TEI embedding model and needs a higher per-instance invocation target to avoid unnecessarily aggressive scaling.
  • A team handles long-running, large, or sparse workloads and wants an async SageMaker endpoint that scales to zero between batches.
  • A deployment operator needs to verify actual inference and diagnose an endpoint that is InService but has a crashing Python worker.
  • An AWS user needs consistent resource tags and an ordered teardown path for later cleanup.

How do you install this skill?

Before you use it
  • Confirm the image, IAM role, environment variables, S3/SNS destinations, and expected cost before deployment; these parameters create external resources and ongoing charges.
  • Real-time deployment registers autoscaling and alarms only after InService; later failures may leave billable resources, so inspect state and run teardown as needed.
  • Avoid combining --no-alarms with async min-capacity=0; the documented wake-from-zero alarm may then be absent.
  • Data capture writes requests and responses to S3 and may contain sensitive data; confirm permissions, retention, and compliance requirements first.
  • The scripts were not executed, so AWS API behavior, dependency versions, and end-to-end results remain unverified; publisher identity is treated as unknown.
Before you start
Your agent needs
  • Shell / CLI
  • Network access
  • Local filesystem
Install first
  • Python
  • AWS CLI
  • AWS account with SageMaker, CloudWatch, S3, and autoscaling permissions

Copy or symlink the hf-cloud-sagemaker-production-defaults folder from the repository's skills/ directory into a standard Codex .agents/skills location, such as $REPO_ROOT/.agents/skills or $HOME/.agents/skills. The README does not document a dedicated install command for this individual skill; the repository supports loading Agent Skills through compatible clients. If the installed copy lacks scripts/, obtain them from the source repository rather than reimplementing them from the description.

Generic route: install into Claude Code manually (macOS / Linux)
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/huggingface/skills.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/skills/hf-cloud-sagemaker-production-defaults" ~/.claude/skills/
rm -rf "$tmp"

Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.

How do you use this skill?

Try saying

Once installed, send your agent any of these to trigger it:

  • Use the SageMaker production defaults skill to deploy this model, then run the smoke test and CloudWatch worker-crash log check.

After AWS context discovery, IAM preparation, and image selection, prompt an agent: “Use the SageMaker production defaults skill to deploy this model, then run the smoke test and CloudWatch worker-crash log check.” For a real-time endpoint, run python scripts/deploy.py --model-name <name> --image-uri <uri> --role-arn <role-arn> --instance-type <type> --region <region>; vLLM deployments also require the model-specific environment variables and --inference-ami-version for cu130+ images. For async inference, run python scripts/deploy_async.py and provide --output-s3-uri. Report completion only after a real request succeeds and the log scan passes. Tear down with python3 scripts/teardown.py <endpoint-name> <region>.

What are this skill's strengths and limitations?

Pros
  • Real-time and async deployments default to autoscaling, CloudWatch alarms, and consistent tags.
  • Async mode genuinely supports `MinCapacity=0` and automatically configures wake-from-zero step scaling.
  • Bundled helpers cover real-time invocation, async invocation, failure diagnosis, and ordered teardown.
  • Data capture is opt-in, avoiding ongoing S3 capture costs unless requested.
Limitations
  • It assumes the image URI, usable IAM role, region, instance type, and required AWS permissions are already available; it is not the full planning or preflight workflow.
  • AWS endpoint, monitoring, storage, and related costs still apply; data capture additionally requires S3 write access.
  • The supplied source provides no automated test suite or cross-platform test results.
  • The default target of 20 invocations per instance can be too low for high-throughput embedding deployments and may require tuning.

How does this skill compare with similar options?

Side by side with related skills; every score comes from the same FSRS standard.

Skill FS score Stars Last updated License
SageMaker Production Defaults this page 46 · Use with care ★ 11k 18d ago Apache-2.0
SageMaker Deployment Planner 50 · Use with care ★ 11k 18d ago Apache-2.0
SageMaker Serving Image Selector 47 · Use with care ★ 11k 18d ago Apache-2.0
SageMaker IAM Preflight 41 · Not recommended ★ 11k 18d ago Apache-2.0
AI Runway AKS Deployment Guide ✓ Microsoft · Official 51 · Use with care ★ 2.7k 3mo ago MIT

The source does not provide a direct comparison with other deployment tools or skills; it positions this skill as the final step in the SageMaker deployment workflow.

How did FollowSkills review this skill?

FollowSkills review · FSRS-2.0
Use with care
46/ 100 5-point scale 2.3 / 5
The upstream repository has new commits since this review. The score still applies to the reviewed revision shown and may not cover the latest changes.
1Trust12 / 25 · 2.4/5

The evidence shows autoscaling, CloudWatch alarms, consistent tagging, opt-in data capture, and a teardown command that preserves IAM, S3 objects, and model artifacts. However, it directly creates billable SageMaker, S3, CloudWatch, and autoscaling resources without an explicit confirmation step, accepts arbitrary image URIs, IAM role ARNs, environment variables, and S3/SNS destinations, and lacks automatic rollback after partial failure, so points are deducted.

2Reliability7 / 20 · 1.8/5

The scripts provide argument validation, endpoint polling, surfaced failure reasons, BOM handling, and staged error feedback, making the happy path reasonably coherent. However, no committed test suite is supplied; boto3/AWS CLI dependencies and API behavior are not verified by the files; later autoscaling or alarm failures can leave orphaned resources; and same-name model reuse does not verify configuration equivalence, so points are deducted.

3Adaptability8 / 15 · 2.7/5

The description clearly covers real-time and async SageMaker endpoints, vLLM/TEI deployments, long-running workloads, sparse traffic, and some Windows usage. Trigger boundaries, non-SageMaker exclusions, and region/network constraints are incomplete, while core operation depends on AWS, S3, CloudWatch, SNS, and potentially the HF Hub; mainland-China reachability and Chinese-language fit are not evidenced, so points are deducted.

4Convention8 / 15 · 2.7/5

The skill has well-layered instructions, a reference design document, scripts, examples, defaults tables, troubleshooting, teardown guidance, stable-looking names, and repository-level Apache-2.0 licensing. The selected skill lacks its own version, changelog, maintainer, update path, complete dependency installation notes, FAQ, test entry point, and documented API compatibility limits, so points are deducted.

5Effectiveness7 / 15 · 2.3/5

The scripts cover model and endpoint configuration, real-time and async deployment, autoscaling, alarms, smoke invocation, and teardown, and emit machine-readable JSON; static evidence supports completion of the core deployment task. However, execution was not performed, smoke testing is not enforced by the deploy scripts, defaults can create avoidable cost or throughput mismatch, and partial cleanup plus the async no-alarms combination create usability risks, so the static ceiling applies and points are deducted.

6Verifiability4 / 10 · 2.0/5

The source exposes auditable code, parameters, defaults, logs, and failure paths, with limited repository-level context from README and LICENSE. It contains no committed tests, CI coverage, or third-party execution records; key AWS/Hugging Face claims remain documentary assertions and cannot be reproduced statically, so points are deducted.

1 2 3 4 5 6

Open a dimension to read why it scored that way

Reviewed Jul 20, 2026 Reviewed revision 86cdeee824b7 Review evidence[1][2][3][4][5][6][7][8][9]

Evidence confidence:Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.

See the full review method →

FAQ

Does this skill prepare the image and IAM role automatically?
No. The image URI, IAM role, region, and instance type should be available before it runs; the source assigns those inputs to image selection, IAM preflight, AWS context discovery, or the user/planner.
Why can an endpoint be InService but still fail inference?
InService only means the container answered `/ping`. Run one real invocation and scan `/aws/sagemaker/Endpoints/<endpoint-name>` for `Worker died`, `Load model failed`, or `ImportError` markers.
Can async endpoints really scale to zero?
Yes. Async deployment defaults to `--min-capacity 0` and registers both backlog target tracking and a `HasBacklogWithoutCapacity`-driven step-scaling policy for the 0→1 transition.
How do I stop endpoint billing after deployment?
Run `python3 scripts/teardown.py <endpoint-name> <region>`. It deletes alarms, autoscaling, the endpoint, its configuration, and the model in order, but preserves the IAM role, S3 objects, SNS topic, and original model artifacts.

More skills from this repository

All from huggingface/skills

Automation & Ops

SageMaker Deployment Planner

Choose a practical Amazon SageMaker deployment path for your model, traffic pattern, and latency needs.

★ 11k FS 50 Use with care 18d ago
Automation & Ops

SageMaker Serving Image Selector

Choose a compatible SageMaker serving container and retrieve the current regional image URI for a Hugging Face model.

★ 11k FS 47 Use with care 18d ago
Automation & Ops

SageMaker IAM Preflight

Find and validate a usable SageMaker execution role before deployment or training.

★ 11k FS 41 Not recommended 18d ago
Automation & Ops

SageMaker Python Environment Setup

Build an isolated, version-aligned Python environment for SageMaker and AWS automation.

★ 11k FS 49 Use with care 18d ago
Data & Analysis

Hugging Face Local Evaluator

Run controlled inspect-ai or lighteval evaluations for Hugging Face Hub models on local hardware.

★ 11k FS 47 Use with care 18d ago
Automation & Ops

AWS Context Discovery

Resolve the active AWS environment before any account operation.

★ 11k FS 50 Use with care 18d ago
Dev & Engineering

LoRA Space Demo Builder

Build and publish a tailored Gradio Space for a Hugging Face LoRA.

★ 11k FS 51 Use with care 18d ago
Dev & Engineering

Hugging Face Spaces Builder

Build, deploy, debug, and maintain machine-learning apps on Hugging Face Spaces.

★ 11k FS 50 Use with care 18d ago
Dev & Engineering

Hugging Face Local Model Runner

Choose compatible GGUF quantizations from Hugging Face and run them locally with llama.cpp.

★ 11k FS 44 Not recommended 18d ago
Dev & Engineering

Hugging Face ZeroGPU Builder

Build dependable Gradio AI demos and GPU workloads on Hugging Face ZeroGPU.

★ 11k FS 56 Use with care 18d ago
Data & Analysis

Hugging Face Dataset Explorer

Explore, query, and extract Hugging Face datasets through the Dataset Viewer API.

★ 11k FS 49 Use with care 18d ago
Dev & Engineering

Hugging Face Paper Research Assistant

Read AI papers and retrieve structured metadata from Hugging Face paper pages.

★ 11k FS 49 Use with care 18d ago
Data & Analysis

Hugging Face LLM Trainer

Train, fine-tune, and preserve models on Hugging Face cloud GPUs.

★ 11k FS 48 Use with care 18d ago
Data & Analysis

TRL Language Model Trainer

Train and fine-tune transformer language models with TRL’s CLI for SFT, preference alignment, and reinforcement learning.

★ 11k FS 48 Use with care 18d ago
Data & Analysis

Sentence Transformers Training Assistant

Train retrieval, similarity, reranking, and sparse-search models with Sentence Transformers.

★ 11k FS 47 Use with care 18d ago
Data & Analysis

Transformers.js Inference Skill

Run Hugging Face models directly inside JavaScript applications.

★ 11k FS 46 Use with care 18d ago
Data & Analysis

Hugging Face Model Memory Estimator

Estimate inference memory without downloading model weights.

★ 11k FS 44 Not recommended 18d ago
Data & Analysis

Trackio Experiment Tracking

Log, monitor, and query machine-learning experiments with dashboards, diagnostic alerts, and automation-ready output.

★ 11k FS 43 Not recommended 18d ago
Data & Analysis

Hugging Face Vision Trainer

Train detection, classification, and segmentation models on Hugging Face Jobs GPUs and persist the results to the Hub.

★ 11k FS 43 Not recommended 18d ago
Dev & Engineering

Hugging Face Hub CLI Skill

Give agents command-line control over Hugging Face Hub models, datasets, Spaces, jobs, and infrastructure.

★ 11k FS 42 Not recommended 18d ago

Related skills