Azure Content Understanding for Python
Extract structured multimodal content from documents, images, audio, and video with Python.
The skill recommends DefaultAzureCredential, discourages keys, and documents the endpoint and credential environment variables, showing useful credential-security guidance. However, it sends documents, images, audio, and video to an external Azure service by URL without explaining data flows, sensitive-data handling, user confirmation, least-privilege permissions, result protection, or deletion/rollback procedures, so points are deducted.
The core asynchronous workflow, sync/async lifecycle guidance, and long-running-operation warning are reasonably clear. However, the main document's dictionary-based create_analyzer example conflicts with the reference file's begin_create_analyzer/model-based example, and result access is inconsistently shown as result.fields versus content.fields. Exception handling, timeout/retry behavior, pagination, and empty-result handling are also missing. Static evidence is limited, so the score remains conservative.
Triggers, Python SDK audience, document/image/audio/video scenarios, and custom-analysis use cases are explicit, with both sync and async forms covered. It does not define service-region or network reachability, input constraints, privacy boundaries, or non-fit scenarios, and provides no Chinese-language usage guidance, so points are deducted.
The documentation includes installation, environment variables, authentication, workflow guidance, examples, best practices, and references, with MIT licensing, author metadata, and version 1.0.0. It lacks a changelog, explicit maintenance owner, and update path; its references also disagree with the main document on API names and model inheritance, preventing full marks.
The prebuilt analyzers and end-to-end examples can support basic multimodal extraction, with directly useful markdown, transcripts, and key-frame outputs. However, the custom-analyzer examples contain conflicting APIs and result-field access, while error handling, input validation, cost/latency guidance, and alternatives are absent; generated implementations may still require substantial correction.
The files provide concrete code, package names, environment variables, and repository-level CI/test infrastructure, giving some auditability. They do not provide skill-specific tests, actual execution results, or independent third-party corroboration for the key paths, and internal API inconsistencies further limit confidence.
- Before using URL inputs, confirm authorization and whether sensitive data may be sent to Azure and processed in the selected region.
- Do not copy the custom-analyzer examples verbatim; verify the current official SDK method, model types, and result-field access.
- Add production handling for exceptions, timeouts, retries, empty results, input limits, cost, long-running cancellation, and cleanup.
- Confirm that the target environment can reach the Azure endpoint; the skill does not address mainland-China network access, regional availability, or data residency.
What does this skill do, and when should you use it?
This skill guides developers using the Azure AI Content Understanding SDK for Python. It covers multimodal analysis of documents, images, audio, and video, with outputs such as Markdown, transcripts, timing data, key frames, and custom fields. It explains synchronous and asynchronous clients, DefaultAzureCredential, long-running analysis, and analyzer management. It fits RAG and automated workflows, provided the application has an Azure Content Understanding endpoint and the required runtime setup.
It shows how to install azure-ai-contentunderstanding, configure CONTENTUNDERSTANDING_ENDPOINT, and call Azure AI Content Understanding through ContentUnderstandingClient. The examples use prebuilt analyzers to extract document Markdown, image content, audio transcripts, and video transcripts and key frames. It also demonstrates creating custom analyzers with field schemas, listing, retrieving, and deleting analyzers, and reading results from AnalyzeResult.contents or custom fields.
- A Python developer converting PDFs or Office documents to Markdown for a RAG pipeline.
- An application team extracting searchable content from image files.
- An automation developer producing audio transcripts with timing information.
- A video workflow that needs transcripts and key frames from uploaded or hosted media.
- A developer extracting invoice fields such as vendor name, total, and line items with a custom analyzer.
What are this skill's strengths and limitations?
- Covers documents, images, audio, and video in one Python SDK workflow.
- Includes both prebuilt analyzers and custom field schemas.
- Documents synchronous, asynchronous, and analyzer-management patterns.
- Provides guidance on DefaultAzureCredential and deterministic client lifecycle management.
- Requires an Azure Content Understanding endpoint and network access to the Azure service.
- Audio and video analysis can take minutes, so callers must handle long-running operations.
- The source provides no independent test result, pricing information, or detailed permission matrix for this individual skill.
- Examples focus on URL inputs and do not document a detailed local-file upload workflow.
How do you install this skill?
Run pip install azure-ai-contentunderstanding in a terminal. To install from the collection, run npx skills add microsoft/skills and select this skill in the wizard. Alternatively, copy .github/plugins/azure-sdk-python/skills/azure-ai-contentunderstanding-py into the target agent's skill directory; the exact directory depends on the agent.
How do you use this skill?
Set CONTENTUNDERSTANDING_ENDPOINT=https://<resource>.cognitiveservices.azure.com/. Prompt a coding agent with a request such as: “Use azure-ai-contentunderstanding to analyze this PDF URL and return the document Markdown.” Production code should prefer DefaultAzureCredential, start analysis with begin_analyze(), and obtain results with poller.result(). For asynchronous workflows, use the aio client and credentials from azure.identity.aio.