Software Development Skills

Write effective, consistent annotations on LLM/agent spans and traces, and coach the user on annotation practice. Load this whenever you are about to record structured feedback with the batchspanannotate tool, or when the user asks how to annotate, label, score, or review spans/traces, build a failure taxonomy, or set up human/LLM review. Do NOT load for: pure analysis with no intent to save feedback (use debug-trace), latency or cost statistics, or prompt authoring (use playground).

Diagnose failure modes by systematically investigating traces. Trigger when the user explicitly asks for cross-trace diagnosis: "what's going wrong?", "were there errors?", "debug this", "where is my agent struggling?". Do NOT trigger on: (1) advice questions ("what should I do?"), (2) statistical questions ("what's the average latency?"), (3) summarize requests, (4) trace filtering ("show me traces with errors"), (5) vague questions ("is there a problem?"), (6) unrelated requests.

Author or refine a Phoenix evaluator — code or LLM-as-a-judge — that scores a run's output. Trigger when the user wants to create a new evaluator, improve an existing one's logic or rubric, choose labels, or decide what to measure on a dataset or experiment. Do NOT trigger on: (1) manual prompt drafting (use playground), (2) running or comparing experiments themselves (use experiments), (3) cross-trace failure diagnosis with no evaluator in scope (use debug-trace).

Design and implementation guide for the Phoenix CLI (px). Covers the noun-verb command structure, dual-audience design (humans and coding agents), Commander.js patterns, configuration resolution, output formats, exit codes, and conventions for adding or modifying commands. Triggers when working on phoenix-cli commands — adding new commands, modifying existing ones, refactoring command structure, or reviewing CLI code. Also triggers on mentions of px commands, CLI design, or adding a new resource to the CLI.

Guide for the phoenix-client TypeScript package — experiment lifecycle, tracer provider management, and test conventions.

Design system conventions for the Phoenix frontend — layout, dialogs, error display, BEM CSS class naming, and CSS design tokens. Use when building UI, naming CSS classes, creating or consuming tokens, handling errors, or designing dialog interactions in app/src/.

Audit documentation gaps across the Phoenix repo by analyzing recent commits to main (default: last 7 days). Use this skill whenever the user asks to find undocumented features, identify docs gaps, audit what shipped without docs, check which recent changes need documentation, review stale docs against current code, or mentions "documentation debt", "doc coverage", "undocumented APIs", or "what's missing from /docs". Also trigger on requests like "what from last week needs docs", "find stale READMEs", or "check docstring coverage for recent changes". Covers /docs (Mintlify), package READMEs, package-level built-in docs (Sphinx, TypeDoc), Python docstrings, TSDoc, and code comments.

Create a new built-in classification evaluator for Phoenix evals. Use this skill whenever the user asks to create a new eval, build a new metric, add a new builtin evaluator, create an LLM-as-a-judge metric, or add a new classification evaluator to Phoenix.

Frontend development guidelines for the Phoenix AI observability platform. Use when writing, reviewing, or modifying React components, TypeScript code, styles, or UI features in the app/ directory. Triggers on any frontend task — new components, UI changes, styling, accessibility fixes, form handling, or component refactoring. Also use when the user asks about frontend conventions or component patterns for this project. For design system rules (error display, layout, dialogs, tokens), use the phoenix-design skill.

Manage GitHub issues, labels, and project boards for the Arize-ai/phoenix repository. Use when filing roadmap issues, triaging bugs, applying labels, managing the Phoenix roadmap project board, or querying issue/project state via the GitHub CLI.

Write efficient GraphQL queries against the Phoenix API. Load this skill in two cases: (1) before composing any non-trivial GraphQL query yourself for data analysis (via the phoenix-gql bash command) — it contains schema entrypoints and patterns that eliminate the need for introspection; (2) when the user asks for help writing GraphQL queries for their own scripts, tools, or integrations against Phoenix — it covers the endpoint, authentication, and client examples.

Generates onboarding code snippets for Phoenix tracing integrations and wires them into the project onboarding UI. Produces install dependencies and implementation sections for SDKs like OpenAI, LangChain, Vercel AI SDK, and others. Supports Python and TypeScript. Use when asked to create onboarding code, tracing setup snippets, quickstart examples, or getting-started code for a framework integration.

Maintain the Phoenix llms.txt documentation index at docs/phoenix/llms.txt — the machine-readable docs map used by AI agents and the px docs fetch CLI. Use this skill whenever adding, auditing, or reorganizing llms.txt entries. Trigger when the user mentions llms.txt, docs index, px docs, or LLM-friendly documentation.

Open-source AI observability platform for LLM tracing, evaluation, and monitoring.

Guide for the phoenix-otel TypeScript package — OTel registration, stack-based global provider management, and provider lifecycle.

Write Playwright E2E tests for the Phoenix AI observability platform. Use when creating, updating, or debugging Playwright tests, or when the user asks about testing UI features, writing E2E tests, or automating browser interactions for Phoenix.

Screenshot a running Phoenix feature and attach images to a GitHub PR. Builds the frontend, starts Phoenix with env vars, uses agent-browser to capture screenshots, uploads to GCS, and updates the PR body.

Development guide for the Phoenix PXI agent. Use when modifying PXI-specific frontend or backend behavior, extending PXI tool wiring, updating PXI runtime capabilities, or changing the PXI agent request/dispatch flow. Start here for PXI-specific workflows, then read the relevant resource file for the layer you are changing.

Write, extend, and debug PXI Playwright E2E tests for Phoenix. Use when adding PXI agent frontend specs, authoring LLM-as-judge rubrics, asserting PXI tool use, persisting PXI test runs as Phoenix experiments, or debugging PXI E2E failures.

Create Phoenix release documentation grounded in actual code changes. Use this skill whenever the user asks to write release notes, document a release, update release documentation, or mentions undocumented releases. Also trigger when the user wants to update GitHub release descriptions, add entries to the release notes page, or asks what changed in a recent Phoenix version.

Bump the next release-please version for a Phoenix Python package (arize-phoenix, arize-phoenix-client, arize-phoenix-evals, arize-phoenix-otel) by opening a PR with a Release-As commit footer. Use this skill when the user asks to "set the release-please version", "force a 2.0.0 release", "release X as Y", "skip a version", or otherwise wants release-please to propose a specific version on its next run instead of the version it would pick from conventional commits.

REST API development for Phoenix. Use when adding, modifying, or reviewing endpoints in src/phoenix/server/api/routers/v1/.

Backend development guide for the Phoenix AI observability platform (Strawberry GraphQL, SQLAlchemy async, FastAPI). Use this skill when writing or modifying Python server code in the phoenix repo — adding mutations, types, migrations, or tests. Trigger on any backend task touching src/phoenix/server/, src/phoenix/db/, or tests/unit/server/.

Audit recent changes to Phoenix's user-facing surfaces (Python clients, TypeScript clients, CLI, REST/GraphQL APIs) and patch the three external-facing agent skills — phoenix-tracing, phoenix-cli, and phoenix-evals — so they stay in sync with what actually shipped. Use this skill whenever a user asks to update those skills, sync the skills with recent changes, audit skill drift, check what client/CLI/API changes need to land in the skills, or mentions "skill freshness", "skill drift", "stale skills", or "are the skills up to date". Also trigger when shipping a notable client/CLI/API change and the user asks "do the skills need updating?". Default window is the last 7 days on origin/main; user may override.