Back to directory

Pydantic Evals

Agent skill
1 skill

Test and evaluate AI agents and LLM outputs using code-first evaluation framework with strong typing. Use when the user wants to: (1) Create evaluation datasets with test cases for AI agents, (2) Define evaluators (deterministic, LLM-as-Judge, custom, or span-based), (3) Run evaluations and generate reports, (4) Compare model performance across experiments, (5) Integrate evaluations with Pydantic AI agents, (6) Set up observability with Logfire, (7) Generate test datasets using LLMs, (8) Implement regression testing for AI systems.

Install with
Claude CodeCodexCursorGitHub CopilotGemini CLI

Supported by the Atlas install prompt. Skill-specific requirements vary.

Skill stats

Skills1CategoryScience & researchStars0

About this skill

Test and evaluate AI agents and LLM outputs using code-first evaluation framework with strong typing. Use when the user wants to: (1) Create evaluation datasets with test cases for AI agents, (2) Define evaluators (deterministic, LLM-as-Judge, custom, or span-based), (3) Run evaluations and generate reports, (4) Compare model performance across experiments, (5) Integrate evaluations with Pydantic AI agents, (6) Set up observability with Logfire, (7) Generate test datasets using LLMs, (8) Implement regression testing for AI systems.

Quick start

  1. Copy the install prompt. Use the green button above this README.
  2. Send it to your agent. Paste it into your coding agent’s chat and send.
  3. Review the setup. Your agent checks the skill’s instructions, requirements, and installation scope.
Review, license & listing details
Review the source before using this skill

This is an imported listing. We haven’t certified its safety, license, or compatibility. Skills may include executable scripts.

Automated security scan: Partial

Only SKILL.md was scanned. Supporting files and scripts were not checked. No safety grade has been assigned.

0 potential findings · Scanned 11/4/2026 · Scanner 0.2.1

Content SHA-256: 06e5a500b0ea54ca11f2779b8bc2198a32da10ec3439a1c9ec270206fd803929

About scan coverage and limitations

About this listing

Category
Science & research
Reported license
MIT
Listing language (estimated)
English
Review status
Not individually reviewed
Agent compatibility
Atlas installation supports Claude Code, Codex, and Cursor. Skill-specific dependencies and features are checked during setup.