Designing, running, and interpreting controlled experiments to decide whether a change is genuinely better — online A/B tests, randomized controlled trials, and offline hypothesis tests. Covers hypothesis framing, randomization and unit-of-assignment, sample-size and statistical-power calculation, metric design (primary, guardrail, invariant, and the overall evaluation criterion), variance reduction (CUPED/CUPAC) for sensitivity, significance testing and confidence intervals, sequential and anytime-valid testing for the peeking problem, multiple-comparison correction, the trust checklist led by sample-ratio-mismatch, interference-aware designs (cluster and switchback), long-term holdouts and reverse experiments, bandit/adaptive-allocation boundaries, experiment ethics and blast-radius limits, quasi-experimental fallbacks when randomization is infeasible, and the validity threats that quietly invalidate results (novelty and primacy effects, interference/SUTVA violations, Simpson's paradox, and survivorship).
Install with
Claude CodeCodexCursorGitHub CopilotGemini CLI
Supported by the Atlas install prompt. Skill-specific requirements vary.
Skill stats
Skills1CategoryScience & researchStars0
About this skill
Designing, running, and interpreting controlled experiments to decide whether a change is genuinely better — online A/B tests, randomized controlled trials, and offline hypothesis tests. Covers hypothesis framing, randomization and unit-of-assignment, sample-size and statistical-power calculation, metric design (primary, guardrail, invariant, and the overall evaluation criterion), variance reduction (CUPED/CUPAC) for sensitivity, significance testing and confidence intervals, sequential and anytime-valid testing for the peeking problem, multiple-comparison correction, the trust checklist led by sample-ratio-mismatch, interference-aware designs (cluster and switchback), long-term holdouts and reverse experiments, bandit/adaptive-allocation boundaries, experiment ethics and blast-radius limits, quasi-experimental fallbacks when randomization is infeasible, and the validity threats that quietly invalidate results (novelty and primacy effects, interference/SUTVA violations, Simpson's paradox, and survivorship).
Quick start
Copy the install prompt. Use the green button above this README.
Send it to your agent. Paste it into your coding agent’s chat and send.
Review the setup. Your agent checks the skill’s instructions, requirements, and installation scope.
Designing, running, and interpreting controlled experiments to decide whether a change is genuinely better — online A/B tests, randomized controlled trials, and offline hypothesis tests. Covers hypothesis framing, randomization and unit-of-assignment, sample-size and statistical-power calculation, metric design (primary, guardrail, invariant, and the overall evaluation criterion), variance reduction (CUPED/CUPAC) for sensitivity, significance testing and confidence intervals, sequential and anytime-valid testing for the peeking problem, multiple-comparison correction, the trust checklist led by sample-ratio-mismatch, interference-aware designs (cluster and switchback), long-term holdouts and reverse experiments, bandit/adaptive-allocation boundaries, experiment ethics and blast-radius limits, quasi-experimental fallbacks when randomization is infeasible, and the validity threats that quietly invalidate results (novelty and primacy effects, interference/SUTVA violations, Simpson's paradox, and survivorship).
About this listing
Category
Science & research
Reported license
CC-BY-4.0
Listing language (estimated)
English
Review status
Not individually reviewed
Agent compatibility
Atlas installation supports Claude Code, Codex, and Cursor. Skill-specific dependencies and features are checked during setup.
Claude Code
Codex
Cursor
GitHub Copilot
Gemini CLI