Diagnose slow, expensive, or regressed Apache Spark and PySpark applications by comparing runtime evidence against a healthy run. Use for long stages, stragglers, skew, shuffle, spill, garbage collection, poor parallelism, small files, slow scans, scheduler delay, executor imbalance, and unexplained compute-cost growth.
EnglishDiagnose failed Apache Spark and PySpark applications from History Server evidence, logs, and cluster-manager state. Use for driver or executor crashes, out-of-memory errors, fetch failures, task exceptions, timeouts, repeated retries, aborted stages, and intermittent production failures.
EnglishOptimize Apache Spark SQL and DataFrame queries using the final Adaptive Query Execution plan and runtime statistics rather than source code alone. Use to reduce runtime, shuffle, spill, scan cost, skew, join amplification, Python UDF overhead, poor partitioning, or unnecessary work while preserving query semantics.
English
AGENT SKILLS ATLASIncludes Debug Slow Spark Job, Debug Spark Failure, and Optimize Spark SQL Plan.
Claude Code
Codex
Cursor
GitHub Copilot
Gemini CLISupported by the Atlas install prompt. Skill-specific requirements vary.
Pack stats
What’s included
Debug Slow Spark Job
Diagnose slow, expensive, or regressed Apache Spark and PySpark applications by comparing runtime evidence against a healthy run. Use for long stages, stragglers, skew, shuffle, spill, garbage collection, poor parallelism, small files, slow scans, scheduler delay, executor imbalance, and unexplained compute-cost growth.
Debug Spark Failure
Diagnose failed Apache Spark and PySpark applications from History Server evidence, logs, and cluster-manager state. Use for driver or executor crashes, out-of-memory errors, fetch failures, task exceptions, timeouts, repeated retries, aborted stages, and intermittent production failures.
Optimize Spark SQL Plan
Optimize Apache Spark SQL and DataFrame queries using the final Adaptive Query Execution plan and runtime statistics rather than source code alone. Use to reduce runtime, shuffle, spill, scan cost, skew, join amplification, Python UDF overhead, poor partitioning, or unnecessary work while preserving query semantics.
Quick start
- Choose your skills. Select individual skills above, or choose the entire pack.
- Copy the install prompt. Use Copy install prompt above the list.
- Send it to your agent. Paste into your coding agent’s chat. Your agent reviews the source and guides the setup.
Try it with your agent
After installation, start with a request like:
Use the Debug Slow Spark Job skill to help with my task.
Ask me for the context you need before getting started.Requirements & license
- A coding agent that can access your project files and install skills.
- Your agent checks each skill’s instructions, dependencies, and permissions during setup.
- Reported license for the skills shown: Apache-2.0. The author’s terms apply.