Search a vLLM serving model's config space for its best inference performance: a perftunereport campaignrun matrix sweep over concurrency x maxnumbatchedtokens x kv-cache-dtype x cudagraphmode, plus a same-node controlled A/B (run-controlled-ab.sh) for any decode-latency claim, gated by perf-baseline-diff. Picks a champion config that beats the baseline on the run's focus metric (throughput tok/s OR median TPOT/ITL), honoring warm/cold + eager/cudagraph methodology rules. This is the tuning phase of inference-model-optimize, usable standalone. Not a Bayesian optimizer -- it sweeps an operator-named grid (an --optimizer hook is stubbed). Triggers on "tune the vllm config", "sweep maxnumbatchedtokens", "find the best serving config", "vllmArgs sweep", "cudagraphmode A/B", "kv-cache-dtype sweep", "tune concurrency", "config search for ", or any combination of "tune / sweep / optimize / search / A-B" with "vllm / config / vllmArgs / concurrency / batched-tokens / cudagraph".
Install with
Claude CodeCodexCursorGitHub CopilotGemini CLI
Supported by the Atlas install prompt. Skill-specific requirements vary.
Skill stats
Skills1CategoryGeneralStars0
About this skill
Search a vLLM serving model's config space for its best inference performance: a perftunereport campaignrun matrix sweep over concurrency x maxnumbatchedtokens x kv-cache-dtype x cudagraphmode, plus a same-node controlled A/B (run-controlled-ab.sh) for any decode-latency claim, gated by perf-baseline-diff. Picks a champion config that beats the baseline on the run's focus metric (throughput tok/s OR median TPOT/ITL), honoring warm/cold + eager/cudagraph methodology rules. This is the tuning phase of inference-model-optimize, usable standalone. Not a Bayesian optimizer -- it sweeps an operator-named grid (an --optimizer hook is stubbed). Triggers on "tune the vllm config", "sweep maxnumbatchedtokens", "find the best serving config", "vllmArgs sweep", "cudagraphmode A/B", "kv-cache-dtype sweep", "tune concurrency", "config search for ", or any combination of "tune / sweep / optimize / search / A-B" with "vllm / config / vllmArgs / concurrency / batched-tokens / cudagraph".
Quick start
Copy the install prompt. Use the green button above this README.
Send it to your agent. Paste it into your coding agent’s chat and send.
Review the setup. Your agent checks the skill’s instructions, requirements, and installation scope.
Search a vLLM serving model's config space for its best inference performance: a `perftunereport campaign_run` matrix sweep over concurrency x max_num_batched_tokens x kv-cache-dtype x cudagraph_mode, plus a same-node controlled A/B (`run-controlled-ab.sh`) for any decode-latency claim, gated by `perf-baseline-diff`. Picks a champion config that beats the baseline on the run's focus metric (throughput tok/s OR median TPOT/ITL), honoring warm/cold + eager/cudagraph methodology rules. This is the tuning phase of `inference-model-optimize`, usable standalone. Not a Bayesian optimizer -- it sweeps an operator-named grid (an --optimizer hook is stubbed). Triggers on "tune the vllm config", "sweep max_num_batched_tokens", "find the best serving config", "vllmArgs sweep", "cudagraph_mode A/B", "kv-cache-dtype sweep", "tune concurrency", "config search for <model>", or any combination of "tune / sweep / optimize / search / A-B" with "vllm / config / vllmArgs / concurrency / batched-tokens / cudagraph".
About this listing
Category
General
Reported license
MIT
Listing language (estimated)
English
Review status
Not individually reviewed
Agent compatibility
Atlas installation supports Claude Code, Codex, and Cursor. Skill-specific dependencies and features are checked during setup.
Claude Code
Codex
Cursor
GitHub Copilot
Gemini CLI