Add PyTorch profiler support to vllm_bench.sh

[npalaska]

Added support for running torch profiles

[coderabbitai]

Walkthrough

Adds Torch Profiler support to the benchmarking workflow, including per-model/per-concurrency trace generation, environment export, and output management. Documents multi-tool profiling (Nsight Systems, Nsight Compute, PyTorch Profiler). Introduces a sample YAML config enabling Torch Profiler alone or combined with nsys, and updates README instructions accordingly.

Changes

Cohort / File(s)	Summary of Changes
Documentation: Profiling guidance README.md	Rewrites overview for single vLLM server; adds Profiling Tools and Profiling Options sections; documents Nsight Systems, Nsight Compute, PyTorch Profiler; includes config examples and output artifacts; updates output directory wording.
Configuration: Torch profiling presets configs/bench_torch_profiling.yaml	New example config enabling Torch Profiler with per-model toggles; options for record_shapes, profile_memory, with_stack, with_flops; optional combined nsys + Torch profiling; sample concurrencies, result prefixes, and compilation_config.
Benchmark runner: Torch Profiler integration vllm_bench.sh	Adds Torch Profiler enablement via config; exports env vars (dir, record_shapes, profile_memory, with_stack, with_flops); creates per-model/per-concurrency profiler directories; names per-concurrency trace files; integrates --profile flow into vLLM bench command; prints destinations and lists generated traces; nests profiling info in per-model manifest (nsys, torch_prof).

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant U as User
  participant B as vllm_bench.sh
  participant C as Config (YAML)
  participant S as vLLM Server/Bench
  participant TP as Torch Profiler
  participant NS as Nsight Systems (nsys)

  U->>B: Run benchmark with profiling config
  B->>C: Read per-model settings
  loop For each model
    B->>B: Create model output dirs (incl. TORCH_PROFILER_DIR)
    alt Torch Profiler enabled
      B->>B: Export VLLM_TORCH_PROFILER_* env vars<br/>(record_shapes, profile_memory, with_stack, with_flops)
    end
    alt nsys enabled
      B->>NS: Prepare nsys launch/start args
    end
    loop For each concurrency
      B->>B: Set TORCH_PROFILE_FILE (per-concurrency)
      B->>S: Invoke vllm bench [--profile] with model opts
      par Profiling
        S-->>TP: Collect PyTorch trace (json/pt)
        opt If nsys enabled
          S-->>NS: Record system trace (.qdrep)
        end
      end
    end
    B->>B: List generated Torch Profiler traces
  end
  B-->>U: Output paths and summary

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

I tap-tap logs with twitchy paws,
New traces bloom from benchmark laws.
Nsight gleams, Torch sings along—
Concurrencies drum a profiling song.
In tidy burrows of dirs I keep,
qdrep and json drift to sleep. 🐇✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title precisely identifies the main enhancement—adding PyTorch profiler support—and specifies the affected script, vllm_bench.sh, matching the substantive changes in the pull request without unnecessary detail.
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch add_torch_profiling

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2c2d74d and 8239c2e.

📒 Files selected for processing (3)

• README.md (2 hunks)
• configs/bench_torch_profiling.yaml (1 hunks)
• vllm_bench.sh (5 hunks)