Only run get_attr_docs if generating help text

[hmellor]

The docstrings for config class attributes are only actually needed if we're generating --help text or documentation.

This saves 235ms on my machine by completely eliminating the cost of extracting the docstrings on start up.

---

$ vllm serve --help
INFO 08-27 10:36:28 [__init__.py:241] Automatically detected platform cuda.
INFO 08-27 10:36:30 [arg_utils.py:165] Computed kwargs for FrontendArgs in 0.0038s (cumulative: 0.0038s)
INFO 08-27 10:36:30 [arg_utils.py:165] Computed kwargs for ModelConfig in 0.0383s (cumulative: 0.0421s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for LoadConfig in 0.0263s (cumulative: 0.0684s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for DecodingConfig in 0.0302s (cumulative: 0.0985s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for ParallelConfig in 0.0061s (cumulative: 0.1046s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for CacheConfig in 0.0032s (cumulative: 0.1078s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for MultiModalConfig in 0.0284s (cumulative: 0.1362s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for LoRAConfig in 0.0283s (cumulative: 0.1645s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for ObservabilityConfig in 0.0302s (cumulative: 0.1947s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for SchedulerConfig in 0.0034s (cumulative: 0.1980s)
INFO 08-27 10:36:31 [arg_utils.py:165] Computed kwargs for VllmConfig in 0.0370s (cumulative: 0.2350s)
...

$ vllm serve 
INFO 08-27 10:36:57 [__init__.py:241] Automatically detected platform cuda.
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for FrontendArgs in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for ModelConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for LoadConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for DecodingConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for ParallelConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for CacheConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for MultiModalConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for LoRAConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for ObservabilityConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for SchedulerConfig in 0.0000s (cumulative: 0.0000s)
INFO 08-27 10:37:00 [arg_utils.py:165] Computed kwargs for VllmConfig in 0.0000s (cumulative: 0.0000s)
...

[gemini-code-assist]

Code Review

This pull request aims to optimize startup time by avoiding the extraction of docstrings when not generating help text. While the performance improvement is valuable, the current implementation introduces a critical caching bug by making a cached function's behavior dependent on a global variable (sys.argv). My review provides a detailed explanation of this bug and a robust solution to fix it, ensuring both performance and correctness.

[mgoin]

Thank you!

[ZJY0516]

Hi @hmellor . I was wondering how to measure time like this?

INFO 08-27 10:36:30 [arg_utils.py:165] Computed kwargs for FrontendArgs in 0.0038s (cumulative: 0.0038s)

[hmellor]

I added some temporary logs that I did not commit, the code was:

cumulative_time = 0

@functools.lru_cache(maxsize=30)
def _compute_kwargs(cls: ConfigType, needs_help: bool) -> dict[str, Any]:
    import time
    global cumulative_time
    start = time.perf_counter()
    # Save time only getting attr docs if we're generating help text
    cls_docs = get_attr_docs(cls) if NEEDS_HELP else {}
    duration = time.perf_counter() - start
    cumulative_time += duration
    logger.info(f"Computed kwargs for {cls.__name__} in {duration:.4f}s (cumulative: {cumulative_time:.4f}s)")
    ...

[hmellor]

RTD docs build currently isn't triggering get_attr_docs just fixing it then I'll re-enable auto-merge

[hmellor]

Docs look good again

[ProExpertProg]

Perhaps a follow up but could we also extract this during build time? Because now we pay this cost during vllm help when we could avoid it all together.

[hmellor]

Yeah this could be a build time constant. I'm not sure how it would work though (I've never tried to do it before, I'm sure it can be done)