🚀 The feature, motivation and pitch

Currently, in the Attention layer, we check if process_weights_after_loading exists and then call it conditionally, and after that we apply flashinfer-specific logic.

Instead, we should just add a process_weights_after_loading method to AttentionImpl (no-op) by default, call it from Attention.process_weights_after_loading, and override it in FlashInferAttentionImpl.

Alternatives

No response

Additional context

#23016 (comment)

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.