[Enhance] Add LoRA rank args in train_text_to_image_lora

[okotaku]

What does this PR do?

・Add LoRA rank and network_alpha args in train_text_to_image_lora.py

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[patrickvonplaten]

Ok for me! @sayakpaul wdyt?

[sayakpaul]

Thanks for the PR! Could you help me understand why the addition of network_alpha is necessary? IIUC it's only needed for loading the A1111 CivitAI LoRA checkpoints. Cc: @takuma104 as well.

[okotaku]

@sayakpaul Yes. network_alpha has the same meaning as the diffusers' alpha. However, I have noticed that some repositories use network_alpha during training, and I'm not sure of the exact reasons behind this.

For example,

python3 sd-scripts/train_network.py \
  --network_module lycoris.kohya \
  --network_dim "DIM_FOR_LINEAR" --network_alpha "ALPHA_FOR_LINEAR"\
  --network_args "conv_dim=DIM_FOR_CONV" "conv_alpha=ALPHA_FOR_CONV" \
  "dropout=DROPOUT_RATE" "algo=locon" \

https://github.com/KohakuBlueleaf/LyCORIS

--network_alpha: Specifies the alpha value for stable learning and preventing underflow. The default is 1. Specifying the same value as network_dim results in the same behavior as previous versions.

https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md

[sayakpaul]

Thanks for providing the explanation! I guess we can safely ignore that when training using diffusers. WDYT?

[okotaku]

@sayakpaul Sounds good. I fixed.

[sayakpaul]

Gracias 🙏