Training detail

[SummerRainQ]

Thanks for your great work! I have a question about your training process.

In the paper, you mentioned that you did not modify the LLM architecture, but only added 8192 trainable embeddings for discrete image tokens. However, I found the use of LoRA in the training code you wrote, and there is no embedding layer in the LoRA target. Doesn't this mean that the embedding is not trained?

Is it that all parameters are trained during pretraining, and LoRA is used during SFT?