Llama training finetuning interface

[howard0su]

Like to get early feedback to adding fine tunning interface to llama.h.

1. Leverage same code for compute graph
2. leverage same model class to store the weights including LoRA weights

Questions to get feedback:

1. shall we have finetune context instead of llama_context today?
2. shall we have simple function to complete hide training? I feel it may miss the feature like using grammar CFG to finetune model.

[ggerganov]

Looks fine, though it's hard to tell what's the best approach.
I see you are applying LoRA at runtime. Can you make a prototype for loading the optional LoRA tensors and measure what is the performance difference compared to applying the LoRA once at the start?

[ggerganov]

Suggested change

	LLAMA_API int llama_enable_finetune(struct llama_context * ctx, enum llama_finetune_type flags);
	LLAMA_API int llama_finetune_enable(struct llama_context * ctx, enum llama_finetune_type flags);