Download
Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of pre-trained language models (PLMs) to various downstream applications without fine-tuning all the model's parameters. Fine-tuning large-scale PLMs is often prohibitively costly. In this regard, PEFT methods only fine-tune a small number of (extra) model parameters, thereby greatly decreasing the computational and storage costs. Recent State-of-the-Art PEFT techniques achieve performance comparable to that of full fine-tuning.
Features
- Accelerate for large scale models leveraging DeepSpeed and Big Model Inference
- Get comparable performance to full finetuning by adapting LLMs to downstream tasks using consumer hardware
- GPU memory required for adapting LLMs on the few-shot dataset
- Parameter Efficient Tuning of Diffusion Models
- GPU memory required by different settings
- Parameter Efficient Tuning of LLMs for RLHF components such as Ranker and Policy
Project Samples
License
Apache License V2.0Follow PEFT
Other Useful Business Software
Get Avast Free Antivirus | Your top-rated shield against malware and online scams
Our antivirus software scans for security and performance issues and helps you to fix them instantly. It also protects you in real time by analyzing unknown files before they reach your desktop PC or laptop — all for free.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of PEFT!