Skip to content

Commit 0c97a55

Browse files
authored
Update README.md
1 parent f86412b commit 0c97a55

File tree

1 file changed

+7
-6
lines changed

1 file changed

+7
-6
lines changed

README.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,33 +10,33 @@ Create a new conda environment and activate it
1010
conda create -n env
1111
conda activate env
1212
```
13-
Find the pytorch version compatible with your version of cuda [here](https://pytorch.org/get-started/previous-versions/), for example the following command work with cuda 11.6
13+
Install the `pytorch` version compatible with your version of cuda [here](https://pytorch.org/get-started/previous-versions/), for example the following command works with cuda 11.6
1414
```bash
1515
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.6 -c pytorch -c nvidia
1616
```
17-
Install datasets, accelerate and huggingface_hub
17+
Install `datasets`, `accelerate` and `huggingface_hub`
1818

1919
```bash
2020
conda install -c huggingface conda-forge datasets
2121
conda install -c conda-forge accelerate
2222
conda install -c conda-forge huggingface_hub
2323
```
2424

25-
Install the latest version of transformers by cloning the repository
25+
Install the latest version of `transformers` by cloning the repository
2626

2727
```bash
2828
git clone https://github.com/huggingface/transformers
2929
cd transformers
3030
pip install -e .
3131
```
32-
Install the latest version of peft by cloning the repository
32+
Install the latest version of `peft` by cloning the repository
3333

3434
```bash
3535
git clone https://github.com/huggingface/peft
3636
cd peft
3737
pip install -e .
3838
```
39-
Finally, install bitsandbytes and wandb
39+
Finally, install `bitsandbytes` and `wandb`
4040
```bash
4141
pip install bitsandbytes
4242
pip install wandb
@@ -49,10 +49,11 @@ Before you run any of the scripts make sure you are logged in and can push to th
4949
```bash
5050
huggingface-cli login
5151
```
52-
Make sure you are logged in wandb:
52+
Make sure you are logged in `wandb`:
5353
```bash
5454
wandb login
5555
```
56+
Now that everything is done, you can clone the repository and get into the corresponding directory.
5657

5758
## Fine-Tuning (`finetune.py`)
5859
💫 StarCoder can be fine-tuned to achieve multiple downstream tasks. Our interest here is to fine-tune StarCoder in order to make it follow instructions. [Instruction fine-tuning](https://arxiv.org/pdf/2109.01652.pdf) has gained a lot of attention recently as it proposes a simple framework that teaches language models to align their outputs with human needs. That procedure requires the availability of quality instruction datasets, which contain multiple `instruction - answer` pairs. Unfortunately such datasets are not ubiquitous but thanks to Hugging Face 🤗's [datasets](https://github.com/huggingface/datasets) library we can have access to some good proxies. To fine-tune cheaply and efficiently, we use Hugging Face 🤗's [PEFT](https://github.com/huggingface/peft) as well as Tim Dettmers' [bitsandbytes](https://github.com/TimDettmers/bitsandbytes).

0 commit comments

Comments
 (0)