Skip to content

Commit 6d1798e

Browse files
authored
Update README.md
1 parent c19aa29 commit 6d1798e

File tree

1 file changed

+48
-9
lines changed

1 file changed

+48
-9
lines changed

README.md

Lines changed: 48 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ huggingface-cli login
2020
## Fine-Tuning (`finetune.py`)
2121
💫 StarCoder can be fine-tuned to achieve multiple downstream tasks. Our interest here is to fine-tune StarCoder in order to make it follow instructions. [Instruction fine-tuning](https://arxiv.org/pdf/2109.01652.pdf) has gained a lot of attention recently as it proposes a simple framework that teaches language models to align their outputs with human needs. That procedure requires the availability of quality instruction datasets, which contain multiple `instruction - answer` pairs. Unfortunately such datasets are not ubiquitous but thanks to Hugging Face 🤗's [datasets](https://github.com/huggingface/datasets) library we can have access to some good proxies. To fine-tune cheaply and efficiently, we use Hugging Face 🤗's [PEFT](https://github.com/huggingface/peft) as well as Tim Dettmers' bitsandbytes.
2222

23-
### Code Alpaca
23+
### Code Alpaca (CA)
2424
[Code Alpaca](https://huggingface.co/datasets/HuggingFaceH4/CodeAlpaca_20K) is a dataset of about 20K `prompt - completion` pairs generated by the technique presented in the [self-instruct](https://arxiv.org/abs/2212.10560) paper. Each prompt describes a task that is asked by a user and the corresponding completion is the answer to that task as generated by `text-davinci-003`.
2525

2626
To execute the fine-tuning script run the following command:
@@ -57,7 +57,7 @@ python -m torch.distributed.launch \
5757
--weight_decay 0.05\
5858
--output_dir="./checkpoints"
5959
```
60-
### Stack Exchange
60+
### Stack Exchange (SE)
6161
[Stack Exchange](https://en.wikipedia.org/wiki/Stack_Exchange) is a well-known network of Q&A websites on topics in diverse fields. It is a place where a user can ask a question and obtain answers from other users. Those answers are scored and ranked based on their quality. [Stack exchange instruction](https://huggingface.co/datasets/ArmelR/stack-exchange-instruction) is a dataset that was obtained by scrapping the site in order to build a collection of Q&A pairs. A language model can then be fine-tuned on that dataset to make it elicit strong and diverse question-answering skills.
6262

6363
To execute the fine-tuning script run the following command:
@@ -114,24 +114,63 @@ For example
114114
python finetune/merge_peft_adapters.py --model_name_or_path bigcode/large-model --peft_model_path checkpoints/checkpoint-1000 --push_to_hub
115115
```
116116

117-
### Example outputs
117+
### Example outputs (CA)
118118

119-
**Instruction**: Write a function to output the harmonic mean of a list of elements. Make sure to check if the list is empty and ignore the elements that are equal to 0.
119+
**Instruction**: Write a function to compute the geometric mean of a list of numbers.
120120

121121
**StarCoder's answer**:
122122

123-
**Instruction**:
123+
```python
124+
def geometric_mean(nums):
125+
product = 1
126+
for num in nums:
127+
product *= num
128+
return product**(1/len(nums))
124129

125-
**StarCoder's answer**:
130+
```
126131

127-
```bash
132+
**Instruction**:
133+
what is the purpose of the following code?
134+
```python
135+
def function(n):
136+
s = ""
137+
while n > 0 :
138+
s = str(n%3)+s
139+
n = n//3
140+
return s
141+
```
128142

143+
**StarCoder's answer**:
144+
```
145+
The purpose of the code is to convert a given number n to a string of its base 3 representation. This is done by dividing the number by 3 and converting the remainder to a string and then adding it to the string of the previous iteration. This process is repeated until the number is 0.
129146
```
130147

131148
**Instruction**:
149+
Translate this function is Java :
150+
151+
```python
152+
def function(a, b, c):
153+
if a > b :
154+
print("You lost.")
155+
else :
156+
if c > a :
157+
print("Nice try.")
158+
else :
159+
print("You won.")
160+
```
132161

133162
**StarCoder's answer**:
134163

135-
```bash
136-
164+
```java
165+
public static void function(int a, int b, int c) {
166+
if (a > b) {
167+
System.out.println("You lost.");
168+
} else {
169+
if (c > a) {
170+
System.out.println("Nice try.");
171+
} else {
172+
System.out.println("You won.");
173+
}
174+
}
175+
}
137176
```

0 commit comments

Comments
 (0)