Skip to content

Commit 97b2881

Browse files
authored
Update README.md
- specify need to log to HF hub in disclaimer - mention use of fp16/bf16 to save memory when loading model - remove `trust_remote_code` when loading model
1 parent 3839ae4 commit 97b2881

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

README.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,10 @@
55

66
# Disclaimer
77

8-
Before you can use the model go to `hf.co/bigcode/starcoder` and accept the agreement.
8+
Before you can use the model go to `hf.co/bigcode/starcoder` and accept the agreement. And make sure you are logged into the Hugging Face hub with:
9+
```bash
10+
huggingface-cli login
11+
```
912

1013
# Table of Contents
1114
1. [Quickstart](#quickstart)
@@ -36,7 +39,8 @@ checkpoint = "bigcode/starcoder"
3639
device = "cuda" # for GPU usage or "cpu" for CPU usage
3740

3841
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
39-
model = AutoModelForCausalLM.from_pretrained(checkpoint, trust_remote_code=True).to(device)
42+
# to save memory consider using fp16 or bf16 by specifying torch.dtype=torch.float16 for example
43+
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)
4044

4145
inputs = tokenizer.encode("def print_hello_world():", return_tensors="pt").to(device)
4246
outputs = model.generate(inputs)

0 commit comments

Comments
 (0)