Skip to content

Commit d398eb1

Browse files
committed
Update readme instructions for onnx export
1 parent 5219f67 commit d398eb1

File tree

1 file changed

+29
-6
lines changed

1 file changed

+29
-6
lines changed

demo/README.md

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ Navigate to [`http://localhost:8081/`](http://localhost:8081/)
1414

1515
Move your cursor around to see the mask prediction update in real time.
1616

17-
## Change the image, embedding and ONNX model
17+
## Export the image embedding
1818

1919
In the [ONNX Model Example notebook](https://github.com/facebookresearch/segment-anything/blob/main/notebooks/onnx_model_example.ipynb) upload the image of your choice and generate and save corresponding embedding.
2020

@@ -37,18 +37,41 @@ image_embedding = predictor.get_image_embedding().cpu().numpy()
3737
np.save("dogs_embedding.npy", image_embedding)
3838
```
3939

40-
Save the new image and embedding in `/assets/data`and update the following paths to the files at the top of`App.tsx`:
40+
Save the new image and embedding in `/assets/data`.
41+
42+
## Export the ONNX model
43+
44+
You also need to export the quantized ONNX model from the [ONNX Model Example notebook](https://github.com/facebookresearch/segment-anything/blob/main/notebooks/onnx_model_example.ipynb).
45+
46+
Run the cell in the notebook which saves the `sam_onnx_quantized_example.onnx` file, download it and copy it to the path `/model/sam_onnx_quantized_example.onnx`.
47+
48+
Here is a snippet of the export/quantization code:
49+
50+
```
51+
onnx_model_path = "sam_onnx_example.onnx"
52+
onnx_model_quantized_path = "sam_onnx_quantized_example.onnx"
53+
quantize_dynamic(
54+
model_input=onnx_model_path,
55+
model_output=onnx_model_quantized_path,
56+
optimize_model=True,
57+
per_channel=False,
58+
reduce_range=False,
59+
weight_type=QuantType.QUInt8,
60+
)
61+
```
62+
63+
**NOTE: if you change the ONNX model by using a new checkpoint you need to also re-export the embedding.**
64+
65+
## Update the image, embedding, model in the app
66+
67+
Update the following file paths at the top of`App.tsx`:
4168

4269
```py
4370
const IMAGE_PATH = "/assets/data/dogs.jpg";
4471
const IMAGE_EMBEDDING = "/assets/data/dogs_embedding.npy";
4572
const MODEL_DIR = "/model/sam_onnx_quantized_example.onnx";
4673
```
4774

48-
Optionally you can also export the ONNX model. Currently the example ONNX model from the notebook is saved at `/model/sam_onnx_quantized_example.onnx`.
49-
50-
**NOTE: if you change the ONNX model by using a new checkpoint you need to also re-export the embedding.**
51-
5275
## ONNX multithreading with SharedArrayBuffer
5376

5477
To use multithreading, the appropriate headers need to be set to create a cross origin isolation state which will enable use of `SharedArrayBuffer` (see this [blog post](https://cloudblogs.microsoft.com/opensource/2021/09/02/onnx-runtime-web-running-your-machine-learning-model-in-browser/) for more details)

0 commit comments

Comments
 (0)