You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
See the examples notebooks on [using SAM with prompts](/notebooks/predictor_example.ipynb) and [automatically generating masks](/notebooks/automatic_mask_generator_example.ipynb) for more details.
@@ -75,7 +77,7 @@ See the examples notebooks on [using SAM with prompts](/notebooks/predictor_exam
75
77
SAM's lightweight mask decoder can be exported to ONNX format so that it can be run in any environment that supports ONNX runtime, such as in-browser as showcased in the [demo](https://segment-anything.com/demo). Export the model with
See the [example notebook](https://github.com/facebookresearch/segment-anything/blob/main/notebooks/onnx_model_example.ipynb) for details on how to combine image preprocessing via SAM's backbone with mask prediction using the ONNX model. It is recommended to use the latest stable version of PyTorch for ONNX export.
@@ -85,14 +87,55 @@ See the [example notebook](https://github.com/facebookresearch/segment-anything/
85
87
Three model versions of the model are available with different backbone sizes. These models can be instantiated by running
86
88
```
87
89
from segment_anything import sam_model_registry
88
-
sam = sam_model_registry["<name>"](checkpoint="<path/to/checkpoint>")
90
+
sam = sam_model_registry["<model_type>"](checkpoint="<path/to/checkpoint>")
89
91
```
90
-
Click the links below to download the checkpoint for the corresponding model name. The default model in bold can also be instantiated with `build_sam`, as in the examples in [Getting Started](#getting-started).
92
+
Click the links below to download the checkpoint for the corresponding model type.
91
93
92
94
***`default` or `vit_h`: [ViT-H SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth)**
93
95
*`vit_l`: [ViT-L SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_l_0b3195.pth)
94
96
*`vit_b`: [ViT-B SAM model.](https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth)
95
97
98
+
## Dataset
99
+
See [here](https://ai.facebook.com/datasets/segment-anything/) for an overview of the datastet. The dataset can be downloaded [here](https://ai.facebook.com/datasets/segment-anything-downloads/). By downloading the datasets you agree that you have read and accepted the terms of the SA-1B Dataset Research License.
100
+
101
+
We save masks per image as a json file. It can be loaded as a dictionary in python in the below format.
102
+
103
+
104
+
```python
105
+
{
106
+
"image" : image_info,
107
+
"annotations" : [annotation],
108
+
}
109
+
110
+
image_info {
111
+
"image_id" : int, # Image id
112
+
"width" : int, # Image width
113
+
"height" : int, # Image height
114
+
"file_name" : str, # Image filename
115
+
}
116
+
117
+
annotation {
118
+
"id" : int, # Annotation id
119
+
"segmentation" : dict, # Mask saved in COCO RLE format.
120
+
"bbox" : [x, y, w, h], # The box around the mask, in XYWH format
121
+
"area" : int, # The area in pixels of the mask
122
+
"predicted_iou" : float, # The model's own prediction of the mask's quality
123
+
"stability_score" : float, # A measure of the mask's quality
124
+
"crop_box" : [x, y, w, h], # The crop of the image used to generate the mask, in XYWH format
125
+
"point_coords" : [[x, y]], # The point coordinates input to the model to generate the mask
126
+
}
127
+
```
128
+
129
+
Image ids can be found in sa_images_ids.txt which can be downloaded using the above [link](https://ai.facebook.com/datasets/segment-anything-downloads/) as well.
See [here](https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/mask.py) for more instructions to manipulate masks stored in RLE format.
137
+
138
+
96
139
## License
97
140
The model is licensed under the [Apache 2.0 license](LICENSE).
98
141
@@ -105,3 +148,16 @@ See [contributing](CONTRIBUTING.md) and the [code of conduct](CODE_OF_CONDUCT.md
105
148
The Segment Anything project was made possible with the help of many contributors (alphabetical):
If you use SAM or SA-1B in your research, please use the following BibTeX entry.
155
+
156
+
```
157
+
@article{kirillov2023segany,
158
+
title={Segment Anything},
159
+
author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross},
Copy file name to clipboardExpand all lines: notebooks/automatic_mask_generator_example.ipynb
+6-14Lines changed: 6 additions & 14 deletions
Original file line number
Diff line number
Diff line change
@@ -214,19 +214,6 @@
214
214
"To run automatic mask generation, provide a SAM model to the `SamAutomaticMaskGenerator` class. Set the path below to the SAM checkpoint. Running on CUDA and with the default model is recommended."
Copy file name to clipboardExpand all lines: notebooks/predictor_example.ipynb
+6-13Lines changed: 6 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -229,18 +229,6 @@
229
229
"First, load the SAM model and predictor. Change the path below to point to the SAM checkpoint. Running on CUDA and using the default model are recommended for best results."
0 commit comments