Wow, this is pretty epic. I put it through its paces on a pretty wide variety of images that have tripped up recent zero-shot models[1] and am thoroughly impressed.
We have a similar "Smart Polygon" tool[2] built into Roboflow but this is next level. Having the model running in the browser makes it so much more fun to use. Stoked it's open source; we're going to work on adding it to our annotation tool ASAP.
The SAM model is small (4m params) , but requires image embedding to be computed from what is I think a 600m params model. Right now the demo uploads the image to get the embeddings, then runs the actual segmentation locally.
We have a similar "Smart Polygon" tool[2] built into Roboflow but this is next level. Having the model running in the browser makes it so much more fun to use. Stoked it's open source; we're going to work on adding it to our annotation tool ASAP.
[1] Some examples from Open Flamingo last week https://news.ycombinator.com/item?id=35348500
[2] https://blog.roboflow.com/automated-polygon-labeling-compute...