Implementing image search is now very easy, I think recoll should implement it
Hi,
I've been playing around for some time with sentence-BERT and hugging face and I think we reached the point where implementing image search by textual description is actually super easy.
Here's some documentation: https://www.sbert.net/examples/applications/image-search/README.html
here's a toy example: (after pip install -U sentence-transformers
and pip install PIL
)
from sentence_transformers import SentenceTransformer, util
from PIL import Image
#Load CLIP model
model = SentenceTransformer('clip-ViT-B-32-multilingual-v1')
#Encode an image:
img_emb = model.encode(Image.open('two_dogs_in_snow.jpg'))
#Encode text descriptions
text_emb = model.encode(['Two dogs in the snow', 'A cat on a table', 'A picture of London at night'])
#Compute cosine similarities
cos_scores = util.cos_sim(img_emb, text_emb)
print(cos_scores)
What do you think of this idea? I think querying things like "image-search:"selfie next to a boat" that bypasses the regular search to only focus on images would be great!
Looking forward to hearing your opinion on this :)
Edited by thiswillbeyourgithub