hyper³labs

hyper3-clip-v1

Research paper · Website · Discord

hyper3-clip-v1 is a hyperbolic vision–language model for visual search. It brings images and text into a shared embedding space for broad-to-specific retrieval and compositional matching.

• Broad-to-specific retrieval: Search for a broad concept such as “animal” to find different types of animals, then narrow your search with a specific description.
• Compositional matching: Find images by combinations of objects, attributes, and relationships—for example, “a red car beside a tree.” This helps distinguish similar scenes where a color, object, or relationship changes.

Results

Model	SugarCrepe macro accuracy
hyper3-clip-v1	79.54%
Jina CLIP v2	75.02%
OpenAI CLIP ViT-B/16	73.06%

Image retrieval — mAP (%) on evaluation subsets of each dataset.

Dataset	hyper3-clip-v1	OpenAI CLIP ViT-B/32
Amazon Berkeley Objects — product types	58.2	55.2
DeepFashion In-Shop — same item	63.5	35.2
COCO — object categories	55.4	53.2

Quick start

Complete the short access form on this page, then install and sign in.

pip install "torch>=2.2" "transformers>=4.49,<5" "timm>=1.0" \
  "safetensors>=0.4" "Pillow>=10" "huggingface_hub>=0.34"
hf auth login

For SentenceTransformers, also run pip install "sentence-transformers>=5.5.1,<6".

This example ranks three descriptions of a sofa. The example image downloads automatically.

from PIL import Image
from huggingface_hub import hf_hub_download
from transformers import AutoModel

model_id = "hyper3labs/hyper3-clip-v1"
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).eval()
image = Image.open(hf_hub_download(model_id, "examples/grey-velvet-sofa.jpg")).convert("RGB")

descriptions = ["a grey velvet sofa", "a blue velvet sofa", "a wooden chair"]
scores = model.score(image, descriptions)

for score, text in sorted(zip(scores.tolist(), descriptions), reverse=True):
    print(f"{score:.3f}  {text}")

The default is Lorentz scoring. Scores are negative; higher means a closer match.

To try your own, use image = Image.open("your-image.jpg").convert("RGB") and change the descriptions — broad words such as “furniture” work alongside specific ones.

Scoring

Use scoring="lorentz", "cosine", or "cone" with model.score(image, descriptions, scoring=...).

Scoring	What it measures	Use
Lorentz (default)	Closeness in hyperbolic space; gives the same ranking as hyperbolic distance	General image–text retrieval
Cosine	Similarity of embedding directions	Standard vector similarity search
Cone	How well an image fits a description's general-to-specific region	Directional and compositional matching

Higher is better for all three. Their numerical scales differ. The SugarCrepe result above uses cone scoring.

Use with Qdrant

Install the client and start a local Qdrant server:

pip install "qdrant-client>=1.19,<2"
docker run --rm -p 127.0.0.1:6333:6333 qdrant/qdrant:v1.19.1

In Python, continue from the quick start:

from qdrant_client import QdrantClient

index = model.qdrant(QdrantClient(url="http://localhost:6333"), "hyper3_images")
index.create_collection()
index.upsert([image], ids=[1], payloads=[{"label": "grey velvet sofa"}])

for hit in index.search("a grey sofa", limit=5):
    print(hit.id, hit.score, hit.payload["label"])

The helper stores one search vector and two geometry values per image. Qdrant finds candidates with its index, then ranks them by Lorentz score in the same request. Only the final results are returned.

For a hosted deployment, pass your server URL and API key to QdrantClient. index.search accepts query_filter for filtering and candidates (default 200) to tune shortlist recall. For image-to-image search, pass an encode_image_lorentz embedding to index.search_embedding. Create payload indexes for your filters before bulk ingestion. Use a separate collection for each model revision.

License and community

hyper3-clip-v1 is free to use under OpenMDW-1.0. The short access form helps us understand how people use the model.

Discord · Website · Attribution notice