fusion-embedding-1

What is Fusion Embedding 1?

Fusion Embedding 1 is an open-weight multimodal embedding model that maps five modalities — text, image, video, audio, and PDF — into a single shared vector space. It is built by Eximius-Labs for retrieval, RAG, clustering, and cross-modal search, and is designed to be fully self-hostable.

How it works

Instead of training a multimodal embedder from scratch, Fusion Embedding 1 takes Qwen3-VL-Embedding, an open state-of-the-art text/image/video embedding model, and freezes it byte-for-byte. It then adds an audio pathway using a frozen Qwen2.5-Omni audio tower feeding a small trained connector called the FusionResampler (about 16M parameters, under 1% of the base). Audio is aligned to text contrastively, and because text, image, and video already share the base model's vector space, audio-to-image and audio-to-video alignment emerge through the text bridge.

Key features

• Five modalities, one space: text, image, video, audio, and PDF are embedded into a single vector space for retrieval in any direction between modalities.
• Frozen-base architecture: only the connector and a temperature parameter are trained, so the base model's retrieval quality is inherited unchanged (asserted via base_drift == 0 on every training run).
• Matryoshka embeddings: truncate to dimensions of 2048, 1536, 1024, 512, 256, 128, or 64 and re-normalize, with embeddings staying consistent at every size.
• Instruction-aware audio: the same audio clip embeds differently depending on the task, such as sound description, spoken content, or speaker and emotion.
• Self-hostable: no API and no rate limits; it runs quantized on a single consumer GPU for inference.
• Test-first engineering: the pipeline runs end-to-end on CPU-only stand-ins with 125+ unit and end-to-end tests.

Who is it for?

Fusion Embedding 1 is aimed at developers and researchers building retrieval-augmented generation (RAG) systems, cross-modal search engines, and clustering pipelines who need to search and compare text, images, video, audio, and PDFs within one unified vector space. Because it is open-weight and self-hostable, it also suits teams that want to avoid API dependencies and rate limits for embedding workloads.

Availability

A research preview checkpoint, fusion-embedding-1-2b-preview, is available on Hugging Face with connector weights, a model card with benchmarks, and a packaged inference API. The source code and training pipeline are published on GitHub under the Apache-2.0 license.