Vector Similarity Search Definition

Understanding vectors in the context of search

A vector is simply a list of numbers. In the context of AI and search, vectors serve as compact mathematical representations of content — images, text, audio, or video. An AI model processes a piece of content and outputs a vector (often called an embedding) with hundreds or thousands of dimensions, where each dimension captures some aspect of meaning.

The key insight is that content with similar meaning produces vectors that are close together in this mathematical space. A photo of a golden retriever and the text "friendly dog playing in a park" will have vectors that are near each other, even though one is an image and the other is text. This property enables cross-modal search — finding images using text descriptions, or finding similar videos using an example clip.

How vector similarity search differs from keyword search

Keyword search is exact and literal. A search for "automobile" will not find documents that only use the word "car" unless synonyms are explicitly programmed. Keyword search fails entirely for visual content because images and video have no inherent text to match against.

Vector similarity search operates on meaning rather than literal matches. It handles synonyms naturally ("automobile" and "car" have similar vectors), understands conceptual relationships ("vehicle" is related to both), and works across modalities (a photo of a car has a vector similar to the text "car"). This makes it fundamentally better suited for searching visual media.

The mathematics behind similarity

Similarity between vectors is typically measured using cosine similarity — the cosine of the angle between two vectors in high-dimensional space. Vectors pointing in the same direction (cosine similarity close to 1.0) represent highly similar content. This metric is elegant because it focuses on the direction of vectors (their meaning) rather than their magnitude (which might vary based on content length or other irrelevant factors).

Efficient vector search at scale uses specialized data structures like HNSW (Hierarchical Navigable Small World) graphs or IVF (Inverted File) indexes. These allow searching millions of vectors in milliseconds by intelligently narrowing the search space rather than comparing against every vector.

Applications in video search

For video search, vector similarity enables queries that would be impossible with traditional metadata. You can search by describing a scene, by providing an example frame, or even by describing an abstract concept like "tension" or "celebration." The system finds shots whose vector representations are closest to your query's vector, returning results ranked by semantic relevance.

How ShotAI leverages vector similarity search

ShotAI builds a local vector index from your video library, where each shot is represented as a high-dimensional embedding. When you type a natural language query, it is converted into the same vector space and compared against all indexed shots using optimized similarity algorithms. Results return in milliseconds, even across libraries containing millions of shots.