Video OCR Definition

Why video OCR matters

A large share of the information in video is written, not spoken. A conference recording carries its agenda on slides. A product demo shows menu names and error messages. A sports broadcast puts the score, clock, and player names in a graphic. A documentary identifies interviewees with lower thirds. None of that text appears in a transcript, because nobody said it aloud.

Video OCR recovers it. Once on-screen text is extracted and time-coded, a library becomes searchable by words that were only ever displayed: find the frame where a specific slide title appears, locate every shot containing a competitor's logo, or jump to the moment a particular street sign is visible.

How video OCR works

The pipeline samples frames — typically one every one to two seconds, or one per detected shot, since sampling every frame is wasteful when text persists for seconds at a time. Each sampled frame goes through text detection, which locates rectangular regions likely to contain characters, followed by recognition, which converts the pixels in each region into character strings.

A deduplication stage then collapses repeated detections. A slide visible for forty seconds should produce one time-ranged record, not twenty identical ones. The output is a set of text strings, each with a bounding box and a start and end timecode.

Video OCR is harder than document OCR. Text in video is rendered over moving imagery, at arbitrary angles, in decorative fonts, at low contrast, sometimes motion-blurred, and often compressed to the point where thin strokes disappear. Recognition confidence varies far more than it does on a scanned page.

Common applications

Slide and screencast indexing: making webinar and training libraries searchable by slide content rather than by video title.

Sports and broadcast: reading scoreboards, clocks, and name graphics to derive structured event data from a raw game feed.

Brand and compliance monitoring: detecting logos, disclaimers, and required legal text and confirming it appeared for the mandated duration.

Archive recovery: extracting names, dates, and locations from slates and title cards in legacy footage that has no usable metadata.

Accuracy considerations

Sampling rate is the core trade-off. Sparse sampling misses text that flashes briefly — a one-second graphic can fall between frames. Dense sampling multiplies cost. Shot-aware sampling, which takes frames at scene changes plus a periodic interval, is usually the best compromise.

Keep confidence scores rather than discarding them. Low-confidence strings still support fuzzy search even when they are not clean enough to display. Combine OCR output with transcription and visual tags rather than treating it as a separate index; the strongest search results come from matching across all three signals at once.

How ShotAI relates to video OCR

ShotAI extracts on-screen text during indexing and stores it with frame-accurate timing alongside speech and visual descriptors, so a natural-language query can match words that were displayed on screen and return the precise shots where they appeared.