Analyse what is on screen in a YouTube video, not just the captions
A transcript is a recording of the audio. If the answer you need was on a slide, in a chart, in a code sample or in a menu the presenter clicked without narrating, the transcript does not contain it — and no summariser built on top of a transcript can recover it.
The questions captions cannot answer
These all fail against transcript text, however good the model reading it is:
- “What were the four bullets on the slide at 12:40?”
- “Which menu item did they click before the error appeared?”
- “What numbers are on the y-axis of that chart?”
- “What was the exact command typed in the terminal?”
Presenters say “as you can see here” and move on. The information was shown, never spoken, so it never reached the caption track. This is the single most common reason a video summary is confidently useless.
How frame-level analysis differs
Reading the video means sampling actual frames and interpreting them alongside the speech, so on-screen text, slides, diagrams, UI state and demonstrations become answerable. VidWords does this as AI Watch: it produces chapter-by-chapter summaries, real frames from the video, verbatim transcript excerpts, study notes with a self-test, and question answering over the whole thing.
Every citation is checked before you see it
This is the part worth understanding, because it is where most visual-analysis output goes wrong. A model asked what was on a slide will describe a plausible slide. The check is what makes the answer worth quoting:
- A visual claim must match a stored frame within 8 seconds of the timestamp it cites.
- A spoken claim must land on a real transcript segment.
- A claim that matches neither is discarded rather than shown.
- If nothing survives, the answer says the evidence is insufficient and offers to re-watch that stretch — instead of guessing.
An unverifiable answer is worse than no answer when you intend to cite it, which is the whole reason the check exists.
Using it
In the browser, paste a link into AI Watch. Programmatically:
POST /api/watch → returns a job id
GET /api/watch/:jobId → poll it
POST /api/watch/:jobId/ask → ask a question of the finished analysisAnalysis takes minutes, which is why it is asynchronous. From an AI assistant the same capability arrives through the MCP server as analyze_video, get_analysis and ask_video, so the assistant can inspect the video itself. Modes are quick, smart, deep (Pro and Team) and auto, which picks from the video’s length and visual pace.
What it costs
Standard Watch uses three AI Processing Units per analysed minute; Deep Watch uses fifteen. A run refused before it starts costs nothing, and re-starting the same video returns the existing analysis rather than charging twice.
A free account includes 200 AI Processing Units a month, enough for roughly 66 minutes of Standard Watch. Paid plans include 3,000, 10,000 and 30,000 AI Units a month.
The alternatives, honestly
This is a thin field and the options are genuinely different from one another.
- Gemini — its API accepts a public YouTube URL directly as video input, so for a developer who wants raw multimodal access with no intermediate service it is the shortest path. You get model output; verification, chaptering, citation checking and storage are yours to build.
- TwelveLabs — a video-understanding platform that indexes visuals, non-speech audio and speech separately and searches across them. The right answer for serious video search at volume, and priced and scoped as infrastructure rather than as a tool.
- Cloud video intelligence APIs — Google Cloud Video Intelligence and Azure AI Video Indexer do frame-level work at enterprise scale, with the integration effort that implies.
- VidWords AI Watch — a finished product rather than a building block: paste a link, get verified answers with timestamps, through a browser, a REST API or MCP.
If you are building a pipeline, use an API. If you want an answer about a video in the next two minutes, use a tool.
When a transcript is enough
Most of the time, and it costs a third as much. If the video is an interview, a podcast, a lecture with no meaningful visuals, or anything where the speech carries the content, extract the transcript and stop there. Frame analysis earns its cost on tutorials, screencasts, conference talks with slides, product demos and anything where the screen is doing the explaining.
Questions people ask
Does it read text on screen accurately?
Slides, titles, captions burned into the frame, code and terminal output are generally legible. Small UI labels, low-resolution uploads and fast-moving text are where it degrades — and where the citation check tends to discard a claim rather than assert it.
Does the video need captions?
Frame analysis does not depend on a caption track the way transcript extraction does, because it reads the video’s own speech and frames. A video with no captions is still analysable.
Can it tell me the timestamp where something appeared?
Yes — that is the normal output shape. Answers cite the moment, and the visual citations have been matched against a stored frame within 8 seconds of that timestamp before being shown.
How is this different from asking ChatGPT to summarise a video?
ChatGPT has no native YouTube video input; it works from a transcript you supply, so it inherits the transcript’s blind spot for anything shown rather than said. We wrote that up separately in why ChatGPT cannot watch a YouTube video.
Can an AI assistant do this for me directly?
Yes. Connect the MCP server and the assistant calls analyze_video and ask_video itself, with citations already verified before they reach it.
Related
To find where something is discussed across many videos rather than to inspect one deeply, see searching a whole channel’s transcripts.