YouTube transcripts for LangChain and LlamaIndex
Retrieve the transcript first, then pass its text and timestamp metadata into your RAG pipeline. Separating those steps lets you report an unavailable transcript clearly instead of returning an empty document that looks like a successful load.
Fetch timestamped text
The function below uses Python’s standard library to retrieve a batch. Keep your API token in VIDWORDS_TOKEN on the server or local machine running the pipeline.
import json, os, urllib.request
def get_transcripts(video_ids, language=None):
payload = {"ids": video_ids}
if language:
payload["lang"] = language
request = urllib.request.Request(
"https://vidwords.com/api/transcripts",
data=json.dumps(payload).encode(),
headers={"Authorization": "Basic " + os.environ["VIDWORDS_TOKEN"],
"Content-Type": "application/json"},
)
with urllib.request.urlopen(request, timeout=120) as response:
return json.load(response)["results"]Prepare documents with useful citations
This example creates ordinary dictionaries you can adapt to your installed framework’s document type. Every text segment keeps its video id and a link to its start time.
documents = []
for result in get_transcripts(["dQw4w9WgXcQ"]):
if "error" in result:
print(result["id"], result["error"])
continue
for segment in result["segments"]:
start = segment["start"]
documents.append({
"text": segment["text"],
"metadata": {
"video_id": result["id"],
"language": result["languageCode"],
"start_seconds": start,
"source": f'https://www.youtube.com/watch?v={result["id"]}&t={int(start)}s',
},
})Connect it to your framework
In LangChain, map text to a document’s page content and keep the metadata alongside it. In LlamaIndex, use the same text and metadata when creating documents or nodes. Check your installed framework’s documentation for its current constructor and import path.
A caption segment can be too short for useful retrieval. Group nearby segments into chunks appropriate to your questions, preserving the earliest timestamp and source video for each chunk. Keep different videos separate unless the chunk explicitly identifies both.
Make answers verifiable
Ask the answer-generation step to cite retrieved source links. Keep unavailable videos out of the index and report them separately. A transcript covers speech; questions about slides, charts or visible code need video analysis as well.
Limits and failed lookups
The REST API accepts up to 50 video ids per call. A delivered caption transcript uses one Cloud Request. A free account includes 100 one-time Developer Requests; see current plans and top-ups for continued use. Audio transcription requires separate AI Units and explicit consent through the API’s allowAsr option.
Read each result’s error field: one unavailable video does not make the rest of a batch fail. A lookup that returns no transcript is not charged. Hosted retrieval can still be refused by YouTube; it does not promise access to private, restricted or unavailable videos. See the API reference for response fields and limits.