This documentation page is also available as an interactive notebook. You can launch the notebook in
Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
above links.
Extract the audio track from video files for transcription, analysis, or
processing.
Problem
You have video files but need to work with just the audio track—for
transcription, speaker analysis, or audio processing. Extracting audio
manually with ffmpeg is tedious and doesn’t integrate with your data
pipeline.
Solution
What’s in this recipe:
Extract audio from video as a computed column
Choose audio format (mp3, wav, flac)
Chain with transcription for automatic video-to-text
You use the extract_audio function to create an audio column from
video. This integrates seamlessly with transcription and other audio
processing.
Setup
%pip install -qU pixeltable 'numpy<2.4'
import pixeltable as pxtfrom pixeltable.functions.video import extract_audio# Create a fresh directorypxt.drop_dir('audio_extract_demo', force=True)pxt.create_dir('audio_extract_demo')
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
Created directory ‘audio_extract_demo’.
<pixeltable.catalog.dir.Dir at 0x1061fc510>
# Add computed column to extract audio as MP3videos.add_computed_column( audio=extract_audio(videos.video, format='mp3'))
Added 0 column values with 0 errors.
No rows affected.
# Insert a sample video (from multimedia-commons with audio)video_url = 's3://multimedia-commons/data/videos/mp4/ffe/ffb/ffeffbef41bbc269810b2a1a888de.mp4'videos.insert([{'title': 'Sample Video', 'video': video_url}])
# Install whisper for transcription%pip install -qU openai-whisper
from pixeltable.functions import whisper# Add transcription of the extracted audiovideos.add_computed_column( transcription=whisper.transcribe(videos.audio, model='base.en'))
Added 1 column value with 0 errors.
1 row updated, 1 value computed.
# Extract the transcript textvideos.add_computed_column(transcript=videos.transcription.text)
Added 1 column value with 0 errors.
1 row updated, 1 value computed.
# View the full pipeline resultsvideos.select(videos.title, videos.transcript).collect()
Explanation
Audio format options:Pipeline flow:
Video → extract_audio → Audio → whisper.transcribe → Transcript
Each step is a computed column. When you insert a new video: