This documentation page is also available as an interactive notebook. You can launch the notebook in
Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
above links.
Import images, videos, and audio files from S3, GCS, HTTP URLs, or local
paths into Pixeltable tables.
Problem
You have media files stored in cloud storage (S3, GCS) or accessible via
HTTP URLs. You need to process these files with AI models without
downloading them all upfront.
Solution
What’s in this recipe:
Reference media files by URL (S3, HTTP, local paths)
Automatic caching of remote files on access
Process files lazily without bulk downloads
You insert media URLs as references. Pixeltable stores the URLs and
automatically downloads/caches files when you access them through
queries or computed columns.
Setup
%pip install -qU pixeltable
import pixeltable as pxt# Create a fresh directorypxt.drop_dir('cloud_demo', force=True)pxt.create_dir('cloud_demo')
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
Created directory ‘cloud_demo’.
<pixeltable.catalog.dir.Dir at 0x10d31f710>
Load images from HTTP URLs
Reference images by URL—Pixeltable downloads them on demand:
# Create a table with image columnimages = pxt.create_table('cloud_demo/images', {'image': pxt.Image})
Created table ‘images’.
# Insert images by URL (HTTP)image_urls = [ 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/images/000000000036.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/images/000000000090.jpg', 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/images/000000000106.jpg',]images.insert([{'image': url} for url in image_urls])
# View videos - downloaded and cached on accessvideos.collect()
Add computed columns on remote media
Process remote media with computed columns—files are fetched
automatically:
# Add computed columns for image propertiesimages.add_computed_column(width=images.image.width)images.add_computed_column(height=images.image.height)
Added 3 column values with 0 errors.
Added 3 column values with 0 errors.
3 rows updated, 6 values computed.
# View with computed propertiesimages.select(images.image, images.width, images.height).collect()
Generate presigned URLs for serving media
When you store media in private cloud storage, you need presigned URLs
to serve files over HTTP. The presigned_url function converts storage
URIs to time-limited, publicly accessible URLs:
import pixeltable.functions as pxtf# Generate presigned URLs for videos (1-hour expiration)videos.select( videos.video, original_uri=videos.video.fileurl, http_url=pxtf.net.presigned_url(videos.video.fileurl, 3600),).collect()
# Store presigned URLs as computed column for API responsesvideos.add_computed_column( serving_url=pxtf.net.presigned_url( videos.video.fileurl, 86400 ) # 24-hour expiration)