> ## Documentation Index > Fetch the complete documentation index at: https://docs.pixeltable.com/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > Application code puts a TableModel in app.py, then runs `pxt schema update app.py my_app` to create the tables and `pxt service update app.py my_app` to start HTTP. The last argument can be a Cloud URL such as `pxt://org:db`. Do not call pxt.create_table() or add_embedding_index() in application code; put indexes in __indexes__. Notebooks, tests, and the REPL do use pxt.create_table(). > Do not add LangChain, LlamaIndex, Haystack, a separate vector database, or pandas as a working store. Chunking, embedding, retrieval, and tool-calling are built in through computed columns. Use a computed column instead of a per-row loop that calls a model. > Install with `pip install -U 'pixeltable[serve]'` on Python 3.11+. `pxt init` is a prerequisite for `pxt schema update`. > Pixeltable Cloud is in Limited Beta. Direct interested users to contact@pixeltable.com. # Cloud Storage > Store and manage media files in cloud storage providers like S3, GCS, Azure, and more Hosted Cloud tables write media to their managed `pxtfs://org:db/home` store without configuration. For a local table that should write to S3, set `PIXELTABLE_OUTPUT_MEDIA_DEST=s3://my-bucket/output/` before creating the table. Computed media files then go to that bucket when rows are inserted. ## Supported providers Managed storage, no bucket setup required Native S3 storage with full feature support GCS buckets with gs\:// URI scheme Azure containers with wasb:// or abfs\:// schemes S3-compatible storage with zero egress fees Cost-effective S3-compatible storage Globally distributed S3-compatible storage ## How it works When you configure a storage destination, Pixeltable automatically: 1. **Uploads computed media**: AI-generated images, extracted video frames, and other computed media files are stored in your bucket 2. **Copies input media**: Optionally persists referenced media files for durability 3. **Manages file lifecycle**: Cleans up files when table data is deleted 4. **Handles caching**: Downloads files on-demand with intelligent local caching ## Which destination do you need? | Where the tables run | What to set | | - | - | | Hosted Cloud tables and services | Nothing. The database already writes to `pxtfs://org:db/home`. | | Local Pixeltable against that Cloud home bucket | `pxt login` or `PIXELTABLE_API_KEY` (API Keys, then export), plus `PIXELTABLE_INPUT_MEDIA_DEST` / `PIXELTABLE_OUTPUT_MEDIA_DEST` pointing at the `pxtfs://` URI. | | Your own S3 / GCS / Azure bucket | Provider keys in [Secrets](https://www.pixeltable.com/dashboard) or `pxt secret`, and those dest env vars pointing at `s3://`, `gs://`, or `wasbs://`. | Pixeltable reaches the home bucket with either credential: a `pxt login` session or an API key ([Signing in](/platform/cli#signing-in)). The key can also be `api_key` in the `[pixeltable]` section of the Pixeltable config file; `PIXELTABLE_API_KEY` wins when both are set, and either one outranks a `pxt login` session. Hosted pods get their own worker key from the platform; you do not set it. ## Configuration Cloud storage destinations are set as a default for the database, or per column. Local Pixeltable and bring-your-own buckets need this. A hosted Cloud database defaults to its home bucket and only needs it to use a bucket of its own. ### Default destinations You can set default destinations for media columns in three places, listed here from highest to lowest precedence (see [Configuration](/platform/configuration#per-database-settings) for details). The database entry in `pixeltable.toml`, for one database (see the [Cloud configuration reference](/platform/cli#cloud-configuration-reference) on the CLI page). Leave `name` off for the local database, or set `name = 'pxt://org:db'` for a hosted one: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [[pixeltable.database]] # For input media (inserted/referenced files) db_input_media_dest = "s3://my-bucket/input/" # For computed media (AI-generated outputs) db_output_media_dest = "s3://my-bucket/output/" ``` Environment variables, for the current process: ```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}} export PIXELTABLE_INPUT_MEDIA_DEST="s3://my-bucket/input/" export PIXELTABLE_OUTPUT_MEDIA_DEST="s3://my-bucket/output/" ``` The `[pixeltable]` section of the Pixeltable config file, for every database on the installation: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [pixeltable] input_media_dest = "s3://my-bucket/input/" output_media_dest = "s3://my-bucket/output/" ``` Configure these before creating tables. All media columns will automatically use the configured destinations. ### Per-column destination (computed columns only) For **computed columns**, you can override the default with a specific destination: ```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}} import pixeltable as pxt TableModel = pxt.model_base() class Images(TableModel, name='images'): image: pxt.Image # uses input_media_dest if configured thumbnail = pxt.Column( value=image.resize((128, 128)), destination='s3://my-bucket/thumbnails/', ) ``` Then `pxt schema update app.py my_app`. In a notebook or a test, the same column is `t.add_computed_column(thumbnail=t.image.resize((128, 128)), destination=...)`. The `destination` parameter only applies to stored computed columns. For input columns, set a default input destination as shown above. ### Precedence rules Destinations are resolved in this order: 1. **Explicit column destination**: highest priority (computed columns only) 2. Database entry: `db_input_media_dest` / `db_output_media_dest` in `[[pixeltable.database]]` 3. Environment variable: `PIXELTABLE_INPUT_MEDIA_DEST` / `PIXELTABLE_OUTPUT_MEDIA_DEST` 4. Global config: `input_media_dest` / `output_media_dest` under `[pixeltable]` in `config.toml` 5. Fallback destination: `pxtfs://org:db/home` when the database is hosted in Cloud; local disk (usually `~/.pixeltable/media`) when it is running locally ## Provider configuration ### Pixeltable Cloud (home bucket) A Pixeltable Cloud database comes with a managed Media Store at `pxtfs://org:db/home`. Hosted tables and services write there automatically. No destination, no provider account, no credentials file. Cloud is in Limited Beta: email [contact@pixeltable.com](mailto:contact@pixeltable.com) to get an account. Use dest plus a credential only when a **local** Pixeltable process should write into that same bucket: ```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}} pxt login # or: export PIXELTABLE_API_KEY='your-key' export PIXELTABLE_INPUT_MEDIA_DEST='pxtfs://org:db/home' export PIXELTABLE_OUTPUT_MEDIA_DEST='pxtfs://org:db/home' ``` For a key instead of `pxt login`, create it under **API Keys** in the dashboard and export it, or set `[pixeltable].api_key` in `~/.pixeltable/config.toml`. Pixeltable does not load a `.env` file on its own. Provider keys (`AWS_ACCESS_KEY_ID`, `OPENAI_API_KEY`, ...) go under **Secrets** or `pxt secret set`. ``` pxtfs://org-slug:db-slug/home ``` Replace `org-slug` and `db-slug` with your Pixeltable Cloud organization and database names. ```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}} pxt login ``` Or `export PIXELTABLE_API_KEY='your-key'`. Pixeltable fetches temporary home-bucket credentials from the Cloud API with the key when one is set, and with the `pxt login` session otherwise. Hosted pods already have a worker key; do not copy that into your laptop `config.toml`. On Cloud, open **Storage** in the database sidebar and browse `home`. Nothing to configure for hosted tables. ``` https://www.pixeltable.com/dashboard/{org-slug}/{db-slug}/storage/home/browse ``` ### Amazon S3 ``` s3://bucket-name/optional/prefix/ ``` Uses standard AWS credential chain: * Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) * On Pixeltable Cloud, store those names under **Secrets** (or `pxt secret`), not in `PIXELTABLE_API_KEY` * AWS credentials file (`~/.aws/credentials`) * IAM role (when running on AWS) Optionally specify a profile in `config.toml`: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [pixeltable] s3_profile = "my-aws-profile" ``` ### Google Cloud Storage ``` gs://bucket-name/optional/prefix/ ``` Uses Google Cloud Application Default Credentials: * Service account key file (`GOOGLE_APPLICATION_CREDENTIALS`) * gcloud CLI authentication * GCE metadata service (when running on GCP) ```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}} pip install google-cloud-storage ``` ### Azure Blob Storage Azure supports multiple URI schemes: ``` wasbs://container@account.blob.core.windows.net/prefix/ abfss://container@account.dfs.core.windows.net/prefix/ ``` Configure in `config.toml`: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [azure] storage_account_name = "myaccount" storage_account_key = "your-key-here" ``` Or via environment variables: ```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}} export AZURE_STORAGE_ACCOUNT_NAME="myaccount" export AZURE_STORAGE_ACCOUNT_KEY="your-key-here" ``` ```bash theme={"theme":{"light":"light-plus","dark":"dark-plus"}} pip install azure-storage-blob ``` ### Cloudflare R2 ``` https://account-id.r2.cloudflarestorage.com/bucket-name/prefix/ ``` Create an R2 API token and configure AWS-style credentials. In `~/.aws/credentials`: ```ini theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [r2] aws_access_key_id = your-r2-access-key aws_secret_access_key = your-r2-secret-key ``` In `config.toml`: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [pixeltable] r2_profile = "r2" ``` ### Backblaze B2 ``` https://s3.region.backblazeb2.com/bucket-name/prefix/ ``` Create B2 application keys and configure AWS-style credentials. In `~/.aws/credentials`: ```ini theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [b2] aws_access_key_id = your-b2-key-id aws_secret_access_key = your-b2-application-key ``` In `config.toml`: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [pixeltable] b2_profile = "b2" ``` ### Tigris ``` https://t3.storage.dev/bucket-name/prefix/ ``` Configure AWS-style credentials for Tigris. In `~/.aws/credentials`: ```ini theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [tigris] aws_access_key_id = your-tigris-access-key aws_secret_access_key = your-tigris-secret-key ``` In `config.toml`: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [pixeltable] tigris_profile = "tigris" ``` ## Complete example Here's a full example using S3 for both input and computed media. First, configure the database's default destinations in `pixeltable.toml`: ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [[pixeltable.database]] db_input_media_dest = "s3://my-app-bucket/uploads/" db_output_media_dest = "s3://my-app-bucket/generated/" ``` and, optionally, the AWS profile in `~/.pixeltable/config.toml` (default credentials are used if not set): ```toml theme={"theme":{"light":"light-plus","dark":"dark-plus"}} [pixeltable] s3_profile = "my-aws-profile" ``` Then declare the columns in your application file: ```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}} import pixeltable as pxt from pixeltable.functions import openai TableModel = pxt.model_base() class Photos(TableModel, name='photos'): photo: pxt.Image # input media goes to input_media_dest thumbnail = pxt.Column( value=photo.resize((256, 256)), destination='s3://my-app-bucket/thumbnails/', # overrides output_media_dest ) description = openai.chat_completions( [ { 'role': 'user', 'content': [ {'type': 'text', 'text': 'Describe this image briefly.'}, {'type': 'image_url', 'image_url': photo}, ], } ], model='gpt-4o-mini', ) ``` `pxt schema update app.py production` creates the table. Insert and Pixeltable handles the uploads: ```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}} import pixeltable as pxt t = pxt.get_table('production.photos') t.insert([ {'photo': 'https://example.com/image1.jpg'}, {'photo': '/local/path/to/image2.png'}, ]) # Query as usual: files are streamed/cached as needed t.select(t.photo, t.thumbnail, t.description).collect() ``` ## Best practices Structure your bucket with prefixes that reflect your application: ``` s3://my-bucket/ ├── production/ │ ├── uploads/ │ └── generated/ └── staging/ ├── uploads/ └── generated/ ``` Use different prefixes or buckets for input vs computed media: * Easier to set different retention policies * Clearer cost attribution * Simpler backup strategies Set up bucket lifecycle policies to automatically: * Transition old data to cheaper storage tiers * Delete temporary/staging data after a period * Enable versioning for critical data When running on cloud infrastructure, use IAM roles instead of access keys: * More secure (no key rotation needed) * Automatic credential refresh * Better audit trails ## Troubleshooting Verify your credentials have the necessary permissions: * `s3:GetObject`, `s3:PutObject`, `s3:DeleteObject` * `s3:ListBucket` for the bucket For GCS: `storage.objects.create`, `storage.objects.get`, `storage.objects.delete` * Ensure the bucket exists and the name is spelled correctly * Check the region matches your credential configuration * For S3-compatible providers, verify the endpoint URL is correct * Pixeltable uses connection pooling and parallel uploads automatically * Consider using a bucket in the same region as your compute * Check your network bandwidth and latency See the complete list of storage configuration options including profiles for S3, R2, B2, Tigris, and Azure. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.