This documentation page is also available as an interactive notebook. You can launch the notebook in
Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
above links.
Load data from CSV and Excel files into Pixeltable tables for processing
and analysis.This notebook uses pxt.create_table() so you can run cells without a
project. An app puts the same columns on a TableModel in app.py and
creates the tables with pxt schema update.
Problem
You have data in CSV or Excel files that you want to process with AI
models, add computed columns to, or combine with other data sources.
Solution
What’s in this recipe:
Import CSV files directly into tables
Import from Pandas DataFrames
Handle different data types
You use pxt.create_table() with a source parameter to create a table
from a CSV file, or insert DataFrame rows into an existing table.
Setup
%pip install -qU pixeltable pandas
import pandas as pdimport pixeltable as pxt# Create a fresh directorypxt.drop_dir('import_demo', force=True)pxt.create_dir('import_demo')
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
Created directory ‘import_demo’.
<pixeltable.catalog.dir.Dir at 0x141eca110>
Import CSV directly
Use create_table with source to create a table from a CSV file:
# Import CSV from URLcsv_url = 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/world-population-data.csv'population = pxt.create_table('import_demo/population', source=csv_url)
Created table ‘population’.Inserting rows into `population`: 0 rows [00:00, ? rows/s]
Inserting rows into `population`: 234 rows [00:00, 9032.63 rows/s]
Inserted 234 rows with 0 errors.
# View the imported datapopulation.head(5)
Import from Pandas DataFrame
You can also create a DataFrame first and insert it:
Created table ‘users’.Inserting rows into `users`: 0 rows [00:00, ? rows/s]
Inserting rows into `users`: 3 rows [00:00, 923.31 rows/s]
Inserted 3 rows with 0 errors.
3 rows inserted, 6 values computed.
# View the datausers.collect()
Explanation
Source types supported:Type inference:Pixeltable automatically infers column types from CSV data. You can
override types using schema_overrides.Large files:For very large CSV files, consider: