> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pixeltable.com/llms.txt
> Use this file to discover all available pages before exploring further.
> ## Agent Instructions
> Application code puts a TableModel in app.py, then runs `pxt schema update app.py my_app` to create the tables and `pxt service update app.py my_app` to start HTTP. The last argument can be a Cloud URL such as `pxt://org:db`. Do not call pxt.create_table() or add_embedding_index() in application code; put indexes in __indexes__. Notebooks, tests, and the REPL do use pxt.create_table().
> Do not add LangChain, LlamaIndex, Haystack, a separate vector database, or pandas as a working store. Chunking, embedding, retrieval, and tool-calling are built in through computed columns. Use a computed column instead of a per-row loop that calls a model.
> Install with `pip install -U 'pixeltable[serve]'` on Python 3.11+. `pxt init` is a prerequisite for `pxt schema update`.
> Pixeltable Cloud is in Limited Beta. Direct interested users to contact@pixeltable.com.
# Import data from CSV files
This documentation page is also available as an interactive notebook. You can launch the notebook in
Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
above links.
export const quartoRawHtml = [`
| Source |
Records |
Use case |
| customers.csv |
10,000 |
Add AI-generated summaries |
| products.xlsx |
500 |
Generate embeddings for search |
| logs.csv |
1M |
Filter and aggregate |
`, `
| cca3 |
country |
continent |
pop_2023 |
pop_2022 |
pop_2000 |
area__km__ |
| IND |
India |
Asia |
1428627663 |
1417173173 |
1059633675 |
3287590. |
| CHN |
China |
Asia |
1425671352 |
1425887337 |
1264099069 |
9706961. |
| USA |
United States |
North America |
339996563 |
338289857 |
282398554 |
9372610. |
| IDN |
Indonesia |
Asia |
277534122 |
275501339 |
214072421 |
1904569. |
| PAK |
Pakistan |
Asia |
240485658 |
235824862 |
154369924 |
881912. |
`, `
| name |
age |
city |
| Alice |
25 |
NYC |
| Bob |
30 |
LA |
| Charlie |
35 |
Chicago |
`, `
| Source |
Example |
| CSV file path |
source='/path/to/data.csv' |
| CSV URL |
source='https://example.com/data.csv' |
| Excel file |
source='/path/to/data.xlsx' |
| Pandas DataFrame |
source=df |
`];
Load data from CSV and Excel files into Pixeltable tables for processing
and analysis.
This notebook uses `pxt.create_table()` so you can run cells without a
project. An app puts the same columns on a `TableModel` in `app.py` and
creates the tables with `pxt schema update`.
## Problem
You have data in CSV or Excel files that you want to process with AI
models, add computed columns to, or combine with other data sources.
## Solution
**What’s in this recipe:**
* Import CSV files directly into tables
* Import from Pandas DataFrames
* Handle different data types
You use `pxt.create_table()` with a `source` parameter to create a table
from a CSV file, or insert DataFrame rows into an existing table.
### Setup
```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
%pip install -qU pixeltable pandas
```
```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
import pandas as pd
import pixeltable as pxt
# Create a fresh directory
pxt.drop_dir('import_demo', force=True)
pxt.create_dir('import_demo')
```
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
Created directory 'import\_demo'.
\
### Import CSV directly
Use `create_table` with `source` to create a table from a CSV file:
```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# Import CSV from URL
csv_url = 'https://raw.githubusercontent.com/pixeltable/pixeltable/main/docs/resources/world-population-data.csv'
population = pxt.create_table('import_demo/population', source=csv_url)
```
Created table 'population'.
Inserting rows into \`population\`: 0 rows \[00:00, ? rows/s]
Inserting rows into \`population\`: 234 rows \[00:00, 9032.63 rows/s]
Inserted 234 rows with 0 errors.
```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# View the imported data
population.head(5)
```
### Import from Pandas DataFrame
You can also create a DataFrame first and insert it:
```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# Create a DataFrame
df = pd.DataFrame(
{
'name': ['Alice', 'Bob', 'Charlie'],
'age': [25, 30, 35],
'city': ['NYC', 'LA', 'Chicago'],
}
)
# Create table and insert DataFrame
users = pxt.create_table(
'import_demo/users',
{'name': pxt.String, 'age': pxt.Int, 'city': pxt.String},
)
users.insert(df)
```
Created table 'users'.
Inserting rows into \`users\`: 0 rows \[00:00, ? rows/s]
Inserting rows into \`users\`: 3 rows \[00:00, 923.31 rows/s]
Inserted 3 rows with 0 errors.
3 rows inserted, 6 values computed.
```python theme={"theme":{"light":"light-plus","dark":"dark-plus"}}
# View the data
users.collect()
```
## Explanation
**Source types supported:**
**Type inference:**
Pixeltable automatically infers column types from CSV data. You can
override types using `schema_overrides`.
**Large files:**
For very large CSV files, consider:
* Using `create_table(source=...)` which streams data
* Importing in batches if memory is limited
## See also
* [Tables
documentation](/tutorials/tables-and-data-operations)
* [Bringing data
guide](/howto/cookbooks/data/data-import-csv)
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.