This documentation page is also available as an interactive notebook. You can launch the notebook in
Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
above links.
Create training, validation, and test splits with random or stratified
sampling.
Problem
You have a large dataset and need to create subsets for ML
training—random samples for quick experiments, stratified samples for
balanced classes, or reproducible splits for benchmarking.
Solution
What’s in this recipe:
Random sampling with sample(n=...)
Percentage-based sampling with sample(fraction=...)
Stratified sampling with stratify_by=
You use query.sample() to create random subsets, with optional
stratification for balanced class distribution.
Setup
%pip install -qU pixeltable
import pixeltable as pxt# Create a fresh directorypxt.drop_dir('sampling_demo', force=True)pxt.create_dir('sampling_demo')
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
Created directory ‘sampling_demo’.
<pixeltable.catalog.dir.Dir at 0x1471b08e0>
# Create a persistent table from a sample for dev/testtrain_sample = data.sample(fraction=0.8, seed=42)test_sample = data.sample(fraction=0.2, seed=43)# Persist as new tablestrain_table = pxt.create_table('sampling_demo/train', source=train_sample)test_table = pxt.create_table('sampling_demo/test', source=test_sample)
Created table ‘train’.Inserting rows into `train`: 0 rows [00:00, ? rows/s]
Inserting rows into `train`: 9 rows [00:00, 3080.27 rows/s]
Created table ‘test’.Inserting rows into `test`: 0 rows [00:00, ? rows/s]
Inserting rows into `test`: 3 rows [00:00, 1333.92 rows/s]