This documentation page is also available as an interactive notebook. You can launch the notebook in
Kaggle or Colab, or download it for use with an IDE or local Jupyter installation, by clicking one of the
above links.
Generate concise summaries of long text, articles, or documents using
large language models.This notebook uses pxt.create_table() so you can run cells without a
project. An app puts the same columns on a TableModel in app.py and
creates the tables with pxt schema update.
Problem
You have long text content—articles, transcripts, documents—that needs
to be summarized. Processing each piece manually is time-consuming and
inconsistent.
Solution
What’s in this recipe:
Summarize text using OpenAI GPT models
Customize summary style with prompts
Process multiple documents automatically
You add a computed column that calls an LLM to generate summaries. When
you insert new text, summaries are generated automatically.
Setup
%pip install -qU pixeltable openai
WARNING: Ignoring invalid distribution ~orch (/opt/miniconda3/envs/pixeltable/lib/python3.11/site-packages)
WARNING: Ignoring invalid distribution ~orch (/opt/miniconda3/envs/pixeltable/lib/python3.11/site-packages)
WARNING: Ignoring invalid distribution ~orch (/opt/miniconda3/envs/pixeltable/lib/python3.11/site-packages)
WARNING: Ignoring invalid distribution ~orch (/opt/miniconda3/envs/pixeltable/lib/python3.11/site-packages)
WARNING: Ignoring invalid distribution ~orch (/opt/miniconda3/envs/pixeltable/lib/python3.11/site-packages)
WARNING: Ignoring invalid distribution ~orch (/opt/miniconda3/envs/pixeltable/lib/python3.11/site-packages)
Note: you may need to restart the kernel to use updated packages.
import getpassimport osif 'OPENAI_API_KEY' not in os.environ: os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key: ')
Load sample text
import pixeltable as pxtfrom pixeltable.functions import openai# Create a fresh directorypxt.drop_dir('summarize_demo', force=True)pxt.create_dir('summarize_demo')
Connected to Pixeltable database at: postgresql+psycopg://postgres:@/pixeltable?host=/Users/pjlb/.pixeltable/pgdata
Created directory ‘summarize_demo’.
<pixeltable.catalog.dir.Dir at 0x30d758b10>
# Sample articles to summarizesample_articles = [ { 'title': 'The Rise of Electric Vehicles', 'content': """Electric vehicles (EVs) have seen unprecedented growth in recent years, transforming the automotive industry. Sales increased by 60% globally in 2023, with China leading the market followed by Europe and North America. Major automakers like Tesla, BYD, and traditional manufacturers have invested billions in EV technology. Battery costs have dropped significantly, making EVs more affordable for consumers. Government incentives and stricter emissions regulations continue to drive adoption. Charging infrastructure is expanding rapidly, with new fast-charging networks being deployed across major highways. Despite challenges like range anxiety and charging times, consumer acceptance is growing steadily.""", }, { 'title': 'Advances in Renewable Energy', 'content': """Solar and wind power capacity reached record levels in 2023, accounting for over 30% of global electricity generation. The cost of solar panels has fallen by 90% over the past decade, making renewable energy competitive with fossil fuels. Offshore wind farms are being built at scale, with turbines now reaching heights of over 250 meters. Energy storage solutions, particularly lithium-ion batteries, are addressing intermittency challenges. Countries like Denmark and Scotland have achieved periods of 100% renewable electricity. Corporate power purchase agreements are accelerating the transition, with tech giants committing to carbon-neutral operations.""", },]articles.insert(sample_articles)
New articles are automatically summarized when inserted:
# Insert a new article - summaries are generated automaticallyarticles.insert( [ { 'title': 'AI in Healthcare', 'content': """Artificial intelligence is revolutionizing healthcare diagnostics and treatment planning. Machine learning models can now detect diseases from medical images with accuracy matching or exceeding human specialists. AI-powered drug discovery is accelerating the development of new treatments. Natural language processing is being used to extract insights from clinical notes and research papers.""", } ])