DuckDB in Action

Mark Needham, Michael Hunger, Michael Simons

• Publisher: Manning
• Year: 2024
• Topic: Computer Science
• eBook ISBN:
  9781638355595
• Subtopic: Data Mining
• Index: Computer Science

Dive into DuckDB and start processing gigabytes of data with ease—all with no data warehouse.

DuckDB is a cutting-edge SQL database that makes it incredibly easy to analyze big data sets right from your laptop. In DuckDB in Action you’ll learn everything you need to know to get the most out of this awesome tool, keep your data secure on prem, and save you hundreds on your cloud bill. From data ingestion to advanced data pipelines, you’ll learn everything you need to get the most out of DuckDB—all through hands-on examples.

Open up DuckDB in Action and learn how to:

• Read and process data from CSV, JSON and Parquet sources both locally and remote
• Write analytical SQL queries, including aggregations, common table expressions, window functions, special types of joins, and pivot tables
• Use DuckDB from Python, both with SQL and its "Relational"-API, interacting with databases but also data frames
• Prepare, ingest and query large datasets
• Build cloud data pipelines
• Extend DuckDB with custom functionality

Pragmatic and comprehensive, DuckDB in Action introduces the DuckDB database and shows you how to use it to solve common data workflow problems. You won’t need to read through pages of documentation—you’ll learn as you work. Get to grips with DuckDB's unique SQL dialect, learning to seamlessly load, prepare, and analyze data using SQL queries. Extend DuckDB with both Python and built-in tools such as MotherDuck, and gain practical insights into building robust and automated data pipelines.

About the technology

DuckDB makes data analytics fast and fun! You don’t need to set up a Spark or run a cloud data warehouse just to process a few hundred gigabytes of data. DuckDB is easily embeddable in any data analytics application, runs on a laptop, and processes data from almost any source, including JSON, CSV, Parquet, SQLite and Postgres.

About the book

DuckDB in Action guides you example-by-example from setup, through your first SQL query, to advanced topics like building data pipelines and embedding DuckDB as a local data store for a Streamlit web app. You’ll explore DuckDB’s handy SQL extensions, get to grips with aggregation, analysis, and data without persistence, and use Python to customize DuckDB. A hands-on project accompanies each new topic, so you can see DuckDB in action.

What's inside

• Prepare, ingest and query large datasets
• Build cloud data pipelines
• Extend DuckDB with custom functionality
• Fast-paced SQL recap: From simple queries to advanced analytics

About the reader

For data pros comfortable with Python and CLI tools.

About the author

Mark Needham is a blogger and video creator at @?LearnDataWithMark. Michael Hunger leads product innovation for the Neo4j graph database. Michael Simons is a Java Champion, author, and Engineer at Neo4j.

Table of contents

• DuckDB in Action
• copyright
• dedication
• contents
• foreword
• preface
• acknowledgments
• about this book
• about the authors
• about the cover illustration
• 1 An introduction to DuckDB
• 2 Getting started with DuckDB
• 3 Executing SQL queries
• 4 Advanced aggregation and analysis of data
• 5 Exploring data without persistence
• 6 Integrating with the Python ecosystem
• 7 DuckDB in the cloud with MotherDuck
• 8 Building data pipelines with DuckDB
• 9 Building and deploying data apps
• 10 Performance considerations for large datasets
• 11 Conclusion
• appendix Client APIs for DuckDB
• index