Modern Data Architectures with Python
A practical guide to building and deploying data pipelines, data warehouses, and data lakes with Python
- 318 pages
- English
- ePUB (mobile friendly)
- Available on iOS & Android
Modern Data Architectures with Python
A practical guide to building and deploying data pipelines, data warehouses, and data lakes with Python
About this book
Build scalable and reliable data ecosystems using Data Mesh, Databricks Spark, and Kafka
Key Features
- Develop modern data skills used in emerging technologies
- Learn pragmatic design methodologies such as Data Mesh and data lakehouses
- Gain a deeper understanding of data governance
- Purchase of the print or Kindle book includes a free PDF eBook
Book Description
Modern Data Architectures with Python will teach you how to seamlessly incorporate your machine learning and data science work streams into your open data platforms. You'll learn how to take your data and create open lakehouses that work with any technology using tried-and-true techniques, including the medallion architecture and Delta Lake.Starting with the fundamentals, this book will help you build pipelines on Databricks, an open data platform, using SQL and Python. You'll gain an understanding of notebooks and applications written in Python using standard software engineering tools such as git, pre-commit, Jenkins, and Github. Next, you'll delve into streaming and batch-based data processing using Apache Spark and Confluent Kafka. As you advance, you'll learn how to deploy your resources using infrastructure as code and how to automate your workflows and code development. Since any data platform's ability to handle and work with AI and ML is a vital component, you'll also explore the basics of ML and how to work with modern MLOps tooling. Finally, you'll get hands-on experience with Apache Spark, one of the key data technologies in today's market.By the end of this book, you'll have amassed a wealth of practical and theoretical knowledge to build, manage, orchestrate, and architect your data ecosystems.
What you will learn
- Understand data patterns including delta architecture
- Discover how to increase performance with Spark internals
- Find out how to design critical data diagrams
- Explore MLOps with tools such as AutoML and MLflow
- Get to grips with building data products in a data mesh
- Discover data governance and build confidence in your data
- Introduce data visualizations and dashboards into your data practice
Who this book is for
This book is for developers, analytics engineers, and managers looking to further develop a data ecosystem within their organization. While they're not prerequisites, basic knowledge of Python and prior experience with data will help you to read and follow along with the examples.
]]>
Frequently asked questions
- Essential is ideal for learners and professionals who enjoy exploring a wide range of subjects. Access the Essential Library with 800,000+ trusted titles and best-sellers across business, personal growth, and the humanities. Includes unlimited reading time and Standard Read Aloud voice.
- Complete: Perfect for advanced learners and researchers needing full, unrestricted access. Unlock 1.4M+ books across hundreds of subjects, including academic and specialized titles. The Complete Plan also includes advanced features like Premium Read Aloud and Research Assistant.
Please note we cannot support devices running on iOS 13 and Android 7 or earlier. Learn more about using the app.
Information
Table of contents
- Modern Data Architectures with Python
- Contributors
- Preface
- Part 1:Fundamental Data Knowledge
- 1
- 2
- Part 2: Data Engineering Toolset
- 3
- 4
- 5
- Part 3:Modernizing the Data Platform
- 6
- 7
- 8
- 9
- Part 4:Hands-on Project
- 10
- 11
- 12
- Index
- Other Books You May Enjoy