Data Engineering Fundamentals
eBook - ePub

Data Engineering Fundamentals

Building scalable data solutions with ETL pipelines and strategic data architecture design (English Edition)

  1. English
  2. ePUB (mobile friendly)
  3. Available on iOS & Android
eBook - ePub

Data Engineering Fundamentals

Building scalable data solutions with ETL pipelines and strategic data architecture design (English Edition)

About this book

Description
In today's data-driven world, mastering data engineering is crucial for anyone looking to build robust data pipelines and extract valuable insights. This book simplifies complex concepts and provides a clear pathway to understanding the core principles that power modern data solutions. It bridges the gap between raw data and actionable intelligence, making data engineering accessible to everyone.This book walks you through the entire data engineering lifecycle. Starting with foundational concepts and data ingestion from diverse sources, you will learn how to build efficient data lakes and warehouses. You will learn data transformation using tools like Apache Spark and the orchestration of data workflows with platforms like Airflow and Argo Workflow. Crucial aspects of data quality, governance, scalability, and performance monitoring are thoroughly covered, ensuring you understand how to maintain reliable and efficient data systems. Real-world use cases across industries like e-commerce, finance, and government illustrate practical applications, while a final section explores emerging trends such as AI integration and cloud advancements.By the end of this book, you will have a solid foundation in data engineering, along with practical skills to help enhance your career. You will be equipped to design, build, and maintain data pipelines, transforming raw data into meaningful insights.

What you will learn
? Understand data engineering base concepts and build scalable solutions.
? Master data storage, ingestion, and transformation.
? Orchestrates data workflows and automates pipelines for efficiency.
? Ensure data quality, governance, and security compliance.
? Monitor, optimize, and scale data solutions effectively.
? Explore real-world use cases and future data trends.

Who this book is for
This book is for aspiring data engineers, analysts, and developers seeking a foundational understanding of data engineering. Whether you are a beginner or looking to deepen your expertise, this book provides you with the knowledge and tools to succeed in today's data engineering challenges.

Table of Contents
1. Understanding Data Engineering
2. Data Ingestion and Acquisition
3. Data Storage and Management
4. Data Transformation and Processing
5. Data Orchestration and Workflows
6. Data Governance Principles
7. Scaling Data Solutions
8. Monitoring and Performance
9. Real-world Data Engineering Use Cases
10. Future Trends in Data Engineering

Frequently asked questions

Yes, you can cancel anytime from the Subscription tab in your account settings on the Perlego website. Your subscription will stay active until the end of your current billing period. Learn how to cancel your subscription.
No, books cannot be downloaded as external files, such as PDFs, for use outside of Perlego. However, you can download books within the Perlego app for offline reading on mobile or tablet. Learn more here.
Perlego offers two plans: Essential and Complete
  • Essential is ideal for learners and professionals who enjoy exploring a wide range of subjects. Access the Essential Library with 800,000+ trusted titles and best-sellers across business, personal growth, and the humanities. Includes unlimited reading time and Standard Read Aloud voice.
  • Complete: Perfect for advanced learners and researchers needing full, unrestricted access. Unlock 1.4M+ books across hundreds of subjects, including academic and specialized titles. The Complete Plan also includes advanced features like Premium Read Aloud and Research Assistant.
Both plans are available with monthly, semester, or annual billing cycles.
We are an online textbook subscription service, where you can get access to an entire online library for less than the price of a single book per month. With over 1 million books across 1000+ topics, we’ve got you covered! Learn more here.
Look out for the read-aloud symbol on your next book to see if you can listen to it. The read-aloud tool reads text aloud for you, highlighting the text as it is being read. You can pause it, speed it up and slow it down. Learn more here.
Yes! You can use the Perlego app on both iOS or Android devices to read anytime, anywhere — even offline. Perfect for commutes or when you’re on the go.
Please note we cannot support devices running on iOS 13 and Android 7 or earlier. Learn more about using the app.
Yes, you can access Data Engineering Fundamentals by Zhaolong Liu in PDF and/or ePUB format, as well as other popular books in Computer Science & Computer Science General. We have over one million books available in our catalogue for you to explore.

Table of contents

  1. Cover
  2. Title Page
  3. Copyright Page
  4. Dedication Page
  5. About the Author
  6. About the Reviewers
  7. Acknowledgement
  8. Preface
  9. Table of Contents
  10. 1. Understanding Data Engineering
  11. 2. Data Ingestion and Acquisition
  12. 3. Data Storage and Management
  13. 4. Data Transformation and Processing
  14. 5. Data Orchestration and Workflows
  15. 6. Data Governance Principles
  16. 7. Scaling Data Solutions
  17. 8. Monitoring and Performance
  18. 9. Real-world Data Engineering Use Cases
  19. 10. Future Trends in Data Engineering
  20. Index