Limitless Analytics with Azure Synapse

Prashant Kumar Mishra, Mukesh Kumar

• Publisher: Packt Publishing
• Year: 2021
• eBook ISBN: 9781800206977
• Edition: 1
• Topic: Computer Science
• Subtopic: Data Modelling & Design
• Index: Computer Science

Leverage the Azure analytics platform's key analytics services to deliver unmatched intelligence for your dataKey Features• Learn to ingest, prepare, manage, and serve data for immediate business requirements• Bring enterprise data warehousing and big data analytics together to gain insights from your data• Develop end-to-end analytics solutions using Azure SynapseBook DescriptionAzure Synapse Analytics, which Microsoft describes as the next evolution of Azure SQL Data Warehouse, is a limitless analytics service that brings enterprise data warehousing and big data analytics together. With this book, you'll learn how to discover insights from your data effectively using this platform. The book starts with an overview of Azure Synapse Analytics, its architecture, and how it can be used to improve business intelligence and machine learning capabilities. Next, you'll go on to choose and set up the correct environment for your business problem. You'll also learn a variety of ways to ingest data from various sources and orchestrate the data using transformation techniques offered by Azure Synapse. Later, you'll explore how to handle both relational and non-relational data using the SQL language. As you progress, you'll perform real-time streaming and execute data analysis operations on your data using various languages, before going on to apply ML techniques to derive accurate and granular insights from data. Finally, you'll discover how to protect sensitive data in real time by using security and privacy features. By the end of this Azure book, you'll be able to build end-to-end analytics solutions while focusing on data prep, data management, data warehousing, and AI tasks.What you will learn• Explore the necessary considerations for data ingestion and orchestration while building analytical pipelines• Understand pipelines and activities in Synapse pipelines and use them to construct end-to-end data-driven workflows• Query data using various coding languages on Azure Synapse• Focus on Synapse SQL and Synapse Spark• Manage and monitor resource utilization and query activity in Azure Synapse• Connect Power BI workspaces with Azure Synapse and create or modify reports directly from Synapse Studio• Create and manage IP firewall rules in Azure SynapseWho this book is forThis book is for data architects, data scientists, data engineers, and business analysts who are looking to get up and running with the Azure Synapse Analytics platform. Basic knowledge of data warehousing will be beneficial to help you understand the concepts covered in this book more effectively.

Section 1: The Basics and Key Concepts

The objective of this section is to introduce you to the key concepts, download supporting data, and introduce you to example scenarios.

This section comprises the following chapters:

• Chapter 1, Introduction to Azure Synapse
• Chapter 2, Considerations for Your Compute Environment

Chapter 1: Introduction to Azure Synapse

Azure Synapse Analytics, formerly known as Azure SQL Data Warehouse, is not a mere data warehouse anymore. Azure Synapse is an amalgamation of big data analytics with an enterprise data warehouse. It provides two different types of compute environments for different workloads: one is the SQL compute environment, which is called a SQL pool, and the other one is the Spark compute environment, which is called a Spark pool. Now developers can choose their compute environment as per their business needs. Azure Synapse also provides a unified portal called Synapse Studio for developers that creates a workspace for data prep, data management, data exploration, data warehousing, big data, and AI tasks.

This chapter covers an introduction to Azure Synapse and guides you on starting to use Synapse Studio. You will learn how to create an Azure Synapse workspaces and get acquainted with the components of Azure Synapse. You can start using Synapse with the sample data and queries provided in the Azure portal itself.

In this chapter, our topics will include the following:

• Introducing the components of Azure Synapse
• Creating a Synapse workspace
• Understanding Azure Data Lake
• Exploring Synapse Studio

Technical requirements

In this chapter, you are going to learn how to create your first Synapse workspace in the Azure portal. In order to do this, there are certain prerequisites before you start working on Azure Synapse.

It would be beneficial to have basic knowledge of the Azure portal, as well as an understanding of SQL and Spark. Knowledge of Azure Data Factory and Power BI would be helpful but not essential.

You must have your own Azure subscription or access to an Azure subscription with appropriate permissions. If you are new to Azure, you can go through the following link to create a free Azure account: https://azure.microsoft.com/en-us/free/.

Once you have your Azure subscription created, you can proceed further with the main topics of this chapter.

Introducing the components of Azure Synapse

Azure Synapse is a limitless analytics service on the Azure platform. It bundles together data warehousing and big data analytics with deep integration of Azure Machine Learning and Power BI. Azure Synapse brings together relational and non-relational data and helps in querying files in the data lake without looking for any other service.

One of the best features that has been introduced with Azure Synapse is code-free data orchestration where you can build ETL/ELT processes to bring data to Synapse from various sources.

Important note

Synapse provides various layers of security for the data stored; however, you need to follow the security guidelines to keep your data secured. For example, do not expose the username and password in any publicly accessible place – you will invite the biggest threat to your data by doing so. It is important to understand that Azure gives you the power to secure your data, but it is in your hands to best use that power.

What happens when we embrace a new technology in an organization?

We need to look out for a resource that already has knowledge of it, which brings extra costs on top of the cost of the technical implementation. However, Azure Synapse supports various programming languages, such as T-SQL, Python, Scala, Spark, SQL, and .NET, making it easy for people who are already familiar with those languages to learn. In this chapter, we will show a demo for T-SQL, but we will cover examples for other languages in upcoming chapters.

The following diagram represents all the components of Azure Synapse and how all these components are tied together within Synapse Analytics:

Figure 1.1 – The components of Azure Synapse

The preceding diagram represents all components of Azure Synapse, which includes Analytics runtimes, supported languages, form factors, data integration, and Power BI workspaces. We will cover all these topics in upcoming chapters.

Important note

Although Azure Synapse is ...

Table of contents

• Limitless Analytics with Azure Synapse
• Foreword
• Preface
• Section 1: The Basics and Key Concepts
• Chapter 1: Introduction to Azure Synapse
• Chapter 2: Considerations for Your Compute Environment
• Section 2: Data Ingestion and Orchestration
• Chapter 3: Bringing Your Data to Azure Synapse
• Chapter 4: Using Synapse Pipelines to Orchestrate Your Data
• Chapter 5: Using Synapse Link with Azure Cosmos DB
• Section 3: Azure Synapse for Data Scientists and Business Analysts
• Chapter 6: Working with T-SQL in Azure Synapse
• Chapter 7: Working with R, Python, Scala, .NET, and Spark SQL in Azure Synapse
• Chapter 8: Integrating a Power BI Workspace with Azure Synapse
• Chapter 9: Perform Real-Time Analytics on Streaming Data
• Chapter 10: Generate Powerful Insights on Azure Synapse Using Azure ML
• Section 4: Best Practices
• Chapter 11: Performing Backup and Restore in Azure Synapse Analytics
• Chapter 12: Securing Data on Azure Synapse
• Chapter 13: Managing and Monitoring Synapse Workloads
• Chapter 14: Coding Best Practices
• Other Books You May Enjoy