Threat Hunting with Elastic Stack

Solve complex security challenges with integrated prevention, detection, and response

Andrew Pease

• Publisher: Packt Publishing
• Year: 2021
• Print ISBN: 9781801073783
• eBook ISBN: 9781801079808
• Edition: 1
• Topic: Informatica
• Subtopic: Sicurezza informatica

Learn advanced threat analysis techniques in practice by implementing Elastic Stack security features

Key Features

• Get started with Elastic Security configuration and features
• Leverage Elastic Stack features to provide optimal protection against threats
• Discover tips, tricks, and best practices to enhance the security of your environment

Book Description

Threat Hunting with Elastic Stack will show you how to make the best use of Elastic Security to provide optimal protection against cyber threats. With this book, security practitioners working with Kibana will be able to put their knowledge to work and detect malicious adversary activity within their contested network.

You'll take a hands-on approach to learning the implementation and methodologies that will have you up and running in no time. Starting with the foundational parts of the Elastic Stack, you'll explore analytical models and how they support security response and finally leverage Elastic technology to perform defensive cyber operations.

You'll then cover threat intelligence analytical models, threat hunting concepts and methodologies, and how to leverage them in cyber operations. After you've mastered the basics, you'll apply the knowledge you've gained to build and configure your own Elastic Stack, upload data, and explore that data directly as well as by using the built-in tools in the Kibana app to hunt for nefarious activities.

By the end of this book, you'll be able to build an Elastic Stack for self-training or to monitor your own network and/or assets and use Kibana to monitor and hunt for adversaries within your network.

What you will learn

• Explore cyber threat intelligence analytical models and hunting methodologies
• Build and configure Elastic Stack for cyber threat hunting
• Leverage the Elastic endpoint and Beats for data collection
• Perform security data analysis using the Kibana Discover, Visualize, and Dashboard apps
• Execute hunting and response operations using the Kibana Security app
• Use Elastic Common Schema to ensure data uniformity across organizations

Who this book is for

Security analysts, cybersecurity enthusiasts, information systems security staff, or anyone who works with the Elastic Stack for security monitoring, incident response, intelligence analysis, or threat hunting will find this book useful. Basic working knowledge of IT security operations and network and endpoint systems is necessary to get started.

Section 1: Introduction to Threat Hunting, Analytical Models, and Hunting Methodologies

This section will introduce you to the concepts of cyber threat intelligence and how to use analysis to create intelligence beyond simply uploading indicators of compromise.

This part of the book comprises the following chapters:

• Chapter 1, Introduction to Cyber Threat Intelligence, Analytical Models, and Frameworks
• Chapter 2, Hunting Concepts, Methodologies, and Techniques

Chapter 1: Introduction to Cyber Threat Intelligence, Analytical Models, and Frameworks

Generally speaking, there are a few "shiny penny" terms in modern IT terminology – blockchain, artificial intelligence, and the dreaded single pane of glass are some classic examples. Cyber Threat Intelligence (CTI) and threat hunting are no different. While all of these terminologies are tremendously valuable, they are commonly used for figurative hand-waving by marketing and sales teams to procure a meeting with a C-suite. With that in mind, let's discuss what CTI and threat hunting are in practicality, versus as umbrella terms for all things security.

Through the rest of this book, we'll refer back to the theories and concepts that we will cover here. This chapter will focus a lot on critical thinking, reasoning processes, and analytical models; understanding these is paramount because threat hunting is not linear. It involves constant adaption with a live adversary on the other side of the keyboard. As hard as you are working to detect them, they are working just as hard to evade detection. As we'll discover as we progress through the book, knowledge is important, but being able to adapt to a rapidly changing scenario is crucial to success.

In this chapter, we'll go through the following topics:

• What is cyber threat intelligence?
• The Intelligence Pipeline
• The Lockheed Martin Cyber Kill Chain
• Mitre's ATT&CK Matrix
• The Diamond Model

What is cyber threat intelligence?

My experiences have led me to the opinion that CTI and threat hunting are processes and methodologies tightly coupled with, and in support of, traditional security operations (SecOps).

When we talk about traditional SecOps, we're referring to the deployment and management of various types of infrastructure and defensive tools – think firewalls, intrusion detection systems, vulnerability scanners, and antiviruses. Additionally, this includes some of the less exciting elements, such as policy, and processes such as privacy and incident response (not to say that incident response isn't an absolute blast). There are copious amounts of publications that describe traditional SecOps and I'm certainly not going to try and re-write them. However, to grow and mature as a threat hunter, you need to understand where CTI and threat hunting fit into the big picture.

When we talk about CTI, we mean the processes of collection, analysis, and production to transition data into information, and lastly, into intelligence (we'll discuss technologies and methodologies to do that later) and support operations to detect observations that can evade automated detections. Threat hunting searches for adversary activity that cannot be detected through the use of traditional signature-based defensive tools. These mainly include profiling and detecting patterns using endpoint and network activity. CTI and threat hunting combined are the processes of identifying adversary techniques and their relevance to the network being defended. They then generate profiles and patterns within data to identify when someone may be using these identified techniques and – this is the often overlooked part – lead to data-driven decisions.

A great example would be identifying that abusing authorized binaries, such as PowerShell or GCC, is a technique used by adversaries. In this example, both PowerShell and GCC are expected to be on the system, so their existence or usage wouldn't cause a host-based detection system to generate an alert. So CTI processes would identify that this is a tactic used by adversaries, threat hunting would profile how these binaries are used in a defended network, and finally, this information would be used to inform active response operations or recommendations to improve the enduring defensive posture.

Of particular note is that while threat hunting is an evolution from traditional SecOps, that isn't to say that it is inherently better. They are two sides of the same coin. Understanding traditional SecOps and where intelligence analysis and threat hunting should be folded into it is paramount to being successful as a technician, responder, analyst, or leader. In this chapter, we'll discuss the different parts of traditional security operations and how threat hunting and analysis can support SecOps, as well as how SecOps can support threat hunting and incident response operations:

Figure 1.1 – The relationship between IT and cyber security

In the following chapters, we'll discuss several models, both industry-standard ones as well as my own, along with my thoughts on them, what their individual strengths and weaknesses are, and their applicability. It is important to remember that models and frameworks are just guides to help identify research and defensive prioritizations, incident response processes, and tools to describe campaigns, incidents, and events. Analysts and operators get into trouble when they try to use models as one-size-fits-all solutions that, in reality, are purely linear and inflexibly rigid.

The models and frameworks that we'll discuss are as follows:

• The Intelligence Pipeline
• The Lockheed Martin Kill Chain
• The MITRE ATT&CK Matrix
• The Diamond Model

Finally, we'll discuss how the models and frameworks are most impactful when they are chained together instead of being used independently.

The Intelligence Pipeline

Threat hunting is more than comparing provided indicators of compromise (IOCs) to collected data and finding a "known bad." Threat hunting relies on the application and analysis of data into information and then into intelligence – this is known as the Intelligence Pipeline. To process data through the pipeline, there are several proven analytical models that can be used to understand where an adversary is in their campaign, where they'll need to go next, and how to prioritize threat hunting resources (mainly, time) to disrupt or degrade an intrusion.

The Intelligence Pipeline isn't my invention. I first read about it in an extremely nerdy traditional intelligence-doctrine publication from the United States Joint Chiefs of Staff, JP 2-0 (https://www.jcs.mil/Portals/36/Documents/Doctrine/pubs/jp2_0.pdf). In this document, this process is referred to as the Relationship of Data, Information, and Intelligence process. However, as I've taken it out of that document and made some adjustments to fit my experiences and the cyber domain, I feel that the Intelligence Pipeline is more apt. It is the pipeline and process that you use to inform data-driven decisions:

Figure 1.2 – The Intelligence Pipeline

The idea of the pipeline is to introduce the theory that intelligence is made, and generally not provided. This is an anathema to vendors selling the product of actionable intelligence. I should note that selling data or information isn't wrong (in fact, it's really required in one form or another), but you should know precisely what you're getting – that is, data or information, not intelligence.

As illustrated, the operating environment is everything – your environment, the environment of your trust relationships, the environment of your MSSP, and so on. From here, events go through the following processes:

1. Events are collected and processed to turn them into data.
2. Context and enrichment are added to turn the dat...

Table of contents

• Threat Hunting with Elastic Stack
• Contributors
• Preface
• Section 1: Introduction to Threat Hunting, Analytical Models, and Hunting Methodologies
• Chapter 1: Introduction to Cyber Threat Intelligence, Analytical Models, and Frameworks
• Chapter 2: Hunting Concepts, Methodologies, and Techniques
• Section 2: Leveraging the Elastic Stack for Collection and Analysis
• Chapter 3: Introduction to the Elastic Stack
• Chapter 4: Building Your Hunting Lab – Part 1
• Chapter 5: Building Your Hunting Lab – Part 2
• Chapter 6: Data Collection with Beats and Elastic Agent
• Chapter 7: Using Kibana to Explore and Visualize Data
• Chapter 8: The Elastic Security App
• Section 3: Operationalizing Threat Hunting
• Chapter 9: Using Kibana to Pivot Through Data to Find Adversaries
• Chapter 10: Leveraging Hunting to Inform Operations
• Chapter 11: Enriching Data to Make Intelligence
• Chapter 12: Sharing Information and Analysis
• Assessments
• Other Books You May Enjoy