Mastering Machine Learning for Penetration Testing

Develop an extensive skill set to break self-learning systems using Python

Chiheb Chebbi

• Publisher: Packt Publishing
• Year: 2018
• Print ISBN: 9781788997409
• eBook ISBN: 9781788993111
• Edition: 1
• Topic: Computer Science
• Subtopic: Computer Science General
• Index: Computer Science

Become a master at penetration testing using machine learning with Python

Key Features

• Identify ambiguities and breach intelligent security systems
• Perform unique cyber attacks to breach robust systems
• Learn to leverage machine learning algorithms

Book Description

Cyber security is crucial for both businesses and individuals. As systems are getting smarter, we now see machine learning interrupting computer security. With the adoption of machine learning in upcoming security products, it's important for pentesters and security researchers to understand how these systems work, and to breach them for testing purposes.

This book begins with the basics of machine learning and the algorithms used to build robust systems. Once you've gained a fair understanding of how security products leverage machine learning, you'll dive into the core concepts of breaching such systems. Through practical use cases, you'll see how to find loopholes and surpass a self-learning security system.

As you make your way through the chapters, you'll focus on topics such as network intrusion detection and AV and IDS evasion. We'll also cover the best practices when identifying ambiguities, and extensive techniques to breach an intelligent system.

By the end of this book, you will be well-versed with identifying loopholes in a self-learning security system and will be able to efficiently breach a machine learning system.

What you will learn

• Take an in-depth look at machine learning
• Get to know natural language processing (NLP)
• Understand malware feature engineering
• Build generative adversarial networks using Python libraries
• Work on threat hunting with machine learning and the ELK stack
• Explore the best practices for machine learning

Who this book is for

This book is for pen testers and security professionals who are interested in learning techniques to break an intelligent security system. Basic knowledge of Python is needed, but no prior knowledge of machine learning is necessary.

Introduction to Machine Learning in Pentesting

Currently, machine learning techniques are some of the hottest trends in information technology. They impact every aspect of our lives, and they affect every industry and field. Machine learning is a cyber weapon for information security professionals. In this book, readers will not only explore the fundamentals behind machine learning techniques, but will also learn the secrets to building a fully functional machine learning security system. We will not stop at building defensive layers; we will illustrate how to build offensive tools to attack and bypass security defenses. By the end of this book, you will be able to bypass machine learning security systems and use the models constructed in penetration testing (pentesting) missions.

In this chapter, we will cover:

• Machine learning models and algorithms
• Performance evaluation metrics
• Dimensionality reduction
• Ensemble learning
• Machine learning development environments and Python libraries
• Machine learning in penetration testing – promises and challenges

Technical requirements

In this chapter, we are going to build a development environment. Therefore, we are going to install the following Python machine learning libraries:

• NumPy
• SciPy

• TensorFlow
• Keras
• pandas
• MatplotLib
• scikit-learn
• NLTK
• Theano

You will also find all of the scripts and installation guides used in this GitHub repository: https://github.com/PacktPublishing/Mastering-Machine-Learning-for-Penetration-Testing/tree/master/Chapter01.

Artificial intelligence and machine learning

Making a machine think like a human is one of the oldest dreams. Machine learning techniques are used to help make predictions based on experiences and data.

Machine learning models and algorithms

In order to teach machines how to solve a large number of problems by themselves, we need to consider the different machine learning models. As you know, we need to feed the model with data; that is why machine learning models are divided, based on datasets entered (input), into four major categories: supervised learning, semi-supervised learning, unsupervised learning, and reinforcement. In this section, we are going to describe each model in a detailed way, in addition to exploring the most well-known algorithms used in every machine learning model. Before building machine learning systems, we need to know how things work underneath the surface.

Supervised

We talk about supervised machine learning when we have both the input variables and the output variables. In this case, we need to map the function (or pattern) between the two parties. The following are some of the most often used supervised machine learning algorithms.

Bayesian classifiers

According to the Cambridge English Dictionary, bias is the action of supporting or opposing a particular person or thing in an unfair way, allowing personal opinions to influence your judgment. Bayesian machine learning refers to having a prior belief, and updating it later by using data. Mathematically, it is based on the Bayes formula:

One of the simplest Bayesian problems is randomly tossing a coin and trying to predict whether the output will be heads or tails. That is why we can identify Bayesian methodology as being probabilistic. Naive Bayes is very useful when you are using a small amount of data.

Support vector machines

A support vector machine (SVM) is a supervised machine learning model that works by identifying a hyperplane between represented data. The data can be represented in a multidimensional space. Thus, SVMs are widely used in classification models. In an SVM, the hyperplane that best separates the different classes will be used. In some cases, when we have different hyperplanes that separate different classes, identification of the correct one will be performed thanks to something called a margin, or a gap. The margin is the nearest distance between the hyperplanes and the data positions. You can take a look at the following representation to check for the margin:

The hyperplane with the highest gap will be selected. If we choose the hyperplane with the shortest margin, we might face misclassification problems later. Don't be distracted by the previous graph; the hyperplane will not always be linear. Consider a case like the following:

In the preceding situation, we can add a new axis, called the z axis, and apply a transformation using a kernel trick...

Table of contents

• Title Page
• Copyright and Credits
• Dedication
• Packt Upsell
• Contributors
• Preface
• Introduction to Machine Learning in Pentesting
• Phishing Domain Detection
• Malware Detection with API Calls and PE Headers
• Malware Detection with Deep Learning
• Botnet Detection with Machine Learning
• Machine Learning in Anomaly Detection Systems
• Detecting Advanced Persistent Threats
• Evading Intrusion Detection Systems
• Bypassing Machine Learning Malware Detectors
• Best Practices for Machine Learning and Feature Engineering
• Assessments
• Other Books You May Enjoy