Python Forensics

A Workbench for Inventing and Sharing Digital Forensic Technology

Chet Hosmer

• Publisher: Syngress
• Year: 2014
• Print ISBN: 9780124186767
• eBook ISBN: 9780124186835
• Topic: Computer Science
• Subtopic: Cyber Security
• Index: Computer Science

Python Forensics provides many never-before-published proven forensic modules, libraries, and solutions that can be used right out of the box. In addition, detailed instruction and documentation provided with the code samples will allow even novice Python programmers to add their own unique twists or use the models presented to build new solutions.Rapid development of new cybercrime investigation tools is an essential ingredient in virtually every case and environment. Whether you are performing post-mortem investigation, executing live triage, extracting evidence from mobile devices or cloud services, or you are collecting and processing evidence from a network, Python forensic implementations can fill in the gaps.Drawing upon years of practical experience and using numerous examples and illustrative code samples, author Chet Hosmer discusses how to: - Develop new forensic solutions independent of large vendor software release schedules- Participate in an open-source workbench that facilitates direct involvement in the design and implementation of new methods that augment or replace existing tools- Advance your career by creating new solutions along with the construction of cutting-edge automation solutions to solve old problems- Provides hands-on tools, code samples, and detailed instruction and documentation that can be put to use immediately- Discusses how to create a Python forensics workbench- Covers effective forensic searching and indexing using Python- Shows how to use Python to examine mobile device operating systems: iOS, Android, and Windows 8- Presents complete coverage of how to use Python scripts for network investigation

Chapter 1

Why Python Forensics?

Abstract

This chapter sets the stage and expectations for the book. Specifically, this chapter addresses why Python is the right development environment to address the immediate challenges facing digital investigators. We address the gap that exists between computer and social science and how bringing these two very important groups together will serve as a catalyst for new innovations.

Keywords

Python

Forensics

Computer science

Social science

Daubert

Data

Semantics

Test-then code

The digital crime scene

Platform independent

Open source

Developer community

Chapter Contents

Introduction 1

Cybercrime Investigation Challenges 2

How can the Python Programming Environment Help Meet these Challenges? 6

Global Support for Python 6

Open Source and Platform Independence 8

Lifecycle Positioning 8

Cost and Barriers to Entry 8

Python and the Daubert Evidence Standard 9

Organization of the Book 10

Chapter Review 10

Summary Questions 11

Additional Resources 11

Introduction

The Python programming language and environment has proven to be easy to learn and use and is adaptable to virtually any domain or challenge problem. Companies like Google, Dropbox, Disney, Industrial Light and Magic, and YouTube just to mention a handful are using Python within their operations. Additionally, organizations like NASA’s Jet Propulsion Lab; the National Weather Service; The Swedish Meteorological and Hydrological Institute (SMHI); and Lawrence Livermore National Laboratories rely on Python to build models, make predictions, run experiments, and control critical operational systems.

Before diving straight in, I am sure you would like a little more information about what I will be covering and how a programming environment like Python matches up with digital investigations. Also, you might be interested to know what you will be learning about, generally what the scope of this book is, and how you can apply the concepts and practical examples presented.

The primary purpose and scope of the book is to show you how Python can be used to address problems and challenges within the cybercrime and digital investigation domain. I will be doing this by using real examples and providing the full source code along with detailed explanations. Thus the book will become a set of reference implementations, a cookbook of sorts, and at the end of the day, will hopefully get you involved in developing your own Python forensic applications.

I will be presenting the material without any preconceived notion about your programming expertise (or lack thereof). I only expect that you have an interest in using the examples in the book, expanding on them, or developing derivatives that will fit your situation and challenge problems. On the other hand, this is not a how to programming book, many of those exist for Python along with a plethora of online resources.

So, let us get started by defining just some of the challenges we face in cybercrime and digital investigation. These challenges after all were the catalyst behind the book and have come from the past two decades of working on solutions to assist law enforcement; defense and corporate entities collect and analyze digital evidence.

Cybercrime investigation challenges

Some of the challenge problems that we face in cybercrime investigation include:

The changing nature of investigations: Much of the work over the past two decades has focused on the postmortem acquisition, search, format, and display of information contained on various types of media. I can clearly remember the phone call I received almost two decades ago from Ron Stevens and Tom Hurbanek at the New York State Police. They were investigating a case that involved a Linux computer and were quite concerned about files and other data that might have been deleted that could be impeding the investigation. At that point no technology existed to extract deleted files or fragments that were buried away inside deleted Linux inodes, although several solutions existed for the Windows platform at the time. We worked together to develop algorithms that eventually became a tool named “extractor” that we provided free to law enforcement.

The move from simply extracting data, recovering deleted files, and scouring unallocated or slack space from computers has rapidly shifted just in the last couple of years. Today we focus most of our attention on smart mobile devices, dynamically changing memory, cloud applications, real-time network forensics, automotive data analysis, and weather-based forensics, just to mention a few. In addition, new work is addressing the association of direct digital forensic evidence with a broad range of instantly available electronic information. Whether this information comes from text messages, Facebook posts, tweets, Linkedin associations, metadata embedded in digital photographs or movies, GPS data that tracks our movements or the digital fingerprints left from every Web site we surf, all may be relevant and used in civil or criminal cases. The question is how do we connect these dots while maintaining forensic efficacy?

The widening gap between technology developers and investigators: Investigators, examiners, incident response personnel, auditors, compliance experts tend to come into this field with a background in social science, whereas technology developers tend to have backgrounds in computer science and engineering. Clearly, there are some excellent examples of crossovers in both directions, but the vocabulary, thought process, and approach to problem solving can be quite different. Our goal, as depicted in Figure 1.1, is to leverage Python forensic solutions to close that gap and create a collaborative nonthreatening environment whereby computer science and social science can come together.

Figure 1.1 Narrowing the gap.

The challenge is to develop a platform, vernacular environment where both social scientists and computer scientists can comfortably communicate and equally participate in the process of developing new forensic solutions. As you will see, the Python environment provides a level playing field, or common ground at least, where new innovations and thought can emerge. This has already shown to be true in other scientific fields like Space Flight, Meteorology, Hydrology, Simulation, Internet Technology advancement, and Experimentation. Python is already providing valuable contributions in these domains.

Cost and availability of new tools: With a couple of exceptions (for example, EnCase® App Central), most new innovations and capabilities that come through vendor channels take time to develop and can add significant cost to the investigator’s toolkit. In the past, investigators carried with them just a handful of hardware and software tools that they used to extract and preserve digital evidence. Today, to address the wide range of situations they may encounter, 30-40 software products may be necessary just to perform acquisition and rudimentary analysis of the digital crime scene. Of course this is just the start of the investigative process and the number and variety of analytic tools continues to grow.

The true cost and cost of ownership of these technologies can be staggering, especially when you factor in education and training. The barrier to entry into the field can easily reach high five or even six figures. This is in a field where backlogs continue to grow at law enforcement a...

Table of contents

• Cover image
• Title page
• Table of Contents
• Copyright
• Dedication
• Acknowledgments
• Endorsements
• List of figures
• About the Author
• About the Technical Editor
• Foreword
• Preface
• Chapter 1: Why Python Forensics?
• Chapter 2: Setting up a Python Forensics Environment
• Chapter 3: Our First Python Forensics App
• Chapter 4: Forensic Searching and Indexing Using Python
• Chapter 5: Forensic Evidence Extraction (JPEG and TIFF)
• Chapter 6: Forensic Time
• Chapter 7: Using Natural Language Tools in Forensics
• Chapter 8: Network Forensics: Part I
• Chapter 9: Network Forensics: Part II
• Chapter 10: Multiprocessing for Forensics
• Chapter 11: Rainbow in the Cloud
• Chapter 12: Looking Ahead
• Index