Python for Secret Agents

Steven F. Lott

• Editorial: Packt Publishing
• Año: 2014
• ISBN:
  9781783980420
• Edición: 1
• Categoría: Informatica
• Categoría: Programmazione open source

Python is an easy-to-learn and extensible programming language that allows secret agents to work with a wide variety of data in a number of ways. It gives beginners a simple way to start programming, but Python's standard library also provides numerous packages that allow Python-using secret agents to easily utilize very sophisticated information processing.

This book will guide new field agent trainees through putting together a Python-based toolset to gather, analyze, and communicate data. It starts by covering the basics and then moves on to sections such as file exchange, image processing, geocoding, simple trigonometry, and more sensitive statistical processing. You will then learn how to use polynomials to encode and decode data in different representations. Furthermore, this book shows you how to add tools to a Python environment, work with images, and parse HTML web pages to extract meaningful data. The idea of adding packages to Python is central to how an agent will leverage these tools for data processing.

Python for Secret Agents

---

Table of Contents

Python for Secret Agents

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Our Espionage Toolkit

Getting the tools of the trade – Python 3.3

Windows secrets

Mac OS X secrets

Getting more tools – a text editor

Getting other developer tools

Getting a tool to get more Python components

Confirming our tools

How do we stop?

Using the help() system

Mac OS and GNU/Linux secrets

Windows secrets

Using the help mode

Background briefing – math and numbers

The usual culprits

The ivory tower of numbers

Integer numbers

Rational numbers

Floating-point numbers

Decimal numbers

Complex numbers

Outside the numbers

Assigning values to variables

Writing scripts and seeing output

Gathering user input

Handling exceptions

Looping and trying again

Handling text and strings

Converting between numbers and strings

Parsing strings

Organizing our software

Working with files and folders

Creating a file

Reading a file

Defining more complex logical conditions

Solving problems – recovering a lost password

Reading a word corpus

Reading a ZIP archive

Using brute-force search

Summary

2. Acquiring Intelligence Data

Accessing data from the Internet

Background briefing – the TCP/IP protocols

Using http.client for HTTP GET

Changing our client information

Using FTP in Python

Downloading a file via FTP

Using our FTP get() function

Using urllib for HTTP, FTP, or file access

Using urllib for FTP access

Using a REST API in Python

Getting simple REST data

Using more complex RESTful queries

Saving our data via JSON

Organizing collections of data

Using a Python list

Using list index operations

Using a Python tuple

Using generator expressions with list of tuples

Using a Python dictionary mapping

Using the dictionary access methods

Transforming sequences with generator functions

Using the defaultdict and counter mappings

Using a Python set

Using the for statement with a collection

Using Python operators on collections

Solving problems – currency conversion rates

Summary

3. Encoding Secret Messages with Steganography

Background briefing – handling file formats

Working with the OS filesystem

glob

os

Processing simple text files

Working with ZIP files

Working with JSON files

Working with CSV files

JPEG and PNG graphics – pixels and metadata

Using the Pillow library

Adding the required supporting libraries

GNU/Linux secrets

Mac OS X secrets

Windows secrets

Installing and confirming Pillow

Decoding and encoding image data

Manipulating images – resizing and thumbnails

Manipulating images – cropping

Manipulating images – enhancing

Manipulating images – filtering

Manipulating images – ImageOps

Some approaches to steganography

Getting the red-channel data

Extracting bytes from Unicode characters

Manipulating bits and bytes

Assembling the bits

Encoding the message

Decoding a message

Detecting and preventing tampering

Using hash totals to validate a file

Using a key with a digest

Solving problems – encrypting a message

Unpacking a message

Summary

4. Drops, Hideouts, Meetups, and Lairs

Background briefing – latitude, longitude, and GPS

Coping with GPS device limitations

Handling politics – borders, precincts, jurisdictions, and neighborhoods

Finding out where we are with geocoding services

Geocoding an address

Reverse geocoding a latitude-longitude point

How close? What direction?

Combining geocoding and haversine

Compressing data to make grid codes

Creating GeoRef codes

Decoding a GeoRef code

Creating Maidenhead grid codes

Decoding the Maidenhead grid codes

Creating natural area codes

Decoding natural area codes

Solving problems – closest good restaurant

Creating simple Python objects

Working with HTML web services – tools

Working with HTML web services – getting the page

Working with HTML web services – parsing a table

Making a simple Python object from columns of data

Enriching Python objects with geocodes

Enriching Python objects with health scores

Combining the pieces and parts

Working with clean data portals

Making a simple Python object from a JSON document

Combining different pieces and parts

Final steps

Understanding the data – schema and metadata

Summary

5. A Spymaster's More Sensitive Analyses

Creating statistical summaries

Parsing the raw data file

Finding an average value

Understanding generator expressions

Finding the value in the middle

Finding the most popular value

Creating Python modules and applications

Creating and using a module

Creating an application module

Creating a hybrid module

Creating our own classes of objects

Using a class definition

Comparisons and correlations

Computing the standard deviation

Computing a standardized score

Comparing a sequence and an iterable

Computing a coefficient of correlation

Writing high-quality software

Building a self-testing module and a test module

Creating more sophisticated tests

Adding doctest cases to a class definition

Solving problems – analyzing some interesting datasets

Getting some more data

Further research

Summary

Index

Python for Secret Agents

Copyright © 2014 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: August 2014

Production reference: 1190814

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78398-042-0

www.packtpub.com

Cover image by Jarek Blaminsky (<[email protected]>)

Credits

Author

Steven F. Lott

Reviewers

Nick Glynn

Steven Heinrich

Sanjeev Jaiswal

Maurice HT Ling

Acquisition Editor

Sam Wood

Content Development Editor

Prachi Bisht

Technical Editors

Veena Pagare

Anand Singh

Copy Editors

Insiya Morbiwala

Say...