ArcPy and ArcGIS - Second Edition

Silas Toms, Dara O'Beirne

• Publisher: Packt Publishing
• Year: 2017
• ISBN:
  9781787280410
• Edition: 2
• Topic: Computer Science
• Subtopic: Cloud Computing
• Index: Computer Science

Use Python modules such as ArcPy, ArcREST and the ArcGIS API for Python to automate the analysis and mapping of geospatial data.About This Book• Perform GIS analysis faster by automating tasks.• Access the spatial data contained within shapefiles and geodatabases and transform between spatial reference systems.• Automate the mapping of geospatial analyses and production of map books.Who This Book Is ForIf you are a GIS student or professional who needs an understanding of how to use ArcPy to reduce repetitive tasks and perform analysis faster, this book is for you. It is also a valuable book for Python programmers who want to understand how to automate geospatial analyses and implement ArcGIS Online data management. What You Will Learn• Understand how to integrate Python into ArcGIS and make GIS analysis faster and easier.• Create Python script using ArcGIS ModelBuilder.• Learn to use ArcGIS online feature services and the basics of the ArcGIS REST API• Understand the unique Python environment that is new with ArcGIS Pro• Learn about the new ArcGIS Python API and how to use Anaconda and Jupyter with it• Learn to control ArcGIS Enterprise using ArcPyIn DetailArcGIS allows for complex analyses of geographic information. The ArcPy module is used to script these ArcGIS analyses, providing a productive way to perform geo-analyses and automate map production.The second edition of the book focuses on new Python tools, such as the ArcGIS API for Python. Using Python, this book will guide you from basic Python scripting to advanced ArcPy script tools.This book starts off with setting up your Python environment for ArcGIS automation. Then you will learn how to output maps using ArcPy in MXD and update feature class in a geodatabase using arcpy and ArcGIS Online. Next, you will be introduced to ArcREST library followed by examples on querying, updating and manipulating ArcGIS Online feature services. Further, you will be enabling your scripts in the browser and directly interacting with ArcGIS Online using Jupyter notebook. Finally, you can learn ways to use of ArcPy to control ArcGIS Enterprise and explore topics on deployments, data quality assurances, data updates, version control, and editing safeguards.By the end of the book, you will be equipped with the knowledge required to create automated analysis with administration reducing the time-consuming nature of GIS.Style and approachThe book takes a pragmatic approach, showing ways to automate repetitive tasks and utilizing features of ArcPy with ArcGIS Pro and ArcGIS online.

Introduction to Python for ArcGIS

In this chapter, we will discuss the development of Python as a programming language from its introduction in the late 1980s to its current state. We will discuss the creator of the language and the philosophy of design that spurred its development. We will also touch on important modules that will be used throughout the rest of the book, and especially focus on the modules built into the Python Standard Library. We will configure Python and the computer to execute Python scripts. The structure of the Python folder will be discussed, as will the location of the ArcPy module within the ArcGIS folder structure. We will also discuss Integrated Development Environments (IDEs)--programs designed to assist in code creation and code execution--comparing and contrasting existing IDEs to determine what benefits each IDE can offer when scripting Python code.

This chapter will cover the following topics:

• Quick Overview of Python: what it is and what it does, who created it, where is it now
• Important Python modules, both built-in and third party
• Python core concepts including data types, data containers, and looping
• The location of the Python interpreter, and how it is called to execute a script
• Adjusting the computer's environment variables to ensure correct code execution
• Integrated Development Environments (IDEs)
• Python's folder structure, and where the modules are stored

Python as a programming language

Over the last 40+ years of computer science, programming languages have developed from assembly and machine code towards high-level abstracted languages, which are much closer to English. The Python programming language, begun by Guido van Rossum in 1989, was designed to overcome issues that programmers were dealing with in the 1980s: slow development time, overly complicated syntax, and horrible readability. He developed Python as a language that would enable rapid code development and testing, have beautiful (or at least readable) syntax, and produce results with fewer lines of code, in less time. The first version of Python (0.9.0) was released in 1991, and it has always been free to download and use.

Go to https://www.python.org/ to explore Python documentation, try tutorials, get help, find useful Python code libraries, and download Python. Python has multiple major and minor versions. For much of the book, we are using Python 2.7, which is installed automatically along with ArcGIS for Desktop. For chapters on ArcGIS Pro, we will use Python 3.5.

Interpreted language

Python is an interpreted language. It is written in C, a compiled language, and the code is "interpreted" from Python into C before it is executed. Practically, this means that the code is executed as soon as it is converted and compiled. While the code interpretation can have speed implications for the execution of Python-based programs, this has very little real-world implications for its use with ArcGIS. Testing of code snippets is much faster in an interpretive environment, and it is perfect for creating scripts to automate basic, repeatable computing tasks.

Standard (built-in) library

Python, when installed, has a basic set of functions that are referred to as the built-in library. These built-in tools allow Python to perform string manipulations, math computations, HTTP calls, and URL parsing along with many other functions. Some of the tool libraries, or modules, are available as soon as Python is started, while others must be explicitly called using the "import" keyword to make their functions and classes available. Other modules have been developed by third parties, and can be downloaded and installed onto the Python installation as needed.

Glue language

Python is often called a "glue" language. This term describes the use of Python code to control other software programs by sending inputs to their Application Programming Interface (API) and collecting outputs, which are then sent to another program to repeat the process. A GIS example would be to use Python's urllib2 to download zipped shapefiles from a website, unzipping the files, processing the files using ArcToolbox, and compiling the results into an Excel spreadsheet. All of this is accomplished using freely available modules that are either included in Python's built-in library, or added on when ArcGIS is installed.

Wrapper modules

The ArcPy module is a "wrapper" module. It "wraps" a Python code interface over the existing ArcGIS tools and source code, allowing us to access these tools within our scripts. Wrapper modules are common in Python, and are so named because they "wrap" Python onto the tools we will need. They allow us to use Python to interface with programs written in C or other programming languages using the API of those programs. The wrapper module os allows for operating system operations.

For example, wrapper modules make it possible to extract data from an Excel spreadsheet, transform the data into another format such as a shapefile, and load it into an MXD as a layer. Not all modules are wrappers; some modules are written in "pure Python", and perform their analysis and computations using Python syntax. Either way, the end result is that a computer and its programs are available to be manipulated and controlled using Python.

The basics of Python programming

Python has a number of language requirements and conventions, which allow for the control of modules and the structuring of code. Following are a number of important basic concepts which will be used throughout this book, and when crafting scripts for use with geospatial analysis.

To test these examples, open the IDLE (Python GUI) program from the Start Menu/ArcGIS/Python2.7 folder after installing ArcGIS for Desktop. It has a built-in "interpreter" or code entry interface, indicated by the triple chevron >>> and a blinking cursor. To create a script in IDLE to save your code, click on to the File menu and then click New File. Save any script with a .py extension. Otherwise, just enter commands into the interpreter and push Enter to execute or add the next line.

Import statements

Import statements are used to augment the power of Python by calling other modules for use in the script. These modules can be a part of the standard Python library of modules, such as the math module (used to do higher mathematical calculations), or, importantly, can be like ArcPy, which will allow us to interact with ArcGIS. Import statements can be located anywhere before the module is used, but, by convention, they are located at the top of a script.

There are three ways to create an import statement. The first, and most standard, is to import the whole module as follows:

import arcpy

Using this method, we can even import more than one module on the same line. Next, we will import three modules: arcpy, os (the operating system module), and sys (the Python system module):

import arcpy, os, sys 

The next method of importing a script is to import a specific portion of a module instead of importing the entire module using the from <module> import <submodule> syntax:

from arcpy import mapping

This method is used when only a portion of the code from ArcPy is needed; it has the practical effect of limiting the amount of memory used by the module when it is called. We can also import multiple portions of the module in the same fashion.

from arcpy import mapping, da

The third way to import a module is to write the from <module> import <submodule> syntax, but use an asterisk * to import all parts of the module as follows:

from arcpy import *

This method is still used, but it is discouraged as it can have unknown effects--the main one is that the names of the variables in the module might conflict with another variable in another module. For this reason, it is best to avoid this third method. However, lots of existing scripts include import statements in this format, so it is good to know that it exists.

Variables

Variables are a part of all programming languages. They are used to reference data objects stored in memory for using later in a script. There are a lot of arguments over the best method of naming variables. No variable standard has been developed for Python scripting for ArcGIS, so I will describe some common practices to use when naming variables here:

1. Making them descriptive: Don't just name a variable, x; that variable will be useless later when the script is reviewed, and there is no way of knowing what it is used for, or why. They should be longer rather than shorter, and should explain what they do, or even what type of data they hold. For example:

shapefilePath = "C:/Data/shapefile.shp"

2. Using CamelCase to make the variable readable: Camel case is a term used for variables that start with a lowercase letter but have uppercase letters in the middle, resembling a camel's hump. For example:

camelCase = 'camel case is twoWords stuck together like this'

3. Using an underscore to separate parts of the name: This makes the name longer, but adds some clarity when reading the variable name, like this:

location_address = '100 Main St'

4. Including the data type in the variable name: If the variable contains a string, call it variableString or variable_string. This is not standard, and will not be used in this book, but it can help organize the script, and is helpful for others who will read these scripts. Python is dynamically typed instead of statically typed, a programming language distinction, which means that a variable does not have to be declared before it can be used, unlike Visual Basic or other statically typed languages. For example:

variableString = 'this is a string'

For loops

Built into all programming languages is the ability to iterate over a dataset to perform an operation on the data, thus transforming the data or extracting data that meets specific criteria. The dataset must be iterable to be used in a for loop. We will ...