Mastering Python Networking

Your one-stop solution to using Python for network automation, DevOps, and Test-Driven Development, 2nd Edition

Eric Chou

• Verlag: Packt Publishing
• Jahr: 2018
• ISBN:
  9781789130263
• Thema: Computer Science
• Thema: Computer Networking

Key Features

• Explore the power of Python libraries to tackle difficult network problems efficiently and effectively
• Use Python for network device automation, DevOps, and software-defined networking
• Become an expert in implementing advanced network-related tasks with Python

Book Description

Networks in your infrastructure set the foundation for how your application can be deployed, maintained, and serviced. Python is the ideal language for network engineers to explore tools that were previously available to systems engineers and application developers. In this second edition of Mastering Python Networking, you'll embark on a Python-based journey to transition from traditional network engineers to network developers ready for the next-generation of networks. This book begins by reviewing the basics of Python and teaches you how Python can interact with both legacy and API-enabled network devices. As you make your way through the chapters, you will then learn to leverage high-level Python packages and frameworks to perform network engineering tasks for automation, monitoring, management, and enhanced security. In the concluding chapters, you will use Jenkins for continuous network integration as well as testing tools to verify your network. By the end of this book, you will be able to perform all networking tasks with ease using Python.

What you will learn

• Use Python libraries to interact with your network
• Integrate Ansible 2.5 using Python to control Cisco, Juniper, and Arista eAPI network devices
• Leverage existing frameworks to construct high-level APIs
• Learn how to build virtual networks in the AWS Cloud
• Understand how Jenkins can be used to automatically deploy changes in your network
• Use PyTest and Unittest for Test-Driven Network Development

Who this book is for

Mastering Python Networking is for network engineers and programmers who want to use Python for networking. Basic familiarity with Python programming and networking-related concepts such as Transmission Control Protocol/Internet Protocol (TCP/IP) will be useful.

]]>

Network Monitoring with Python – Part 1

Imagine you get a call at 2:00 a.m. in the morning. The person on the other end says: "Hi, we are facing a difficult issue that is impacting production services. We suspect it might be network-related. Can you check this for us? For this type of urgent, open-ended question, what would be the first thing you do?" Most of the time, the thing that comes to mind would be: What changed between the time when the network was working until something went wrong? Chances are you would check your monitoring tool and see if any of the key metrics changed in the last few hours. Better yet is if you have received any monitoring alerts from any metric baseline deviation.

Throughout this book, we have been discussing various ways to systematically make predictable changes to our network, with the goal of keeping the network running as smoothly as possible. However, networks are not static – far from it – they are probably one of the most fluid parts of the entire infrastructure. By definition, a network connects different parts of the infrastructure together, constantly passing traffic back and forth. There are lots of moving parts that can cause your network to stop working as expected: hardware failures, software with bugs, human mistakes despite their best intentions, and many more. It is not a question of whether things would go wrong, but rather a question of when and what went wrong when it happens. We need ways to monitor our network to make sure it works as expected and hopefully be notified when it does not.

In upcoming two chapters, we will look at various ways to perform network monitoring tasks. Many of the tools we have looked at thus far can be tied together or directly managed by Python. Like many tools we have looked at, network monitoring has to do with two parts. First, we need to know what information the equipment is capable of transmitting. Second, we need to identify what useful information we can interpret from them.

We will look at a few tools that allow us to monitor the network effectively:

• The Simple Network Management Protocol (SNMP)
• Matplotlib and Pygal visualization
• MRTG and Cacti

This list is not exhaustive, and there is certainly no lack of commercial vendors in the network monitoring space. The basics of network monitoring that we will look at, however, carry well for both open source and commercial tools.

Lab setup

The lab for this chapter is similar to the one in Chapter 6, Network Security with Python, but with this difference: both of the network devices are IOSv devices. Here's an illustration of this:

The two Ubuntu hosts will be used to generate traffic across the network so that we can look at some non-zero counters.

SNMP

SNMP is a standardized protocol used to collect and manage devices. Although the standard allows you to use SNMP for device management, in my experience, most network administrators prefer to keep SNMP as an information collection mechanism only. Since SNMP operates on UDP, which is connectionless, and considering the relatively weak security mechanism in versions 1 and 2, making device changes via SNMP tends to make network operators a bit uneasy. SNMP version 3 has added cryptographic security and new concepts and terminologies to the protocol, but the way the technology is adapted varies among network device vendors.

SNMP is widely used in network monitoring and has been around since 1988 as part of RFC 1065. The operations are straightforward, with the network manager sending GET and SET requests toward the device and the device with the SNMP agent responding with the information per request. The most widely adopted standard is SNMPv2c, which is defined in RFC 1901 – RFC 1908. It uses a simple community-based security scheme for security. It has also introduced new features, such as the ability to get bulk information. The following diagram displays the high-level operation for SNMP:

SNMP operations

The information residing in the device is structured in the Management Information Base (MIB). The MIB uses a hierarchical namespace containing an Object Identifier (OID), which represents the information that can be read and fed back to the requester. When we talk about using SNMP to query device information, we are really talking about using the management station to query the specific OID that represents the information we are after. There is a common OID structure, such as systems and interfaces OID, that is shared among vendors. Besides common OID, each vendor can also supply an enterprise-level OID that is specific to them.

As an operator, you are required to put some effort into consolidating information into an OID structure in your environment to retrieve useful information. This can sometimes be a tedious process of finding one OID at a time. For example, you might be making a request to a device OID and receive a value of 10,000. What is that value? Is that interface traffic? Is it in bytes or bits? Or maybe it is a number of packets? How do we know? We will need to consult either the standard or the vendor documentation to find out. There are tools that help with this process, such as a MIB browser that can provide more metadata to the value. But, at least in my experience, constructing an SNMP-based monitoring tool for your network can sometimes feel like a cat-and-mouse game of trying to find that one missing value.

Some of the main points to take away from the operation are as follows:

• The implementation relies heavily on the amount of information the device agent can provide. This, in turn, relies on how the vendor treats SNMP: as a core feature or an added feature.
• SNMP agents generally require CPU cycles from the control plane to return a value. Not only is this inefficient for devices with, say, large BGP tables, it is also not feasible to use SNMP to query the data at small intervals.
• The user needs to know the OID in order to query the data.

Since SNMP has been around for a while, my assumption is that you have some experience with it already. Let's jump directly into package installation and our first SNMP example.

Setup

First, let's make sure that we have the SNMP managing device and agent work in our setup. The SNMP bundle can be installed on either the hosts (client or server) in our lab or the managing device on the management network. As long as the SNMP manager has IP reachability to the device and the managed device allows the inbound connection, SNMP should work. In production, you should only install the software on the management host and only allow SNMP traffic in the control plane.

In this lab, we have installed SNMP on both the Ubuntu host on the management network and the client host in the lab to test security:

$ ...