Hands-On System Programming with C++

Build performant and concurrent Unix and Linux systems with C++17

Dr. Rian Quinn

• Publisher: Packt Publishing
• Year: 2018
• ISBN:
  9781789131772
• Edition: 1
• Topic: Computer Science
• Subtopic: Programming in C++
• Index: Computer Science

A hands-on guide to making system programming with C++ easy

Key Features

• Write system-level code leveraging C++17
• Learn the internals of the Linux Application Binary Interface (ABI) and apply it to system programming
• Explore C++ concurrency to take advantage of server-level constructs

Book Description

C++ is a general-purpose programming language with a bias toward system programming as it provides ready access to hardware-level resources, efficient compilation, and a versatile approach to higher-level abstractions.

This book will help you understand the benefits of system programming with C++17. You will gain a firm understanding of various C, C++, and POSIX standards, as well as their respective system types for both C++ and POSIX. After a brief refresher on C++, Resource Acquisition Is Initialization (RAII), and the new C++ Guideline Support Library (GSL), you will learn to program Linux and Unix systems along with process management. As you progress through the chapters, you will become acquainted with C++'s support for IO. You will then study various memory management methods, including a chapter on allocators and how they benefit system programming. You will also explore how to program file input and output and learn about POSIX sockets. This book will help you get to grips with safely setting up a UDP and TCP server/client.

Finally, you will be guided through Unix time interfaces, multithreading, and error handling with C++ exceptions. By the end of this book, you will be comfortable with using C++ to program high-quality systems.

What you will learn

• Understand the benefits of using C++ for system programming
• Program Linux/Unix systems using C++
• Discover the advantages of Resource Acquisition Is Initialization (RAII)
• Program both console and file input and output
• Uncover the POSIX socket APIs and understand how to program them
• Explore advanced system programming topics, such as C++ allocators
• Use POSIX and C++ threads to program concurrent systems
• Grasp how C++ can be used to create performant system applications

Who this book is for

If you are a fresh developer with intermediate knowledge of C++ but little or no knowledge of Unix and Linux system programming, this book will help you learn system programming with C++ in a practical way.

Programming Linux/Unix Systems

The goal of this chapter is to explain the foundations of programming on Linux/Unix-based systems. This will provide a more complete picture of how a program executes on a Unix/Linux system, how to write more efficient code, and where to look when hard-to-find bugs arise.

To that end, this chapter starts by taking a comprehensive look at the Linux ABI, or more specifically, the System V ABI. In this section, we will review everything from the register and stack layout, to the System V calling conventions and ELF binary object specification.

The next section will briefly cover the Linux filesystem, including the standard layout and permissions. We will then provide a comprehensive review of Unix processes and how to program them, including considerations such as forking new processes and interprocess communication.

Finally, this chapter will conclude with a brief overview of Unix-based signals and how to work with them (both sending them and receiving them).

In this chapter, we will address the following:

• The Linux ABI
• The Unix filesystem
• Unix process APIs
• Unix signal APIs

Technical requirements

In order to follow the examples in this chapter, you must have the following:

• A Linux-based system capable of compiling and executing C++17 (for example, Ubuntu 17.10+)
• GCC 7+
• CMake 3.6+
• An internet connection

To download all the code in this chapter, including the examples and code snippets, go to the following link: https://github.com/PacktPublishing/Hands-On-System-Programming-with-CPP/tree/master/Chapter05.

The Linux ABI

In this section, we will discuss the Linux ABI (which is actually called the System V ABI), as well as the ELF standard and its use in Linux/Unix.

We will also dive into some of the details associated with ELF files, how to read and interpret them, and some of the implications of specific components within an ELF file.

The System V ABI

Unix System V was one of the first versions of Unix available, and largely defined Unix for years. Under the hood, System V leveraged the System V ABI. As Linux and BSD (Unix-like operating systems) became more widely used, the popularity of System V declined. However, the System V ABI remained popular, as operating systems such as Linux adopted this specification for Intel-based PCs.

In this chapter, we will focus on the System V ABI for Intel platforms on the Linux operating system. It should be noted, however, that other architectures and operating systems might use different ABIs. For example, ARM has its own ABI, which is largely based on System V (and, oddly, the Itanium 64 specification), but has several key differences.

The goal of this section is to expose the inner workings of a single Unix ABI, which in turn should make learning other ABIs easier, if needed.

Most of the specifications discussed in this chapter can be found at the following link: https://refspecs.linuxfoundation.org/.

The System V ABI defines most of the low-level details of a program (which in turn define the interfaces for system programming), including:

• The register layout
• The stack frame
• Function prologs and epilogs
• The calling convention (that is, parameter passing)
• Exception handling
• Virtual memory layout
• Debugging
• The binary object format (in this case, ELF)
• Program loading and linking

In Chapter 2, Learning the C, C++17, and POSIX Standards, we discussed the details of program linking and dynamic loading, and we devoted an entire section to the binary object format (ELF).

The following is a brief description of the remaining details of the System V specification, with respect to the Intel 64-bit architecture.

The register layout

For the purpose of keeping this topic simple, we will focus on Intel 64-bit. A whole book could be written on the different register layouts for each ABI, operating system, and architecture combination.

The Intel 64-bit architecture (which is usually referred to as AMD64, as AMD actually wrote it) defines several registers, of which a few have defined meanings within the instruction set.

The instruction pointer rip defines a program's current location in executable memory. Specifically, as a program executes, it executes from the location stored in rip, and each time an instruction is retired, rip advances to the next instruction.

The stack pointer and the base pointer (rsp and rbp respectively) are used to define the current location in the stack, as well as the location of the beginning of a stack frame (we will provide more information on this later).

The following are the remaining general-purpose registers. These have different meanings, which will be discussed in the rest of this section: rax, rbx, rcx, rdx, rdi, rsi, r8, r9, r10, r11, r12, r13, r14, and r15.

It should be noted before we continue that there are several other registers defined on the system that have very specific purposes, including floating-point registers and wide registers (which are used by special instructions designed to speed up certain types of calculations; for example, SSE and AVX). These are out of scope for the purpose of this discussion.

Finally, some of the registers end with letters, while others end with numbers, because versions of Intel's x86 processors only had letter-based registers, and the only true, general-purpose registers were AX, BX, CX, and DX.

When 64-bit was introduced by AMD, the number of general-purpose registers doubled, and to keep things simple, the register names were given numbers.

The stack frame

The stack frame is used to store the return address of each function, and to store function parameters and stack-based variables. It is a resource used heavily by all program, and it takes the following form:

high |----------| <- top of stack
 | |
 | Used |
 | |
 |----------| <- Current frame (rbp)
 | | <- Stack pointer (rsp)
 |----------|
 | |
 | Unused |
 | |
 low |----------|

The stack frame is nothing more than an array of memory that grows from top to bottom. That is to say, on an Intel PC, pushing to the stack subtracts from the stack pointer, while popping from the stack adds to the stack pointer—which means that memory actually grows down (assuming your view is that memory grows upward as an address increases, as in the previous diagram).

The System V ABI states that the stack is made up of stack frames. Each frame looks like the following:

high |----------| 
 | .... |
 |----------| 
 | arg8 | 
 |----------| 
 | arg7 | 
 |----------| 
 | ret addr | 
 |----------| <- Stack pointer (rbp)
 | |
 low |----------|

Each frame represents a function call, and starts with any arguments to a function beyond the first six being called (the first six arguments are passed as registers—this will be discussed in more detail later). Finally, the return address is pushed to the stack, and the function is called.

Memory after the return address belongs to variables that are scoped to the function itself. This is why we call variables defined in a function stack-based variables. The remaining stack is used by functions that will be called in the future. Each time one function calls another, the stack grows, while each time a function returns, the stack shrinks.

It is the job of the operating system to manage the size of the stack, ensuring that it always has enough memory. For example, if an application is trying to use too much memory, the operating system will kill the program.

Finally, it should be noted that on most CPU architectures, special instructions are provided that return from a function call and automatically pop the return address of the stack. In the case of Intel, the call instruction will jump to a function and push the current rip to the stack as the return address, and then ret will pop the return address from the stack and jump the address that was popped.

Function prologs and epilogs

Each function comes with a st...