Mastering Rust

Learn about memory safety, type system, concurrency, and the new features of Rust 2018 edition, 2nd Edition

Rahul Sharma, Vesa Kaihlavirta

• Publisher: Packt Publishing
• Year: 2019
• ISBN:
  9781789341188
• Edition: 2
• Topic: Computer Science
• Subtopic: Programming Languages
• Index: Computer Science

Become proficient in designing, developing and deploying effective software systems using the advanced constructs of Rust

Key Features

• Improve your productivity using the latest version of Rust and write simpler and easier code
• Understand Rust's immutability and ownership principle, expressive type system, safe concurrency
• Deep dive into the new doamins of Rust like WebAssembly, Networking and Command line tools

Book Description

Rust is an empowering language that provides a rare combination of safety, speed, and zero-cost abstractions. Mastering Rust – Second Edition is filled with clear and simple explanations of the language features along with real-world examples, showing you how you can build robust, scalable, and reliable programs.

This second edition of the book improves upon the previous one and touches on all aspects that make Rust a great language. We have included the features from latest Rust 2018 edition such as the new module system, the smarter compiler, helpful error messages, and the stable procedural macros. You'll learn how Rust can be used for systems programming, network programming, and even on the web. You'll also learn techniques such as writing memory-safe code, building idiomatic Rust libraries, writing efficient asynchronous networking code, and advanced macros. The book contains a mix of theory and hands-on tasks so you acquire the skills as well as the knowledge, and it also provides exercises to hammer the concepts in.

After reading this book, you will be able to implement Rust for your enterprise projects, write better tests and documentation, design for performance, and write idiomatic Rust code.

What you will learn

• Write generic and type-safe code by using Rust's powerful type system
• How memory safety works without garbage collection
• Know the different strategies in error handling and when to use them
• Learn how to use concurrency primitives such as threads and channels
• Use advanced macros to reduce boilerplate code
• Create efficient web applications with the Actix-web framework
• Use Diesel for type-safe database interactions in your web application

Who this book is for

The book is aimed at beginner and intermediate programmers who already have familiarity with any imperative language and have only heard of Rust as a new language. If you are a developer who wants to write robust, efficient and maintainable software systems and want to become proficient with Rust, this book is for you. It starts by giving a whirlwind tour of the important concepts of Rust and covers advanced features of the language in subsequent chapters using code examples that readers will find useful to advance their knowledge.

Memory Management and Safety

Memory management is a fundamental concept to understand for anyone working with a low-level programming language. Low-level languages don't come with automatic memory reclamation solutions like a built-in garbage collector, and it's the responsibility of the programmer to manage memory that's used by the program. Having knowledge of where and how memory gets used in a program enables programmers to build efficient and safe software systems. A lot of bugs in low-level software are due to improper handling of memory. At times, it's the programmer's mistake. The other times, it's the side effect of the programming language used, such as C and C++, which are infamous for a lot of memory vulnerability reports in software. Rust offers a better, compile-time solution to memory management. It makes it hard to write software that leaks memory unless you explicitly intend to! Programmers who have done a fair amount of development with Rust eventually come to the realization that it discourages bad programming practices and directs the programmer toward writing software that uses memory safely and efficiently.

In this chapter, we go into the nitty-gritty details of how Rust tames the memory that's used by resources in a program. We'll give a brief introduction to processes, memory allocation, memory management, and what we mean by memory safety. Then, we'll go through the memory safety model provided by Rust and understand the concepts that enable it to track memory usage at compile time. We'll see how traits are used to control where types reside in memory and when they get freed. We'll also delve into various smart pointer types that provide abstractions to manage resources in the program.

The topics that are covered in this chapter are as follows:

• Programs and memory
• Memory allocations and safety
• Memory management
• Stack and Heap
• Trifecta of safety—Ownership, borrowing, and lifetimes
• Smart pointer types

Programs and memory

"If you’re willing to restrict the flexibility of your approach, you can almost always do something better."

– John Carmack

As a motivation to understand memory and its management, it's important for us to have a general idea of how programs are run by the operating system and what mechanisms are in place that allow it to use memory for its requirements.

Every program needs memory to run, whether it's your favorite command-line tool or a complex stream processing service, and they have vastly different memory requirements. In major operating system implementations, a program in execution is implemented as a process. A process is a running instance of a program. When we execute ./my_program in a shell in Linux or double-click on my_program.exe on Windows, the OS loads my_program as a process in memory and starts executing it, along with other processes, giving it a share of CPU and memory. It assigns the process with its own virtual address space, which is distinct from the virtual address space of other processes and has its own view of memory.

During the lifetime of a process, it uses many system resources. First, it needs memory to store its own instructions, then it needs space for resources that are demanded at runtime during instruction execution, then it needs a way to keep track of function calls, any local variables, and the address to return to after the last invoked function. Some of these memory requirements can be decided ahead at compile time, like storing a primitive type in a variable, while others can only be satisfied at runtime, like creating a dynamic data type such as Vec<String>. Due to the various tiers of memory requirements, and also for security purposes, a process's view of memory is divided into regions known as the memory layout.

Here, we have an approximate representation of the memory layout of a process in general:

This layout is divided into various regions based on the kind of data they store and the functionality they provide. The major parts we are concerned with are as follows:

• Text segment: This section contains the actual code to be executed in the compiled binary. The text segment is a read-only segment and any user code is forbidden to modify it. Doing so can result in a crash of the program.
• Data segment: This is further divided into subsections, that is, the initialized data segment and uninitialized data segment, which is historically known as Block Started by Symbol (BSS), and holds all global and static values declared in the program. Uninitialized values are initialized to zero when they are loaded into memory.
• Stack segment: This segment is used to hold any local variables and the return addresses of functions. All resources whose sizes are known in advance and any temporary/intermediary variables that a program creates are implicitly stored on the stack.
• Heap segment: This segment is used to store any dynamically allocated data whose size is not known up front and can change at runtime depending on the needs of the program. This is the ideal allocation place when we want values to outlive their declaration within a function.

How do programs use memory?

So, we know that a process has a chunk of memory dedicated for its execution. But, how does it access this memory to perform its task? For security purposes and fault isolation, a process is not allowed to access the physical memory directly. Instead, it uses a virtual memory, which is mapped to the actual physical memory by the OS using an in-memory data structure called pages, which are maintained in page tables. The process has to request memory from the OS for its use, and what it gets is a virtual address that is internally mapped to a physical address in the RAM. For performance reasons, this memory is requested and processed in chunks. When virtual memory is accessed by the process, the memory management unit does the actual conversion from virtual to physical memory.

The whole series of steps through which memory is acquired by a process from the OS is known as memory allocation. A process requests a chunk of memory from the OS by using system calls, and the OS marks that chunk of memory in use by that process. When the process is done using the memory, it has to mark the memory as free so other processes can use it. This is called de-allocation of memory. Major operating system implementations provide abstractions through system calls (such as brk and sbrk in Linux), which are functions that talk directly to the OS kernel and can allocate memory requested by the process. But these kernel-level functions are very low-level, so they are further abstracted by system libraries such as the glibc library, which is C's standard library in Linux including the implementation of the POSIX APIs, facilitating low-level interactions with the OS from the C language.

POSIX is an acronym for Portable Operating System Interface, a term coind by Richard Stallman. It is a set of standards that emerged with the need to standardize what functionality, a Unix-like operating system should provide, what low level APIs they should expose to languages such as C, what command-line utilities they should include, and many other aspects.

Glibc also provides a memory allocator API, exposing functions such as malloc, calloc, and realloc for allocating memory and the free function for de-allocating memory. Even though we have a fairly high-level API for allocating/de-allocating memory, we still have to manage memory ourselves when using low-level programming languages.

Memory management and its kinds

The RAM in your computer is a limited resource and is shared by all running programs. It's a necessity that when a program is done executing its instructions, it is expected to release any memory used so that the OS can reclaim it and hand it to other processes. When we talk about memory management, a prominent aspect we care about is the reclamation of used memory and how that happens. The level of management required in deallocating used memory is different in different languages. Up until the mid-1990s, the majority of programming languages relied on manual memory management, which required the programmer to call memory allocator APIs such as malloc and free in code to allocate and deallocate memory, respectively. Around 1959, John McCarthy, the creator of Lisp, invented Garbage Collectors (GC), a form of automatic memory management and Lisp was the first language to use one. A GC runs as a daemon thread as part of the running program and analyzes the memory that is no longer being referenced by any va...