Hands-On Concurrency with Rust
eBook - ePub

Hands-On Concurrency with Rust

Confidently build memory-safe, parallel, and efficient software in Rust

Brian L. Troutwine

Share book
  1. 462 pages
  2. English
  3. ePUB (mobile friendly)
  4. Available on iOS & Android
eBook - ePub

Hands-On Concurrency with Rust

Confidently build memory-safe, parallel, and efficient software in Rust

Brian L. Troutwine

Book details
Book preview
Table of contents
Citations

About This Book

Get to grips with modern software demands by learning the effective uses of Rust's powerful memory safety.About This Bookā€¢ Learn and improve the sequential performance characteristics of your softwareā€¢ Understand the use of operating system processes in a high-scale concurrent systemā€¢ Learn of the various coordination methods available in the Standard libraryWho This Book Is ForThis book is aimed at software engineers with a basic understanding of Rust who want to exploit the parallel and concurrent nature of modern computing environments, safely.What You Will Learnā€¢ Probe your programs for performance and accuracy issuesā€¢ Create your own threading and multi-processing environment in Rustā€¢ Use coarse locks from Rust's Standard libraryā€¢ Solve common synchronization problems or avoid synchronization using atomic programmingā€¢ Build lock-free/wait-free structures in Rust and understand their implementations in the crates ecosystemā€¢ Leverage Rust's memory model and type system to build safety properties into your parallel programsā€¢ Understand the new features of the Rust programming language to ease the writing of parallel programsIn DetailMost programming languages can really complicate things, especially with regard to unsafe memory access. The burden on you, the programmer, lies across two domains: understanding the modern machine and your language's pain-points. This book will teach you to how to manage program performance on modern machines and build fast, memory-safe, and concurrent software in Rust. It starts with the fundamentals of Rust and discusses machine architecture concepts. You will be taken through ways to measure and improve the performance of Rust code systematically and how to write collections with confidence. You will learn about the Sync and Send traits applied to threads, and coordinate thread execution with locks, atomic primitives, data-parallelism, and more.The book will show you how to efficiently embed Rust in C++ code and explore the functionalities of various crates for multithreaded applications. It explores implementations in depth. You will know how a mutex works and build several yourself. You will master radically different approaches that exist in the ecosystem for structuring and managing high-scale systems.By the end of the book, you will feel comfortable with designing safe, consistent, parallel, and high-performance applications in Rust.Style and approachReaders will be taken through various ways to improve the performance of their Rust code.

Frequently asked questions

How do I cancel my subscription?
Simply head over to the account section in settings and click on ā€œCancel Subscriptionā€ - itā€™s as simple as that. After you cancel, your membership will stay active for the remainder of the time youā€™ve paid for. Learn more here.
Can/how do I download books?
At the moment all of our mobile-responsive ePub books are available to download via the app. Most of our PDFs are also available to download and we're working on making the final remaining ones downloadable now. Learn more here.
What is the difference between the pricing plans?
Both plans give you full access to the library and all of Perlegoā€™s features. The only differences are the price and subscription period: With the annual plan youā€™ll save around 30% compared to 12 months on the monthly plan.
What is Perlego?
We are an online textbook subscription service, where you can get access to an entire online library for less than the price of a single book per month. With over 1 million books across 1000+ topics, weā€™ve got you covered! Learn more here.
Do you support text-to-speech?
Look out for the read-aloud symbol on your next book to see if you can listen to it. The read-aloud tool reads text aloud for you, highlighting the text as it is being read. You can pause it, speed it up and slow it down. Learn more here.
Is Hands-On Concurrency with Rust an online PDF/ePUB?
Yes, you can access Hands-On Concurrency with Rust by Brian L. Troutwine in PDF and/or ePUB format, as well as other popular books in Computer Science & Programming in C++. We have over one million books available in our catalogue for you to explore.

Information

Year
2018
ISBN
9781788478359

High-Level Parallelism ā€“ Threadpools, Parallel Iterators and Processes

In previous chapters, we introduced the basic mechanisms of concurrency in the Rustā€”programming language. In Chapter 4, Sync and Send ā€“ the Foundation of Rust Concurrency, we discussed the interplay of the type system of Rust with concurrent programs, how Rust ensures memory safety in this most difficult of circumstances. In Chapter 5, Locks ā€“ Mutex, Condvar, Barriers and RWLock, we discussed the higher, so-called coarse, synchronization mechanisms available to us, common among many languages. In Chapter 6, Atomics ā€“ the Primitives of Synchronization, and Chapter 7, Atomics ā€“ Safely Reclaiming Memory, we discussed the finer synchronization primitives available on modern machines, exposed through Rust's concurrent memory model. This has all been well and good but, though we've done deep-dives into select libraries or data structures, we have yet to see the consequences of all of these tools on the structure of programs, or how you might choose to split up your workloads across CPUs depending on need.
In this chapter, we will explore higher-level techniques for exploiting concurrent machines without dipping into manual locking or atomic synchronization. We'll examine thread pooling, a technique common in other programming languages, data parallelism with the rayon library, and demonstrate multiprocessing in the context of a genetic programming project that will carry us into the next chapter, as well.
By the end of this chapter, we will have:
  • Explored the implementation of thread pool
  • Understood how thread pooling relates to the operation of rayon
  • Explored rayon's internal mechanism in-depth
  • Demonstrated the use of rayon in a non-trivial exercise

Technical requirements

This chapter requires a working Rust installation. The details of verifying your installation are covered in Chapter 1, Preliminaries ā€“ Machine Architecture and Getting Started with Rust. A pMARS executable is required and must be on your PATH. Please follow the instructions in the pMARS (http://www.koth.org/pmars/) source tree for building instructions.
You can find the source code for this book's projects on GitHub: https://github.com/PacktPublishing/Hands-On-Concurrency-with-Rust. The source code for this chapter is under Chapter08.

Thread pooling

To this point in the book, whenever we have needed a thread, we've simply called thread::spawn. This is not necessarily a safe thing to do. Let's inspect two projects that suffer from a common defectā€”potential over-consumption of OS threads. The first will be obviously deficient, the second less so.

Slowloris ā€“ attacking thread-per-connection servers

The thread-per-connection architecture is a networking architecture that allocates one OS thread per inbound connection. This works well for servers that will receive a total number of connections relatively similar to the number of CPUs available to the server. Depending on the operating system, this architecture tends to reduce time-to-response latency of network transactions, as a nice bonus. The major defect with thread-per-connection systems has to do with slowloris attacks. In this style of attack, a malicious user opens a connection to the serverā€“a relatively cheap operation, requiring only a single file-handler and simply holds it. Thread-per-connection systems can mitigate the slowloris attack by aggressively timing out idle connections, which the attacker can then mitigate by sending data through the connection at a very slow rate, say one byte per 100 milliseconds. The attacker, which could easily be just buggy software if you're deploying solely inside a trusted network, spends very little resources to carry out their attack. Meanwhile, for every new connection into the system, the server is forced to allocate a full stack frame and forced to cram another thread into the OS scheduler. That's not cheap.

The server

Let's put together a vulnerable server, then blow it up. Lay out your Cargo.toml like so:
[package] name = "overwhelmed_tcp_server" version = "0.1.0" authors = ["Brian L. Troutwine <[email protected]>"] [dependencies] clap = "2.31" slog = "2.2" slog-term = "2.4" slog-async = "2.3" [[bin]] name = "server" [[bin]] name = "client"
The library slog (https://crates.io/crates/slog) is a structured logging library that I highly recommend in production systems. Slog itself is very flexible, being more of a framework for composing a logging system rather than a set thing. Here we'll be logging to terminal, which is what slog-term is for. The slog-async dependency defers actual writing to the terminal, in our case, to a dedicated thread. This dedication is more important when slog is emitting logs over a network, rather than quickly to terminal. We won't go in-depth on slog in this book but the documentation is comprehensive and I imagine you'll appreciate slog if you aren't already using it. The library clap (https://crates.io/crates/clap) is a command-line parsing library, which I also highly recommend for use in production systems. Note, finally, that we produce two binaries in this project, called server and client. Let's dig into server first. As usual, our projects start with a preamble:
#[macro_use] extern crate slog; extern crate clap; extern crate slog_async; extern crate slog_term; use clap::{App, Arg}; use slog::Drain; use std::io::{self, BufRead, BufReader, BufWriter, Write}; use std::net::{TcpListener, TcpStream};
use std::sync::atomic::{AtomicUsize, Ordering}; use std::thread; static TOTAL_STREAMS: AtomicUsize = AtomicUsize::new(0);
By this point in the book, there's nothing surprising here. We import the external libraries we've just discussed as well as a host of standard library material. The networking-related imports are unfamiliar, with regard to the previous content of this book, but we'll go into that briefly below. We finally create a static TOTAL_STREAMS at the top of the program that the server will use to track the total number of TCP streams it has connected. The main function starts off setting up slog:
fn main() { let decorator = slog_term::TermDecorator::new().build(); let drain = slog_term::CompactFormat::new(decorator).build().fuse(); let drain = slog_async::Async::new(drain).build().fuse(); let root = slog::Logger::root(drain, o!());
The exact details here are left to the reader to discover in slog's documentation. Next, we set up the clap and parse program arguments:
 let matches = App::new("server") .arg( Arg::with_name("host") .long("host") .value_name("HOST") .help("Sets which hostname to listen on") .takes_value(true), ) .arg( Arg::with_name("port") .long("port") .value_name("PORT") .help("Sets which port to listen on") .takes_value(true), ) .get_matches(); let host: &str = matches.value_of("host").unwrap_or("localhost"); let port = matches .value_of("port") .unwrap_or("1987") .parse::<u16>() .expect("port-no not valid");
The details here are also left to the reader to discover in clap's documentation, but hopefully the intent is clear enough. We're setting up two arguments, host and port, which the server will listen for connections on. Speaking of:
 let listener = TcpListener::bind((host, port)).unwrap();
Note that we're unwrapping if it's not possible for the server to establish a connection. This is user-hostile and in a production-worthy application you should match on the error and print out a nice, helpful message to explain the error. Now that the server is listening for new connections, we make a server-specific logger and start handling incoming connections:
 let server = root.new(o!("host" => host.to_string(),
"port" => port)); info!(server, "Server open for business! :D"); let mut joins = Vec::new(); for stream in listener.incoming() { if let Ok(stream) = stream { let stream_no = TOTAL_STREAMS.fetch_add(1,
Ordering::Relaxed); let log = root.new(o!("stream-no" => stream_no, "peer-addr" => stream.peer_addr()
.expect("no peer address")
.to_string())); let writer = BufWriter::new(
stream.try_clone()
.expect("could not clone stream")); let reader = BufReader::new(stream); match handle_client(log, reader, writer) { Ok(handler) => { joins.push(handler); } Err(err) => { error!(server,
"Could not make client handler. {:?}",
err); } } } else { info!(root, "Shutting down! {:?}", stream); } } info!( server, "No more incoming connections. Draining existing connections." ); for jh in joins { if let Err(err) = jh.join() { info!(server,
"Connection handler died with error: {:?}",
err); } } }
The key thing here is handle_client(log, reader, writer). This function accepts the newly created stream :: TcpStreamā€“ in its buffered reader and writer guiseā€”and returns std::io::Result<JoinHandler<()>. We'll see the implementation of this function directly. Before that and somewhat as an aside, it's very important to remember to add buffering to your IO. If we did not have BufWriter and BufReader in place here, every read and write to TcpStream would result in a system call on most systems doing per-byte transmissions over the network. It is significantly more efficient for everyone involved to do reading and writing in batches, which the BufReader and BufWriter implementations take care of. I have lost count of how many overly slow programs I've seen fixed with a judicious application of buffered-IO. Unfortunately, we won't dig into the implementations of BufReader and BufWriter here as they're outside the scope of this book. If you've read this far and I've done alright job explaining, then you've learned everything you need to understand the implementations and you are encouraged to dig in at your convenience. Note that here we're also allocating a vector for JoinHandler<()>s returned by handle_client. This is not necessarily ideal. Consider if the first connection were to be long-lived and every subsequent connection short. None of the handlers would be cleared out, though they were completed, and the program would gradually grow in memory consumption. The resolution to this problem is going to be program-specific but, at least here, it'll be sufficient to ignore the handles and force mid-transaction exits on worker threads. Why? Because the server is only echoing:
fn handle_client( log: slog::Logger, mut reader: BufReader<TcpStream>, mut writer: BufWriter<TcpStream>, ) -> io::Result<thread::JoinHandle<()>> { let builder = thread::Builder::new(); builder.spawn(move || { let mut buf = String::with_capacity(2048); while let Ok(sz) = reader.read_line(&mut buf) { info!(log, "Received a {} bytes: {}", sz, buf); writer .write_all(&buf.as_bytes()) .expect("could not write line"); buf.clear(); } TOTAL_STREAMS.fetch_sub(1, Ordering::Relaxed); }) }
A network protocol must, anyway, be resilient to hangups on either end, owing to the unreliable nature of networks. In fact, the reader who has enjoyed this book and especially Chapter 6, Atomics ā€“ the Primitives of Synchronization, and Chapter 7, Atomics ā€“ Safely Reclaiming Memory, will be delighted to learn that distributed systems face many of the same difficulties with the added bonus of unreliable transmission. Anyway, note that handle_client isn't doing all that much, merely using the thread::Builder API to construct threads, which we discussed in Chapter 5, Locks ā€“ Mutex, Condvar, Barriers and RWLock. Otherwise, it's a fairly standard TCP echo.
Let's see the server in operation. In the top of the project, run cargo run --release --bin server and you should be rewarded with something much like:
> cargo run --release --bin server Finished release [optimized] target(s) in 0.26 secs Running `target/release/server` host: localhost port: 1987 Apr 22 21:31:14.001 INFO Server open for business! :D
So far so good. This server is listening on localhost, port 1987. In some other terminal, you can run a telnet to the server:
> telnet localhost 1987 Trying ::1... Connected to localhost. Escape character is '^]'. hello server
I've sent hello server to my running instance and have yet to receive a response because of the behavior of the write buffer. An explicit flush would correct this, at the expense of worse performance. Whether a flush should or should not be placed will depend entirely on setting. Here? Meh.
The server dutifully logs the interaction:
stream-no: 0 peer-addr: [::1]:65219 Apr 22 21:32:54.943 INFO Received a 14 bytes: hello server

The client

The real challenge for our server comes with a specially constructed client. Let's go through it before we see the client in action. The preamble is typical:
#[macro_use] extern crate slog; extern crate clap; extern crate slog_async; extern crate slog_term; use clap::{App, Arg}; use slog::Drain; use std::net::TcpStream; use std::sync::atomic::{AtomicUsize, Ordering}; use std::{thread, time}; static TOTAL_STREAMS: AtomicUsize = AtomicUsize::new(0);
In fact, this client preamble hews fairly close to that of the server. So, too, the main function, as we'll see shortly. As with many other programs in this book, we dedicate a thread to reporting on the behavior of the program. In this client, this thread runs the long-lived report:
fn report(log: slog::Logger) { let delay = time::Duration::from_millis(1000); let mut total_streams = 0; loop { let streams_per_second = TOTAL_STREAMS.swap(0, Ordering::Relaxed); info!(log, "Total connections: {}", total_streams); info!(log, "Connections per second: {}", streams_per_second); total_streams += streams_per_second; thread::sleep(delay); } }
Every second report swa...

Table of contents