Mastering Go

Create Golang production applications using network libraries, concurrency, machine learning, and advanced data structures, 2nd Edition

Mihalis Tsoukalos

• Publisher: Packt Publishing
• Year: 2019
• ISBN:
  9781838555320
• Edition: 2
• Topic: Computer Science
• Subtopic: Programming Languages
• Index: Computer Science

Publisher's Note: This edition from 2019 is outdated and is not compatible with the latest version of Go. A new third edition, updated for 2021 and featuring the latest in Go programming, has now been published.

Key Features

• Second edition of the bestselling guide to advanced Go programming, expanded to cover machine learning, more Go packages and a range of modern development techniques
• Completes the Go developer's education with real-world guides to building high-performance production systems
• Packed with practical examples and patterns to apply to your own development work
• Clearly explains Go nuances and features to remove the frustration from Go development

Book Description

Often referred to (incorrectly) as Golang, Go is the high-performance systems language of the future. Mastering Go, Second Edition helps you become a productive expert Go programmer, building and improving on the groundbreaking first edition.

Mastering Go, Second Edition shows how to put Go to work on real production systems. For programmers who already know the Go language basics, this book provides examples, patterns, and clear explanations to help you deeply understand Go's capabilities and apply them in your programming work.

The book covers the nuances of Go, with in-depth guides on types and structures, packages, concurrency, network programming, compiler design, optimization, and more. Each chapter ends with exercises and resources to fully embed your new knowledge.

This second edition includes a completely new chapter on machine learning in Go, guiding you from the foundation statistics techniques through simple regression and clustering to classification, neural networks, and anomaly detection. Other chapters are expanded to cover using Go with Docker and Kubernetes, Git, WebAssembly, JSON, and more.

If you take the Go programming language seriously, the second edition of this book is an essential guide on expert techniques.

What you will learn

• Clear guidance on using Go for production systems
• Detailed explanations of how Go internals work, the design choices behind the language, and how to optimize your Go code
• A full guide to all Go data types, composite types, and data structures
• Master packages, reflection, and interfaces for effective Go programming
• Build high-performance systems networking code, including server and client-side applications
• Interface with other systems using WebAssembly, JSON, and gRPC
• Write reliable, high-performance concurrent code
• Build machine learning systems in Go, from simple statistical regression to complex neural networks

Who this book is for

Mastering Go, Second Edition is for Go programmers who already know the language basics, and want to become expert Go practitioners.

Code Testing, Optimization, and Profiling

The previous chapter discussed concurrency in Go, mutexes, the atomic package, the various types of channels, race conditions, and how the select statement allows you to use channels as glue to control goroutines and allow them to communicate.

The Go topics in this chapter are very practical and important, especially if you are interested in improving the performance of your Go programs and discovering bugs quickly. This chapter primarily addresses code optimization, code testing, code documentation, and code profiling.

Code optimization is a process where one or more developers try to make certain parts of a program run faster, be more efficient, or use fewer resources. Put simply, code optimization is about eliminating the bottlenecks of a program.

Code testing is about making sure that your code does what you want it to do. In this chapter, you will experience the Go way of testing code. The best time to write code to test your programs is during the development phase, as this can help to reveal bugs in your code as early as possible.

Code profiling relates to measuring certain aspects of a program in order to get a detailed understanding of the way the code works. The results of code profiling may help you to decide which parts of your code need to change.

I hope that you already recognize the importance of documenting your code in order to describe the decisions you made while developing the implementation of your program. In this chapter, you will see how Go can help you to generate documentation for the modules that you implement.

Documentation is so important that some developers write the documentation first and the code afterward! However, what is really important is that the documentation and the functionality of the program say and do the same thing, respectively.

In this chapter, you will learn about the following topics:

• Profiling Go code
• The go tool pprof utility
• Using the web interface of the Go profiler
• Testing Go code
• The go test command
• The go tool trace utility
• The handy testing/quick package
• Benchmarking Go code
• Cross-compilation
• Testing code coverage
• Generating documentation for your Go code
• Creating example functions
• Finding unreachable Go code in your programs

About optimization

Code optimization is both an art and a science. This means that there is no deterministic way to help you optimize your Go code, or any other code in any programming language, and that you should use your brain and try many things if you want to make your code run faster.

You should make sure that you optimize code that does not contain any bugs, because there is no point in optimizing a bug. If you have any bugs in your program, you should debug it first.

If you are really into code optimization, you might want to read Compilers: Principles, Techniques, and Tools by Alfred V. Aho, Monica S. Lam, Ravi Sethi, and Jeffrey D. Ullman (Pearson Education Limited, 2014), which focuses on compiler construction. Additionally, all volumes in The Art of Computer Programming series by Donald Knuth (Addison-Wesley Professional, 1998) are great resources for all aspects of programming.

Always remember what Knuth said about optimization:

"The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming."

Also, remember what Joe Armstrong, one of the developers of Erlang, said about optimization:

"Make it work, then make it beautiful, then if you really, really have to, make it fast. 90 percent of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!"

Additionally, generally speaking, only a small percentage of a program needs to be optimized. In such cases, the assembly programming language, which can be used to implement certain Go functions, is a really good candidate and will have a huge impact on the performance of your programs.

Optimizing Go code

As mentioned, code optimization is the process where you try to discover the parts of your code that have a big impact on the performance of the entire program in order to make them run faster or use fewer resources.

The benchmarking section that appears later in this chapter will greatly help you to understand what is going on with your code behind the scenes and which parameters of your program have the greatest impact on the performance of your program. However, do not underestimate the importance of common sense. Put simply, if one of your functions is executed 10,000 times more than the rest of the functions of a program, try to optimize that function first.

The general advice for optimization is that you must optimize bug-free code only. This means that you must optimize working code only. Therefore, you should first try to write correct code even if that code is slow. Finally, the single most frequent mistake that programmers make is trying to optimize the first version of their code, which is the root of most bugs!

Again, code optimization is both an art and a science, which means that it is a pretty difficult task. The next section about profiling Go code will definitely help you with code optimization because the main purpose of profiling is to find the bottlenecks in your code in order to optimize the most important parts of your program.

Profiling Go code

Profiling is a process of dynamic program analysis that measures various values related to program execution in order to give you a better understanding of the behavior of your program. In this section, you are going to learn how to profile Go code in order to understand your code better and improve its performance. Sometimes, code profiling can even reveal bugs!

First, we are going to use the command-line interface of the Go profiler. Then, we will use the web interface of the Go profiler.

The single most important thing to remember is that if you want to profile Go code, you will need to import the runtime/pprof standard Go package, either directly or indirectly. You can find the help page of the pprof tool by executing the go tool pprof -help command, which will generate lots of output.

The net/http/pprof standard Go package

Although Go comes with the low-level runtime/pprof standard Go package, there is also the high-level net/http/pprof package, which should be used when you want to profile a web application written in Go. As this chapter will not talk about creating HTTP servers in Go, you will learn more about the net/http/pprof package in Chapter 12, The Foundations of Network Programming in Go.

A simple profiling example

Go supports two kinds of profiling: CPU profiling and memory profiling. It is not recommended that you profile an application for both kinds at the same time, because these two different kinds of profiling do not work well with each other. The profileMe.go application is an exception, however, because it is used to illustrate the two techniques.

The Go code to be profiled is saved as profileMe.go, and it will be presented in five parts. The first part of profileMe.go is shown in the following Go code:

package main import ( "fmt" "math" "os" "runtime" "runtime/pprof" "time" ) func fibo1(n int) int64 { if n == 0 || n == 1 { return int64(n) } time.Sleep(time.Millisecond) return int64(fibo2(n-1)) + int64(fibo2(n-2)) } 

Notice that it is compulsory to import the runtime/pprof package directly or indirectly for your program to create profiling data. The reason for calling time.Sleep() in the fibo1() function is to slow it down a bit. You will learn why near the end of this section.

The second code segment of profileMe.go follows:

func fibo2(n int) int { fn := make(map[int]int) for i := 0; i <= n; i++ { var f int if i <= 2 { f = 1 } else { f = fn[i-1] + fn[i-2] } fn[i] = f } time.Sleep(50 * time.Millisecond) return fn[n] } 

The preceding code contains the implementation of another Go function that uses a different algorithm for calculating numbers of the Fibonacci sequence.

The third part of profileMe.go contains the following Go code:

func N1(n int) bool { k := math.Floor(float64(n/2 + 1)) for i := 2; i < int(k); i++ { if (n % i) == 0 { return false } } return true } func N2(n int) bool { for i := 2; i < n; i++ { if (n % i) == 0 { return false } } return true } 

Both the N1() and N2() functions are used to find out whether a given integer is a pri...