Basics of Matrix Algebra for Statistics with R

Nick Fieller

• Publisher: Chapman and Hall/CRC
• Year: 2018
• eBook ISBN: 9781315360058
• Topic: Mathematics
• Subtopic: Algebra
• Index: Mathematics

A Thorough Guide to Elementary Matrix Algebra and Implementation in R

Basics of Matrix Algebra for Statistics with R provides a guide to elementary matrix algebra sufficient for undertaking specialized courses, such as multivariate data analysis and linear models. It also covers advanced topics, such as generalized inverses of singular and rectangular matrices and manipulation of partitioned matrices, for those who want to delve deeper into the subject.

The book introduces the definition of a matrix and the basic rules of addition, subtraction, multiplication, and inversion. Later topics include determinants, calculation of eigenvectors and eigenvalues, and differentiation of linear and quadratic forms with respect to vectors. The text explores how these concepts arise in statistical techniques, including principal component analysis, canonical correlation analysis, and linear modeling.

In addition to the algebraic manipulation of matrices, the book presents numerical examples that illustrate how to perform calculations by hand and using R. Many theoretical and numerical exercises of varying levels of difficulty aid readers in assessing their knowledge of the material. Outline solutions at the back of the book enable readers to verify the techniques required and obtain numerical answers.

Avoiding vector spaces and other advanced mathematics, this book shows how to manipulate matrices and perform numerical calculations in R. It prepares readers for higher-level and specialized studies in statistics.

1

Introduction

1.1 Objectives

The aim of this book is to provide a guide to elementary matrix algebra sufficient for undertaking intermediate and advanced statistical courses such as multivariate data analysis and linear models. Starting from a definition of a matrix and covering the basic rules of addition, subtraction, multiplication and inversion, the later topics include determinants, calculation of eigenvectors and eigenvalues and differentiation of linear and quadratic forms with respect to vectors. These later topics are sometimes not included in basic courses on linear algebra but are virtually essential for full discussion of statistical topics in multivariate analysis and linear models. The notes go a little beyond meeting just this need, providing an initial guide to more advanced topics such as generalized inverses of singular and rectangular matrices and manipulation of partitioned matrices. This is to provide a second step for those who need to go a little further than standard lecture courses on advanced statistics, for example, when embarking on a dissertation. As well as describing the basics of matrix algebra, including numerical calculations “by hand”, for example, of matrix multiplication and inversion, the notes give guidance on how to do numerical calculations in R (R Core Team 2014). R is broadly similar in operation to the package MATLAB® but oriented specifically towards statistical applications rather than more general areas of applied mathematics. R is an open source system and is available free. It is closely similar to the commercial package S-PLUS: the prime difference is that R is command-line driven without the standard menus and dialogue boxes for statistical operations in S-PLUS. Otherwise, most code written for the two systems is interchangeable. There are however a few differences, for example there may be differences in the available optional arguments for some functions. Generally, all options available in S-PLUS are also available in R but not necessarily vice versa. These are quickly verified by use of the help system. The sites from which R and associated software (extensions and libraries) and manuals can be found are listed at http://www.ci.tuwien.ac.at/R/mirrors.html.

There are many available user interfaces for R, such as RStudio (Verzani, 2011), some of which provide a menu-driven facility, but this text assumes use of only the standard version of R without any such special facility.

Free versions of full manuals for R (mostly in PDF format) can be found at any of these mirror sites. There is a wealth of contributed documentation.

Some sections are considerably more advanced in concept and technical requirements than others, for example, those on partitioned matrices (§2.6.1) and generalized inverses (§8.3), but these have not been starred or marked since what may seem technically complex to one may seem straightforward to another with a different background. It is left to the readers to glean what they can from them on initial readings and then return later if a complete understanding is wanted. Some topics often covered in texts such as this one are not included (for example linear spaces) which means that there are a few results (e.g. equality of row and column rank, §3.1) which will need to be taken on trust or else checked in one of the more specialist texts referenced in §1.2

Since the applications envisaged for the majority of users of these notes are within statistics, most emphasis is given to real matrices and mostly to real symmetric matrices. Unless specifically indicated, all matrices are presumed to be real.

In addition to the algebraic manipulation of matrices the notes give numerical examples since it is necessary to be able to do numerical calculations, both by hand and by using R. Much of the exposition is by presenting examples with solutions. There are additionally some examples for self study and solutions are available separately (for the desperate) to these. For numerical questions, the calculations are given only in R and in some cases only the numerical answers are given. A brief guide to installing and running R is included.

1.2 Further Reading

1.2.1 Matrix algebra

There are many books on matrix algebra and a search will reveal hundreds if not thousands aimed at readers in the later years at school, throughout university and at advanced levels of mathematical research. A particular one consulted in the preparation of this text is Abadir and Magnus (2005) which consists largely of worked exercises. This will complete many of the details omitted from this book (in particular complex matrices) but, as is common with all post-school level texts, it does require a good understanding of linear spaces. It is not specifically written for any particular discipline but does contain several sections on statistical topics such as least squares and maximum likelihood estimation. A rather more theoretical book by Magnus and Neudecker (1988) goes into further details on vector and matrix calculus. Many statistical texts on topics such as linear models and multivariate analysis, such as Draper and Smith (1998), Seber and Lee (2012), Anderson (2003), Cox (2005) and Mardia et al. (1979), contain brief sections or appendices on matrix algebra but there are also many books aimed at the statistical reader. Three such are Searle (1982), Basilevsky (2013) and Harville (2008). A comprehensive account of linear algebra...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• Preface
• 1 Introduction
• 2 Vectors and Matrices
• 3 Rank of Matrices
• 4 Determinants
• 5 Inverses
• 6 Eigenanalysis of Real Symmetric Matrices
• 7 Vector and Matrix Calculus
• 8 Further Topics
• 9 Key Applications to Statistics
• Outline Solutions to Exercises
• Bibliography
• Index