Chemometrics in Spectroscopy

Howard Mark, Jerry Workman Jr., Jerry Workman, Jr.

• Publisher: Academic Press
• Year: 2010
• eBook ISBN: 9780080548388
• Topic: Physical Sciences
• Subtopic: Analytic Chemistry

Chemometrics in Spectroscopy builds upon the statistical information covered in other books written by these leading authors in the field by providing a broader range of mathematics and progressing into the fundamentals of multivariate and experimental data analysis. Subjects covered in this work include: matrix algebra, analytic geometry, experimental design, calibration regression, linearity, design of collaborative laboratory studies, comparing analytical methods, noise analysis, use of derivatives, analytical accuracy, analysis of variance, and much more are all part of this chemometrics compendium. Developed in the form of a tutorial offering a basic hands-on approach to chemometric and statistical analysis for analytical scientists, experimentalists, and spectroscopists. Without using complicated mathematics, Chemometrics in Spectroscopy demonstrates the basic principles underlying the use of common experimental, chemometric, and statistical tools. Emphasis has been given to problem-solving applications and the proper use and interpretation of data used for scientific research.- Offers basic hands-on approach to chemometric and statistical analysis for analytical scientists, experimentalists, and spectroscopists- Useful for analysts in their daily problem solving, as well as detailed insights into subjects often considered difficult to thoroughly grasp by non-specialists- Provides mathematical proofs and derivations for the student or rigorously-minded specialist

1

A New Beginning. . .

Why do we title this chapter “A New Beginning …”? Well, there are a lot of reasons. First of all, of course, is the simple fact that that is just the way we do things. Secondly, is the fact that we developed this book in much the same way we developed our previous book Statistics in Spectroscopy (SiS). Those of you out there who have followed the series of articles published in Spectroscopy magazine since 1986 know that for the most part, each column in the series was pretty much self-contained and could stand alone, yet also fit into that series in the appropriate place and contributed to the flow of information in that series as a whole.

We hope to be able to reproduce that on a larger scale. Just as the series Statistics in Spectroscopy (this is too long to write out each time, from here on we will abbreviate it SiS) was self-contained and stood alone, so too will we try to make this new series stand alone, and at the same time be a worthy successor to SiS, and also continue to develop the concepts we began there.

Thirdly is the fact that we are finally starting to write again. To you, our readership, it may seem like we have been writing continuously since we began SiS, but in fact we have been running on backlog for a longer time than you would believe. That was advantageous in that it allowed us time to pursue our personal and professional lives including such other projects as arranging for SiS to be published as a book [1],

The downside of our getting ahead of ourselves, on the other hand, is that we were not able to keep you abreast on the latest developments related to our favorite topic. However, since the last time we actually wrote something, there have been a number of noteworthy developments.

Our last series dealt only with the elementary concepts of statistics related to the general practice of calibration used for UV-VIS-NIR and occasionally for IR spectroscopy. Our purpose in writing SiS was to help provide a small foot bridge to cross the gap between specialized chemometrics literature written at the expert level and those general statistics articles and texts dealing with examples and questions far removed from chemistry or spectroscopic practice. Since the beginning of the “Statistics” series in 1986, several reviews, tutorials, and textbooks have been published to begin the construction of a major highway bridging this gap. Most notably, at least in our minds, have been tutorial articles on classical least squares (CLS), principal components regression (PCR), and partial least squares regression (PLSR) by Haaland and Thomas [2, 3]. Other important work includes textbooks on calibration and chemometrics by Naes and Martens [4], and Mark [5]. Chemometric reviews discussing the progress of tutorial and textbook literature appear regularly in Analytical Chemistry, Critical Review issues. Another recent series of articles on chemometric concepts termed “The Chemometric Space” by Naes and Isaksson has appeared [6]. In addition, there is a North American chapter of the International Chemometrics Society (NAmICS) which we are told has over 300 members. Those interested in joining or obtaining further information may contact Professor Thomas O’Haver at the Department of Chemistry and Biochemistry, University of Maryland, College Park, Maryland 20742 (Donald B. Dahlberg, 1993, personal communication).

All the foregoing was true as of when the Chemometrics column began in 1993. Now in 2006, when we are preparing this for book publication, there are many more sources of information about Chemometrics. However, since this is not a review of the field, we forebear to list them all, but will correct one item that has changed since then: to obtain information about NAmICS, or to join the discussion group, contact David Duewer at NIST ([email protected])) or send a message to the discussion group ([email protected]).

Finally, since imitation is the sincerest form of flattery (or so they tell us), we are pleased to see that others have also taken the route of printing longer tutorial discussions in the form of a series of related articles on a given topic. Two series that we have no qualms recommending, on topics related to ours, have appeared in some of the sister publications of Spectroscopy [7–15] (note: there have been recent indications that the series in Spectroscopy International has continued beyond the ones we have listed. If we can obtain more information we will keep you posted – Spectroscopy International has also undergone some transformations and it is not always easy to get copies).

So, overall the chemometrics bridge between the lands of the overly simplistic and severely complex is well under construction; one may find at least a single lane open by which to pass. So why another series? Well, it is still our labor of love to deal with specific issues that plague ourselves and our colleagues involved in the practice of multivariate qualitative and quantitative spectroscopic calibration. Having collectively worked with hundreds of instrument users over 25 combined years of calibration problems, we are compelled, like bees loaded with pollen, to disseminate the problems, answers, and questions brought about by these experiences. Then what would a series named “Chemometrics in Spectroscopy” hope to cover which is of interest to the readers of “Spectroscopy”?

We have been taken to task (with perhaps some justice) for using the broader title label “Chemometrics in Spectroscopy” for what we have claimed will be discussions of the somewhat narrower range of topics included in the field of multivariate statistical algorithms applied to chemical problems, when the term “Chemometrics” actually applies to a much wider range of topics. Nevertheless, we will use this title, for a number of reasons. First, that is what we said we were going to do, and we hate to not follow through, even on such a minor point. Secondly, we have said before (with all due arrogance) that this is our column, and we have been pretty fortunate that th...

Table of contents

• Cover image
• Title page
• Table of Contents
• Preface
• Note to Readers
• Chapter 1: A New Beginning. . .
• Chapter 2: Elementary Matrix Algebra: Part 1
• Chapter 3: Elementary Matrix Algebra: Part 2
• Chapter 4: Matrix Algebra and Multiple Linear Regression: Part 1
• Chapter 5: Matrix Algebra and Multiple Linear Regression: Part 2
• Chapter 6: Matrix Algebra and Multiple Linear Regression: Part 3 – The Concept of Determinants
• Chapter 7: Matrix Algebra and Multiple Linear Regression: Part 4 – Concluding Remarks
• Chapter 8: Experimental Designs: Part 1
• Chapter 9: Experimental Designs: Part 2
• Chapter 10: Experimental Designs: Part 3
• Chapter 11: Analytic Geometry: Part 1 – The Basics in Two and Three Dimensions
• Chapter 12: Analytic Geometry: Part 2 – Geometric Representation of Vectors and Algebraic Operations
• Chapter 13: Analytic Geometry: Part 3 – Reducing Dimensionality
• Chapter 14: Analytic Geometry: Part 4 – The Geometry of Vectors and Matrices
• Chapter 15: Experimental Designs: Part 4 – Varying Parameters to Expand the Design
• Chapter 16: Experimental Designs: Part 5 – One-at-a-time Designs
• Chapter 17: Experimental Designs: Part 6 – Sequential Designs
• Chapter 18: Experimental Designs: Part 7 – β, the Power of a Test
• Chapter 19: Experimental Designs: Part 8 – β, the Power of a Test (Continued)
• Chapter 20: Experimental Designs: Part 9 – Sequential Designs Concluded
• Chapter 21: Calculating the Solution for Regression Techniques: Part 1 – Multivariate Regression Made Simple
• Chapter 22: Calculating the Solution for Regression Techniques: Part 2 – Principal Component(s) Regression Made Simple
• Chapter 23: Calculating the Solution for Regression Techniques: Part 3 – Partial Least Squares Regression Made Simple
• Chapter 24: Looking Behind and Ahead: Interlude
• Chapter 25: A Simple Question: The Meaning of Chemometrics Pondered
• Chapter 26: Calculating the Solution for Regression Techniques: Part 4 – Singular Value Decomposition
• Chapter 27: Linearity in Calibration
• Chapter 28: Challenges: Unsolved Problems in Chemometrics
• Chapter 29: Linearity in Calibration: Act II Scene I
• Chapter 30: Linearity in Calibration: Act II Scene II – Reader’s Comments …
• Chapter 31: Linearity in Calibration: Act II Scene III
• Chapter 32: Linearity in Calibration: Act II Scene IV
• Chapter 33: Linearity in Calibration: Act II Scene V
• Chapter 34: Collaborative Laboratory Studies: Part 1 – A Blueprint
• Chapter 35: Collaborative Laboratory Studies: Part 2 – using ANOVA
• Chapter 36: Collaborative Laboratory Studies: Part 3 – Testing for Systematic Error
• Chapter 37: Collaborative Laboratory Studies: Part 4 – Ranking Test
• Chapter 38: Collaborative Laboratory Studies: Part 5 – Efficient Comparison of Two Methods
• Chapter 39: Collaborative Laboratory Studies: Part 6 – MathCad Worksheet Text
• Chapter 40: Is Noise Brought by the Stork? Analysis of Noise: Part 1
• Chapter 41: Analysis of Noise: Part 2
• Chapter 42: Analysis of Noise: Part 3
• Chapter 43: Analysis of Noise: Part 4
• Chapter 44: Analysis of Noise: Part 5
• Chapter 45: Analysis of Noise: Part 6
• Chapter 46: Analysis of Noise: Part 7
• Chapter 47: Analysis of Noise: Part 8
• Chapter 48: Analysis of Noise: Part 9
• Chapter 49: Analysis of Noise: Part 10
• Chapter 50: Analysis of Noise: Part 11
• Chapter 51: Analysis of Noise: Part 12
• Chapter 52: Analysis of Noise: Part 13
• Chapter 53: Analysis of Noise: Part 14
• Chapter 54: Derivatives in Spectroscopy: Part 1 – The Behavior of the Derivative
• Chapter 55: Derivatives in Spectroscopy: Part 2 – The “True” Derivative
• Chapter 56: Derivatives in Spectroscopy: Part 3 – Computing the Derivative
• Chapter 57: Derivatives in Spectroscopy: Part 4 – Calibrating with Derivatives
• Chapter 58: Comparison of Goodness of Fit Statistics for Linear Regression: Part 1 – Introduction
• Chapter 59: Comparison of Goodness of Fit Statistics for Linear Regression: Part 2 – The Correlation Coefficient
• Chapter 60: Comparison of Goodness of Fit Statistics for Linear Regression: Part 3 – Computing Confidence Limits for the Correlation Coefficient
• Chapter 61: Comparison of Goodness of Fit Statistics for Linear Regression: Part 4 – Confidence Limits for Slope and Intercept
• Chapter 62: Correction and Discussion Regarding Derivatives
• Chapter 63: Linearity in Calibration: Act III Scene I – Importance of Nonlinearity
• Chapter 64: Linearity in Calibration: Act III Scene II – A Discussion of the Durbin-Watson Statistic, a Step in the Right Direction
• Chapter 65: Linearity in Calibration: Act III Scene III – Other Tests for Nonlinearity
• Chapter 66: Linearity in Calibration: Act III Scene IV – How to Test for Nonlinearity
• Chapter 67: Linearity in Calibration: Act III Scene V – Quantifying Nonlinearity
• Chapter 68: Linearity in Calibration: Act III Scene VI – Quantifying Nonlinearity, Part II, and a News Flash
• Chapter 69: Connecting Chemometrics to Statistics: Part 1 – The Chemometrics Side
• Chapter 70: Connecting Chemometrics to Statistics: Part 2 – The Statistics Side
• Chapter 71: Limitations in Analytical Accuracy: Part 1 – Horwitz’s Trumpet
• Chapter 72: Limitations in Analytical Accuracy: Part 2 – Theories to Describe the Limits in Analytical Accuracy
• Chapter 73: Limitations in Analytical Accuracy: Part 3 – Comparing Test Results for Analytical Uncertainty
• Chapter 74: The Statistics of Spectral Searches
• Chapter 75: The Chemometrics of Imaging Spectroscopy
• Glossary of Terms
• Index