# Foundations of Statistics for Data Scientists

## With R and Python

## Alan Agresti, Maria Kateri

# Foundations of Statistics for Data Scientists

## With R and Python

## Alan Agresti, Maria Kateri

## About This Book

Foundations of Statistics for Data Scientists: With R and Python is designed as a textbook for a one- or two-term introduction to mathematical statistics for students training to become data scientists. It is an in-depth presentation of the topics in statistical science with which any data scientist should be familiar, including probability distributions, descriptive and inferential statistical methods, and linear modeling. The book assumes knowledge of basic calculus, so the presentation can focus on "why it works" as well as "how to do it." Compared to traditional "mathematical statistics" textbooks, however, the book has less emphasis on probability theory and more emphasis on using software to implement statistical methods and to conduct simulations to illustrate key concepts. All statistical analyses in the book use R software, with an appendix showing the same analyses with Python.

Key Features:

- Shows the elements of statistical science thatareimportant for students who plan to become data scientists.
- Includes Bayesian and regularized fitting of models (e.g., showing an example using the lasso), classification and clustering, and implementing methods with modern software (R and Python).
- Contains nearly 500 exercises.

The book also introduces modern topics that do not normally appear in mathematical statistics texts but are highly relevant for data scientists, such as Bayesian inference, generalized linear models for non-normal responses (e.g., logistic regression and Poisson loglinear models), and regularized model fitting. The nearly 500 exercises are grouped into "Data Analysis and Applications" and "Methods and Concepts." Appendices introduce R and Python and contain solutions for odd-numbered exercises. The book's website ( http://stat4ds.rwth-aachen.de/ )has expanded R, Python, and Matlab appendices and all data sets from the examples and exercises.

## Information

# 1Introduction to Statistical Science

*the methods of statistical science result from a unified theory*, although that theory itself has slight variations in the way it is implemented or interpreted. Another danger of the ubiquity of statistical software is that prospective data scientists might expect that software can automatically perform good data analyses without input from the user. We'll see, however, that careful thought is needed to decide which statistical methods are appropriate for any particular situation, as they all make certain assumptions, and some methods work poorly when the assumptions are violated. Moreover, a data scientist needs to be able to interpret and explain the results that software yields.

*variables*that represent how measured characteristics can vary from observation to observation. We also introduce

*graphical and numerical methods*for describing the data. When a study can use

*randomization*in collecting the data or conducting an experiment, data analysts can exploit the random variation to make reliable estimations and predictions.

## 1.1 Statistical Science: Description and Inference

**means. You regularly hear statistics quoted about sports events, the economy, medical research, and opinions, beliefs, and behaviors of people. In this sense, a statistic is merely a number calculated from**

*statistics***—the observations that provide information about the subject matter. But the field of**

*data***has a much broader sense—as a field that gives us a way of gathering and analyzing the data in an objective manner.**

*statistical science**probability*. The methods enable us to deal with uncertainty and variability in virtually all scientific fields. With statistical methods, we learn from the data while measuring, controlling, and communicating uncertainty.

### 1.1.1 Design, Descriptive Statistics, and Inferential Statistics

: Planning how to gather relevant data for the subject matter of interest.*Design*: Summarizing the data.*Description*: Making evaluations, such as estimations and predictions, based on the data.*Inference*

**refers to planning a study so that it yields useful data. For example, for a poll taken to determine public opinion on some issue, the design specifies how to select the people to interview and constructs the questionnaire for interviews. For a research study to compare an experimental diet with a standard diet to address obesity, the design specifies how to obtain people for the study, how to determine which people use each diet, and specifies the characteristics to measure to compare the diets.**

*Design***refers to summarizing the data, to mine the information that the data provide. For any study, the raw data are a complete listing of observations that can be overwhelming for comprehension. To present the results, we reduce the data to simpler and more understandable form without distorting or losing much information. Graphs, tables, and numerical summaries such as averages and percentages are called**

*Description***.**

*descriptive statistics***refers to using the data to make estimations and other sorts of evaluations, such as predictions. These evaluations take into account random variability that occurs with the characteristics measured and the resulting uncertainty in decision-making. For instance, suppose that in the study comparing two diets, the people on the experimental diet had an average weight loss of 7.0 kilograms. What can we say about the average weight change if hypothetically**

*Inference**all*obese people used this diet? An inferential statistical method provides an interval of numbers within which we can predict that the average weight change would fall. The analysis...

## Table of contents

*Foundations of Statistics for Data Scientists*(1st ed.). CRC Press. Retrieved from https://www.perlego.com/book/3056222/foundations-of-statistics-for-data-scientists-with-r-and-python-pdf (Original work published 2021)

*Foundations of Statistics for Data Scientists*. 1st ed. CRC Press. https://www.perlego.com/book/3056222/foundations-of-statistics-for-data-scientists-with-r-and-python-pdf.

*Foundations of Statistics for Data Scientists*. 1st edn. CRC Press. Available at: https://www.perlego.com/book/3056222/foundations-of-statistics-for-data-scientists-with-r-and-python-pdf (Accessed: 15 October 2022).

*Foundations of Statistics for Data Scientists*. 1st ed. CRC Press, 2021. Web. 15 Oct. 2022.