Regression Analysis with R

Giuseppe Ciaburro

• Publisher: Packt Publishing
• Year: 2018
• Print ISBN: 9781788627306
• eBook ISBN: 9781788622707
• Edition: 1
• Topic: Computer Science
• Subtopic: Data Modelling & Design
• Index: Computer Science

Build effective regression models in R to extract valuable insights from real data

Key Features

• Implement different regression analysis techniques to solve common problems in data science - from data exploration to dealing with missing values
• From Simple Linear Regression to Logistic Regression - this book covers all regression techniques and their implementation in R
• A complete guide to building effective regression models in R and interpreting results from them to make valuable predictions

Book Description

Regression analysis is a statistical process which enables prediction of relationships between variables. The predictions are based on the casual effect of one variable upon another. Regression techniques for modeling and analyzing are employed on large set of data in order to reveal hidden relationship among the variables.

This book will give you a rundown explaining what regression analysis is, explaining you the process from scratch. The first few chapters give an understanding of what the different types of learning are – supervised and unsupervised, how these learnings differ from each other. We then move to covering the supervised learning in details covering the various aspects of regression analysis. The outline of chapters are arranged in a way that gives a feel of all the steps covered in a data science process – loading the training dataset, handling missing values, EDA on the dataset, transformations and feature engineering, model building, assessing the model fitting and performance, and finally making predictions on unseen datasets. Each chapter starts with explaining the theoretical concepts and once the reader gets comfortable with the theory, we move to the practical examples to support the understanding. The practical examples are illustrated using R code including the different packages in R such as R Stats, Caret and so on. Each chapter is a mix of theory and practical examples.

By the end of this book you will know all the concepts and pain-points related to regression analysis, and you will be able to implement your learning in your projects.

What you will learn

• Get started with the journey of data science using Simple linear regression
• Deal with interaction, collinearity and other problems using multiple linear regression
• Understand diagnostics and what to do if the assumptions fail with proper analysis
• Load your dataset, treat missing values, and plot relationships with exploratory data analysis
• Develop a perfect model keeping overfitting, under-fitting, and cross-validation into consideration
• Deal with classification problems by applying Logistic regression
• Explore other regression techniques – Decision trees, Bagging, and Boosting techniques
• Learn by getting it all in action with the help of a real world case study.

Who this book is for

This book is intended for budding data scientists and data analysts who want to implement regression analysis techniques using R. If you are interested in statistics, data science, machine learning and wants to get an easy introduction to the topic, then this book is what you need! Basic understanding of statistics and math will help you to get the most out of the book. Some programming experience with R will also be helpful

Regression Analysis in Practice

In Chapter 1, Getting Started with Regression, we said that regression models are the most well-understood in numerical simulation. Once we've acquired the necessary experience in regression models, we will be able to understand all other machine learning algorithms. Regression models are easily interpretable as they are based on solid mathematical bases. Perhaps this is the reason behind the extreme ease of understanding of such techniques.

After analyzing in detail the different regression algorithms, it is time to put them into practice. This last chapter is meant to be a short reference, covering some of the major regression algorithms. In this chapter, you will just apply what has been learned. You can explore multiple linear regression, logistic regression, random forest regression, neural networks, and much more as applied to datasets resulting from real cases. The basic concepts we have learned from the previous chapters will now be put into practice. If we need to reinforce these concepts, we can safely retrace our footsteps and review the topics already discussed. We will look at practical examples from raw data to extract as much knowledge as possible.

This chapter starts with solving a real-world regression problem. To perform a regression analysis, a random forest technique will be adopted. Then you'll learn how to use logistic regression to classify a pattern. Finally, we'll perform a neural network regression analysis.

We will cover the following topics:

• Random forest
• Logistic regression
• Neural network regression

At the end of the chapter, we will have understood the techniques required to perform fitting analysis, pattern recognition, and neural network regression through a series of real cases. We will learn how to prepare data for regression analysis. We will also know how to perform fitting, pattern recognition, and neural network regression analysis in R, and how to perform preprocessing, postprocessing, and visualization to improve training efficiency and assess model performance.

Random forest regression with the Boston dataset

In this section, we will run a random forest regression for the Boston dataset; the median values of owner-occupied homes are predicted for the test data. The dataset describes 13 numerical properties of houses in Boston suburbs, and is concerned with modeling the price of houses in those suburbs in thousands of dollars. As such, this is a regression predictive modeling problem. Input attributes include features like crime rate, proportion of non-retail business acres, chemical concentrations, and more.

To get the data, we draw on the large collection of data available in the UCI Machine Learning Repository at the following link:
http://archive.ics.uci.edu/ml

The following list shows all the variables, followed by a brief description:

• Number of instances: 506
• Number of attributes: 14 continuous attributes (including the class attribute medv), and one binary-valued attribute

Each of the attributes is detailed as follows:

• crim: Per capita crime rate by town
• zn: Proportion of residential land zoned for lots over 25,000 square feet
• indus: Proportion of non-retail business acres per town
• chas: Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)
• nox: Nitric oxides concentration (parts per ten million)
• rm: Average number of rooms per dwelling
• age: Proportion of owner-occupied units built prior to 1940
• dis: Weighted distances to five Boston employment centers
• rad: Index of accessibility to radial highways
• tax: Full-value property-tax rate per $10,000
• ptratio: Pupil-teacher ratio by town
• black: 1000(B_k - 0.63)^2 where B_k is the proportion of blacks by town
• lstat: Percent of the lower status of the population
• medv: Median value of owner-occupied homes in $1000

Of these, medv is the response variable, while the other 13 variables are possible predictors. The goal of this analysis is to fit a regression model that best explains the variation in medv.

There is a relation between the first 13 columns and the medv response variable. We can predict the medv value based on the input 13 columns.

As said previously, the objective of this example is to predict the median value of owner-occupied homes value. First, we have to get the data. To do this we can use the read.table() function as follows:

BHData <- read.table(url("https://archive.ics.uci.edu/ml/machine-learning-databases/housing/housing.data"), sep = "")

In this function, instead of the filename, we can also enter a complete URL of a file contained on a website repository. Now, we use the names() function to set the names of the dataset according to the list previously viewed:

names(BHData)<- c("crim","zn","indus","chas","nox","rm", 
 "age","dis","rad","tax","ptratio","black","lstat","medv")

The table also includes the names of the variables as they are in the original dataset.

Exploratory analysis

Before starting with data analysis through the multiple linear regression, we conduct an exploratory analysis to understand how the data is distributed and extract preliminary knowledge. Let's start by checking the dataset using the str() function. This function provides a compact display of the internal structure of an object. Ideally, only one line for each basic structure is displayed:

str(BHData)

The results are shown here:

> str(BHData)
'data.frame': 506 obs. of 14 variables:
 $ crim : num 0.00632 0.02731 0.02729 0.03237 0.06905 ...
 $ zn : num 18 0 0 0 0 0 12.5 12.5 12.5 12.5 ...
 $ indus : num 2.31 7.07 7.07 2.18 2.18 2.18 7.87 7.87 7.87 7.87
 ...
 $ chas : int 0 0 0 0 0 0 0 0 0 0 ...
 $ nox : num 0.538 0.469 0.469 0.458 0.458 0.4...

Table of contents

• Title Page
• Copyright and Credits
• Packt Upsell
• Contributors
• Preface
• Getting Started with Regression
• Basic Concepts – Simple Linear Regression
• More Than Just One Predictor – MLR
• When the Response Falls into Two Categories – Logistic Regression
• Data Preparation Using R Tools
• Avoiding Overfitting Problems - Achieving Generalization
• Going Further with Regression Models
• Beyond Linearity – When Curving Is Much Better
• Regression Analysis in Practice
• Other Books You May Enjoy