Scribd
Upload
103 views

Linear Regression-2: Prof. Asim Tewari IIT Bombay

This document discusses simple linear regression. It describes linear regression as modeling a linear relationship between a response variable Y and predictor variable X. It discusses estimating regression coefficients using the least squares approach to minimize residual errors. It also describes assessing the accuracy of coefficient estimates using standard errors and confidence intervals, and performing hypothesis tests on the coefficients using t-statistics and p-values. Finally, it discusses evaluating the accuracy of regression models using measures like residual standard error and the R2 statistic.

Uploaded by

Scion Of Virikvas
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
103 views

Linear Regression-2: Prof. Asim Tewari IIT Bombay

This document discusses simple linear regression. It describes linear regression as modeling a linear relationship between a response variable Y and predictor variable X. It discusses estimating regression coefficients using the least squares approach to minimize residual errors. It also describes assessing the accuracy of coefficient estimates using standard errors and confidence intervals, and performing hypothesis tests on the coefficients using t-statistics and p-values. Finally, it discusses evaluating the accuracy of regression models using measures like residual standard error and the R2 statistic.

Uploaded by

Scion Of Virikvas
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 19

Linear Regression-2

Prof. Asim Tewari


IIT Bombay

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
It assumes that there is approximately a linear
relationship between X and Y
or
β0 and β1 are intercept slope known as the
model coefficients or parameters

Hat symbol, ˆ , to denote the estimated value


for an unknown parameter or coefficient

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
Estimating the Coefficients
• Least squares approach
The least squares approach chooses parameters
to minimize the residual sum of squares (RSS)
represents ith residual

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
Estimating the Coefficients

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
Assessing the Accuracy of the Coefficient
Estimates
Standard Errors associated with coefficients

95% confidence interval associated with


coefficients

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
Assessing the Accuracy of the Coefficient
Estimates
Standard Errors associated with coefficients

95% confidence interval associated with


coefficients

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
Assessing the Accuracy of the Coefficient
Estimates
Standard Errors associated with coefficients

95% confidence interval associated with


coefficients

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
Hypothesis tests on the coefficients

versus the alternative hypothesis

Mathematically, this corresponds to testing


versus
For this we calculate t statistics which measures the
number of standard deviations that is away from
0.

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression

p-value is defined as

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression

p-value is defined as

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression

p-value is defined as

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
p-value is defined as

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
P-Value is the probability of observing any value
equal to |t| or larger for a t-distribution with
n−2 degrees of freedom

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
• The p-value represents the chance your results
could be random (i.e. happened by chance).

• So a small p-value means that there is a small


chance that your results are random. Thus, they
are not random. So we can infer that there is an
association between the predictor and the
response (i.e we reject the null hypothesis)

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
For the Advertising data, the least squares fit for the regression of sales onto TV is shown.
The fit is found by minimizing the sum of squared errors. Each grey line segment represents
an error, and the fit makes a compromise by averaging their squares. In this case a linear fit
captures the essence of the relationship, although it is somewhat deficient in the left of the
plot.

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
P-value

For the Advertising data, coefficients of the least squares model for the regression of number
of units sold on TV advertising budget. An increase of $1,000 in the TV advertising budget is
associated with an increase in sales by around 50 units (Recall that the sales variable is in
thousands of units, and the TV variable is in thousands of dollars).

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
SE of a mean of a RV

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
SE of a mean of a RV

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining
Simple Linear Regression
Assessing the Accuracy of the Model
Residual Standard Error (RSE)

R2 Statistic: The RSE provides an absolute measure, R2


provides a relative measure

R=

Asim Tewari, IIT Bombay ME 781: Statistical Machine Learning and Data Mining

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy