Join Telegram Channel: https://t.me/hsslive Downloaded from www.Hsslive.

in ®

NOTES ON
REGRESSION ANALYSIS
This notes contain the following topics

1. Meaning of Regression

2. Regression Equations

 Regression equation of Y on X
 Regression equation of X on Y

3. Regression coefficients

 Regression coefficient of Y on X
 Regression coefficient of X on Y
 Properties of regression coefficients

4. Properties of regression lines

5. Identification of regression lines

6. Comparison between correlation and regression

 Join Telegram Channel: https://t.me/hsslive Downloaded from www.Hsslive.in ®

REGRESSION ANALYSIS

1. Meaning of Regression
Regression analysis is a mathematical measure of the nature of relationship
between two or more variables. It is the functional form of the nature of relation.
Regression analysis is the study of the cause and effect relationship.

Regression analysis was introduced by Sir Francis Galton.

There are different types of regression. We have to study only the linear regression
between two variables. In this type of regression only linear nature is considered. That is the
two variables are assumed to be linearly dependent. R
The following are some uses of regression analysis. e

 It is used to determine the relationship between ind

independent
ependent and g
dependent variables.
 The most important use is to predict and forecast. r

e
2. Regression Equations
s
Here we discuss only the linear regression between two variables as mentioned
s
earlier. Suppose we have a bivariate data. We
can plot this data in scatter diagram. If the points i
in the scatter diagram concentrate around a o
straight line, that line is called the regression line
n
or the line of best fit.. The equation to that line is
A
called the regression equation.. This equation is a
first degree equation in the variables X and Y n
(say). The variables X and Y are not reversible in regression analysis. So we have two a
regression lines. The regression lines are fitted by the assumption that one variable under
l
analysis is an independent variable and the other is a dependent variable. If X is the
y
independent variable and Y is the dependent variable we get a regression line called the
regression line of Y on X and if Y is the independent variable and X is the dependent s
variable we get another regression line called the regression line of X on Y.
Y This two lines i
are not reversible. Hence we have two regression lines.
s
 Join Telegram Channel: https://t.me/hsslive Downloaded from www.Hsslive.in ®

Regression equation of Y on X

The regression equation of Y on X is constructed by the assumption that X is the

independent variable and Y is the dependent variable.

The regression equation of Y on X is of the form,

y  y  bYX  x  x 
Cov ( X , Y )
Where bYX is called the regression coefficient of Y on X, defined as bYX  . (more on
V (X )
regression coefficients is described in next session).

Regression equation of X on Y

The regression equation of X on Y is constructed by the assumption that Y is the

independent variable and X is the dependent variable.

The regression equation of X on Y is of the form,

x  x  bXY  y  y 

Cov ( X , Y )
Where bXY is called the regression coefficient of X on Y, defined as bXY  .
V (Y )

3. The regression coefficients

As mentioned in the above, there are two types of regression coefficients.

Regression coefficient of Y on X

Cov ( X , Y ) n  
x  x  y  y 
1
It is defined as bYX   .
x  x 
V (X ) 1 2

The simplified formula used for calculation is:

n  xy   x  y
bYX 
n x 2    x 
2

Regression coefficient of X on Y

Cov ( X , Y ) n  
x  x  y  y 
1
It is defined as bXY   .
 y  y 
V (Y ) 1 2

n
 Join Telegram Channel: https://t.me/hsslive Downloaded from www.Hsslive.in ®

The simplified formula used for calculation is:

n xy   x  y
bXY 
n y 2    y 
2

Properties of regression coefficients

 In the regression line of Y on X, bYX is the regression coefficient of Y on X.

 In the regression line of X on Y, bXY is the regression coefficient of X on Y.
 If the regression line of Y on X is expressed in the form y  ax  b , then bYX  a .
 If the regression line of X on Y is expressed in the form x  cy  d , then bXY  c .
 The sign of both regression coefficients will be the same. Either both are +ve or both are
–ve.
 The product of both the coefficients is less than or equal to 1.
 If one of the regression coefficients is greater than 1, then the other must be less than 1.
Both of them can be less than 1 simultaneously.
 Correlation coefficient is the geometric mean of the regression coefficients.
ie, r  bYX  bXY

Explanation

Cov ( X , Y ) Cov ( X , Y )
We have bYX  and bXY 
V (X ) V (Y )

Cov( X , Y ) Cov( X , Y )  Cov ( X , Y ) 

2

bYX  bXY   
V (X ) V (Y ) V ( X )  V (Y )

 Cov( X , Y )   Cov( X , Y ) 
2 2

ie, bYX  bXY    r

2

 X2   Y2   X  Y 

ie, r 2  bYX  bXY  r   bYX  bXY

Y 
 bYX  r  and bXY  r  X
X Y
Explanation
Cov( X , Y )
We have the correlation coefficient, r
 X Y
 Cov ( X , Y )  r   X   Y
Cov( X , Y ) Cov ( X , Y ) r   X   Y 
Now, bYX     r Y
V (X ) X 2
X2
X
 Join Telegram Channel: https://t.me/hsslive Downloaded from www.Hsslive.in ®

Cov( X , Y ) Cov( X , Y ) r   X   Y 
Similarly. bXY     r X
V (Y ) Y2
Y2
Y
 Regression coefficients are not symmetric. ie, in general, bYX  bXY .

4. Properties of regression lines

We have two regression lines, the regression line Y on X and the regression line of X on Y.

The following are some properties of regression lines.

 The two lines intersect at the point ( x , y ).

 The regression lines will be perpendicular if the correlation coefficient r  0.
 The regression lines will coincide if the correlation coefficient is perfect ( r  1 ).

5. Identification of regression lines

Suppose we are given two regression lines. Sometimes we have to identify them. That is
we have to identify which is the regression line of Y on X and which the regression line of X on
Y.

At the first step we have to assume one of them as the regression line of Y on X and the other
as the regression line of X on Y.

Express the regression line of Y on X in the form y  ax  b . Then bYX  a

Express the regression line of X on Y in the form x  cy  d . Then bXY  c

Find bYX  bXY . If it is less than or equal to 1, our assumption is right.

6. Comparison between correlation and regression

Regression Correlation
Regression deals with the study Correlation deals with the
of the nature of relationship study of the degree of
between the variables relationship between the
variables.
Regression is not symmetric Correlation is symmetric
Regression is the cause and Correlation is the association
effect relationship between between the variables.
the variables
Regression is used for Correlation is not used for
prediction prediction
Regression is not reversible Correlation is reversible