LINEAR REGRESSION MODELS-STA531

Introductory note on Linear Regression Models

CIA 1 - Component 1
Diya Jose-1740838

Regression is a statistical measurement that attempts to determine the strength of the relationship
between one dependent variable (usually denoted by Y) and a series of other changing variables
(known as independent variables). Linear regression is one of the most common techniques of
regression analysis.

NOTE:

 The two basic types of regression are linear regression and multiple linear regression,
although there are non-linear regression methods for more complicated data and analysis.
 Regression helps investment and financial managers to value assets and understand the
relationships between variables
 Regression can help finance and investment professionals as well as professionals in
other businesses.

The theory and fundamentals of linear models lay the foundation for developing the tools for
regression analysis that are based on valid statistical theory and concepts.
Steps in regression analysis
• Statement of the problem under consideration• Choice of relevant variable • Collection of data
on relevant variables • Specification of model • Choice of method for fitting the data • Fitting of
model • Model validation and criticism • Using the chosen model(s) for the solution of the posed
problem.
In order to make regression analysis work, you must collect all the relevant data. It can be
presented on a graph, with an x-axis and a y-axis.

Examples and some applications:

Regression is often used to determine how many specific factors such as the price of a
commodity, interest rates, particular industries or sectors influence the price movement of an
asset. Most companies use regression analysis to explain a phenomenon they want to understand
(e.g. why did customer service calls drop last month?); predict things about the future (e.g. what

1
will sales look like over the next six months?); or to decide what to do (e.g. should we go with
this promotion or a different one?).

The residual is defined as the difference between the observed and fitted value of study variable.

𝒆 = 𝒚𝒊 ~ŷ𝒊 , 𝒊 = 𝟏, 𝟐, … . . 𝒏 , where yi is an observation and ŷ is the corresponding fitted value.

An error term is a residual variable produced by a statistical or mathematical model, which is

created when the model does not fully represent the actual relationship between the independent
variables and the dependent variables.

The Difference between Error Terms and Residuals

An error term is generally unobservable and a residual is observable and calculable, making it
much easier to quantify and visualize. In effect, while an error term represents the way observed
data differs from the actual population, a residual represents the way observed data differs from
sample population data.

Adequacy of the regression model

The fitting of linear regression model, estimation of parameters testing of hypothesis properties
of the estimator are based on following major assumptions: 1. The relationship between the study
variable and explanatory variables is linear, atleast approximately 2. The error term has zero
mean 3. The error term has constant variance 4. The errors are uncorrelated 5. The errors are
normally distributed.

The validity of these assumption is needed for the results to be meaningful. If these assumptions
are violated, the result can be incorrect and may have serious consequences. If these departures
are small, the final result and conclusions may not be affected much. But if the departures are
large, the model obtained may become unstable in the sense that a different sample could lead to
an entirely different model with different conclusions. So such underlying assumptions have to
be verified before attempting to regression modeling. To check the assumption of linearity
between study variable and explanatory variables, the scatter plot matrix of the data can be used.

If there is any departure from the assumptions on random errors, then it should be shown up by
the residual. Analysis of residual helps in finding the model inadequacies.

2
References
(n.d.). Retrieved from Investopedia: www.investopedia.com
(n.d.). Retrieved from home.iitk: http://home.iitk.ac.in/~shalab/regression/Chapter1-Regression-
Introduction.pdf
(n.d.). Retrieved from Math for college:
http://nm.mathforcollege.com/mws/gen/06reg/mws_gen_reg_spe_adequacy.pdf
G. Geoffrey Vining, E. A. (2008). Introduction to Linear Regression Analysis.