Download Time Series Regression - Econometrics - Lecture Notes and more Study notes Econometrics and Mathematical Economics in PDF only on Docsity!
Time Series Regression
Serial Correlation and Unit Root
Time series data are obtained from observations for different time periods. Examples are unemployment rates, inflation rate, GDP, stock price, etc. Another common type of data is cross sectional data, which is obtained from the observations for different individuals or different states or different cities, or different countries for the same time period.
Many time series data are readily available from various websites, and we tend to have more chances to use time series in regression analysis.
Time series data are denoted by { y 1 , y 2 , ..., yT }. Subscript denotes the time periods, and T is the
total number of observations. Although this is more popular notation, sometimes people use n instead of T to denote the sample size.
The most distinguished characteristic of time series data is in the serial correlation. The value of next period observation depends on the value of the current period observation. This is very different from the cross sectional data where all the observations are independent from each other. (There is no such thing like order of observations in cross-section data.)
For example, individual income data could be obtained for different people, then it is a cross- sectional data. It is safe to assume that the individual income is independent of each other. But, we can imagine the income data of the same person observed over time. Then, certainly the next period income will depend on the income of the current period --- namely, the data are serially correlated.
It is useful to model the serial correlations of time series data. The simplest model, but still is very popular, is called autoregressive process:
yt = ρ yt − 1 + ε t ,
where ε t is the unobserved disturbance term. We typically assume that the error terms
{ ε 1 , ε 2 , ε 3 , ... ε T } are independent mean zero process.
In the above model, the value of ρ indicates the degree of serial correlation. Higher ρ value
signifies higher correlation across different time period. For example, if ρ = 0, yt = ε t , so the
series is independent. If ρ is positive, the series is positively correlated: If yt-1 is big, yt tends to be big. If ρ is negative, the series is negatively correlated: If yt-1 is big, yt tends to be small.
The following picture shows the movement of the four series where values of ρ = 0, 0.5, 1, -0.5. In any time series that I know of, the value of ρ does not exceed 1 in absolute value. If ρ is greater than one, the series explode as time period extends. As is seen in the picture, time series is more jagged for smaller value of ρ. The series is the most jagged when ρ = -0.5 in the picture, and is most smooth when ρ = 1.
Two cases when ρ = 0, so serially independent case and ρ = 1, which is called unit root process, are practically the most interesting cases.
Serially independent series is important because we hope in the regression analysis that the errors are serially independent, and after we model the series such as autoregressive form, as above, what we hope for again is the error term (or disturbance term) is serially independent.
Unit root case is interesting because many time series data in practice resemble unit root process. The following is the scatter plot of unemployment data we used in the class.
rho = 0
0
2
4
6
8
0 20 40 60 80 100
rho = 0.
rho = 1
0
2
4
6
8
0 20 40 60 80 100
rho = -0.
Dependent Variable = D_UnRate R Square 0. Observations 435 Coefficients Standard Error t Stat Intercept 0.0613243 0.035216318 1. UnRate(-1) - 0.009407 0.005618552 - 1.
This is a lower tail test because we exclude the possibility of β greater than 1.
This regression is called Dickey-Fuller regression and the t-test based on this Dickey-Fuller regression is called Dickey-Fuller test following the names of Dickey and Fuller who conducted this test first. It is important to note that the distribution of t-statistic is not normal in Dickey- Fuller regression. It follows Dickey-Fuller distribution --- they put this name to celebrate these two people. The critical value at 5% significance level of Dickey-Fuller distribution is -2.86, which is much smaller than -1.64 of the lower tail of the standard normal distribution.
In the above example, we cannot reject the null hypothesis of unit root because the t-statistic in the Dickey-Fuller regression is greater than -2.86, and we conclude that the unemployment rate follows unit root.
The above autoregressive process is called autoregressive of order one or AR(1) in the sense that yt depends only on yt-1. We may imagine the case yt depends more than one lagged value of y , which is yt-1, yt-2, …, yt-p. So we need to include yt-1, yt-2, …, yt-p in the regression. Let the number of lags p = 3 in the unemployment example, so we have
unempt = α + β 1 unempt − 1 + β 2 unempt − 2 + β 3 unempt − 3 +ε t
Now the unit root hypothesis is β 1 + β 2 + β 3 = 1, against β 1 + β 2 + β 3 < 1. Through re- parameterization, we have
unempt = α + ( β 1 + β 2 + β 3 ) unempt − 1 − β 2 ( unempt − 1 − unempt − 2 ) − β 3 ( unempt − 1 − unempt − 3 ) +ε t
Further re-parameterization gives
∆ unempt = α + ( β 1 + β 2 + β 3 − 1) unempt − 1 − β 2 ∆ unempt − 1 − β 3 ∆ unemp t − 2 +ε t
where ∆ unemp t = unempt − unempt − 1. Therefore, we run the regression ∆unempt on the
regressors (1, unempt-1, ∆unempt-1 ∆unempt-2), and use the t-statistic for the slope of unempt-1 for unit root test. We obtain the following regression results: Dependent Variable = D_UnRate R Square 0. Observations 436 Coefficients Standard Error t Stat Intercept 0.0729676 0.034271633 2. UnRate(-1) - 0.011569 0.005463024 - 2. D_UnRte(-1) 0.1245459 0.04668557 2. D_UnRte(-2) 0.2286078 0.046794642 4.
Note that if β 2 and β 3 are not zero in the above regression, then AR(1) model is wrong, and the resulting unit root test is not reliable.
In AR(3) model, β 2 and β 3 are estimated to be 0.12 and 0.23, and both are significant. Therefore, the new test based on AR(3) model is more reliable. We now have unit root test statistic -2.12, which still is greater than -2.86. We cannot reject the unit root hypothesis of unemployment rate series.
We need to proceed to include more lags until the coefficient are estimated to be insignificant to for more conclusive evidence whether unemployment rate follows unit root process.
One important implication of unit root hypothesis is that any shock (or error or disturbance) to the time series, in this case to the unemployment rate, will remain in the series. We can see this from
yt = y t − 1 + ε t = yt − 2 + ε t − 1 + ε t = ... = ε t + ε t − 1 + ε t − 2,....
Namely, the current unemployment rate is accumulation of all the shocks. This is not very suggestive, but as we have seen through auto-regression results, unemployment rate process is near unit root.
Time Trend
Many time series are trended. As time goes it grows. Examples are GDP, stock prices, etc. Then, it may be more appropriate to model it adding time trend into it. The simplest model of trended time series would be:
yt = α + γ t + ρ yt − 1 +ε t
Here, “ t ” is the time trend, so that we expect that the time series grows by γ in each period on average.
Example:
GDP grows exponentially. Following graph shows historical U.S. GDP data in 1996 U.S. dollar.
GDP in billions of chained 1996 dollars
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
1920 1940 1960 1980 2000 2020
of the heights of tree A on the heights of tree B. Although they are independent, we will easily have a significant result. The reason behind is that the height of two trees share the common trend. The significance will disappear after we include the time trend as a regressor if both trees grow linearly in time. We may consider the problem as an omitted variable problem, and the omitted variable is the time trend.
We include time trend in regression when we use time series that shows linear time trend.
Prediction:
Suppose that the unemployment follows the unit root process: unempt = unempt − 1 + ε t , and
we want to predict the future unemployment rate based on the past unemployment rates. If we let the current period T , unemployment of the next ( T+1 ) period will be
unempT + 1 = unempT + ε T + 1. Here, the error term (or disturbance term), ε T + 1 , is something that
occurs in the future, and is the source of uncertainty. To predict the future, we need to model
ε T + 1. The simplest assumption is that { ε 1 , ε 2 , ε 3 , ..., ε T , ε T + 1 , ...} follows independent normal
distribution with mean zero and variance σ^2. Then, we can construct the confidence interval for the T+1 period unemployment rate.
We need to estimate σ^2. Since ε t = unempt − unempt − 1. Variance of ε can be estimate using
the data (^) { ∆ unemp (^) 1 (^) , ∆ unemp (^) 2 (^) , ∆ unemp (^) 3 (^) − unemp 2 (^) , ∆ unemp (^) T (^) − 1 , ∆ unemp (^) T }as
( ) 2 2 1
1 T
t t
s unemp
T =
= (^) ∑ ∆
From the data, we have s = 0.18. Unemployment rate of March 2004 is 5.7%. Therefore, 95% confidence interval of April unemployment rate is (5.7 – 1.96×0.18, 5.7 + 1.96×0.18) = (5.35, 6.05).
What about the May unemployment rate? The error term for the May unemployment rate is
ε T + 1 + ε T + 2 , which is normal distribution with mean zero and variance 2σ
(^2) , and the standard
deviation is 2 σ. It is estimated by 2 s = 0.255. Therefore, the 95% confidence interval is
(5.7 – 1.96×0.255, 5.7 + 1.96×0.255) = (5.2, 6.2). Adding a few more periods, we could draw a picture for the 95% confidence interval of the future unemployment rates as follows: DATE Low High 3/1/2004 5.7 5. 4/1/2004 5.35 6. 5/1/2004 5.20 6. 6/1/2004 5.09 6. 7/1/2004 4.99 6. 8/1/2004 4.91 6. 9/1/2004 4.84 6. 10/1/2004 4.77 6. 11/1/2004 4.70 6. 12/1/2004 4.64 6.
Following the AR(1) unit root model, this December’s unemployment rate will be at the interval (4.64%, 6.76%) with 95% confidence.
We can present the above prediction in graph as
Unemployment Rate Prediction (95% Interval)
0
1
2
3
4
5
6
7
8
5/24/2002 12/10/2002 6/28/2003 1/14/2004 8/1/2004 2/17/ Time
Unemployment Rate (%)
Prediction provided above is very preliminary. Government officers and researchers use more complicated model in prediction of unemployment rate.
However, it is important to remember that we have to deal with the future error terms that grow bigger for farther future. All we did is estimation of the standard deviation of these errors using the historical data. Then we assume that these errors follow normal distribution, so use 1.96 as 95% z-score.
We may find some variable that is helpful in prediction of these errors. If we find some relevant information or data that are correlated with these future errors, we can reduce the variance of our prediction, so can better predict the future. It is rather an art to find these information and data that are correlated with future error term of the unemployment rate. But, we may find and verify that they are useful in prediction of the errors using the historical data. What Alan Greenspan and other government economists do is to find the variables that are helpful in prediction, and
incorporate them in prediction.
Prediction of Trend
For trended time series, it is much more important to predict the trend than the stochastic error terms.
As we saw in the graph for the log of GDP, ln(GDP) has a linear trend. The average growth rate is about 3.36% and the standard deviation is 5%. Log of 2003 U.S. GDP (in billion dollar) is 9.25. Log of 2004 is predicted to be 9.25 + 0.0336 = 9.2836, and the 95% confidence interval can be given as (9.2836 – 1.960.05, 9.2836 + 1.960.05) = (9.1856, 9.3816), which becomes, after taking exponent, (97756,11867) billion dollars.
Growth Rate of GDP
-0.
-0.
-0.
-0.
0
1930 1950 1970 1990 2010
Time
Growth Rate
We can see that the series is positively correlated, implying that the higher growth rate
tends to be followed by higher growth rate.
When we run regression of ln(GDPt) on ln(GDPt-1), we have
Dependent Variable = ∆ln(GDPt). R Square 0. Observations 73
Coefficients
Standard Error t Stat P-value Intercept 0.01777004 0. 005813254 3.056814 0. ∆ln(GDPt- 1 ) 0.52095072 0.095413549 5.459924 6.63E- 07
Unlike unit root test, t-statistic follows standard normal distribution under the null
hypothesis that the growth rate, ∆ln(GDPt), is serially independent. Null hypothesis is rejected,
leading to a conclusion that the growth rates are positively correlated.
95% confidence interval for the prediction of future growth rate is based on the assumption of normality and serial independence. Since the growth rate is serially correlated, 95% confidence interval reported above is not valid. There is a way to do it, but is out of scope of this course.
Spurious Regression
Suppose both the dependent variable and independent variables follow unit root process, and they are independent of each other. Therefore, if we run the regression, the true slope is zero. However, the least squares estimator of slope is not estimated appropriately, and the t-statistic tend to get bigger as the sample size grows, and R-square gets close to one. In another words, estimation and the inference based on least squares will lead to a very wrong conclusion. The following result is obtained from the 1000 computer generated numbers (in Excel), where the X and Y are independent, but both follow unit root process.
R Square 0. Standard Error 17. Observations 1000 Coefficients Standard Error t Stat P-value Intercept - 26.353208 1.21434859 - 21.7015182 7.54E- 86 X 1.20822339 0.075305836 16.04421988 1.04E- 51
As is seen, following the t-statistic, the null hypothesis of β = 0 is decisively rejected. This regression is called spurious regression in the sense that the regression result shows that X and Y are correlated while they are in fact independent.
The question is how we know whether time series regression is spurious or not. A simple way is to see whether the residuals from the regression follow unit root. Whenever the residual follow unit root, then the regression is spurious.
Example
A regression is run to see whether tax exemption for each child has effect of the birth rate. Yearly data on 1913 – 1945 were collected on the gfr = births per 1000 women 15-44 years old, pe = per child tax exemption in real dollar over the years. In the regression WW2 = world war 2 dummy which is 1 for 1941-1945, year denotes the year of the data (so it is the trend variable), and pill is the dummy variable for birth control pill, which is one from 1963 and zeros before 1963. They obtained
R Square 0. Observations 72 Coefficients Standard Error T Stat P-value Intercept 2310.32472 361.4199 6.392355 1.82E- 08 Pe 0.278877806 0.04002 6.968485 1.72E- 09 Year - 1.149872015 0.187904 - 6.11947 5.49E- 08 Pill 0.997446981 6.26163 0.159295 0. WW2 - 35.59228379 6.297377 - 5.65192 3.53E- 07
From the regression we see that tax exemption has a significant effect on the birth rate, a very nice result. However when we plot the residuals on the time to have
residuals
0
10
20
1910 1920 1930 1940 1950 1960 1970 1980 1990
What is the problem? Obviously, the residual shows a highly persistent pattern. Therefore, it is constructive to check whether the residuals follow unit root.
Regression Using Time Series Data
The underlying problem behind of spurious regression is that the high serial correlations in the error. But, even when the errors does not follow unit root, but is correlated more moderately, there remains problem. Most difficult problem is in the estimation of the standard errors. There are a few different ways of estimating standard errors in the presence of moderate serial correlations. But, details of this should be left to more advanced level.
