Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
The autoregressive process of order p or AR(p) is defined by the equation ... MA(q) can define correlated noise structure in our data.
Typology: Exams
1 / 37
This page cannot be seen from the preview
Don't miss anything!
AR, MA and ARMA models
The autoregressive process of order
(^) p (^) or
(^) AR
(p ) is defined
by the equation
p
j=1 ∑
φ j X t− j
(^) ω t
where
(^) ω t ∼ (^) N (^) ( , σ 2 )
φ (^) = (
φ 1 , φ 2 ,... , φ
p ) is the vector of model coefficients and
p (^) is a non-negative integer.
The AR model establishes that a realization at time
(^) t (^) is a
linear combination of the
(^) p (^) previous realization plus some
noise term.
For
(^) p (^) = 0,
(^) ω t and there is no autoregression term.
53
The lag operator is denoted by
and used to express
lagged values of the process so
(^) X t− 1 ,
2 X t = (^) X t− 2 , B 3 X t =
t− 3 ,...
, B d X t− d .
If we define Φ( B ) = 1
p
j=1 ∑
φ j B j = 1
(^) − (^) φ 1 B (^) − (^) φ 2 B (^2) −
(^)...
(^) − (^) φ p B p
Φ( the AR(p) process is given by the equation B ) X t = (^) ω t; (^) t (^) = 1
,... , n
) is known as the
(^) characteristic
(^) polynomial of the
stationary or not.process and its roots determine when the process is
The moving average process of order
(^) q (^) or
(^) M A
q ) is
54
The ARMA process of orders
(^) p (^) and
(^) q is defined as
p
j=1 ∑
φ j X t− j
q
j=1 ∑
θ j ω t− j
(^) ω t
In lag operator notation, the
(p, q
) process is
given by Φ(
t = Θ(
ω t, t (^) = 1
,... , n
polynomial.Lets focus on the AR process and its characteristic
The characteristic polynomial can be expressed as:
p
i=1 ∏
( (^) − (^) α iB )
where the
(^) α s′ (^) are the reciprocal roots.
If (^) β 1 , β 2 ,... , β
p (^) are such that Φ(
β i) = 0 (roots of the
56
polynomial) then
(^) β 1 (^) = 1
/α 1 , β 2 (^) = 1
/α 2 ,... , β
p (^) = 1
/α p
Theorem:
If (^) X t ∼ (^) AR
p ), (^) X t is a stationary process if and
polynomial are greater than one, i.e. ifonly if the modulus of all the roots of the characteristic
β i|| (^) > (^) 1 for all
i = 1
, 2 ,... , p
(^) or equivalently if
α i|| (^) < (^1) , i (^) = 1
,... p
The
(^) α is′ (^) are also known as the
(^) poles
(^) of the AR process.
This theorem follows from the
(^) general linear process
theory.
distinguish between the 2 cases.and some can be complex numbers and we need toSome of the poles or reciprocal roots can be real number
57
β (^) = 1
/φ (^) (assuming
(^) φ (^6) = 0).
The AR(1) process is stationary if only if
φ | < (^) 1 or
(^) < φ <
The case where
(^) φ (^) = 1 corresponds to a Random Walk
process with a zero drift,
(^) X t− 1 (^) + (^) ω t
This is a non-stationary explosive process.
X Walk process can be expressed asIf we recursive apply the AR(1) equation, the Random t = (^) ω t
(^) ω t− 1 (^) + (^) ω t− (^2)
(^)... . Then,
V ar
t) =
(^) ∑
t=0∞
(^) σ 2 =^ (^) ∞
.
Example. AR(2) process
(^) φ 1 X t− (^1)
(^) φ 2 X t− 2 (^) + (^) ω t
The characteristic polynomial is now
59
(^) φ 1 B (^) − (^) φ 2 B 2 )
The solutions to Φ(
) = 0 are
β (^1) = − φ 1 (^) +
√ φ 1 2
φ 2
2 φ 2 ; β 2
φ (^1) − √ φ 12 (^) + 4
φ 2
φ 2
The reciprocal roots are
α 1 (^) =
φ 1 (^) +
√ φ 1 2
φ 2
α 2 (^) = (^) φ 1 (^) −
√ φ 12 (^) + 4
φ 2
The AR(2) is stationary if and only if
α 1 || (^) < (^) 1 and
α 2 || (^) < (^1)
These two conditions imply that
α 1 α 2 || (^) = (^) | φ 2 | < (^) 1 and
α (^1)
(^) α 2 || (^) = (^) | φ 1 | < (^) 2 which means
(^) < φ
(^2) < (^) 1 and
(^) < φ
1 (^) < (^) 2.
60
form,
j=0 ∑ ∞
a j ω t− j
where
(^) ω t is a white noise sequence with variance
(^) σ 2 .
expressionIn lag operator notation, the general linear is given by the
t = Φ(
B ) − 1 ω t
where Φ(
− 1 =^ (^) ∑
j=0∞
(^) a j B j .
E Note firstly that by the definition of the linear process, (X t) = 0.
Then, the covariance between
t and
s is
tX s ] =
j=0 ∑ ∞
l=0 ∑ ∞
a j a lE [X t− j X s− l]
62
σ 2 j=0 ∑ ∞
a j a j+ s− t ; ( s (^) ≥ (^) t )
The last expression depends on
(^) t (^) and
(^) s (^) only through the
difference
(^) s (^) − (^) t
. Therefore, the process is stationary if
∑ j=0∞
(^) a j a j+ k is finite for all non-negative integers
(^) k
Setting
(^) k (^) = 0 we require that
(^) ∑
(^) a j 2 < (^) ∞
Given that a correlation is always between
1 and 1,
|γ k | ≤
(^) γ 0
so if
(^) γ 0 (^) < (^) ∞
(^) then
(^) ∑
j=0∞
(^) a j a j+ k (^) < (^) ∞
.
Then
t is stationary if and only if
(^) ∑
j=0∞
(^) a j 2 < (^) ∞
The MA(
q ) process can be written as a
(^) general linear
63
Definition:
A process
t is invertible if
j=1 ∑ ∞
a j X t− j
(^) ω t
with the restriction that
(^) ∑
j=1∞
(^) a j 2 < (^) ∞
autoregression.Basically, an invertible process is an infinite
By definition the AR(
p ) is invertible. We can set
a (^1) = (^) φ 1 , a 2 (^) = (^) φ 2 ,... a
p (^) = (^) φ p (^) and
(^) a j = 0
, j > p
. Then
∑ j=1∞
(^) a j 2 = (^) ∑
=1p (^) φ j 2 which is finite.
For an MA(
q ) process we have
t = Θ(
ω t. If we find a
polynomial Θ(
− (^1) such that Θ(
− (^1) = 1 then we
can invert the process since Θ(
B ) − 1 X t =
(^) ω t
65
The MA(
q ) process is invertible if and only if the roots of
) have all modulus greater than one.
X To illustrate this last point consider the MA(1) process t = (
(^) θB
) ω t
If (^) | θ | < (^) 1 then
− 1 =^
(^) θB
j=0 ∑ ∞
θ j B j
Since
θ | < (^) 1 then
(^) ∑
j=0∞
(^) θ j < (^) ∞
(^) and so the process is
invertible and has the representation
j=1 ∑ ∞
θ j X t− j
(^) ω t
The ARMA(
p ,q ) process is invertible whenever the MA
66
The coefficients
(^) φ k 1 , φ k 2 ,... , φ
kk
define the PACF.
can be obtained via Cramer’s Rule.We have a set of linear equations for which the solution
acf(x,type=’’partial’’)R/S-plus include an option to compute the PACF.
68
Suppose thatThe partial autocorrelation can be derived as follows.
t is zero mean stationary process.
Consider a regression model where
t+ k is regressed on
(^) k
lagged variables
t+ k − 1 , Z t+ k − 2 ,... , Z
t, i.e.,
t+ k (^) = (^) φ k 1 Z t+ k − (^1)
(^) φ k 2 Z t+ k − (^2)
(^)...
(^) + (^) φ kk Z t
(^) ω t+ k
φ ki (^) denotes the i-th regression parameter and
(^) ω t+ k is a
normal error term uncorrelated with
t+ k − j for (^) j (^) ≥ (^) 1.
Multiplying
t+ k − j on both sides of the above regression
equation and taking the expectation, we get
γ j = (^) φ k 1 γ j− (^1)
(^) φ k 2 γ j− 2 (^) +
(^)...
(^) + (^) φ kk γ j− k
69
φ 22
=
∣ ∣∣∣ ∣ ∣ 1
ρ 1
ρ (^1)
ρ 2 ∣ ∣∣∣ ∣ ∣
∣∣ ∣∣ ∣ ∣ 1
ρ 1
ρ (^1)
ρ 1 ∣∣ ∣∣ ∣ ∣
φ 33
=
∣ ∣∣∣ ∣∣ ∣ ∣ 1
ρ 1
ρ 1
ρ (^1)
1
ρ 2
ρ (^2)
ρ 1
ρ 3 ∣ ∣∣∣ ∣∣ ∣ ∣
∣ ∣∣∣ ∣∣ ∣ ∣ 1
ρ 1
ρ 2
ρ (^1)
1
ρ 1
ρ (^2)
ρ 1
1
∣ ∣∣∣ ∣∣ ∣ ∣
71
φ kk
=
∣ ∣∣ ∣∣∣ ∣∣ ∣∣∣ ∣ 1
ρ (^1)
ρ (^2)
...
ρ k − (^2)
ρ 1
ρ 1
1
ρ (^1)
...
ρ k − (^3)
ρ 2
... . .. . .. . .. . ..
...
ρ k − (^1)
ρ k − (^2)
ρ k − (^3)
...
ρ (^1)
ρ k ∣ ∣∣ ∣∣∣ ∣∣ ∣∣∣ ∣
∣∣ ∣∣∣ ∣∣ ∣∣∣ ∣ ∣ 1
ρ (^1)
ρ (^2)
...
ρ k − (^2)
ρ k − 1
ρ (^1)
1
ρ (^1)
...
ρ k − (^3)
ρ k − 2
... . .. . .. . .. . ..
...
ρ k − (^1)
ρ k − (^2)
ρ k − 3
...
ρ (^1)
1
∣∣ ∣∣∣ ∣∣ ∣∣∣ ∣ ∣
As a function of
(^) k , φ kk
is usually referred to as the
partial autocorrelation function (PACF).
A computer package will produce an estimate of
(^) φ k,k
using ˆ
ρ k
72
