Introduction to Time Series Analysis of Macroeconomic- and Financial-Data. Lecture 5: Trends, Model Selection, and Summary

Introduction Introduction to Time Series Analysis of Macroeconomic- and Financial-Data Felix Pretis Programme for Economic Modelling Oxford Martin School, University of Oxford Lecture 5: Trends, Model Selection, and Summary Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 1 / 88

Re-Cap So far: Auto-regressive Models Forecasting works well if nothing changes over time...(i.e. world is stationary) But...things change all the time! (think back 10, 20, 100 years) World is non-stationary!...how to model? Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 2 / 88

Today AR(1) model has three elements: (1) Where Y t was the last time period. (2) The unexpected event ɛ t. (3) Constant term allowing mean of Y t to be non-zero. Y t = β }{{} 1 + β 2 Y }{{ t 1 } (3) (1) + ɛ t }{{} (2), ɛ t N0, σ 2. (1) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 3 / 88

Do things remain the same, or change through time? We expect Y t to change through time (hence ɛ t ). But we expect it to revert to its equilibrium over time: Equilibrium value = mean value. But what about structural change? Financial crises? Does economic progress mean that equilibria change? Stationarity: Underlying distribution of Y t is time invariant. All aspects of distribution: Mean, variance, skewness, kurtosis. In practice we focus on Mean and variance. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 4 / 88

A Stationary Series Always has the same data generating process: No structural change taking place. Model as: Y t = β 1 + β 2 Y t 1 + ɛ t, iid(0, σ 2 ). (2) Stationarity implies that β 2 < 1: Shocks die away. Stationarity implies that: E(Y t ) = µ y = β 1 1 β 2. (3) Long-run equilibrium that economic variable returns to. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 5 / 88

Stationary Series Forecasts DLCPI US Inflation 0.01 0.00-0.01 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020 Without shocks we expect series to settle down to equilibrium. For US inflation that mean is (EY t ) = ˆβ 0 /(1 ˆβ 1 ) = 0.003. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 6 / 88

Forms of Non-Stationarity Stationary is the exception not the norm! Types of Non-stationarity: like going to zoo to look at non-elephants 1... Many many forms... = Three common forms encountered and tame-able: 1: Linear Trends 2: Structural Breaks (sudden change in parameters) 3: Unit-roots/Stochastic trends (Advanced) 1 with thanks to Anders Rahbek Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 7 / 88

1: Linear Trends 750 Wheat Price Index UK 500 250 1800 1820 1840 1860 1880 1900 1920 1940 1960 1980 2000 Example: Price of Wheat in the UK from 1800-1996 Series trending downwards - need to account for trend Can include a linear trend as regressor Y t = β 1 + β 2 t + ɛ t (4) Can treat as normal regressor (standard inference applies) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 8 / 88

Model Fit: Trend vs. Constant 750 500 3 Wheat Fitted r:wheat (scaled) 2 1 0-1 250-2 1800 1850 1900 1950 2000 1800 1850 1900 1950 2000 750 500 250 Wheat Fitted 2 r:wheat (scaled) 1 0-1 1800 1850 1900 1950 2000 1800 1850 1900 1950 2000 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 9 / 88

Spurious Correlation What happens if we use non-stationary trending variables in our regression models? Y t = β 1 + β 2 X t + ɛ t (5) Suppose β 2 = 0 (no relationship between Y t and X t : if: Y t and X t are stationary (stable) no problem if Y t is trending and X t is trending risk of spurious relationship! Find more spurious correlations: http://tylervigen.com/spurious-correlations Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 10 / 88

Spurious Relationship: Example Japanese GDP and Cumulative Rainfall in Fortaleza (NE Brazil) 500000 Japan RGDP 6000 Cumulative Rainfall Fortaleza Brazil 400000 5000 4000 300000 3000 200000 2000 100000 1000 1960 1970 1980 1990 2000 1960 1970 1980 1990 2000 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 11 / 88

Model RGDP using Rainfall 400000 Japan RGDP Model Fit 200000 1955 1960 1965 1970 1975 1980 1985 1990 2 Residuals 0 1955 1960 1965 1970 1975 1980 1985 1990 Model Jap.RDGP t = 23352.2 (4938) + 67.89 (1.45) cumrainfall Brazil t R 2 =0.98, F(1,36) = 2206 [0.000]** Cumulative Rainfall highly significant, explains 98% of variation in Japanese GDP! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 12 / 88

Taming the trend Need to account for the trend - two options: Model the trend: include a trend variable Remove the trend: difference the data, use Y t, X t 20000 DJapan GDP 0 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 300 DCumulativeRainfall_Brazil 200 100 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 13 / 88

Option 1: Model the Trend Jap.RGDP = 9467 (8497) + 11 (28.9) cumrainfall Brazil t + 9222 (4679) Trend t ˆβ 2 = 11 with (se = 28.9) not statistically significant anymore! The estimation sample is: 1955-1992 Coefficient Std.Error t-value t-prob Constant 9467.74 8497. 1.11 0.2728 cumrainfall_brazil 11.0163 28.89 0.381 0.7053 Trend 9222.44 4679. 1.97 0.0567 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 14 / 88

Option 2: Remove the Trend by Differencing By taking differences we can remove the trend: Subtract Y t 1 from both sides: Y t = β 1 + β 2 X t + β 3 t + ɛ t (6) Y t Y t 1 = β 1 + β 2 X t + β 3 t + ɛ t Y t 1 This is equal to: Y t = β 1 + β 2 X t + β 3 t + ɛ t (β 1 + β 2 X t 1 + β 3 (t 1) + ɛ t 1 ) and simplifies to: The trend is gone! Y t = β 2 X t + β 3 + ɛ t 1 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 15 / 88

Option 2: Differencing Japanese RGDP and Brazilian Rainfall 400000 Japan RGDP 5000 Cumulative Rainfall 200000 2500 20000 0 1960 1970 1980 1990 2000 300 D Japan RGDP 200 100 1960 1970 1980 1990 2000 1960 1970 1980 1990 2000 D Cumulative Rainfall 1960 1970 1980 1990 2000 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 16 / 88

Option 2: Differencing Model in differences: Jap.RGDP = 10792.5 (2976) + 0.125 (18.3) cumrainfall Brazil t Coefficient Std.Error t-value t-prob Constant 10792.5 2976. 3.63 0.0009 DcumRainfall_Brazil 0.125866 18.29 0.00688 0.9945 20000 D Japan RGDP Fitted 10000 0 1955 1960 1965 1970 1975 1980 1985 1990 No significant relationship! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 17 / 88

Model Building How to build a model: 1 Come up with an interesting question (hard!) 2 Collect the data 3 Plot the data! relationship and functional forms Over time: trending? stable? Scatter plots: relationship? linear? non-linear? Time series properties: PACF 4 Build general model estimate Diagnostic tests satisfied? Can the model explain the data? Yes? Hooray! (very very rare...) No? Modify model and repeat. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 18 / 88

Automatic Model Selection Build general model estimate modify: labour intensive! Automatic Model Selection Start with very general model ( General Unrestricted Model ) Drop variables through path search General to Specific Making sure diagnostic tests satisfied Finds simplest possible econometric model that is well specified Autometrics in OxMetrics Note: Does not replace you: garbage in, garbage out! Reduces hard work selecting correct model. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 19 / 88

Autometrics in Practice 1 Start with general model: All possibly important variables and lags (can fix by setting as U - fixed ). 2 At next menu select Autometrics box; choose p-value. Drops variables until all significant at chosen p α (e.g. 0.05) 3 Click OK and OK and wait for OxMetrics to select final model. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 20 / 88

What does Autometrics do? Tree-Search drop variables and make sure models well specified. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 21 / 88

Autometrics General Model Many lags, many potentially important variables Search Algorithm Estimates many models starting from the general model Tests model diagnostics Tests variables for significance Final Model Smallest, well-specified model Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 22 / 88

Automatic Model Selection So far: Japanese Kuznets curve model mis-specified: Coefficient Std.Error t-value t-prob Lco2_pc_1 0.759387 0.07734 9.82 0.0000 Constant -8.24547 4.841-1.70 0.0953 Lrgdp_pc 1.69733 0.9750 1.74 0.0884 Lrgdp_pc_sq -0.0820198 0.04775-1.72 0.0926 AR 1-2 test: F(2,44) = 3.7621 [0.0310]* ARCH 1-1 test: F(1,48) = 0.85793 [0.3590] Normality test: Chiˆ2(2) = 0.054841 [0.9730] Hetero test: F(5,44) = 2.1576 [0.0762] Hetero-X test: F(8,41) = 2.8564 [0.0128]* RESET23 test: F(2,44) = 5.9879 [0.0050]** How can we fix it? let the data speak: automatic model selection. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 23 / 88

Automatic Model Selection Start with general model for log(co2): log(co2) t 1 (for auto-correlation) log(rgdp) t, log(rgdp) t 1 log(rgdp) 2 t, log(rgdp)2 t 1 log(rgdp) t, log(rgdp) t 1 (economic growth?) General Model: log(co2) t = β 1 + β 2 log(co2) t 1 + β 3 log(gdp) t +β 4 log(gdp) t 1 + β 5 log(gdp) 2 t +β 6 log(gdp) 2 t 1 + β 7 log(gdp) t +β 8 log(gdp) t 1 + ɛ t using Autometrics... Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 24 / 88

Automatic Model Selection Autometrics Selection at p = 0.01 (1%): General Model: log(co2) t = β 1 + β 2 log(co2) t 1 + β 3 log(gdp) t +β 4 log(gdp) t 1 + β 5 log(gdp) 2 t +β 6 log(gdp) 2 t 1 + β 7 log(gdp) t +β 8 log(gdp) t 1 + ɛ t Estimated 14 different models until reduced to Specific Model: log(co2) t = β 1 + β 2 log(co2) t 1 + β 7 log(gdp) t + ɛ t What is left? Persistence and Economic growth! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 25 / 88

Automatic Model Selection Coefficient Std.Error t-value t-prob Lco2_pc_1 0.995420 0.02190 45.5 0.0000 DLrgdp_pc 1.20914 0.2062 5.86 0.0000 Constant U -0.00800367 0.04953-0.162 0.8723 AR 1-2 test: F(2,44) = 2.0199 [0.1448] ARCH 1-1 test: F(1,47) = 0.86555 [0.3569] Normality test: Chiˆ2(2) = 2.8113 [0.2452] Hetero test: F(4,44) = 0.87707 [0.4854] Hetero-X test: F(5,43) = 0.68684 [0.6360] RESET23 test: F(2,44) = 2.0449 [0.1415] log(co2) Fitted scaled residual 2 2.0 0 1.5-2 1960 1970 1980 1990 2000 2010 1960 1970 1980 1990 2000 2010 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 26 / 88

Specific Model log(co2) = 0.995 (0.0219) T=49 R 2 =0.987 log(co2) t 1 + 1.21 (0.206) AR 1-2 test: F(2,44) = 2.0199 [0.1448] ARCH 1-1 test: F(1,47) = 0.86555 [0.3569] Normality test: Chiˆ2(2) = 2.8113 [0.2452] Hetero test: F(4,44) = 0.87707 [0.4854] Hetero-X test: F(5,43) = 0.68684 [0.6360] RESET23 test: F(2,44) = 2.0449 [0.1415] Maybe no Kuznets curve? High persistence: ˆβ 1 =0.99 Economic Growth Effect: log(rgdp) t, ˆβ 7 =1.21 increased CO2 emissions during boom decreased emission during recession log(rgdp) t 0.008 (0.0495) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 27 / 88

CO2 Emissions and Economic Growth 10.0 7.5 kg CO2 Real GDP 1961-2006 CO2 Real GDP 2007-2011 1961-2006 2007-2011 5.0 7500 10000 12500 15000 17500 20000 22500 25000 27500 30000 32500 35000 Real GDP 0.1 D log(real GDP) log(usd) 0.0 Recession 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 28 / 88

Autometrics has more uses Automatic Model Selection to detect which variables matter to find well-specified models to detect when changes/structural breaks happen? Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 29 / 88

Structural Breaks Parameters not constant: β β e.g. Temporary shift in mean Already encountered two potential examples: Japanese Exports 0.5 D12LJapanExports Fitted Residuals 2.5 0.0 0.0-0.5-2.5 1990 2000 2010 1990 2000 2010 Baseball Attendance %Attendance Fitted 0.0100 0.0075 0.0050 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2.5 Residuals (scaled) Large Residual! 0.0-2.5 Large Residual! 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 30 / 88

Correcting for Breaks Dummy Variable: = 1 during break = 0 otherwise accounts for all variation of these observations = as if removed from the sample and model estimated outside of the break period E.g. Dummy: 2008:6 2011:2 0.50 0.25 D12LJapanExports 0.00-0.25-0.50 1980 1985 1990 1995 2000 2005 2010 1.00 Dummy: 2008:6--2011:2 0.75 0.50 0.25 0.00 1980 1985 1990 1995 2000 2005 2010 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 31 / 88

Detection of Breaks How can we detect and control for breaks? Presence of breaks affects parameter estimates Parameter estimates affect whether we can detect a break simultaneity problem, must do estimation and detection jointly! Impulse Indicator Saturation: add indicator (0/1) variable for every observation Keep only significant ones easy to do with Autometrics (Automatic Model Selection Algorithm in OxMetrics/PcGive) Choose: Outlier and Break Detection IIS Searches over full set of dummy variables keeps significant Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 32 / 88

Japanese Exports Model: T 12 Y t = α 12 Y t 1 + µ + 1 i=t δ i i=1 Estimated Model & Detected Breaks (at 0.1%): D12LJapanExports = 0.9 (0.019) + 0.17 (0.049) + 0.0022 (0.0026) D12LJapanExports t 1 0.24 (0.049) I:2009(11) t + 0.17 (0.049) I:2008(11) t 0.22 (0.049) I:2009(12) t + 0.24 (0.049) I:2010(1) t I:2009(1) t identifies break period from: 2008:11-2010(1) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 33 / 88

Japenese Exports 0.5 D12LJapanExports Fitted 0.0-0.5 1980 1985 1990 1995 2000 2005 2010 2.5 r:d12ljapanexports (scaled) 0.0-2.5 1980 1985 1990 1995 2000 2005 2010 1.0 I:2008(11) I:2009(1) I:2009(11) I:2009(12) I:2010(1) Detected Breaks: 0.5 1980 1985 1990 1995 2000 2005 2010 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 34 / 88

Baseball Attendance %Attendance Fitted 0.0100 0.0075 0.0050 2.5 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 Residuals (scaled) Large Residual! 0.0-2.5 Large Residual! 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 Breaks? Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 35 / 88

Baseball Attendance %Attend. t = α 1 %Attend. t 1 + β 1 + β 2 PCT t + β 3 Unemp. t + ɛ t %Attend. = 0.5815 (0.0913) + 0.00006 (0.000104) %Attend. t 1 0.003985 (0.00171) Unempl. t + 0.01307 (0.00335) PCT t AR 1-2 test: F(2,45) = 0.60392 [0.5510] ARCH 1-1 test: F(1,49) = 0.67357 [0.4158] Normality test: Chiˆ2(2) = 11.261 [0.0036]** Hetero test: F(6,44) = 0.53186 [0.7810] Hetero-X test: F(9,41) = 0.43033 [0.9109] RESET23 test: F(2,45) = 0.10631 [0.8994] What now?? Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 36 / 88

Baseball: Break detection Use Impulse Indicator Saturation: %Attend. t = α 1 %Attend. t 1 +β 1 +β 2 PCT t +β 3 Unemp. t + Detected two outliers or breaks: in 1967 and 1981! %Attend. t = 0.65 (0.087) 0.48 (0.16) + 0.16 (0.31) α 1 %Attend. t 1 + 0.33 (0.097) + 1.1 (0.28) PCT t 1 PCT t + 0.017 (0.0083) T 1 i=t δ i +ɛ t i=1 I:1967 t 0.42 (0.09) Unemp. t I:1981 t Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 37 / 88

Baseball Attendance Breaks detected: 1967 (+0.33), 1981 (-0.42) what happened? 1967: Red Sox reach World Series for first time in 20 years 1981: Players Strike (38% Games cancelled!) Corrected for Breaks - Effects: Coefficient on unemployment significant! ˆβ 3 = 0.017, se=0.0083, = p ˆβ 3 = 0.049 Diagnostic Tests AR 1-2 test: F(2,42) = 0.32262 [0.7260] ARCH 1-1 test: F(1,49) = 1.0971 [0.3001] Normality test: Chiˆ2(2) = 0.64421 [0.7246] Hetero test: F(8,40) = 0.66101 [0.7220] Hetero-X test: F(14,34) = 1.6272 [0.1216] RESET23 test: F(2,42) = 1.7242 [0.1907] Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 38 / 88

Structural Breaks Sudden changes/shocks are common! Need to be controlled for! Can be detected (IIS, many other ways!) Correct parameter estimates Interpretation of shock itself! (e.g. baseball, recession...) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 39 / 88

Practical: Automatic Selection Automatic Model Selection - Kuznets Curve Use Autometrics at 1% General Model: Constant (U-fixed) log(co2) t 1 log(rgdp) t, log(rgdp) t 1 log(rgdp) 2 t, log(rgdp)2 t 1 log(rgdp) t, log(rgdp) t 1 Does model selection ever result in an exact Kuznets curve specification? Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 40 / 88

Practical: First Differences Model in first differences Estimate the Kuznets curve in first differences! Interpret significance and diagnostics! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 41 / 88

Practical: Structural Breaks Dealing with structural breaks: Constructing Dummy Variables: common approach if known time period is problematic (e.g. World War II) Detection of Breaks: finding breaks/disturbances without prior knowledge of their occurrence Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 42 / 88

Practical: Baseball Model Load data red sox 1948.in7 Estimate the ADL model: %Attend. t = α 1 %Attend. t 1 + β 1 + β 2 PCT t + β 3 Unemp. t + ɛ t Recall two drastic events: 1967: World Series for first time! 1981: Player s Strike Construct 2 dummy variables using calculator: I:1967 I:1981 Estimate the model with dummies: %Attend. t = α 1 %Attend. t 1 + β 1 + β 2 PCT t + β 3 Unemp. t +δ 1 I : 1967 + δ 2 I : 1981 + ɛ t Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 43 / 88

Practial: Fitted Model 1.00 att_pop100 Fitted Note the perfect fit, observations "dummied" out! 0.75 0.50 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2 r:att_pop100 (scaled) 1 0-1 -2 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 44 / 88

How could we have found these breaks? Using break detection methods: e.g. IIS (Impulse Indicator Saturation) Re-estimate the above model using IIS Mark all regressors as U - fixed. Select IIS in Automatic Model Selection Choose p α = 0.01 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 45 / 88

Practial: Additional Useful Features: Useful features of OxMetrics (not covered so far): Aggregate: change frequencies of measurement Batch files: store model code in file to be replicable (ALT+B) Test - Further Output: write model results in equation format Ox Programming Language: more complex models Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 46 / 88

Re-Cap of the Course Themes: OLS Regression Model Mis-specification Dependence over time Forecasting Non-Stationarity Cointegration Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 47 / 88

OLS Regression Assumptions: (i) (Y t, X t, Z t ) independent across t. (ii) Identical conditional distribution: (Y t X t, Z t ) (β 1 + β 2 X t + β 3 Z t, σ 2 ). (iii) X t and Z t exogenously determined for Y t. (iv) A parameter space exists: β 1, β 2, β 3, σ 2 R R +, Gives model: Y t = β 1 + β 2 X t + β 3 Z t + ɛ t, ɛ t N0, σ 2 (7) Parameter interpretation as before except ceteris paribus: EY t X t, Z t X t = β 2, (8) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 48 / 88

Misspecification Conclusions drawn from model only valid if model is well-specified! 1 Heteroskedasticity (variance changes through sample) (Hetero) 2 Errors normally distributed (Normality) 3 Functional form of the model (RESET) 4 Residual Auto-correlation (AR) 5 Variance persistent (ARCH) How to spot: Plot the series, fitted values, and residuals! OxMetrics reports mis-specification tests by default! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 49 / 88

Depdendence over Time Time series = dependence over time between observations. Has to be modelled! How to spot dependence and determine lag length: (Partial) Auto-correlation function Cross-correlation function How to model: Auto-regressive model Auto-regressive distributed lag model (lags of dependent and independent variables) = all models of this type are equilibrium-correction! = Estimate long-run mean in-sample and correct towards it! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 50 / 88

Forecasting Forecasting easy to implement but hard to do well (especially about the future...)! h-step forecasts (use real observations to update) Dynamic forecasts (use forecasted values to update) Do not evaluate a model by its forecast performance! Good models can forecast badly (or well...) Bad models can forecast well (or badly...) no connection between forecast performance and model validity Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 51 / 88

Non-Stationarity and Model Selection The world is non-stationary things change all the time! (think back 50 years ago) Many forms, some tameable: Trends: beware of spurious correlation! add linear trend to model! or take differences! Structural breaks: detect & correct for (dummy variables) Model Building and Model Selection Need to use your judgement and expertise! Can use automated methods (Autometrics) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 52 / 88

Putting it all together... Large set of tools - pick & mix but should follow coherent structure! Project Structure Example: Environmental Kuznets Curve (from Problem Sets) 1) Introduction Introduce the theory: Environmental Kuznets curve suggests inverse U-shape between income and pollution Why? - transition of developing economies Previous literature? Investigate using per capita CO2 emissions and per capita GDP for Japan Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 53 / 88

Data 2) Data per capita CO2 emissions (World Bank, in kg) per capita real GDP (FRED, in 2011 USD) theory suggests proportional relationship: log transform plot the data (raw and in logs) time series: in levels and differences scatter plot: CO2 against GDP Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 54 / 88

Data Plots 2) Data (continued) 10.0 7.5 kg 5.0 CO2 per capita Real GDP per capita log(co2 kg) log(co2 kg) 7500 10000 12500 15000 17500 20000 22500 25000 27500 30000 32500 35000 USD (2011) 2.5 log(co2) 10.5 log(rgdp) 2.0 10.0 1.5 9.5 1.0 9.0 1960 1970 1980 1990 2000 2010 0.2 log(co2) 0.1 0.0 log(usd) log(usd) 1960 1970 1980 1990 2000 2010 0.1 log(rgdp) 0.0 1960 1970 1980 1990 2000 2010 1960 1970 1980 1990 2000 2010 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 55 / 88

Methodology 3) Methodology Theory: inverse U-shape, requires quadratic functional form: log(co2) t = β 1 + β 2 log(rgdp) t + β 3 log(rgdp) 2 t + ɛ t Before we estimate, determine the time series properties: Partial (auto-correlation function) suggests 1-lag of log(co2) We also report a forecasting scenario and model in first differences to account for potential trends: Forecast from 1991 onwards First difference model: log(co2) t = β 1 + β 2 log(rgdp) t + β 3 log(rgdp) 2 t Use Automatic Model Selection to improve on theory model: include more lags and log(rgdp) t (economic growth) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 56 / 88

Results 4) Results Present model estimates in table incl. coefficients, standard errors, #observations, diagnostic tests for each model! plot fitted values and residuals! Table: Modelled Variable: log(co2 Emissions per capita) Variable Estimated Coefficient Constant -46.48 (4.15)** L(RGDP) 9.32 (0.85)** L(RGDP) 2-0.45 (0.044)** R 2 0.96 T 51 AR 1-2 Test 67.647 [0.00]** ARCH 1-1 Test 48.077 [0.00]** Normality Test 1.3641 [0.50] Hetero Test 5.6484 [0.002]** RESET Test 20.552 [0.00]** Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 57 / 88

4) Results continued: 2.25 log(co2 per capita) Fitted Model Residuals 1-step Forecasts 2.30 log(co2 per capita) 2.00 2 2.25 1 2.20 log(co2 kg) 1.75 1.50 SD 0 log(co2 kg) 2.15 2.10 1.25-1 2.05 1960 1980 2000 1960 1980 2000 1980 2000 Label the axes! Label your data! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 58 / 88

4) Results continued: Automatic Model Selection Results general model (incl. lags and economic growth) reduced to: log(co2) = 0.995 (0.0219) T=49 R 2 =0.987 log(co2) t 1 + 1.21 (0.206) AR 1-2 test: F(2,44) = 2.0199 [0.1448] ARCH 1-1 test: F(1,47) = 0.86555 [0.3569] Normality test: Chiˆ2(2) = 2.8113 [0.2452] Hetero test: F(4,44) = 0.87707 [0.4854] Hetero-X test: F(5,43) = 0.68684 [0.6360] RESET23 test: F(2,44) = 2.0449 [0.1415] log(rgdp) t 0.008 (0.0495) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 59 / 88

Discussion 5) Discussion Interpret model results: estimated 9.32% increase in emissions for 1% increase in GDP, diminishing marginal effect: -0.44% t-tests (individual) and F-tests (joint significance) ˆβ 2 > 0, ˆβ 3 < 0, consistent with EKC Estimates imply turning point of 34,500 USD ( 2007 in Japan) BUT: Model mis-specified, Kuznets theory not sufficient to explain CO2 emissions Automatic model selection: well specified model reduced to high persistence and economic growth effect suggests little evidence of Kuznets curve Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 60 / 88

Conclusion & References 6) Conclusion Summarize project: investigated EKC Mixed evidence of EKC for Japan, Japan past turning point But more likely that economic growth effect holds Further research: other countries, more variables 7) References Cite all data sources and referenced literature! e.g.: Stern, D. I. (2004). The rise and fall of the environmental Kuznets curve. World Development, 32 (8), 1419 1439. avoid plagiarism! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 61 / 88

Principles of Modelling Start general and work to specific: Too few variables: Bias, cannot trust any statistics. Too many: too many to cope with? General: Can encompass many theories and test each one. Solution to problem of too many variables: Omit insignificant variables! If t-statistics are very small, omit variables. But time consuming and we can make mistakes. Can use automatic model selection. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 62 / 88

Potential Topics Environmental Kuznets Curve: look at other countries and extend the analysis. When are their estimated turning points? Purchasing Power Parity: investigate whether purchasing power parity holds for given exchange rates between countries Efficient market hypothesis: is it possible to model/predict share prices? The Phillips curve: theory suggests a link and trade-off between unemployment and inflation, test this using econometric methods. Econometric tools can be useful in many fields: temperatures and greenhouse gases temperatures and sea level policy effectiveness effect of interventions?... Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 63 / 88

Last thoughts Plot the data before you conduct any analysis! Have a clear structure in mind! (as in example) Reference all your sources!...remember the deadline: 5. June 2016 Building a sensible model with real data is very hard - don t be disheartened! Email me with any questions! felix.pretis@nuffield.ox.ac.uk Stay in touch and build models! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 64 / 88

Appendix Optional Appendix If you would like to learn more about how to deal with non-linear but rather stochastic trends and unit-roots, the next slides introduce the concept of cointegration! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 65 / 88

Re-Cap Yesterday: Stationary is the exception not the norm! Types of Non-stationarity: like going to zoo to look at non-elephants... Many many forms... = Three common forms encountered and tame-able: 1: Linear Trends 2: Structural Breaks (sudden change in parameters) 3: Unit-roots/Stochastic trends (advanced) can induce spurious correlation! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 66 / 88

Spurious Relationship: Example Japanese GDP and Cumulative Rainfall in Fortaleza (NE Brazil) 500000 Japan RGDP 6000 Cumulative Rainfall Fortaleza Brazil 400000 5000 4000 300000 3000 200000 2000 100000 1000 1960 1970 1980 1990 2000 1960 1970 1980 1990 2000 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 67 / 88

Unit-Root Tests - Japanese RGDP: Unit-root tests The sample is: 1959-1992 (38 observations and 2 variables) Jap_rGDP_90_billionY: ADF tests (T=34, Constant; 5%=-2.95 1%=-3.64) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3 0.8581 1.0098 5162. 1.082 0.2882 17.23 2 1.368 1.0145 5177. -0.7882 0.4368 17.21 0.2882 1 1.152 1.0111 5145. 1.584 0.1233 17.18 0.4186 0 2.440 1.0197 5265. 17.19 0.2544 - Cumulative Rainfall: cumrainfall_brazil: ADF tests (T=34, Constant; 5%=-2.95 1%=-3.64) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3-0.7352 0.99589 51.16-0.6048 0.5500 8.005 2-0.8907 0.99519 50.61-0.02881 0.9772 7.958 0.5500 1-0.9121 0.99517 49.79 1.686 0.1018 7.900 0.8335 0-0.8124 0.99558 51.20 7.929 0.3980 = both series unit-root non-stationary Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 68 / 88

Model RGDP using Rainfall 400000 Japan RGDP Model Fit 200000 1955 1960 1965 1970 1975 1980 1985 1990 2 Residuals 0 1955 1960 1965 1970 1975 1980 1985 1990 Model Jap.RDGP t = 23352.2 (4938) + 67.89 (1.45) cumrainfall Brazil t R 2 =0.98, F(1,36) = 2206 [0.000]** Cumulative Rainfall highly significant, explains 98% of variation in Japanese GDP Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 69 / 88

Addressing Spurious Relationships How can we actually estimate meaningful economic relationships? Method 1: Take differences of our data: If Y t is random walk, Y t = Y t Y t 1 is stationary. Regress Y t on X t. But we lose levels information. Economic theory in levels. Y t = 10792 (2976) + 0.13 (18.29) cumrainfall Brazil t + ˆɛ t. (9) = not significant anymore! Method 2: Cointegration Both series non-stationary Is a linear combination of them stationary? Residuals of regression of Y t on X t stationary: Test of cointegration. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 70 / 88

Cointegration: Drunken walk with dog Random Walk Ɛ 10 Random Walk Ɛ 1 Ɛ 5 Ɛ Stationary Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 71 / 88

Residuals Non-Stationary: 2 Residuals (scaled) 0 1955 1960 1965 1970 1975 1980 1985 1990 Stationary: 2 Residuals 0-2 0 10 20 30 40 50 60 70 80 90 100 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 72 / 88

Cointegration Fundamental concept in econometrics: Engle and Granger won Nobel Prizes. Static regression model: Y t = β 1 + β 2 X t + ɛ t. (10) If Y t I(1) and X t I(1) (both non-stationary) expect combination non-stationary. ɛ t = Y t β 1 β 2 X t so expect ɛ t I(1), non-stationary. Hence residuals from Japan GDP and Brazilian rainfall are I(1). 2 Residuals (scaled) 0 1955 1960 1965 1970 1975 1980 1985 1990 But if ɛ t I(0), stationary, then we have cointegration: Both series move together in a stationary economic relationship. 2.5 r:s (scaled) 0.0-2.5 2009 2010 2011 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 73 / 88

Cointegration exists if: Both Y t and X t are non-stationary and have unit roots. Combination of Y t and X t, Y t β 1 β 2 X t is stationary. Then we say: Y t and X t are cointegrated. If Y t, X t cointegrated then ɛ t = Y t β 1 β 2 X t stationary Engle & Granger test for cointegration: Unit root test on residuals. 1 Test Y t and X t for unit root. 2 2 If both have unit roots, run regression of Y t on constant and X t (static, don t include lags). 3 Save residuals ˆɛ t and run unit root test on them. 2 From Descriptive Statistics after clicking. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 74 / 88

Cointegration Examples Japanese RGDP (Y t ) and Cumulative Rainfall in Fortaleza Brazil (X t ) 1) Test Y t, X t for unit-roots to establish if I(1) - Japanese RGDP: Unit-root tests The sample is: 1959-1992 (38 observations and 2 variables) Jap_rGDP_90_billionY: ADF tests (T=34, Constant; 5%=-2.95 1%=-3.64) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3 0.8581 1.0098 5162. 1.082 0.2882 17.23 2 1.368 1.0145 5177. -0.7882 0.4368 17.21 0.2882 1 1.152 1.0111 5145. 1.584 0.1233 17.18 0.4186 0 2.440 1.0197 5265. 17.19 0.2544 - Cumulative Rainfall: cumrainfall_brazil: ADF tests (T=34, Constant; 5%=-2.95 1%=-3.64) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3-0.7352 0.99589 51.16-0.6048 0.5500 8.005 2-0.8907 0.99519 50.61-0.02881 0.9772 7.958 0.5500 1-0.9121 0.99517 49.79 1.686 0.1018 7.900 0.8335 0-0.8124 0.99558 51.20 7.929 0.3980 = both series unit-root non-stationary Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 75 / 88

2) Regress Y t on X t, store the residuals ˆɛ t Model Jap.RDGP t = 23352.2 (4938) + 67.89 (1.45) cumrainfall Brazil t Keep Residual: ˆɛ t = Y t ˆβ 1 ˆβ 2 X t ˆɛ t = Y t 23352.2 67.89X t 2 Residuals (scaled) 0 1955 1960 1965 1970 1975 1980 1985 1990 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 76 / 88

Test ˆɛ t for unit-root non-stationarity using Dickey-Fuller Test: ˆɛ t = ρˆɛ t 1 + v t Test H0: ρ 1 = 0 ˆɛ t = (ρ 1)ˆɛ t 1 + v t Unit-root tests The sample is: 1959-1992 (38 observations and 1 variables) residuals: ADF tests (T=34, Constant; 5%=-2.95 1%=-3.64) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3-2.520 0.74289 6236. 1.680 0.1037 17.61 2-2.010 0.80149 6423. 0.7200 0.4771 17.65 0.1037 1-1.901 0.83283 6373. 3.109 0.0040 17.60 0.2030 0-0.5074 0.95499 7184. 17.82 0.0105 Test Statistic = -0.507 for no lags, -1.901 for 1 lag Critical value = -2.95 for 5% 1.901 > 2.95 Cannot reject H0: Unit root = residual non-stationary, no cointegration Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 77 / 88

US Short-term and long-term interest rates US short-term (3-month inter-bank) (Y t ) and long-term (10-year bonds) (X t ) interest rates from 1964Q3 2013Q2 In short-run expect random walks Stable relationship in long-run? cointegration? 15 Short Term_US Long Term_US 10 5 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 5.0 DShort Term_US DLong Term_US 2.5 0.0-2.5 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 78 / 88

Order of Integration of Levels? Determine Order of Integration: I(0)? I(1)?... Univariate Unit Root Tests on Y t, X t - US Short-Term Interest Rate The dataset is: new11.in7 The sample is: 1965(4) - 2013(2) (195 observations and 4 variables) ST_US: ADF tests (T=191, Constant; 5%=-2.88 1%=-3.47) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3-2.073 0.95862 0.9114 3.288 0.0012-0.1596 2-1.524 0.96921 0.9351-3.269 0.0013-0.1136 0.0012 1-2.142 0.95645 0.9588 2.988 0.0032-0.06849 0.0000 0-1.638 0.96647 0.9787-0.03256 0.0000 - US Long-Term Interest Rate LT_US: ADF tests (T=191, Constant; 5%=-2.88 1%=-3.47) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3-1.436 0.98067 0.4885 2.241 0.0262-1.407 2-1.115 0.98498 0.4937-1.252 0.2120-1.391 0.0262 1-1.317 0.98244 0.4945 3.502 0.0006-1.393 0.0386 0-0.8573 0.98832 0.5090-1.340 0.0004 = Unit-root non-stationary! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 79 / 88

Order of Integration of First Differences? Determine Order of Integration: I(0)? I(1)?... Univariate Unit Root Tests on First Differences Y t, X t - US Short-Term Interest Rate DST_US: ADF tests (T=191, Constant; 5%=-2.88 1%=-3.47) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3-6.301** 0.18921 0.9219 0.1467 0.8835-0.1369 2-7.027** 0.19783 0.9194-2.967 0.0034-0.1472 0.8835 1-11.33** -0.017949 0.9383 3.619 0.0004-0.1117 0.0137 0-11.35** 0.18901 0.9679-0.05484 0.0001 - US Long-Term Interest Rate DLT_US: ADF tests (T=191, Constant; 5%=-2.88 1%=-3.47) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 3-6.388** 0.23401 0.4901 0.9280 0.3546-1.401 2-6.655** 0.28271 0.4899-2.051 0.0417-1.406 0.3546 1-9.398** 0.15768 0.4940 1.436 0.1527-1.395 0.0823 0-10.79** 0.23720 0.4954-1.394 0.0705 Y t, X t Not Unit-root non-stationary! = Y t, X t are I(1): stationary if differenced once. Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 80 / 88

Static Regression 2) Regress Y t on X t, store the residuals ˆɛ t Model Short Term t = 1.75 (0.29) + 1.14 (0.04) Long Term t Keep Residual: ˆɛ t = Y t ˆβ 1 ˆβ 2 X t ˆɛ t = Y t + 1.75 1.14X t 5.0 Short-Term Model Residuals 2.5 0.0-2.5 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 81 / 88

Test ˆɛ t for unit-root non-stationarity using Dickey-Fuller Test: ˆɛ t = ρˆɛ t 1 + v t Test H0: ρ 1 = 0 ˆɛ t = (ρ 1)ˆɛ t 1 + v t Unit-root tests The sample is: 1965(4) - 2013(2) (196 observations and 1 variables) US_ST_LT_residuals: ADF tests (T=191, Constant; 5%=-2.88 1%=-3.47) D-lag t-adf beta Y_1 sigma t-dy_lag t-prob AIC F-prob 4-4.012** 0.84587 0.7181 0.8310 0.4071-0.6313 3-3.938** 0.85455 0.7175 1.991 0.0480-0.6380 0.4071 2-3.533** 0.87245 0.7232-1.664 0.0977-0.6274 0.1008 1-4.159** 0.85536 0.7266 2.862 0.0047-0.6232 0.0621 0-3.510** 0.87929 0.7403-0.5910 0.0042 Test Statistics below critical values (-2.88 for 5%, -3.47 for 1%) Reject H0: Unit root = residual stationary, series cointegrate! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 82 / 88

Interpretation: Short-run: Short-term and Long-term interest rates behave as random walks Long-run: Stationary relationship between them, never drift too far apart! Equilibrium Correction: e.g. Long-term interest rate changes, short-term interest rate adjusts to new equilibrium Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 83 / 88

Practical Computer Lab 5: Test for unit-root non-stationarity Test for cointegration Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 84 / 88

Short-Term and Long-Term Interest Rates Link between UK Short-term and Long-term interest rates Load data: interest rates.in7 Plot the data in levels and first differences (construct) Test for order of integration/unit-root non-stationarity 15 LT_UK 10 5 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 15 ST_UK 10 5 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 85 / 88

Test for cointegration Estimate static regression model LT t = β 1 + β 2 ST t + ɛ t. Store the residual Test for unit-root non-stationarity on the residual 2.5 ST_UK_residuals 0.0-2.5 1975 1980 1985 1990 1995 2000 2005 2010 2015 Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 86 / 88

Cointegration between Stock Prices Closing price for Toyota, Honda, Ford load data car closing price.in7 Yesterday: found closing prices to be random walks Do the series cointegrate? = test for cointegration! Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 87 / 88

Cointegration between Stock Prices Steps: Test for order of integration/unit root non-stationarity Estimate static regression model & store residual Test for cointegration (unit-root test on residual) Felix Pretis (Oxford) Time Series Akita Intl. University, 2016 88 / 88