Real-Time Electricity Pricing Xi Chen, Jonathan Hosking and Soumyadip Ghosh IBM Watson Research Center / Northwestern University Yorktown Heights, NY, USA X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 1 / 37
Agenda 1 Introduction Project Background Motivation Data Summary 2 Modeling and Estimation General Framework A Single Household RTP Average Household 14 RTP Households 3 Conclusions and Future Work 4 Acknowledgment 5 Questions X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 2 / 37
Olympic Peninsula Project Project Background Why To investigate distributed real-time control of the grid and to test impacts of time-varying rates Who and What Residential, commercial and municipal test sites, and several distributed generator were coordinated to manage the constrained feeder electrical distribution Where In and near Sequim and Port Angeles, Washington, on the Olympic Peninsula When From 4/1/2006 through 3/31/2007 X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 3 / 37
Transactive Control Figure: Control of Imposed Distribution Constraint Using Transactive Control, courtesy of PNNL. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 4 / 37
Pricing Contract Programs The Residential Customers 112 households are divided into four contract groups: 1 Fixed price: Price remains constant. a X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 5 / 37
Pricing Contract Programs The Residential Customers 112 households are divided into four contract groups: 1 Fixed price: Price remains constant. a 2 TOU/CPP: Critical peak rate > On-peak rate > off-peak rate; during each period, price remains the same. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 5 / 37
Pricing Contract Programs The Residential Customers 112 households are divided into four contract groups: 1 Fixed price: Price remains constant. a 2 TOU/CPP: Critical peak rate > On-peak rate > off-peak rate; during each period, price remains the same. 3 RTP: Price varies every fifteen minutes; customers set automatic response to price changes; they can override their pre-set response settings at any time X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 5 / 37
Pricing Contract Programs The Residential Customers 112 households are divided into four contract groups: 1 Fixed price: Price remains constant. a 2 TOU/CPP: Critical peak rate > On-peak rate > off-peak rate; during each period, price remains the same. 3 RTP: Price varies every fifteen minutes; customers set automatic response to price changes; they can override their pre-set response settings at any time 4 Control: No program account and no bills. Customers received $150 in appreciation of their participation. a The unit of price is $/kwh X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 5 / 37
Pricing Contract Programs The Residential Customers 112 households are divided into four contract groups: 1 Fixed price: Price remains constant. a 2 TOU/CPP: Critical peak rate > On-peak rate > off-peak rate; during each period, price remains the same. 3 RTP: Price varies every fifteen minutes; customers set automatic response to price changes; they can override their pre-set response settings at any time 4 Control: No program account and no bills. Customers received $150 in appreciation of their participation. a The unit of price is $/kwh In our study, we focus on analyzing the relationship between real-time price and electricity usage of the RTP group. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 5 / 37
Data Summary: A Quick Overview Problems with data on the residential customers 1 Randomly missing data entries (especially for RTP group) X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 6 / 37
Data Summary: A Quick Overview Problems with data on the residential customers 1 Randomly missing data entries (especially for RTP group) 2 Unequal lengths of the RTP households data files X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 6 / 37
Data Summary: A Quick Overview Problems with data on the residential customers 1 Randomly missing data entries (especially for RTP group) 2 Unequal lengths of the RTP households data files 3 Failed to keep a record on changes of households mode settings X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 6 / 37
Data Summary: A Quick Overview Problems with data on the residential customers 1 Randomly missing data entries (especially for RTP group) 2 Unequal lengths of the RTP households data files 3 Failed to keep a record on changes of households mode settings 4 No demographics (PII) of the households available X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 6 / 37
Data Summary: A Quick Overview Problems with data on the residential customers 1 Randomly missing data entries (especially for RTP group) 2 Unequal lengths of the RTP households data files 3 Failed to keep a record on changes of households mode settings 4 No demographics (PII) of the households available 5 Potential preselection bias in the experimental design X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 6 / 37
Data Summary: A Quick Overview Problems with data on the residential customers 1 Randomly missing data entries (especially for RTP group) 2 Unequal lengths of the RTP households data files 3 Failed to keep a record on changes of households mode settings 4 No demographics (PII) of the households available 5 Potential preselection bias in the experimental design 6 Possible data recording errors X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 6 / 37
Data Summary: A Quick Overview Problems with data on the residential customers 1 Randomly missing data entries (especially for RTP group) 2 Unequal lengths of the RTP households data files 3 Failed to keep a record on changes of households mode settings 4 No demographics (PII) of the households available 5 Potential preselection bias in the experimental design 6 Possible data recording errors 7 Incomplete weather data a a According to the weather expert Dr.Lloyd Treinish at IBM, the oceanic climate occurs in most parts of Pacific Northwest including the areas of our interest, which leads to dramatic weather changes throughout a typical year. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 6 / 37
Data Summary: Customer ID 671 month/year missing usage=0 price<0 total 5/2006 4 0 19 23 6/2006 19 50 60 129 7/2006 249 0 4 253 8/2006 0 4 0 4 9/2006 324 0 0 324 10/2006 1 2 0 3 11/2006 0 0 0 0 12/2006 5 6 0 11 1/2007 3 0 0 3 2/2007 0 0 0 0 3/2007 7 0 33 40 Percentage 1.90% 0.19% 0.36% 2.46% X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 7 / 37
Data Summary: 14 Households in the RTP Group month/year missing usage=0 price<0 total 5/2006 56 32 266 354 6/2006 266 78 840 1184 7/2006 3486 68 56 3610 8/2006 0 177 0 177 9/2006 4536 17 0 4553 10/2006 14 27 0 41 11/2006 0 46 0 46 12/2006 70 161 0 231 1/2007 42 57 0 99 2/2007 0 26 0 26 3/2007 98 23 462 583 Percentage 1.90% 0.16% 0.36% 2.42% X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 8 / 37
Time-Series Regression Analysis Basic Picture Figure: Time series of electricity demands and price in 15-min time interval X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 9 / 37
CustID 671: Late Spring, Early Summer Figure: Late Spring: 5/1/2006 6/30/2006 with No spike in price X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 10 / 37
CustID 671: Summer Figure: Summer: 8/1/2006 8/31/2006 with No spike in price X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 11 / 37
CustID 671: Fall Figure: Fall: 10/1/2006 10/31/2006 X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 12 / 37
CustID 671: Winter Figure: Winter: 11/1/2006 2/28/2007 X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 13 / 37
Time-Series Regression Models: Why? Rationale Behind Time-Series Regression Models 1 We are interested in capturing the effect of real-time price on electricity demand, no matter how slight it might be regression models 2 Some factors that impact electricity demand are not available to us omitted variable bias 3 It is desirable to capture the dynamics in electricity demand given the volatility observed in electricity usage of any household including lagged terms of dependent variable into predictor matrix X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 14 / 37
Time-Series Regression Models: How? Some Methods and Concepts in Econometrics Before fitting... Time series involved in this regression: stationary or not? Spurious regression A linear combination of I(1) variables is stationary, then the variables are called cointegrated a long-run relationship between the I(1) variables. a We want to learn about the true causal relationship, so need to do a formal unit root test in each series a Unit root : Y t = θy t 1 + ε t, where θ = 1 Nonstationarity or random walk, or I(1) variable. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 15 / 37
Time-Series Regression Models: How? Some Methods and Concepts in Econometrics Before fitting... Time series involved in this regression: stationary or not? Spurious regression A linear combination of I(1) variables is stationary, then the variables are called cointegrated a long-run relationship between the I(1) variables. a We want to learn about the true causal relationship, so need to do a formal unit root test in each series a Unit root : Y t = θy t 1 + ε t, where θ = 1 Nonstationarity or random walk, or I(1) variable. After fitting... Stationarity and unit root problem in residuals Augmented Dickey Fuller test; Autocorrelations in residuals Box-Pierce and Ljung-Box tests X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 15 / 37
CustID 671: Time Series Regression Models We consider time-series regression models for four seasons: log[q(t)] = β 0 + β i0 1{t is the i-th time interval of a weekday} +f (log[q(t k)]) + h (log[p(t l)]) +β Tmp log[t(t 3)] + β RH log[h(t 3)] + ε(t) (1) where k = 1, 2, 3, 4, 94, 95, 96, 97, 98 and l = 0, 1, 3, 4, 73, 84 (terms vary for four seasons); f( ) and h( ) are linear combinations of back-shifted log[q(t)] and log[p(t)] respectively. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 16 / 37
CustID 671: Time Series Regression Models We consider time-series regression models for four seasons: log[q(t)] = β 0 + β i0 1{t is the i-th time interval of a weekday} +f (log[q(t k)]) + h (log[p(t l)]) +β Tmp log[t(t 3)] + β RH log[h(t 3)] + ε(t) (1) where k = 1, 2, 3, 4, 94, 95, 96, 97, 98 and l = 0, 1, 3, 4, 73, 84 (terms vary for four seasons); f( ) and h( ) are linear combinations of back-shifted log[q(t)] and log[p(t)] respectively. How did we choose the lagged terms in Equation (1)? X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 16 / 37
CustID 671: Time Series Regression Models We consider time-series regression models for four seasons: log[q(t)] = β 0 + β i0 1{t is the i-th time interval of a weekday} +f (log[q(t k)]) + h (log[p(t l)]) +β Tmp log[t(t 3)] + β RH log[h(t 3)] + ε(t) (1) where k = 1, 2, 3, 4, 94, 95, 96, 97, 98 and l = 0, 1, 3, 4, 73, 84 (terms vary for four seasons); f( ) and h( ) are linear combinations of back-shifted log[q(t)] and log[p(t)] respectively. How did we choose the lagged terms in Equation (1)? Vector AutoRegression (VAR) y 1t A 11 (B) A 12 (B) A 13 (B) A 14 (B) y 2t y 3t = A 21 (B) A 22 (B) A 23 (B) A 24 (B) A 31 (B) A 32 (B) A 33 (B) A 34 (B) A 41 (B) A 42 (B) A 43 (B) A 44 (B) y 4t y 1t y 2t y 3t y 4t + C 1 C 2 C 3 C 4 + where y 1t = log[q(t)], y 2t = log[p(t)], y 3t = log[t(t)] and y 4t = log[h(t)]. A ij (B), i, j = 1, 2, 3 and 4 take the form p k=1 a ijk B k, where B is the lag (backshift) operator defined by B k y t = y t k and p = 96. ε 1 ε 2 ε 3 ε 4 (2) Through VAR, we found that... Compared to log[q(t)], log[p(t)] can be better predicted given the historical electricity demands and its own lagged terms. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 16 / 37
CustID 671: Regression and Tests Results We use the Newey-West adjusted heteroscedastic-serial consistent least-squares regression method to fit models specified by Equation (1). statistics spring summer fall winter R 2 0.526 0.489 0.568 0.712 Tests For time-series regression, we need to do tests the stationarity and independence of the regression residuals! p-values Test spring summer fall winter Box-Pierce (p=1) 0.5330 0.8797 0.9929 0.8317 Box-Pierce (p=96) 0.1656 0.1656 0.3883 0.0064 Dickey-Fuller <0.01 (p = 16) <0.01 (p = 14) <0.01 (p = 13) <0.01 (p = 22) Table: Cust ID 671 TS Regression results and tests on residuals More on the tests Box-Pierce Test: H 0 : There is no autocorrelation exists; Dickey-Fuller Test: H 0 : Unit root does exist in the underlying process. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 17 / 37
CustID 671: Price Elasticities and Effects of Weather Factors A glance at own price elasticity of energy demand ( d q(t) / d p(t) ) at time t: q(t) p(t) Season ˆβ for log[p(t)] ˆβ Tmp ˆβ RH Spring Summer Fall Winter (0.0147) 0.0524 (0.1348) 0.0595 (0.2432) 0.0161 (0.0170) 0.0133 (0.2016) 0.0875 (0.008) 0.4769 (0.3724) 0.0689 (<0.0001) 0.1494 (0.0055) 0.0931 (0.003) 0.2136 (0.9777) 0.0012 (0.2588) 0.0409 Table: Price elasticity estimates and coefficient estimates for temperature and RH for four seasons Summary X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 18 / 37
CustID 671: Price Elasticities and Effects of Weather Factors A glance at own price elasticity of energy demand ( d q(t) / d p(t) ) at time t: q(t) p(t) Season ˆβ for log[p(t)] ˆβ Tmp ˆβ RH Spring Summer Fall Winter (0.0147) 0.0524 (0.1348) 0.0595 (0.2432) 0.0161 (0.0170) 0.0133 (0.2016) 0.0875 (0.008) 0.4769 (0.3724) 0.0689 (<0.0001) 0.1494 (0.0055) 0.0931 (0.003) 0.2136 (0.9777) 0.0012 (0.2588) 0.0409 Table: Price elasticity estimates and coefficient estimates for temperature and RH for four seasons Summary 1 Better fit for winter season than non-winter season, especially when there are spikes in price. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 18 / 37
CustID 671: Price Elasticities and Effects of Weather Factors A glance at own price elasticity of energy demand ( d q(t) / d p(t) ) at time t: q(t) p(t) Season ˆβ for log[p(t)] ˆβ Tmp ˆβ RH Spring Summer Fall Winter (0.0147) 0.0524 (0.1348) 0.0595 (0.2432) 0.0161 (0.0170) 0.0133 (0.2016) 0.0875 (0.008) 0.4769 (0.3724) 0.0689 (<0.0001) 0.1494 (0.0055) 0.0931 (0.003) 0.2136 (0.9777) 0.0012 (0.2588) 0.0409 Table: Price elasticity estimates and coefficient estimates for temperature and RH for four seasons Summary 1 Better fit for winter season than non-winter season, especially when there are spikes in price. 2 Weather effects: Temperature : an important factor in determining the electricity demand, especially for summer and winter; Relative humidity can be an important factor for summer, and it also has a slight impact for spring. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 18 / 37
CustID 671: Price Elasticities and Effects of Weather Factors A glance at own price elasticity of energy demand ( d q(t) / d p(t) ) at time t: q(t) p(t) Season ˆβ for log[p(t)] ˆβ Tmp ˆβ RH Spring Summer Fall Winter (0.0147) 0.0524 (0.1348) 0.0595 (0.2432) 0.0161 (0.0170) 0.0133 (0.2016) 0.0875 (0.008) 0.4769 (0.3724) 0.0689 (<0.0001) 0.1494 (0.0055) 0.0931 (0.003) 0.2136 (0.9777) 0.0012 (0.2588) 0.0409 Table: Price elasticity estimates and coefficient estimates for temperature and RH for four seasons Summary 1 Better fit for winter season than non-winter season, especially when there are spikes in price. 2 Weather effects: Temperature : an important factor in determining the electricity demand, especially for summer and winter; Relative humidity can be an important factor for summer, and it also has a slight impact for spring. 3 Price plays a role in determining the electricity demand in winter and spring; its impacts seem minor when compared to the weather factors in magnitude. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 18 / 37
CustID 671: Factors that Impact Electricity Demand with Price Threshold Use the price threshold p(t 2) > $0.2 /kwh, we obtain the following for the winter season: Variable ˆβ for log[p(t)] ˆβ Tmp ˆβ RH log[p(t)] when p(t 2) > $0.2/kWh log[p(t)] when p(t 2) $0.2/kWh (0.0119) 0.0166 (0.0001) 0.0154 (0.0112) 0.0609 (<0.0001) 0.0668 (0.6978) 0.0075 (0.5885) 0.0039 Table: Price elasticity estimates and coefficient estimates for temperature and RH for winter season with price threshold X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 19 / 37
CustID 671: Factors that Impact Electricity Demand with Price Threshold Use the price threshold p(t 2) > $0.2 /kwh, we obtain the following for the winter season: Variable ˆβ for log[p(t)] ˆβ Tmp ˆβ RH log[p(t)] when p(t 2) > $0.2/kWh log[p(t)] when p(t 2) $0.2/kWh (0.0119) 0.0166 (0.0001) 0.0154 (0.0112) 0.0609 (<0.0001) 0.0668 (0.6978) 0.0075 (0.5885) 0.0039 Table: Price elasticity estimates and coefficient estimates for temperature and RH for winter season with price threshold We observe that... 1 When price is above $0.2/kWh, CustID 671 is more price sensitive and cares less about cold weather. 2 When price is below $0.2/kWh, CustID 671 is less price sensitive and more low temperature concerned. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 19 / 37
CustID 671: Factors that Impact Electricity Demand with Price Threshold Use the price threshold p(t 2) > $0.2 /kwh, we obtain the following for the winter season: Variable ˆβ for log[p(t)] ˆβ Tmp ˆβ RH log[p(t)] when p(t 2) > $0.2/kWh log[p(t)] when p(t 2) $0.2/kWh (0.0119) 0.0166 (0.0001) 0.0154 (0.0112) 0.0609 (<0.0001) 0.0668 (0.6978) 0.0075 (0.5885) 0.0039 Table: Price elasticity estimates and coefficient estimates for temperature and RH for winter season with price threshold We observe that... 1 When price is above $0.2/kWh, CustID 671 is more price sensitive and cares less about cold weather. 2 When price is below $0.2/kWh, CustID 671 is less price sensitive and more low temperature concerned. 3 Price does have a significant impact on the electricity demand in winter; however, temperature seems to play a dominating role. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 19 / 37
CustID 671: Time-of-Day Model (Winter) Fit separate time-series regression models for the 96 time-of-day points, for winter and non-winter seasons. Figure: R2 for 96 fitted time-of-day regression models (Winter) X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 20 / 37
CustID 671: Time-of-Day Model (Non-Winter) Figure: R2 for 96 fitted time-of-day regression models (Non-winter) X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 21 / 37
CustID 671: Time-of-Day Model Own Price Elasticities Time-of-Day LB ˆβ UB 01:15-0.1346-0.0686-0.0025 06:45-0.1651-0.0875-0.0099 09:45-0.3587-0.2119-0.0651 10:00-0.5219-0.3199-0.1178 10:15-0.4177-0.2482-0.0787 16:45-0.1948-0.1101-0.0255 18:15 0.0264 0.1237 0.2211 23:45 0.0102 0.0673 0.1244 Table: Cust ID 671 95% C.I. s for significant TOD price elasticities (winter) that do not include 0 Time-of-Day LB ˆβ UB 01:45-0.0632-0.0332-0.0032 06:45-0.1227-0.0649-0.0072 08:30 0.0705 0.1672 0.2638 11:15 0.0047 0.0809 0.1570 14:30-0.1937-0.1106-0.0275 17:15-0.1573-0.0824-0.0075 20:00-0.1504-0.0902-0.0300 Table: Cust ID 671 95% C.I. s for significant TOD price elasticities (non-winter) that do not include 0 X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 22 / 37
CustID 671: Time-of-Day Model Own Price Elasticities Time-of-Day LB ˆβ UB 01:15-0.1346-0.0686-0.0025 06:45-0.1651-0.0875-0.0099 09:45-0.3587-0.2119-0.0651 10:00-0.5219-0.3199-0.1178 10:15-0.4177-0.2482-0.0787 16:45-0.1948-0.1101-0.0255 18:15 0.0264 0.1237 0.2211 23:45 0.0102 0.0673 0.1244 Table: Cust ID 671 95% C.I. s for significant TOD price elasticities (winter) that do not include 0 Time-of-Day LB ˆβ UB 01:45-0.0632-0.0332-0.0032 06:45-0.1227-0.0649-0.0072 08:30 0.0705 0.1672 0.2638 11:15 0.0047 0.0809 0.1570 14:30-0.1937-0.1106-0.0275 17:15-0.1573-0.0824-0.0075 20:00-0.1504-0.0902-0.0300 Table: Cust ID 671 95% C.I. s for significant TOD price elasticities (non-winter) that do not include 0 Observation 1 CustID 671 seems more flexible in shifting its electricity demand to non-peak hours, compared to peak-hours; 2 CustID 671 has relatively inelastic electricity demand in midnight hours than in other non-peak hours. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 22 / 37
CustID 671: TOD Model Temperature Effect Figure: Cust ID 671 95% C.I. s for significant temperature coefficient(non-winter) that do not include 0 Figure: Cust ID 671 95% C.I. s for significant temperature coefficient (Winter) that do not include 0 X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 23 / 37
CustID 671: Time-of-Day Models Summary on TOD models 1 Poor fitting performance for some time-of-day points, especially when price does not vary a lot whereas electricity demand does singularity of predictor matrix. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 24 / 37
CustID 671: Time-of-Day Models Summary on TOD models 1 Poor fitting performance for some time-of-day points, especially when price does not vary a lot whereas electricity demand does singularity of predictor matrix. 2 More difficult to find better regression models for non-winter than for winter X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 24 / 37
CustID 671: Time-of-Day Models Summary on TOD models 1 Poor fitting performance for some time-of-day points, especially when price does not vary a lot whereas electricity demand does singularity of predictor matrix. 2 More difficult to find better regression models for non-winter than for winter 3 Design incentive policy for utilities to explore the differences between price elasticities of on-peak and off-peak hours X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 24 / 37
CustID 671: Time-of-Day Models Summary on TOD models 1 Poor fitting performance for some time-of-day points, especially when price does not vary a lot whereas electricity demand does singularity of predictor matrix. 2 More difficult to find better regression models for non-winter than for winter 3 Design incentive policy for utilities to explore the differences between price elasticities of on-peak and off-peak hours 4 Make inferences about individual household s electricity demand fingerprint (such as occupancy of the house and family s work schedules) and further exploit them X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 24 / 37
RTP Average Household: Regression and Tests Results Fit time-series regression models for four seasons: 1 log[q(t)] = β 0 + β i0 1{t is the i-th time interval of a weekday} +f (log[q(t k)]) + h (log[p(t l)]) +β Tmp log[t(t 3)] + β RH log[h(t 3)] + ε(t) (3) where basically k = 1, 2, 3, 4, 85, 86, 90, 94, 96 and l = 0, 1, 2, 3, 4 (terms vary for four seasons) 1 Newey-West adjusted heteroscedastic-serial consistent Least-squares Regression X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 25 / 37
RTP Average Household: Regression and Tests Results Fit time-series regression models for four seasons: 1 log[q(t)] = β 0 + β i0 1{t is the i-th time interval of a weekday} +f (log[q(t k)]) + h (log[p(t l)]) +β Tmp log[t(t 3)] + β RH log[h(t 3)] + ε(t) (3) where basically k = 1, 2, 3, 4, 85, 86, 90, 94, 96 and l = 0, 1, 2, 3, 4 (terms vary for four seasons) Better regression fitting performance! statistics spring summer fall winter R 2 0.748 0.689 0.803 0.878 1 Newey-West adjusted heteroscedastic-serial consistent Least-squares Regression X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 25 / 37
RTP Average Household: Regression and Tests Results Fit time-series regression models for four seasons: 1 log[q(t)] = β 0 + β i0 1{t is the i-th time interval of a weekday} +f (log[q(t k)]) + h (log[p(t l)]) +β Tmp log[t(t 3)] + β RH log[h(t 3)] + ε(t) (3) where basically k = 1, 2, 3, 4, 85, 86, 90, 94, 96 and l = 0, 1, 2, 3, 4 (terms vary for four seasons) Better regression fitting performance! statistics spring summer fall winter R 2 0.748 0.689 0.803 0.878 Stronger autocorrelation exists in the regression residuals! p-values Test spring summer fall winter Box-Pierce (p=1) 0.587 0.326 0.5962 0.7686 Box-Pierce (p=96) 0.027 0.116 0.327 1.921 10 5 Dickey-Fuller <0.01(p=16) <0.01(p=14) <0.01 (p=14) <0.01 (p=22) Table: RTP Average TS Regression results and tests on residuals 1 Newey-West adjusted heteroscedastic-serial consistent Least-squares Regression X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 25 / 37
RTP Average Household: Price Elasticities and Effects of Weather Factors A glance at own price elasticity of energy usage ( d q(t) q(t) / d p(t) ) at time t: p(t) Season ˆβ for log[p(t)] ˆβ Tmp ˆβ RH Spring Summer Fall Winter (0.1055) 0.0339 (0.1980) 0.0808 (0.4058) 0.0060 (0.0101) 0.0074 (0.0539) 0.0942 (0.0039) 0.2181 (<0.0001) 0.2446 (<0.0001) 0.1396 (0.5163) 0.0100 (0.0662) 0.0584 (0.2015) 0.0215 (0.3139) 0.0138 Table: Price elasticity estimates and coefficient estimates for temperature and RH for four seasons With price threshold p(t 2) > $0.2 /kwh, we obtain the following estimates for the winter season: Variable ˆβ for log[p(t)] ˆβ Tmp ˆβ RH log[p(t)] when p(t 2) > $0.2/kWh log[p(t)] when p(t 2) $0.2/kWh (0.0240) 0.0160 (0.0022) 0.0094 (0.0058) 0.0684 (<0.0001) 0.0692 (0.6466) 0.0179 (0.3054) 0.0142 Table: Price elasticity estimates and coefficient estimates for temperature and RH for winter with price threshold X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 26 / 37
RTP Average Household: Time-of-Day Model (Winter) Fit separate time-series regression models for the 96 time-of-day points, for winter and non-winter seasons. Figure: R2 for the 96 time-of-day models (Winter) X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 27 / 37
RTP Average Household: TOD Model (Non-Winter) Figure: R2 for the 96 time-of-day models (Non-winter) X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 28 / 37
RTP Average Household: TOD Model Elasticities Time-of-Day LB ˆβ UB 00:15-0.4320-0.2184-0.0049 00:30-0.3598-0.2032-0.0467 01:00-0.3408-0.1875-0.0343 01:15-0.3601-0.1810-0.0019 01:30-0.1596-0.0855-0.0113 03:30-0.2923-0.2015-0.1107 07:45 0.0048 0.0285 0.0522 08:15-0.0512-0.0283-0.0054 11:45 0.0257 0.0752 0.1248 12:15-0.1393-0.0763-0.0132 14:30-0.1301-0.0818-0.0335 16:00 0.0118 0.0696 0.1273 17:45-0.0716-0.0375-0.0035 23:15-0.2474-0.1281-0.0087 Table: RTP AVG 95% C.I. s for significant price elasticities (winter) Time-of-Day LB ˆβ UB 02:15 0.0008 0.0156 0.0305 02:30 0.0012 0.0158 0.0303 05:00-0.0282-0.0149-0.0016 06:15-0.0727-0.0453-0.0179 06:30-0.0341-0.0189-0.0036 07:00-0.0401-0.0208-0.0016 09:00-0.0432-0.0218-0.0004 09:30 0.0170 0.0360 0.0550 13:00 0.0134 0.0387 0.0641 14:45-0.0614-0.0397-0.0181 19:00 0.0119 0.0372 0.0626 20:15-0.0399-0.0246-0.0094 20:45 0.0115 0.0309 0.0502 Table: RTP AVG 95% C.I. s for significant price elasticities (non-winter) X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 29 / 37
RTP Average Household: TOD Model Temperature Effect Figure: RTP AVG 95% C.I. s for significant temperature coefficient (Winter) Figure: RTP AVG 95% C.I. s for significant temperature coefficient(non-winter) X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 30 / 37
Models for 14 RTP Households Modeling Perspective Choice to make: Separate vs. Pooled Cross Sectional Models How differently do the 14 households behave from one another? X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 31 / 37
14 RTP Households: Separate Models Fit a common model for each single household use lmlist R. Figure: 95% C.I. s of coefficients for the regressions of log[q(t)] on lagged log[q(t)], log[p(t)] and weather variables. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 32 / 37
14 RTP Households: Separate Models Figure: Fitted vs. True for regressions of log[q(t)] on lagged log[q(t)], log[p(t)] and weather variables. X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 33 / 37
14 RTP Households: Observations and Conclusions 1 Substantial variation in the coefficients 95% C.I. s, especially true for log[q(t 1)] and log[q(t 96)] (both of them are statistically significant). 2 Test of poolability. We use pooltest from the package plm in R. 2 We reject the hypothesis that a pooled model estimation is appropriate. Therefore, we do not consider using pooled fixed effects model on the 14 households. 3 Mixed (or Random) Effects Model: only data of 14 RTP households available; No other exogenous variables available so far except for price and weather factors; The random effects of our interest can be highly correlated with other regressors prohibit us from using random effects model. 2 It compares a model obtained for the full sample (with different intercepts for different households) and a model based on the estimation of a separate equation for each household X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 34 / 37
Summary and Future Work Conclusions We used time-series regression models to obtain the price elasticities of electricity demand for the households in the RTP group; The regression fitting performance is better for predicting demand in winter; For the purpose of reasonably accurate forecasting, it s better to use ARIMA model for electricity demand and let the data speak for itself. The demand is relatively more price elastic in winter compared to non-winter, this is especially true when price spikes are present; Temperature plays an important role in determining electricity demand; according to our results, people are more sensitive to changes in temperature than changes in price; The responsiveness that individual RTP household showed to changes in price differ from one another; their daily electricity usage patterns also differ from each other to a great extent. Future work Fit different regression TOD models for on-peak and peak-hours Analyze the interactions between temperature and price signals Study the TOU/CPP group X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 35 / 37
Acknowledgment We are grateful to Dr. Pu Huang, Dr. Ramesh Natarajan @ IBM and Prof. Jeremy Staum @ Northwestern University for their helpful comments and discussion X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 36 / 37
Questions and Comments? X. Chen, J. Hosking & S. Ghosh (IBM) Real-Time Electricity Pricing IBM / NU, Summer 2010 37 / 37