PUBLICATIONS. Journal of Geophysical Research: Atmospheres

Transcription

1 PUBLICATIONS Journal of Geophysical Research: Atmospheres RESEARCH ARTICLE Key Points: Spatial correlation measures model performance in spatial variations Spatiotemporal correlation measures dynamic behavior of a model An RCM adds value in spatial/temporal patterns Model performance in spatiotemporal patterns of precipitation: New methods for identifying value added by a regional climate model Jiali Wang 1, F. N. U. Swati 2, Michael L. Stein 2, and V. Rao Kotamarthi 1 1 Environmental Science Division, Argonne National Laboratory, Argonne, Illinois, USA, 2 Department of Statistics, University of Chicago, Chicago, Illinois, USA Supporting Information: Readme and Tables S1 S3 Figures S1 S6 Data Set S1 Data Set S2 Data Set S3 Data Set S4 Data Set S5 Data Set S6 Correspondence to: J. Wang, jialiwang@anl.gov Citation: Wang, J., F. N. U. Swati, M. L. Stein, and V. R. Kotamarthi (2015), Model performance in spatiotemporal patterns of precipitation: New methods for identifying value added by a regional climate model, J. Geophys. Res. Atmos., 120, , doi: / 2014JD Received 12 AUG 2014 Accepted 17 JAN 2015 Accepted article online 24 JAN 2015 Published online 18 FEB 2015 Abstract Regional climate models (RCMs) are a standard tool for downscaling climate forecasts to finer spatial scales. The evaluation of RCMs against observational data is an important step in building confidence in the use of RCMs for future projection. In addition to model performance in climatological means and marginal distributions, a model s ability to capture spatiotemporal relationships is important. This study develops two approaches: (1) spatial correlation/variogram for a range of spatial lags, with total monthly precipitation and nonseasonal precipitation components used to assess the spatial variations of precipitation, and (2) spatiotemporal correlation for a wide range of distances, directions, and time lags, with daily precipitation occurrence used to detect the dynamic features of precipitation. These measures of spatial and spatiotemporal dependence are applied to a high-resolution RCM run and to the National Center for Environmental Prediction (NCEP)-U.S. Department of Energy Atmospheric Model Intercomparison Project II reanalysis data (NCEP-R2), which provide initial and lateral boundary conditions for the RCM. The RCM performs significantly better than NCEP-R2 in capturing both the spatial variations of total and nonseasonal precipitation components and the spatiotemporal correlations of daily precipitation occurrences, which are related to dynamic behavior of precipitating systems. The improvements are apparent not only at resolutions finer than that of NCEP-R2 but also when the RCM and observational data are aggregated to the resolution of NCEP-R2. 1. Introduction Projecting future changes in the pattern of precipitation is necessary for a full evaluation of the impacts of climate change, but direct verification of projections decades into the future through comparison to data is not feasible. Therefore, understanding models performance and their errors relative to observations in current periods is critical for increasing confidence in projections. In general, global climate models (GCMs) have reasonably good skill in simulating large-scale ( km) precipitation patterns, but they are not necessarily capable of capturing the regional- or small-scale spatial and/or temporal patterns of precipitation required for regional and national climate change assessments. Regional climate models (RCMs) have proven to be not only reasonably good at large scales [Seth et al., 2007; De Sales and Xue, 2010] but also useful for the downscaling of either GCMs or reanalysis data sets to a few tens of kilometers, because generally, the RCMs add value to the driving data for resolution-dependent surface variables (e.g., precipitation, air temperature, near-surface wind, and sea level/surface pressure) in regions characterized by small-scale orographic features, such as mountainous regions [Feser, 2006; Prommel et al., 2010] and coastal areas [Winterfeldt and Weisse, 2009]. For reviews of the value added by RCMs for different kinds of surface variables, see Maraun et al. [2010], Feseretal. [2011], and Xue et al. [2014]. The evaluation of RCMs against observational data is an important step in building confidenceintheirusefor future projection. When driving an RCM with a GCM, the GCM effects on discrepancies from the observations are difficult to separate from the RCM effects. Kaufman and Sain [2010] developed a framework for functional analysis of variance modeling to separate the effects from a GCM, an RCM itself and the interactions between the GCM and RCM on the results of dynamical downscaling. Maraun et al. [2010, p. 5] noted that using reanalysis data to drive the RCM enables cleaner assessment of its downscaling skills. Maraun et al. [2010] also argued that driving an RCM with reanalysis data enables comparison of day-to-day sequences of weather as an assessment tool. However, Mearns et al. [2012, pp ] noted the difficulty in interpreting comparisons of an observation-assimilated reanalysis and observations versus comparisons of RCMs and observations. We share this concern (see Appendix A) and prefer to compare the statistical distribution of WANG ET AL American Geophysical Union. All Rights Reserved. 1239

2 weather events (e.g., climate) rather than actual day-to-day weather. Many assessments of RCMs in the literature [see Liang et al., 2004; Otte et al., 2012; Bowden et al., 2012; Mearns et al., 2012; Xu and Yang, 2012; Martynov et al., 2013] focus on seasonal means (averages across many years), although Mearns et al. [2012], Otte et al. [2012], and Martynov et al. [2013] also considered interannual variability in seasonal averages. These procedures are valuable for comparing marginal distributions of model output with observations. Analyses that are done separately for each pixel provide little indication of how well spatiotemporal patterns of modeled variables are represented. There are many possible interpretations of spatial correlations when one has spatiotemporal data. Lo et al. [2008] computed correlations between observed and modeled 24 h precipitation time series at each pixel and presented a spatial overview of these correlations in a map. Mearns et al. [2012] considered what were termed spatial correlations by computing correlation coefficients between the vectors of observed and modeled long-term seasonal means, with each component of the vector corresponding to a pixel. These correlations provide a convenient summary of the relationship between observations and model output, but neither of them are what we will call spatial correlations, by which we will mean correlations of quantities measured at different spatial locations. We do not consider correlations between observations and model output. Rather, we compute spatial (and spatiotemporal) correlations separately for observations and model output and then compare these correlations as a measure of how well the model reproduces patterns of spatial dependence in the observations. Since meteorological variables such as precipitation and/or air pollution are controlled by general circulations and vary with time and space, we believe that spatiotemporal correlations (e.g., correlations between some quantity of interest at two different locations and different times, perhaps a day apart) can be a particularly effective tool for assessing the realism of model output. In particular, at shorter (e.g., daily) time scales, spatial and temporal variations of meteorological variables are generally not separable [Cressie and Wikle, 2011] so that comparing spatiotemporal relationships between observations and numerical models, and not purely spatial and purely temporal relationships separately [Sampson and Guttorp, 1999], is critical for evaluating model performance in capturing dynamic behavior. Meiring et al. [1998] and Haas [1998] assessed air quality modelsinaspace-time context. Di Luca et al. [2012] investigated the potential value added by RCMs in considerations of fine-scale spatial/temporal variabilities by comparing distributions of precipitation rates at different scales through aggregation of fine-scale RCM outputs. However, none of these approaches are intended to evaluate the space-time correlation structure or the dynamic behavior of the model outputs. With this in mind, Jun and Stein [2004] developed a spatiotemporal correlation metric to assess the model s ability to capture dynamic aspects of the transportation of sulfate concentration. On the other hand, although the spatial correlation measures the strength and direction of the linear relationship between two locations with a distance shift, it cannot measure the difference in field values between the two locations. Two time series with a huge difference in field values can be highly correlated. Therefore, to describe the difference in precipitation values between two locations, this study also considers spatial variograms. Precipitation patterns over the United States are highly dependent on locations and seasons. For example, precipitation over the Southwest mainly occurs in July and August and is dominated by convective activities of the North American Monsoon (NAM) [Castro et al., 2012]. In contrast, precipitation over the northwest occurs frequently in winter and is controlled by large-/synoptic-scale atmospheric circulations [Dominguez et al., 2012]. Over the east, precipitation is generally heavier in the warm season than in the cold season and is associated with synoptic-scale or mesoscale atmospheric circulations [Konrad, 2001]. Hence, an in-depth evaluation of model performance in spatiotemporal patterns of precipitation over the United States is greatly desired. Model-produced errors (related to real values or patterns) in dependence over space and/or time will result in errors of precipitation over some spatial or temporal scales, even if the marginal distributions of precipitation at a given scale are accurate. This study introduces two general approaches for evaluation of RCMs by measuring spatial variations and spatiotemporal relationships in precipitation. One approach compares spatial correlations and spatial variograms for observed and modeled total monthly precipitation and the nonseasonal monthly precipitation component at a range of spatial lags in both the north-south and east-west directions. The second approach compares spatiotemporal correlations for observed and modeled daily precipitation occurrences with time lags of 1 to 3 days and spatial lags in all directions about each of a set of selected pixels. To examine the value added by the RCM in terms of spatial and temporal dependencies, we apply those two approaches to the RCM output and its WANG ET AL American Geophysical Union. All Rights Reserved. 1240

3 Figure 1. Evaluation domain, with terrain height (m) indicated by shading. The numbers denote 48 reference points selected for calculating spatiotemporal correlations. driving data (see below) and compare the results. We find that the RCM provides more realistic details of spatial correlation/variograms and spatiotemporal correlations at finer scales than the driving data and also more accurate correlations than does the driver at the resolution of the driver, especially over regions with complex terrains. Our methods give researchers new tools for assessing RCMs, thus providing important complement to traditional model-based evaluation procedures. We describe the RCM and data sets in section 2. The methodology is presented in section 3, and section 4 shows the results. The final section discusses some technical issues and summarizes our findings. 2. Model and Data Sets The RCM evaluated in this study is the Weather Research and Forecast model (WRF, version 3.3.1). The model domain is centered at N and W and has dimensions of horizontal grid points in the west-east and south-north directions with grid spacing of 12 km, covering most of North America. The initial and boundary conditions are constructed from the National Center for Environmental Prediction (NCEP)-U.S. Department of Energy (DOE) Atmospheric Model Intercomparison Project II reanalysis data (NCEP-R2) [Kanamitsu et al., 2002]. Model output is saved once every 3 h for the entire period of the simulations. Details about the WRF configuration/setup, model biases in precipitation over the annual cycle, and dominant reasons for the biases are documented by Wang and Kotamarthi [2014]. Precipitation in NCEP-R2 is a variable that is not assimilated but is completely generated by the parameterizations in the model. Thus, NCEP-R2 precipitation provides a reference for WRF s enhancement of downscaling skill because of WRF s refined spatial resolution and physical representations. We use monthly precipitation from Precipitation-Elevation Regressions on Independent Slopes Model (PRISM) as one of the evaluation data sets. PRISM was developed [Daly et al., 1994, 2008] with corrections for systematic elevation effects on precipitation climatology. PRISM provides observation-based precipitation on a grid mesh of 4 km that covers all of the contiguous United States (CONUS). Given the strong precipitation dependence on elevation, the topographic adjustment is critical, because cooperative stations over mountainous regions are preferentially located at lower elevations and thus tend to underestimate the true spatial average. PRISM is therefore used widely for estimates of high-resolution precipitation, especially in topographically complex regions. We also use daily precipitation from the National Oceanic and Atmospheric Administration Climate Prediction Center (CPC). This data set is based on more than 13,000 station reports each day for 1948 to present. The station data are first quality controlled to eliminate duplicates and overlapping stations [Chen et al., 2008] then gridded at over all of CONUS by using a Cressman scheme [Chen et al., 2008; Xie et al., 2007]. The evaluation area in this study is mainly confined to CONUS (Figure 1), because both evaluation data sets are available only over CONUS. The length of all data sets is 31 years ( ) for both the monthly and daily scales. WANG ET AL American Geophysical Union. All Rights Reserved. 1241

4 3. Methodology 3.1. Spatial Correlation As stated in section 1, in a traditional use of correlation, the temporal correlation coefficient between a model s output and observational data is computed for each grid cell. This correlation requires regridding of one of the sources to match the resolution of the other source, and it measures the temporal relationship between the model and the observation only over that particular grid cell [e.g., Lo et al., 2008, Figures 4 and 8; Wang and Kotamarthi, 2014, Figure 6]. Evaluating the temporal variations over each grid cell is useful, but it does not reflect the spatial dependence of either observed or modeled variables. Furthermore, since the goal of downscaling is to produce a more accurate climatology, these correlations might not measure the effectiveness of the RCM (see Appendix A). To investigate WRF s ability to capture local features and spatial variations of precipitation, we compute spatial correlation coefficients at certain spatial lags along longitudes and latitudes by using total monthly precipitation in For spatial correlation along latitude or longitude, a Pearson correlation coefficient between the grid at (L, l) and the grid at (L + s 1, l + s 2 ) from an observation or a simulation is computed by X n x i;l;l x L;l xi;lþs1;lþs 2 x Lþs1;lþs2 r L;l; s 1;s 2 ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi X n s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 X n (1) 2 x i;l;l x L;l x i;lþs1;lþs 2 x Lþs1 ;lþs 2 where the variables denote the following: i, month in the 31 years of record (i =1,2,3,, 372); L, latitude; l, longitude; s 1, the south-to-north lag; and s 2, the west-to-east lag. We use 0.5, 1, 2, and 4 as spatial lags in both directions. x i,l,l is the ith month s total precipitation at (L, l), and x indicates the average of the monthly precipitation totals over at each location. Equation (1) computes the temporal correlations between two grid cells with a certain distance (s 1 along latitude or s 2 along longitude) for observations or simulations, and it reflects the spatial variations or local features for observed or modeled precipitation over the evaluation domain Spatial Variogram Although the spatial correlation defined in equation (1) measures the strength and direction of the linear relationship between two locations with a distance shift, it cannot measure the difference in field values between the two locations. Two time series with a huge difference in field values can be highly correlated. Therefore, to describe the difference in precipitation values between two locations, we calculate spatial variograms. For purely spatial data, the empirical spatial variogram is 1 γðs 1 ; s 2 Þ ¼ 2jNs ð 1 ; s 2 Þj X 2 x L;l x Lþs1;lþs 2 (2) jns ð 1;s 2 Þj where N(s 1, s 2 ) is the number of pairs of spatial locations for x L,l and x Lþs1;lþs 2 with difference in latitude s 1 and difference in longitude s 2 [see Jun and Stein, 2004, Appendix A]. For repeated observations in time at fixed spatial locations (as in the present setting), we can obtain an empirical variogram for two fixed locations by averaging over time. Specifically, we consider the following square root variogram, which has the merit of being in the same units as the observations (here mm): sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 X n 2 γ L;l;s1;s 2 ¼ x i;l;l x i;lþs1;lþs 2n 2 (3) 3.3. Decomposition For computation of spatial correlation with equation (1), we use total monthly precipitation. These totals include at least two important components: the seasonal component and the nonseasonal or residual component. As stated in section 1, the large seasonal precipitation cycle over most of the United States could drive a majority of the spatial correlation patterns. However, we would ask what will happen if we remove seasonal cycles from the data set before computing spatial correlations. Can WRF also capture the spatial WANG ET AL American Geophysical Union. All Rights Reserved. 1242

5 dependencies of the nonseasonal component of monthly precipitation? With this in mind, we decompose the total monthly precipitation into seasonal and nonseasonal components by fitting a harmonic regression model over each pixel, as shown in equation (4). Yt ðþ¼μþ XH h¼1 α h cos 2πht P þ β h sin 2πht P þ εðþ t (4) Here μ is all-month mean precipitation, h represents the order of the harmonic term, and t is the month of a year (t =1,2,3,, 12). Here we take H = 2 to capture the seasonal variations and estimate the parameters of the seasonal cycle using ordinary least squares. If we had more years of data, we might choose a larger value for H or allow for a different mean for each of the 12 months. The term ε is the nonseasonal or residual component. With the nonseasonal components, we recompute spatial correlations with equation (1) to investigate whether the model is able to capture spatial dependence beyond that induced by similar seasonal components in nearby pixels Spatiotemporal Correlation To understand spatiotemporal processes, it is often inadequate to consider spatial and temporal variations separately. For example, an extremely cold air event originating over the Arctic would not affect the northern United States until a few days later. Monthly summaries are not the proper vehicle for studying this kind of pattern, so instead, we look at daily summaries of precipitation. Specifically, to investigate the temporally and spatially dependent dynamic behavior of synoptic-scale and/or mesoscale events, we use spatiotemporal correlation, considering a range of spatial and temporal lags [Jun and Stein, 2004]. We consider correlations in daily precipitation amount and also an indicator of occurrence by using CPC observations, WRF simulations, and NCEP-R2 data in Further discussion of the differences in results between precipitation amount and occurrence is found in section 4.4. We define spatiotemporal correlation as r L;l;t;s1;s 2 X n t x i;l;l x L;l xiþt;lþs1;lþs 2 x Lþs1;lþs2 ¼ sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi X n t s ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2 X n t (5) 2 x i;l;l x L;l x iþt;lþs1;lþs 2 x Lþs1 ;lþs 2 where i denotes the day of the 31 years of record (i =1,2,3,, 11323), t is the time lag, and (s 1, s 2 ) is a vector indicating not only the direction but also the distance. The correlation coefficient is between the reference point at (L, l) on the ith day and the point at (L + s 1, l + s 2 ) on the (i + t)th day. In this study, the range of spatial lags for each pixel s matrix is 10 to +10 along both longitude and latitude Statistical Verifications This study evaluates the value added by an RCM by comparing with evaluation data sets and NCEP-R2 reanalysis data. To quantify the overall differences in spatial correlation/variogram and spatiotemporal correlation between WRF and PRISM (or CPC) as well as between NCEP-R2 and PRISM (or CPC), we compute average absolute differences between models and evaluation data sets over all the pixels on the grid spacing of NCEP-R2, as shown below: d WRF ¼ 1 P d NCEP ¼ 1 P X P X P jwrf i PRISM i j (6) jncep i PRISM i j (7) where i denotes each pixel and P denotes the total number of pixels. WRF i (PRISM i or NCEP i ) denotes either spatial correlation/variogram or spatiotemporal correlation of WRF (PRISM or NCEP-R2) data at ith pixel. For daily scale, PRISM i should be replaced with CPC i. Then we define a difference between d NCEP and d WRF : θ ¼ d NCEP d WRF (8) If θ > 0, then NCEP-R2 shows larger differences from PRISM (or CPC) data than does WRF, and vice versa. WANG ET AL American Geophysical Union. All Rights Reserved. 1243

6 Table 1a. Statistical Summary of the Differences in Figure 3 and Standard Errors for the Differences Between WRF (or NCEP) and PRISM on the Resolution of NCEP-R2 a South to North West to East 2 Apart 4 Apart 2 Apart 4 Apart d WRF d NCEP θ SE a Bold font indicates that the difference is statistically significant. To investigate the statistical significance of this difference, we conduct block bootstrap (see Appendix B) using 12 continuous months as a sample block. For each bootstrap sample, we first compute spatial correlation/variogram for each data set (PRISM or CPC, WRF, and NCEP-R2). Then we estimate the differences between WRF and PRISM (or CPC) as well as between NCEP-R2 and PRISM (or CPC) according to equations (6) (8). So we get eθ r ¼ e d NCEP;r e d WRF;r ; (9) where r =1,2, and R denotes the index of the bootstrap resample; here R = 500 bootstrap resamples are generated for each data set. We estimate the standard error of e θ by vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u 1 X R SE eθ ¼ t 2 eθ r θ ; (10) R 1 X R where θ ¼ R 1 eθ r.if0 < 2SE eθ < θ, then NCEP-R2 shows significantly larger differences from evaluation r¼1 data sets than does WRF. Otherwise, the difference between NCEP-R2 and WRF is not significant, and they may show similar differences from the evaluation data sets (we have only computed bootstrap standard errors for the summary statistics reported in Tables 1a 1c and 2. For these statistics, θ is always positive). To investigate the statistical significance of differences in spatiotemporal correlations, which are calculated from daily data, we conduct a block of blocks bootstrap and estimate the standard errors of the differences between evaluation data sets and models. For details of this approach, see Appendix B. To investigate the statistical significance of the differences in spatial correlation/variogram and spatiotemporal correlation over each pixel between WRF and PRISM (or CPC) as well as between NCEP-R2 and PRISM (or CPC), we calculate Δ i, which is defined by j Δ i ¼ WRF i PRISM i j j or Δ i ¼ NCEP i PRISM i j (11) SE gwrfi PRISMi g SE gncepi PRISMi g where i denotes each pixel and WRF i (PRISM i or NCEP i ) denotes either spatial correlation/variogram or spatiotemporal correlation computed by 31 year WRF (PRISM or NCEP-R2) data at ith pixel. For daily scale results, PRISM i should be replaced with CPC i. gwrf i, PRISM g i,andncep g i denote either spatial correlation/variogram or spatiotemporal correlations generated by 500 bootstrap resamples. SE is the standard error of the differences in spatial correlation/variogram or spatiotemporal correlation between models and data sets. If Δ i > 2, then, at least approximately, the differences are statistically significant at the 5% level. r¼1 Table 1b. Statistical Summary of the Differences in Figure 5 and Standard Errors for the Differences Between WRF (or NCEP) and PRISM on the Resolution of NCEP-R2 a South to North West to East 2 Apart 4 Apart 2 Apart 4 Apart d WRF d NCEP θ SE a Bold font indicates that the difference is statistically significant. WANG ET AL American Geophysical Union. All Rights Reserved. 1244

7 Table 1c. Statistical Summary of the Differences in Figure 7 and Standard Errors for the Differences Between WRF (or NCEP) and PRISM on the Resolution of NCEP-R2 a South to North West to East 2 Apart 4 Apart 2 Apart 4 Apart d WRF d NCEP θ SE a Bold font indicates that the difference is statistically significant. 4. Results 4.1. Spatial Correlations As shown in equation (1), in the absence of a time lag for spatial correlation, we expect the correlation between two grid cells separated by a short distance to be relatively high, because the weather at locations a short distance apart usually shows small differences. Accordingly, the correlations should decrease with increasing lags. On the other hand, two grid cells even a short distance apart over areas with complex topographies (e.g., mountain ranges) could have low correlations. In this section, we compute spatial correlations between one grid cell and another along the south-to-north and west-to-east directions by using total monthly precipitation values from the PRISM, WRF, and NCEP-R2 in Results for the nonseasonal component of monthly precipitation are described in section 4.3. As shown in Figure 2, for the south-to-north spatial correlations (first and second columns) between grid cells with various lags (s 1 in equation (1)) in the PRISM observation (first row), the spatial correlations of precipitation are substantially weaker over the mountain ranges than over relatively flat areas. Even at a lag of ~0.5 (~55 km), the correlations are often only around 0.5 over the Rocky Mountains, but generally around 0.9 over the Great Plains (GP) and eastern CONUS. The correlations are reduced further when the distance is 1 and larger, and there are negative correlations over some locations when the lag is 4. A similar but smaller diminution of correlations occurs at high elevations over southeastern CONUS (near Virginia, North Carolina, and Tennessee). The WRF model (second row) reproduces the decreasing correlations with increasing distances and also captures the strong local features of precipitation correlations over the Rockies and the southern mountains. However, the correlation is lower for the WRF simulation than for PRISM over several states of the western CONUS (e.g., Nevada and Oregon for 2.0 and 4.0 lags), while it is higher than for PRISM over the GP and much of the eastern United States. For the west-to-east spatial correlations between grid cells with various lags (s 2 in equation (1)), as shown in Figure 2 (third and fourth columns), results are qualitatively similar, with some evidence of stronger correlations than in the north-south direction, especially at more northerly latitudes over central CONUS. This stronger correlation might be, at least in part, due to the shrinking distance of a degree longitude as one moves northward, although it might also reflect differences in seasonal cycles and/or climate dynamics at different latitudes. WRF captures the pattern of west-to-east spatial correlations of monthly precipitation, except that (as for north-south correlations) it shows higher correlations than does PRISM over much of the eastern United States. The correlations are reduced from 0.5 apart to 4.0 apart, and there are negative correlations over the Rockies when the lag is 4. Table 2. Statistical Summary of the Differences in Figure 9 and Standard Errors for the Differences Between WRF (or NCEP) and CPC on the Resolution of NCEP-R2 a Region 16 Region 20 Region 35 Region 45 d WRF d NCEP θ SE a Bold font indicates that the difference is statistically significant. WANG ET AL American Geophysical Union. All Rights Reserved. 1245

8 South-to-north Correlations West-to-east Correlations 0.5 apart 1.0 apart 0.5 apart 1.0 apart WRF-PRISM WRF PRISM 2.0 apart 4.0 apart 2.0 apart 4.0 apart WRF-PRISM WRF PRISM Figure 2. Spatial correlation coefficients for PRISM-observed and WRF-simulated total monthly precipitation and the differences (bottom row) between WRF and PRISM on the grid spacing of WRF (12 km). (first and second columns) South-to-north spatial correlation with lags 0.5, 1, 2, and 4. (third and fourth columns) West-to-east spatial correlation with lags 0.5, 1, 2, and 4. To investigate the dominant climate system that drives the patterns of spatial correlations, we conduct the same calculation (equation (1)) but use summer and winter data, respectively (Figures S2 and S3 in the supporting information). The relatively low correlations over the Rockies are mainly induced by winter precipitation, because most of the precipitation over this region occurs in winter rather than in summer. This is why the correlations over WANG ET AL American Geophysical Union. All Rights Reserved. 1246

9 South-to-north Correlations West-to-east Correlations 2.0 apart 4.0 apart 2.0 apart 4.0 apart NCEP-PRISM WRF-PRISM PRISM Figure 3. Spatial correlation coefficients for (top row) the PRISM-observed total monthly precipitation and (middle and bottom rows) the differences between WRF and PRISM as well as between NCEP-R2 and PRISM, on the grid spacing of NCEP-R2 (~ ). (first and second columns) South-to-north spatial correlation with lags 2 and 4. (third and fourth columns) West-to-east spatial correlation with lags 2 and 4. Cross hatching indicates that the differences are statistically significant. the Rockies in summer are not as low as those in winter. On the other hand, the relatively low correlation over the Southwest (e.g., New Mexico and Colorado) is mainly induced by summer precipitation, because the precipitation over this region is mainly generated by NAM, which usually occurs in July and August [Castro et al., 2012; Bukovssky et al., 2013; Lebassi-Habtezion and Diffenbaugh, 2013]. The overestimation of correlations by WRF over the Great Plains and the Southeast is mainly induced by the simulated summer precipitation. On the other hand, the overestimation of correlations over north central CONUS is also attributed to winter precipitation. Over Texas and adjacent areas, precipitation is generated by warm moisture from the Gulf of Mexico and usually occurs before the NAM (May to June). Over southeastern CONUS, precipitation could be generated by tropical cyclones or other mesoscale atmospheric circulation near the surface [Konrad,2001],andprecipitation isusually heavier than in the other two regions during an entire year. In contrast, the precipitation over north central CONUS is mainly generated by upper level troughs, which are large-scale weather systems that induce relatively small local variations [Gutowski et al., 2008; Dominguez et al., 2012]. WRF typically captures the spatial correlations with short distance lags better than those with long distance lags, indicating that WRF is better at explaining small- and local-scale patterns in observed monthly precipitation than overall levels of precipitation. Some details in spatial correlations in PRISM that WRF captures at scales finer than those of NCEP-R2 are evidence of the value added by WRF. To investigate the value added by WRF to the driving data (NCEP-R2) further, Figure 3 compares spatial correlations between PRISM, WRF, and NCEP-R2 on the grid spacing of NCEP-R2 (~ ), with cross hatching indicating that the differences are statistically significant (Δ i > 2in equation (11)). For this comparison, we aggregate the PRISM-observed and WRF-modeled monthly precipitation data from their original spatial resolutions to the resolution of NCEP-R2 using a bilinear regridding approach. The bilinear regridding generates similar spatial correlation patterns to those on their original resolutions, and there are few differences in the results when the data are regridded using an inverse distance squared method. However, regridding using area averaging shows higher correlations (and lower variograms) than the other two approaches. Figure 3 again considers monthly precipitation, now at the longer spatial lags of 2 and 4. Similar to Figure 2, WRF and PRISM on the grid spacing of NCEP-R2 show relatively low correlations over the western mountains for both south-to-north and west-to-east correlations. There are still overestimates of spatial WANG ET AL American Geophysical Union. All Rights Reserved. 1247

10 South-to-north Variograms West-to-east Variograms 0.5 apart 1.0 apart 0.5 apart 1.0 apart WRF-PRISM WRF PRISM 2.0 apart 4.0 apart 2.0 apart 4.0 apart WRF-PRISM WRF PRISM Figure 4. Square root variograms for (first row) PRISM-observed and (second row) WRF-simulated total monthly precipitation and the (third row) differences between WRF and PRISM on the grid spacing of WRF (12 km). (first and second columns) South-to-north spatial correlation with lags 0.5, 1, 2, and 4. (third and fourth columns) West-to-east spatial correlation with lags 0.5, 1, 2, and 4. correlations (especially in south-to-north direction) over the Great Plains and the Southeast by WRF, but most of the differences between WRF and PRISM are not significant. In contrast, the differences between NCEP and PRISM are larger than those between WRF and PRISM, and there are many pixels that show significant differences from PRISM, especially over western mountains and Southeast (Figure 3, bottom row). These results indicate that NCEP-R2 does not capture spatial variations and local features of precipitation over the western WANG ET AL American Geophysical Union. All Rights Reserved. 1248

11 South-to-north Variograms West-to-east Variograms 2.0 apart 4.0 apart 2.0 apart 4.0 apart NCEP-PRISM WRF-PRISM PRISM Figure 5. Variograms for PRISM-observed total monthly precipitation and the differences between WRF and PRISM as well as between NCEP-R2 and PRISM on the grid spacing of NCEP-R2 (~ ). (first and second columns) South-to-north spatial correlation with lags 2 and 4. (third and fourth columns) West-to-east spatial correlation with lags 2 and 4. Cross hatching indicates that the differences are statistically significant. mountains as well as WRF, even at its own coarse resolution. It is natural to ascribe these problems with NCEP-R2 to its inability to resolve small-scale orographic features such as mountainous regions and coastal areas. Thus, two grid cells with a distance lag (i.e., 200 km or 400 km) would not have too many differences in their daily or seasonal variations. Table 1a summarizes the differences in spatial correlations between WRF and PRISM as well as between NCEP-R2 and PRISM and the standard errors of these differences (equations (8) (10)). NCEP-R2 shows significantly larger (θ > 2 SE) differences from PRISM than does WRF in both directions at lags of 2 and 4. We view this result as clear evidence of the value added by the RCM Spatial Variograms We expect the variogram between two nearby grid cells to be relatively small and to increase with distance between cells. However, for regions with complex topographies, monthly precipitation at even nearby locations might be quite different, yielding large variogram values even at short lags. In this section, we compute square root spatial variograms via equation (3) between one grid cell and another along the south-to-north and west-to-east directions. We still use overall monthly precipitation values from PRISM, WRF, and NCEP-R2 in with no seasonal decomposition, although taking spatial differences largely eliminates seasonal variation since neighboring pixels tend to have similar seasonal patterns. As shown in Figure 4, the variograms generally increase with increasing lag magnitude. This implies that the precipitation at a short (long) distance lag shows not only similar (different) temporal variations/annual cycles (Figure 2) but also similar (different) amounts (Figure 4). The variograms over western CONUS (excluding the Cascade ranges and California coastal zones) are typically smaller than those over the Great Plains and eastern CONUS. WRF captures the pattern of variograms but overestimates the variogram values over western and central CONUS in both zonal and meridional directions. This overestimation is partially due to model bias in precipitation amount (Figure S1), which induces bias in the spatial variograms. To investigate the dominant climate systems that induce the patterns of spatial variograms, we conduct the same calculation (equation (3)) but only use summer and winter data, respectively (Figures S4 and S5). Results show that the relatively large variograms over western mountains (e.g., the Rockies) are mainly induced by winter precipitation, while the relatively large variograms over the Southwest (e.g., New Mexico and Colorado) are mainly induced by summer precipitation. WANG ET AL American Geophysical Union. All Rights Reserved. 1249

12 South-to-north Correlations West-to-east Correlations 0.5 apart 1.0 apart 0.5 apart 1.0 apart WRF-PRISM WRF PRISM 2.0 apart 4.0 apart 2.0 apart 4.0 apart WRF-PRISM WRF PRISM Figure 6. Spatial correlation coefficients and differences as in Figure 2 but for nonseasonal components of total monthly precipitation. Figure 5 compares spatial variograms between aggregated PRISM, aggregated WRF, and NCEP-R2 with 2 and 4 lags on the grid spacing of NCEP-R2, with cross hatching indicating that the differences are statistically significant (Δ i > 2 in equation (11)). In comparison with WRF, the NCEP-R2 significantly underestimates the fine-scale (with respect to the NCEP-R2 resolution) spatial variations over western mountains and the Cascade Ranges at both directions and 2.0 and 4.0 lags. However, NCEP-R2 shows closer variograms to PRISM over the Great Plains than does WRF. Over the eastern United States, aggregated WRF and NCEP-R2 perform fairly similarly in reproducing the aggregated PRISM observations, and both of them show insignificant differences from PRISM over most pixels. Table 1b summarizes the overall differences in spatial variograms between WRF WANG ET AL American Geophysical Union. All Rights Reserved. 1250

13 South-to-north Correlations West-to-east Correlations 2.0 apart 4.0 apart 2.0 apart 4.0 apart NCEP-PRISM WRF-PRISM PRISM Figure 7. Spatial correlation coefficients and differences as in Figure 3 but for nonseasonal components of total monthly precipitation. Cross hatching indicates that the differences are statistically significant. and PRISM as well as between NCEP-R2 and PRISM and the standard errors of these differences. NCEP-R2 shows larger absolute differences from PRISM than does WRF at both directions with 2 and 4 apart. However, the difference is only significant at west-to-east direction with 2 apart when we consider all the pixels over CONUS. This is because WRF shows larger and significant absolute difference from PRISM over the Great Plains than does NCEP-R2, although it shows smaller and insignificant absolute difference from PRISM over the Rockies (over ~20 pixels) than does NCEP-R Spatial Correlation for Nonseasonal Components Figure 6 shows spatial correlations (equation (1)) for nonseasonal precipitation components (equation (4)) with various lags. In comparison with Figure 2, which gives correlations before removing a seasonal component, the spatial correlations with only nonseasonal components are reduced (Figure 6) over central and eastern CONUS but are increased over western mountains. This is because both the strong correlations over central and eastern CONUS and the weak correlation over western mountains are largely driven by the seasonal cycle, as indicated by the spatial correlations calculated for summer and winter precipitation (Figures S2 and S3). WRF generates similar spatial patterns of correlations to those of the PRISM for most lags. However, differences between PRISM and WRF at the 4 lag are apparent for south-to-north spatial correlations. This result suggests that WRF is better at capturing small-scale and mesoscale variations of nonseasonal precipitation than large-scale variations, which is also apparent in spatial correlations for total monthly precipitation (Figure 2). Figure 7 compares the spatial correlations for nonseasonal components between the aggregated PRISM, aggregated WRF, and original NCEP-R2 on the grid spacing of NCEP-R2, with cross hatching indicating that the differences are statistically significant. For this comparison, we first aggregate the PRISM-observed and WRF-simulated total monthly precipitation from their original spatial resolutions to the NCEP-R2 resolution then decompose the coarse-resolution data sets according to equation (4). In comparison with WRF, NCEP-R2 shows higher correlations than PRISM over most of CONUS, and the differences between NCEP-R2 and PRISM are significant at 2 lags in both directions, especially over western mountains and eastern CONUS (Figure 7, bottom row). These results indicate that NCEP-R2 captures less of the spatial variations at this scale than does WRF. The differences between WRF and PRISM are smaller than those between NCEP and PRISM WANG ET AL American Geophysical Union. All Rights Reserved. 1251

14 and are not significant over almost the entire CONUS. Table 1c summarizes the overall differences in spatial correlations of nonseasonal components between WRF and PRISM as well as between NCEP-R2 and PRISM and the standard errors of these differences. To estimate the standard errors, we first conduct block bootstrap resampling using 12 continuous months as a sample block. For each bootstrap sample, we fit the harmonic model (equation (4)) and then compute the spatial correlations using the residual component of each sample. We then compute the differences between WRF and PRISM as well as between NCEP-R2 and PRISM for each sample and estimate the standard errors of these differences. NCEP-R2 shows larger differences from PRISM than does WRF at both directions, but the differences are only significant (θ > 2 SE) with 2.0 lag. To summarize sections , in addition to traditional evaluation procedures such as comparison of climatological means, the evaluation of spatial dependence of an RCM can be an important tool for assessing the value added by an RCM to a GCM or a reanalysis data set Spatiotemporal Correlations Usually, the CONUS experiences westerly flows, and weather events move in the same direction. On the other hand, in the warm season over the central and eastern United States, rainfall systems tend to move from south to north because of the low-level jet a low-altitude wind that transports moisture from the Gulf of Mexico north northeastward. Thus, we expect the spatiotemporal correlation to be higher between the reference pixel and pixels toward the east and northeast than between the reference pixel and pixels toward the west and southwest. However, exceptions exist. For example, when an upper level low-pressure system is south of a high-pressure system, the main flow of the jet stream is cut off, and the storm or rain will move in the opposite direction east to west. In addition, over the western United States with its complex terrain (e.g., mountain ranges), the weather is strongly affected by the topography, and a south-to-north low-level jet is rare. In these two cases, we might not be able to detect the expected similar features. To investigate the model s ability to capture these dynamic aspects of synoptic-scale and/or mesoscale weather systems, we select 48 well-spaced subregions, mostly in the CONUS. Each subregion is a area in the zonal and meridional directions, with the reference pixel (denoted by the numbers in Figure 1) located at the center. First, we computed spatiotemporal correlations for daily precipitation amount, which are generally quite small and therefore not very informative. In particular, more than 90% of the correlations are < 0.1 over most of the areas selected for a 1 day lag, in both all-season data and warm-season (May to September) data. This result is perhaps not surprising, because from one day to the next, the precipitation from a weather system could be getting stronger or weaker. For example, if a strong low-pressure system (cyclone) that generates precipitation and is associated with warm, moist air moves over a relatively cold, dry area, it will become weaker and generate less precipitation because it loses the energy needed to stay warm and moist. In contrast, if this cyclone meets with a low-pressure warm system, it could gain more energy and generate more precipitation. Thus, if site 2 is a few degrees to the east or northeast of site 1, we would not expect the precipitation amount at site 2 tomorrow to be the same as that at site 1 today. But, if it rains today at site 1, it will likely rain tomorrow at site 2 as well. With this consideration in mind, we investigate the daily precipitation occurrence by coding the original data (CPC at 0.25 and WRF at 12 km) to 1 or 0, indicating rainy or nonrainy days, respectively, at an occurrence threshold of 1.0, 5.0, and 10.0 mm d 1, respectively. Then we compute spatiotemporal correlations according to equation (5). Figures 8a 8d show spatiotemporal correlations using all-season daily precipitation occurrence at 5.0 mm d 1 between the reference pixels and the surrounding pixels over four selected areas. Similar figures for the other 44 selected pixels and using different thresholds can be found in the supporting information (Table S1). The spatial lags between two pixels range from 10 to 10, with 1 increments in all directions. The four reference points for Figure 8 are numbers 16, 20, 35, and 45, over the Southwest, GP, Southeast, and Northeast, respectively (Figure 1). These four selected regions are representative of four different regions with a range of terrain and/or climate systems. The other subregions around each of these four subregions show similar patterns of spatiotemporal correlations (see supporting information). We focus on presenting results using 5.0 mm d 1 as a threshold to define rainy days, because the gridded data CPC generated based on Cressman interpolation method [Chen et al., 2008] significantly overrepresent the frequency of light rain ( 1mmd 1 [see Chen et al., 2008, Figures 3 and 5]). For those pixels with no occurrences of precipitation 5mmd 1, which is not available to calculate correlations, we assign grey color in Figures 8 9. WANG ET AL American Geophysical Union. All Rights Reserved. 1252

15 CPC Observations (a) Region around Point 16 (b) Region around Point WRF WRF-CPC Figure 8. Spatiotemporal correlations for CPC-observed and WRF-simulated precipitation occurrences and the differences between CPC and WRF with 1 day lag in all directions around reference (a) point 16, (b) point 20, (c) point 35, and (d) point 45, as identified in Figure 1. The increment of spatial lags is 1. The grey color indicates that the occurrence of precipitation 5mmd 1 over that pixel is zero. The values at the reference points in Figure 8 are the ordinary lag 1 autocorrelations in time. The values around the reference points in the selected subregions are correlations between the reference point and a wide range of spatial lags with a 1 day temporal lag. For example, the spatiotemporal correlations to the northeast of a reference point (above and to the right in Figure 8) are calculated along the southwest-to-northeast direction, while the spatial-temporal correlations to the southwest of a reference point (below and to the left in Figure 8) are computed along the northeast-to-southwest direction. Significant asymmetries are apparent between the correlations to the north and those to the south of the reference points (Figure 8), as also reported by Jun and Stein [2004] for air pollution concentrations. Similarly, the correlations to the west and to the east of a reference point are also asymmetric. Similar to the CPC data, WRF generates higher correlations to the north than to the south of the reference point in all the regions shown here. In addition, correlation patterns for WRF output over the different areas vary similarly to the CPC observations. However, WRF slightly overestimates the correlations over New Mexico and Colorado (reference point 16, Figure 8a), Great Plain (reference point 20, Figure 8b), and West Virginia (reference point 45, Figure 8d). The differences in spatiotemporal correlations between CPC and WRF are somewhat similar when we use 1.0 or 10.0 mm d 1 as a threshold to compute the spatiotemporal correlations, but the values of correlation are larger (smaller) when we use 1.0 (10.0) mm d 1 as thresholds. WANG ET AL American Geophysical Union. All Rights Reserved. 1253