18 Aug In support of Alternative A, no action, based:

Transcription

1 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 1 18 Aug 2014 Comments on CENWP-PM-E / Double-crested cormorant draft EIS TO: Sondra Ruckwardt, U.S. Army Corps of Engineers, Portland District, PO Box 2946, Portland, OR From: Gary Shugart, PhD., Slater Museum of Natural History, University of Puget Sound, Tacoma, WA CORMORANT-EIS@USACE.ARMY.MIL In support of Alternative A, no action, based: Predation estimates for Double-crested Cormorants at East Sand Island are simulated. I had an opportunity to review the inner working of the bioenergetics model used for simulations where is was used in a nearby Tillamook Bay, OR. Problems with the simulation included: o Mistakes in allocation of salmonid proportions resulting in 40% overestimation of consumption in Tillamook Bay. o A tenuous protocol for converting binary data from genetic id to quantitative data for salmonid proportions in the diet. o Mistakes in estimating standard deviations used to compute confidence intervals. o Mistakes in assumption of normality for small proportions o Exorbitant extrapolation from relative few fish in samples to millions. o Internal inconsistencies in calculations. Similar problems probably exist in simulated predation estimates for this DEIS, but this cannot be determined because code for generating the numbers and input data were not provided. Foregut sampling could be abandoned as it is a waste of time and effort without truthing the miniscule samples to the population as a whole. As is, the data simply provides input for a garbage in, garbage out modeling efforts.

2 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 2 Management of piscivores in the Columbia River Estuary is ultimately a numbers game revolving around predator numbers vs how much of the resource is consumed because the impacts of predation on salmon populations are unknown. Consumption estimates are based on a bioenergetics model that generates the number of items consumed. Ultimately these numbers, usually ranging between million salmonids, provide the rationale for the management actions outlined in the DEIS. Little is known of the inner working of the model or input values used to produce the consumption numbers. I had an opportunity to view the inner workings of the model in review of consumption numbers for double-crested cormorants in Tillamook Bay, OR in The analysis revealed that the model was inapplicable for the specific example of Tillamook Bay 2012, and probably for other instances where it has been used, such as East Sand Island, which is the focus of this DEIS. Note that the model and data used to generate numbers for East Sand Island were unavailable being deemed proprietary, thus the Tillamook Bay 2012 example was used as a surrogate. After examination of the model, current and historical consumption numbers are in need of review and recalculation after repair and modification of the model. Even after such an effort this modeling approach appears to be unworkable because of statistical limitation and an alternative approach using simple frequency of occurrence and nonparametric methods are needed in order to inject some scientific validity to the consumption numbers I ve dispensed with the detailed comments on the DEIS and simply agree with bioenergetics guru, Don Lyons and the rest of Roby et al. team, that Despite over a decade of study by scores of biologists, scientific uncertainty remains regarding the significance of avian predation on juvenile salmonids in the Columbia River estuary. We now know that millions of smolts are consumed by (Don Lyons et al. 2010, p. 250) Updated to 2014, we really know that piscivores eat fish and some of these are salmonids, including some that are designated threatened/endangered. Demonstrating a significant impact on salmonid populations is difficult due to effects related to density of predators and prey, compensatory mortality, ocean conditions, condition dependent survival, differential digestibility of prey, experimental design, and psychological/political bias (Welsh et al 2008, Lyons et al 2010, Fort et al. 2011, Göktepe et al. 2012, Hilborn 2013, Rechisky et al. 2013a, Rechisky et al. 2013b, Hilborn 2013, Evans et al. 2014). Alternatively there may be no significant impact on salmonid populations as evidenced by the record runs of late despite record predation in brood years (e.g., CBB Bulletin, Aug 2014), and higher survivorship in the Columbia River than in the Fraser River, which lacks both dams and significant predator populations (Welch et al. 2008). Lacking any demonstrable impacts on salmonid populations, the DEIS and management effort focus on simulations of fish consumed using a bioenergetics model. The model that has been used to generate the number of fish and other prey items eaten by piscivores in the Columbia River Estuary (CRE), East Sand Island (ESI) and elsewhere for about 15 years beginning with Roby et al. s (2003) use on Caspian terns (Hydroprogne caspia). Ultimately estimates from the model, termed best estimates as usually cited, provide the rationale for management. For example in this DEIS (p 2-3), predation estimates provide the

3 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 3 basis for the suggestion of a 3.6% gap in the former vs current steelhead survivorship, which then provides the prime driver for management (Fredricks 2013). However, data and code used to generate estimates are lacking and the model is essentially a black box except for sketchy details in what appears to be the model s original conception (Roby et al. 2003). The inner workings of the model including the code used for calculations and input values should be in this DEIS and should have been provided in yearly reports (see yearly reports from Bird Research NW, Roby et al ). Current usage of the model, apparently retitled Bird Research NW Bioenergetics Model (hereafter BRNW/OSU model), usually provides some best estimates numbers and putative 95% confidence intervals (CIs) referencing Roby et al. (2003). The CIs are an integral part of the science underlying this protocol and they should provide statistically and thus scientifically sound estimates that inoculate estimates from criticism. However, based on this analysis the opposite is true. In analyzing the BRNW/OSU model many errors or lapses are apparent; some may have been specific to the Tillamook Bay 2012 example (Appendix A). However I ll concentrate on what appear to be some the more glaring fundamental problems after an explanation of calculations and content that provide the basis for this analysis. Methods: The input and output of the BRNW/OSU model were received from OSU through Oregon Department of Fish & Wildlife (ODFW) on 9 May 2013 as an Excel workbook 2012TillamookBayDCCOBioenergeticsModel. Within the workbook, data were in the form of worksheets or tabs including: 1. Input Data 2. Input Variable Values 3. Energy Needed 4. Output Numbers by Time Period 5. Output Biomass by Time Period 6. Output Biomass by Prey Type 7. Output Energy by Prey Type 8. Output Graph - Total Salmonids 9. % Consumed I received second copy of the 2012TillamookBayDCCOBioenergeticsModel on 21 June 2013 after a query to Lindsay Adrean (ODFW) and Don Lyons (OSU) regarding some aspects of the original workbook. The two copies did not differ indicating that there had been no updates or change to input or output data. Data used for input to the model were received as three additional workbooks: TMK_DCCO_2012_StomachContents, TillamookSurveysAprilMay2012 and TMK_DCCO_2012_Salmonids (Genetic id s). An additional pdf, OriginalData, provided copies of field stomach dissection results. All workbooks and pdf s were deemed public domain by ODFW. From the workbooks and raw data I verified input variables that appeared in Input Values and Input Variable Values tabs on the 2012TillamookBayDCCOBioenergeticsModel workbook. The macro used to calculate the output was not provided to ODFW or me after specifically requesting it for review because it was deemed proprietary by BRNW or OSU (D. Lyons, pers. com.). However a macro to run the simulation was simple to reconstruct from the embedded formulae that were still in the workbook and from obvious calculations from

4 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 4 inspection of input and output data. I wrote a macro (GWS macro) to run the calculations using the same values in the Input Variable Values tab and output the results for comparison. The model was also run with corrected input values for a comparison (see below). The macro is available upon request. Major steps in the simulation include the calculation of total energy as Total Energy = Population Size x DEE (kj/day) x Days/Assimilation Eff, where Populations Size was based on an indexed number of pairs, DEE was Daily Energy Expenditure based on seven males and three females for nonstandard day lengths (see Lyons et al 2010, Table 3.1), assimilation efficiency was from literature values, and days represented days two-week periods. At some point in calculations, indexed pairs was converted to individuals. Using total energy, the Energy for each prey type = Total Energy x Proportional representation of prey in diet. The latter value was based on DCCO foregut sampling to quantify non-salmonids and overall salmonids, followed by genetic id and conversion of binary data to quantitative data (see below). Energy for each prey type was extrapolated to numbers as Numbers consumed=energy for each prey type/ Energy Density/Average mass of prey type where energy density was kj/gm of prey based on empirical data and estimation yielding biomass for each prey type that was extrapolated to numbers using average mass, which was the average weight of prey items. Values for the above quantities that were input to the BRNW/OSU model for Tillamook Bay 2012 appear on the Input Data tab. These were combinations of simulated, guesstimated, and empirically derived values. In some instances means and SDs were based on a single value. Using these initial values as input (or seeds) to a random numbers generator after converting SDs to standard errors (SEs), 1,000 values were produced for each mean or simulated mean that appear on the Input Variable Values tab. These were then used to calculate numbers consumed 1,000 times in a process referred to as a Monte Carlo simulation. These 1,000 values were then used to calculate SDs and +/-2 SDs were used as the CIs. Given that SDs were simulated for many variables, the SEs represent a second level of simulation and compounded error (see Peralta 2012). Although the type of distribution wasn t specified, I assumed a random normal distribution based on plots of the 1,000 input values and the apparent original basis for the model (Furness 1978). Once numbers were computed, these were then used to estimate the % of that prey items that were consumed based on estimated populations. A more detailed analysis is available upon request and presented in summary form as Appendix A. Obvious problems included: Non-random randomized simulations. The Monte Carlo simulation simply involved generating 1,000 values for each input value using means and SEs then calculating the numbers consumed 1,000 times and using 2 x SD s of the result for CIs. Implicit was that a random numbers generator was used, in this case a random normal generator, and that the values were

5 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 5 thus random. However, an inconsistency occurred in generation of random values for proportions for prey types (Table 1). The proportions were relative values, therefore changing one changes others. In generating 1,000 random values for each proportion, the model forced these to sum to one or each iteration. This violates the assumption of randomness implicit in the Monte Carlo method and constrains variability in the end result thereby minimizing CIs. It appeared that adjustments were made by adding or subtracting small values to proportions until the total equaled one. Incorrect calculations. The SDs and subsequent SEs used to generate 1,000 iterations for prey proportions in the diet that were used to calculate CIs were incorrectly calculated. There were insufficient raw data to compute SDs empirically, so SDs were estimated, but an incorrect formula was used, actually the variance, resulting in gross underestimation of the variation and resulting CIs (Tables 2 & 3). The results using the incorrect SEs are provided in Fig. 1. The results of a run using SEs computed correctly are shown in Fig. 2 where all confidence intervals venture in the negative numbers. This illustrates a novel ecological phenomenon of negative consumption, and is facetious, but provides a striking example of a problem that appears to be hard coded into the model. This is that variation and CIs were underestimated thereby making it seem that the CIs and midpoint best estimates were reasonable. In reality, the CIs reported as 95% range from 10-50% which seriously undermines the credibility of the estimates and has no scientific validity. Without 95% CIs, the best estimates have unknown reliability. Violation of a first lesson in biostatistics. Related to the above is precaution given in Biostats books and classes (e.g., Zar 2010 ), or just Google it, that proportions below ~0.2 or above ~0.8 are not normally distributed and cannot be treated as such without transformation. Almost all the prey proportions in the model were below 0.1 due to small samples and parsing into too many categories, but the authors apparently proceed to generate random normal distributions using assumed parametric values. Even if samples size were adequate, the distribution would not be normal and thus could not be used in this type of modeling without transformation of the variables or calculation of asymmetrical CIs. The model calculations failed internal consistency checks. A problem emerged in examining the input vs output values in that there appeared to be an internal inconsistency in the calculations. Specifically, the proportions on the Input Values and Input Variable Values tab did not match what was used to allocate total energy to prey categories. The calculations were deterministic therefore it should be possible to calculate backward from any intermediate step. For example, assume total energy is 10,000 kj, and the proportional allocation to a prey category is of.3 yielding 3,000 kj for this prey category. A back calculation of 3,000/10,000 should equal.3, but the values deviate from the expected by up to 30% (Table 4, Fig. 3). Because of the extreme extrapolations from few data and large multipliers like population size, the resulting errors in final results would be significant. I discovered this in comparing the results from single calculations and verified it in running the GWS macro using the values provided on the Input Variable Values tab, which although incorrect (see below), were used for comparative purposes. Slight differences might be due to the sequence of calculations combined with rounding errors but this difference wasn t slight. Alternatively there could have been hidden correction factors in

6 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 6 play to adjust proportions, but more likely there was an error in calculations or summations in the coding that currently is in use. To resolve this, code and raw data are needed. Converting quantitative data (Yes/No) to quantitative data precise to 3 decimals. Proportions of individual prey categories in the diet including total salmonids were estimated from foregut samples. Then salmonid samples were further analyzed using genetic identification (Roby et al , Lyons 2013). This protocol was an essential part of the model as it generates the proportions of individual salmonids in the diet that were then used to calculate the numbers consumed. However genetic id provides only Yes/No or binary data. The protocol for converting binary data to quantitative data precise to three decimals was incredulous and no methods or protocol were provided in yearly reports other than genetic id was used estimate proportions of salmonids (Roby et al ). Reconstruction of the protocols was done in inspection of an ancillary Excel workbook entitled TMK_DCCO_2012_Salmonids (Fig. 4). This hidden protocol adds unaccounted error to estimates and lacks scientific rationale or common sense. The protocol is outlined in Fig. 4, correspond to: 1. Compute salmonid vs non-salmonid proportions based on individual stomach proportions (not shown in Fig. 4) 2. Genetic Id of some of the salmonid samples, 11 of 14 in Tillamook Bay Populate a frequency of occurrence (FO) matrix parsed by the number categories found (e.g., if two species or categories were found, each is assigned a FO of 0.5) 4. Sum FO by category 5. FO expressed as proportions of all summed FOs 6. FO proportions were weighted by average weight of fish 7. Weighted FO were expressed as proportions of the total weighted FOs 8. Weighted FO proportions were used to apportion the overall salmonids to category 9. Use proportions from 8 to apportion Total kj/ energy density (kj/gm) to get biomass, then /average mass (gm) to get the number of fish in each category. A simple mistake leads to over estimating salmonid consumption by 40% in Tillamook Bay Perhaps unique to the Tillamook Bay 2012 data, an error was made by using weights for 45 gm and 5 gm for cutthroat and chum respectively, rather than 40 gm and.6 gm used in the model. Using the correct values dropped the total take by 40% from ~51,000 to ~29,000 (Fig. 4, compare 6a-9a to 6b-9b). The initial incorrect calculation were submitted to the Oregon Legislature in support of cormorant control (Lyons 2013), used in a final report (Adrean 2013), and in summary form are in the DEIS. This might be a single mistake, but since the code used for calculations and input data were not provided for ESI, it is impossible to tell. This alone is sufficient reason to review the ESI consumption data because the personnel that produced the Tillamook Bay 2012 numbers also produced those numbers for ESI apparently using the same model and protocols. Small sample sizes were obfuscated by the fog of modeling. A related data disability that emerged from the Tillamook Bay 2012 data were that sample sizes were incredibly small even though 2-10% of the populations were sampled. Using data from workbooks, I calculated that the entire effort was based on a minimum of 29.7 salmonids. These were extrapolated to

7 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 7 ~51,000 salmonids using the incorrect proportions or 29,000 using the corrected proportions (Fig. 4). Similar detailed data were not available in annual draft reports for ESI (Roby et al ) although, some creative summaries can be done to produce similar estimates. Using data from yearly reports (Roby et al ), reverse extrapolation can be done to determine the approximate number of salmonids in the original sample (Table 5) and the extrapolations for each fish in the sample (Fig. 5). Results from the latter shows that each fish was extrapolated from 10,000 to 350,000 fish consumed, depending on the category (Fig. 5). Actual sample sizes and data typically are forgotten in modeling, but these data should have been presented in tabular form in yearly reports and in the DEIS to ground the science in reality. In conducting this analysis, it also occurred to me that there are two categories or Chinook, sub-yearling and yearling, but the protocol indicates salmonids were id d genetically, which raises additional questions regarding methodology and incomplete protocols. I provided a few examples of what appear to be fundamental flaws in the BRNW/OSU bioenergetics model and data that was used to generate the number of prey consumed in Tillamook Bay There were numerous problems specific to the Tillamook example summarized in Appendix A. More significantly, the Tillamook workbook provided a look at the inner workings of the model used to generate the number of prey items consumed by DCCOs at ESI. A problem with modeling is that the details often are obscured or ignored in the quest to run the model. Although the model might be reasonable, the output might simply be garbage due to sketchy input data. In model speak this phenomenon is referred to as garbage in, garbage out or GIGO modeling. This appears to be the case for a specific example I ve examine where the model was used to calculate the take of salmonids in Tillamook Bay, OR in In addition, the model appears to be flawed due to statistical anomalies and mistakes. Although the model and data used in the CRE & ESI were unavailable (D. Lyons, pers. com.), the deficiencies in the model appear hard coded rendering the model as configured unusable where it has been used. The end result of simulation, or best estimates produced by the model, are simply midpoints of putative 95% confidence intervals (CIs). For management and public relations, these are taken as the numbers consumed and CIs are largely ignored (e.g. Fredricks 2013, Lyons 2013, Adrean 2013, this DEIS). For example, Fredricks (2013) naively used the best estimates expressed to three decimal places to calculate that there was a.036 (or 3.6%) deficiency in survival for steelhead between former and present conditions. Reducing DCCOs at ESI to ~5,600 pairs as recommended in management alternative in the DEIS (p 2-3) would theoretically restore the former condition. However, given the imprecision of the estimates based on CIs and sketchy extrapolated numbers (Fig. 5), the estimates were so imprecise that a.036 difference would not be detectable. At some point in the history of using the current modeling approach, the best estimate midpoint was substituted for the interval probably to give managers, public relation officers, and the public a single number to focus on. Without including the CIs in analysis, there is no way to assess the believability of the best estimate midpoints and as such efforts lack statistical and scientific credibility. References

8 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 8 Adrean, L Avian Predation Program 2012 Final Report, 10 pp, ( CBB (Columbia Basin Bulletin), 8/9/2014. EXPECTED RECORD RETURNS, 1.5 MILLION FALL CHINOOK, 638,300 COHO, LIKELY A FISHING SEASON TO REMEMBER. (Intentionally left bold) Evans, A. F., N. J. Hostetter, K. Collis, D. D. Roby & F. J. Loge Relationship between Juvenile Fish Condition and Survival to Adulthood in Steelhead. Transactions of the American Fisheries Society. 143 (4): Fort, J., W.P. Porter and D. Grémillet Energetic modelling: A comparison of the different approaches used in seabirds. Comparative Biochemistry and Physiology Part A: Molecular & Integrative Physiology 158: Furness, R. W Energy requirements of seabird communities: a bioenergetics model. Journal of Animal Ecology 47: Fredricks, G Appendix D in DEIS. Double-crested Cormorant Estuary Smolt Consumption BiOp Analysis. NMFS (National Marine Fisheries Service). Göktepe, O, v P. Hundt, W. Porter and D. Pereira Comparing Bioenergetics Models of Double-Crested Cormorant (Phalacrocorax auritus) Fish Consumption. Waterbirds 35(sp1): Hilborn, R Ocean and dam influences on salmon survival: The Decline of Columbia River Chinook Salmon. PNAS 110 (17): Lyons, D Summary of Preliminary Juvenile Salmonid Consumption Estimates by Double-crested Cormorants in Tillamook Bay during April and May, 2012, 2pp. Submitted to Oregon Legislature by J. Rohleder for Seafood, OR, McKenzie River Guides Assoc in support of a cormorant (species unspecified) control bill. ( Peralta, M Propagation of Errors. How to Mathematically Predict and Measure Errors of First and Second Order. San Bernadino, CA. Ridgway, M. S A review of estimates of daily energy expenditure and food intake in cormorants (Phalacrocorax spp.). J. Great Lakes Research 36: Roby et al , yearly reports

9 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 9 Roby, et al Quantifying the effect of predators on endangered species using a bioenergetics approach: Caspian terns and juvenile salmonids in the Columbia River estuary. Can. J. Zool. 81: Rechisky, E. L., D. W. Welch, A. D. Porter, M. C. Jacobs-Scott, and P. M. Winchell. 2013a. Influence of multiple dam passage on survival of juvenile Chinook salmon in the Columbia River estuary and coastal ocean. PNAS (April) 110 (17): Rechisky, E. L., D. W. Welch, A. D. Porter. 2013b. Reply to Haeseker: Value of controlled scientific experiments to resolve critical uncertainties regarding Snake River salmon survival. PNAS 110 (37): E3465. Trites, A. W. and R. Joy Dietary analysis from fecal samples: How many scats are enough? J. Mammalogy 86 (4): Welch DW, Rechisky EL, Melnychuk MC, Porter AD, and Walters CJ Survival of Migrating Salmon Smolts in Large Rivers With and Without Dams. PLoS Biol 6(10): e265. Zar, J. H Biostatistical Analysis.

10 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 10 Table 1. Input Variable Values tab from 2012TillamookBayDCCOBioenergeticsModel showing prey types 6-22 labeled in row 2 with the values from random generation of values in respective columns. I added columns 143 and 144 showing sums of the 22 random values illustrating the random values were adjusted to sum to 1 at the precision of six decimal places. Using truly random values and the sums vary from.5 to 1.5 depending on the seed values for standard error used to induce variation in the random numbers generator. The end result is that by forcing proportions to sum to 1, the simulation is not random and the variation in the final CIs are constrained.

11 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 11 Table 2. From BRNW Bioenergetics Model workbook 2012 Tillamook Bay DCCO Salmonid Consumption xlsx Input Values tab showing incorrect computation of SDs as the mean x the complement of the mean (Column I * (1-Column I)), which is the variance in a binomial distribution. Only Period 1 of 4 was used as an example but the error was repeated for all periods and presumably was systematic where the model has been used. The formula should have been the square root (SQRT) of the value shown in Std. Dev. From this the error is further compounded by using the incorrect formula for standard error. Note that Col I is mistakenly labeled % when it is actually a proportion. The mislabeling occurred numerous times in the workbook. Column C D I J Prey species % of diet by number fish watch mean*(1-mean) Std. Dev. Salmonidae cutthroat =I143*I144 =I154*(1-I154) chum =I143*I145 =I155*(1-I155) Coho =I143*I146 =I156*(1-I156) unid salmonid =I143*I147 =I157*(1-I157) Steelhead =I143*I148 =I158*(1-I158) Clupiedae Herring, Sardine, Shad 0 =I159*(1-I159) Engraulidae Anchovy 0 =I160*(1-I160) Osmeridae Smelt 0 =I161*(1-I161) Surf Perch, Shiner Embioticidae Perch 0 =I162*(1-I162) Unid Nonsal Unid Nonsal =I163*(1-I163) Gasterosteidae Stickleback 0 =I164*(1-I164) Cottidae Sculpin =I165*(1-I165) Ammodytidae Sandlance 0 =I166*(1-I166) Pholididae Gunnel 0 =I167*(1-I167) Snake Prickleback 0 =I168*(1-I168) Other Prickleback =I169*(1-I169) Prickleback/Gunnel 0 =I170*(1-I170) Pipefish =I171*(1-I171) Rockfish 0 =I172*(1-I172) Greenling 0 =I173*(1-I173) Gadidae Cod 0 =I174*(1-I174) Non-fish Crustaceans, insects =I175*(1-I175)

12 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 12 Table 3. From BRNW Bioengergetics Model workbook 2012 Tillamook Bay DCCO Salmonid Consumption xlsx Input Values tab Period 1 of 4. Showing proportions (mislabeled as %), incorrectly computed SD and SE, correctly computed SD and SE, and differences. By using the incorrect SD and SE, variation is grossly underestimated, which significantly underestimates confidence intervals making it appear that the estimates of consumption were statistically and scientifically valid. Use of the incorrect SE was verified by inspection of summary values on the Input Variable Values tab. % of diet by fish watch Incorrect Std. Dev. Sample Size Incorrect SE=SQRT(n) Correct SE= SQRT( p ( 1 - p)) / n] % difference relative to incorrect original value cutthroat chum Coho unid salmonid Steelhead Herring, Sardine, Shad 14 Anchovy 14 Smelt 14 Surf Perch, Shiner Perch 14 Unid Nonsal Stickleback 14 Sculpin Sandlance 14 Gunnel 14 Snake Prickleback 14 Other Prickleback Prickleback/Gunnel 14 Pipefish Rockfish 14 Greenling 14 Cod 14 Crustaceans, insects

13 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Aug 2014, 13 Table 4. From BRNW/OSU Bioengergetics Model workbook 2012 Tillamook Bay DCCO Salmonid Consumption xlsx. Data suggesting there was an internal inconsistency in the BRNW/OSU model. Percent difference of prey proportions as reported on Input Tab and Input Variable Values tabs vs what was used based on back calculations from Output Biomass by Time Period tab. The numerical differences in the proportions are small but because of the extrapolations, predicted numbers can be large. See Fig 3. Back Calculated from Biomass Period Verified 1&2 on Input Var. Period 1 Period 2 Values tab numerical diff Back Calculated Period Verified 3&4 on % Input Var. difference Period 3 Period 4 Values tab numerical diff % difference cutthroat chum Coho unid salmonid Steelhead Herring, Sardine, Shad Anchovy Smelt Surf Perch, Shiner Perc unid non salmonid Stickleback Sculpin Sandlance Gunnel Snake Prickleback Other Prickleback Prickleback/Gunnel Pipefish Rockfish Greenling Cod Crustaceans, insects

14 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 14 Table 5. Data copy/pasted and parsed (gray cells) from Bird Research NW website ( yearly breakdown of consumption. Using the total gm of fish in yearly samples, the gm of salmonids can be estimated, followed by gm of species or age, divided by average size of fish yieldsg the estimated number of fish in original sample Dividing millions of fish from Roby et al by estimated number in original sample yeilds the extroplation "rate" for each fish (Fig. 5). gm identifia ble fish # fish, minimum* in sample Extrapolated millions of salmonids from Roby et al average gm/fish, from Tillamook Bay 2012 workbook, or Lyons et al # % gm gm of gm/stom millions/ Year stomachs samonids salmonids salmonid proportion ach # of fish 1 fish = , % 2,781 92% id'd "=ColK/ColO" see ColB juvenile salmonids comprised 10.7% of the diet 11.4 million sub yearling Chinook smolts (95% c.i. = million; 70% of total 1, , million coho salmon smolts (95% c.i. = million; 17% of total smolt consumption), , million steelhead smolts (95% c.i. = million; 6%), , million yearling Chinook salmon smolts (95% c.i. = million; 5%), , million sockeye salmon smolts (95% c.i. = million; 1%; Figure 70) ,803 TOTAL 2, , , % 4, million smolts (95% c.i. = million), 10.8 million smolts or 57% were sub yearling Chinook salmon (95% c.i. = million), 2, , million smolts or 26% were coho salmon (95% c.i. = million), 1, , million smolts or 9% were steelhead (95% c.i. = million), , million smolts or 8% were yearling Chinook salmon (95% c.i. = million), and , million smolts or 0.6% were sockeye salmon (95% c.i. = million; ,367 TOTAL 4, , , % 4,710 76% were sub yearling Chinook salmon (best estimate = 15.6 million smolts; 95% c.i. = , ,299 13% were coho salmon (best estimate = 2.7 million smolts; 95% c.i. = million), ,327 6% were steelhead (best estimate = 1.2 million smolts; 95% c.i. = million), ,094 4% were yearling Chinook salmon (best estimate = 0.9 million smolts; 95% c.i. = million , % were sockeye salmon (best estimate = 0.01 million smolts; 95% c.i. = million ,245 TOTAL 4, , , % 3, was 19.2 million smolts (95% c.i. = million), 69.8% were sub yearling Chinook salmon (best estimate = 13.4 million; 95% c.i. = milli 2, , % were coho salmon (best estimate = 3.0 million; 95% c.i. = million), , % were steelhead (best estimate = 1.5 million; 95% c.i. = million), , % were yearling Chinook salmon (best estimate = 1.3 million; 95% c.i. = million), ,493 and 0.2% were sockeye salmon (best estimate = 0.03 million; 95% c.i. = million; ,122 TOTAL 3, , , % 1,965 74% were sub yearling Chinook salmon (best estimate = 8.3 million; 95% c.i. = million 1, ,506 12% were coho salmon (best estimate = 1.4 million; 95% c.i. = million), ,800 7% were steelhead (best estimate = 0.8 million; 95% c.i. = million), ,691 6% were yearling Chinook salmon (best estimate = 0.7 million; 95% c.i. = million), and ,610 < 1% were sockeye salmon (best estimate = 0.02 million; 95% c.i. = million; Figure ,830 TOTAL 1, ,857 *from foregut, so fish mostly whole

15 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 15 Figure 1: From BRNW Bioenergetics Model workbook 2012 Tillamook Bay DCCO Salmonid Consumption xlsx. Incorrectly computed confidence intervals using BRNW/OSU model showing that no intervals venture into negative territory because variation in values was grossly under represented. Compare to Fig. 2 for correct values. 120, ,000 80,000 60,000 40,000 20,000 0 cutthro at chum Coho unid salmon id Steelhe ad Herring, Sardin e, Shad Anchov y Smelt Surf Perch, Shiner Perch Unid Stickle Nonsal back Sculpin Sandla Gunnel nce Snake Prickle back Other Prickle back Prickle Pipefis Rockfis Greenli back/g h h ng unnel Midpoint of CI 7,264 22,512 7, ,399 6,751 7,408 11,004 67,615 33,524 13,559 17, ,455 1,920 27,284 8,634 2,223 3,320 8, , SDs 13,337 41,384 13,148 1,676 18,269 11,129 12,327 18, ,50 54,297 22,675 29, ,463 3,469 49,126 15,455 4,171 5,532 14, ,100-2 SDs 1,191 3,640 2, ,529 2,373 2,490 3,752 26,729 12,750 4,443 6, , ,442 1, ,108 3, ,376 Cod Crustac eans, insects

16 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 16 Figure 2. Predicting negative take of prey. Correctly computed 95% CIs using the corrected BRNW/OSU data from the Input Variable Value tab showing that the confidence intervals include a significant negative take for all prey categories and were 2-10x the original estimates. Although correctly computed in this example, a related issue was that proportions or percents, usually below.2 and above.8, need to be transformed to produce normalized values. This additional statistical problem was brought on by parsing data into too many categories, too many missing values in the raw data, and failure to understand basics statistics. This figure would make a humorous PowerPoint slide at professional meetings and reinforces the belief that modeling is useless when input variables are not controlled for quality and basic statistics are ignored. 160, , , ,000 80,000 60,000 40,000 20, ,000-40,000-60,000-80, ,000 cutthro at chum Coho unid Herring, Steelhe Anchov salmoni Sardine, ad y d Shad Smelt Surf Perch, Shiner Perch Unid Nonsal Stickleb ack Sculpin Sandlan ce Gunnel Snake Prickleb ack Other Prickleb ack Prickleb ack/gun Pipefish Rockfish Greenli ng nel Midpoint of CI 6,343 27,388 6, ,270 5,791 5,655 8,886 66,435 30,597 14,048 19, ,730 1,602 24,375 7,794 2,331 3,033 10, ,080 +2SDs 18, ,554 18,336 4,145 23,690 15,998 18,090 26, ,863 65,039 97,932 42, ,363 8,068 57,032 23,286 8,521 9,780 29, ,176-2SDs -5,466-83,778-4,892-2,343-3,151-4,416-6,780-8,826-2,992-3,845-69,836-4, ,903-4,865-8,283-7,698-3,859-3,713-7, ,016 Cod Crustac eans, insects

17 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 17 Figure 3. Percent difference in prey proportions as reported on Input Tab and Input Variable Values tabs vs what was used based on back calculations from Output Biomass by Time Period (by prey type) tab in from BRNW Bioenergetics Model workbook 2012 Tillamook Bay DCCO Salmonid Consumption xlsx. The numerical differences in the proportions are small but because of the extrapolations, predicted numbers can be large. There should be no differences and the inability to back calculate values suggests there is an internal inconsistency in the BRNW/OSU model Period 1&2 Period 3&4

18 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 18 Figure DCCO Tillamook Bay Genetic Samples, protocol for converting binary data to quantitative data and incorrect proportions. Numbers denoted as 6a-9a are incorrect and 6b-9b are corrected. See text for explanation of sequence, 1 is not shown. GWS notes, MTD15 3 samples, same species; TD22 2 samples same species, TD7 2 samples same species aw data Frequency of occurrence (FO) parsed Cou nt 2 Sample # Samp Type Colony Collectio n Date Sample No. NWFSC Species ID Sample # Date cutthroat chum coho unid steelhead 1 TD10a Stomach Tillamook 04/25/ TD10a omykiss 1 04/12/ TD10b Stomach Tillamook 04/25/ TD10b clarki 2 04/12/ TD10c Stomach Tillamook 04/25/ TD10c unid 7 04/18/ TD11 Stomach Tillamook 04/25/ TD11 clarki 10 04/25/ TD12 Stomach Tillamook 04/25/ TD12 clarki 11 04/25/ TD15 Stomach Tillamook 04/26/ TD15 omykiss Frequency of 12 04/25/ TD15b Stomach Tillamook 04/26/ TD15b omykiss occurrence (FO) 15 04/26/ TD15c Stomach Tillamook 04/26/ TD15c omykiss 22 05/07/ TD1a Stomach Tillamook 04/12/ TD1a omykiss 38 05/17/ TD1b Stomach Tillamook 04/12/ TD1b coho 43 05/17/ May n 11 TD22a Stomach Tillamook 05/07/ TD22a chum 45 05/17/ total n 12 TD22b Stomach Tillamook 05/07/ TD22b chum 4 Summed FO TD2a Stomach Tillamook 04/12/ TD2a omykiss 5 Relative FO (pooling all samples) TD2b Stomach Tillamook 04/12/ TD2b coho (GWS note, not used) 15 TD38 Stomach Tillamook 05/17/ TD38 omykiss april samples TD43 Stomach Tillamook 05/17/ TD43 coho 17 TD45 Stomach Tillamook 05/17/ TD45 coho may samples TD7a Stomach Tillamook 04/18/ TD7a omykiss G. Shugart notes 19 TD7b Stomach Tillamook 04/18/ TD7b omykiss assumed biomass (g) original sticky notes, red upper right commets by Don Ly 6a Relative FO weighted by biomass chum.6 gm as entered on the Input Tab, rather than 5 g Proportions used to apportion to overall salmonids mistakenly 7a using 45 gm for Cutthroat and 5 gm for chum, see below check 6b biomass from Input Data tab (g) FO weighted biomass using IVV tab Proportional breakdown by 7b biomass using 40 gm for cutthroat and.6 gm for chum check 3 From the "Model" worksheet Input Data tab, note row 29 above is identical to cols L & N From here they apply these proportions to.616 &.091 salmon in the diet. Period Period (Apr) (May) 1. Compute salmonid vs non-salmonid proportions based on individual stomach proportions (not shown) Diet 2. Genetic Id of some of the salmonid samples, 11 of 14 in Tillamook Bay 2012 fish watch % observed salmonid Over all salmonid proportions from Input Data tab, note.616 is fro 3. Populate afrequency of occurrence (FO) matrix parsed by the number categories found (e.g., if two species % cutthroat cutthroat From Row 26, incorrect relative FO's weighted by biomass or categories were found, each is assigned a FO of 0.5) % chum chum From Row 26, incorrect relative FO's weighted by biomass 4. Sum FO by category % coho Coho From Row 26, incorrect relative FO's weighted by biomass 5. FO expressed as proportions of all summed FOs % unid salmonid unid salmonid From Row 26, incorrect relative FO's weighted by biomass 6. FO proportions were weighted by average weight of fish % steelhead Steelhead From Row 26, incorrect relative FO's weighted by biomass 7. Weighted FO were expressed as proportions of the total weighted FOs Weight FO proportions were used to apportion the overall salmonids to category Prey Prey % of % of Example of salmonid apportioning 9. Use proportions from 8 to apportion Total kj/ energy density (kj/gm) to get biomass, then /average mass (gm) number species diet by diet by cell N35 x cell N36 = cell N 46 8a to get the number of fish in each category. number number fish watch fish watch Prey 1 Salmonidaecutthroat Incorrect values used in the Tillamook Bay 2012 simulation Prey 2 chum Incorrect values used in the Tillamook Bay 2012 simulation Prey 3 Coho Incorrect values used in the Tillamook Bay 2012 simulation Prey 4 unid salmonid Incorrect values used in the Tillamook Bay 2012 simulation Prey 5 Steelhead Incorrect values used in the Tillamook Bay 2012 simulation Corrected values Diet 8b Figure 5. Computed from summary data from Roby et al , see Table 5. Each fish in DCCO foregut samples was extrapolated to the number on the y-axis illustrating the tenuous Period 1-2 (Apr) Period 3-4 (May) fish watch % observed salmonid % cutthroat cutthroat From Row29, corrected values % chum chum From Row29, corrected values % coho Coho From Row29, corrected values % unid salmonid unid salmonid From Row29, corrected values % steelhead Steelhead From Row29, corrected values Prey Prey % of % of number species diet by diet by number number fish watch fish watch Prey 1 Salmonidaecutthroat Corrected proportions Prey 2 chum Corrected proportions Prey 3 Coho Corrected proportions Prey 4 unid salmonid Corrected proportions Prey 5 Steelhead Corrected proportions Number of fish eaten by category Category Corrected Original Lyons' calculations Salmonidae cutthroat 6,245 7,264 chum 3,432 22,512 Coho 7,332 7,989 unid salmonid Steelhead 11,595 11,399 Total 29,517 50,077 9b 9a

19 Original extrapolated number of fish Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 19 nature for original projection. Reverse extrapolations were necessitated in the absence of raw data. 400,000 Reverse extrapolations, extrapolations from a single fish in samples 350, , ,000 sub-yealing Chinook 200, ,000 coho steelhead yearling Chinook sockeye 100,000 50, Year

20 Comments on CENWP-PM-E / Double-crested cormorant draft EIS, Shugart, 20 Appendix A. Excerpt from: Using unverifiable and simulated data to managing piscivores: An example from Tillamook Bay 2012 with general comments on the Bird Research NW Bioenergetics model and management of Columbia River piscivores. (draft as of 19 July 2014 latest) Summary: Foregut sampling of Double-crested Cormorants (Phalacrocorax auritus) was done in Tillamook Bay, Oregon in April-May 2012 to estimate the consumption of salmonids. Consumption was simulated using Bird Research Northwest s Bioenergetic Model (BRNWM) The model generated confidence intervals of consumption and used midpoints of the intervals as estimates of consumption. Findings were that cormorants consumed about 50,000 salmonids, which represented midpoints of relatively huge putative 95% confidence intervals. In reviewing the input and output from the model, I found it was deficient in many respects. Most importantly the take of salmonids was overestimated by 40% due to a mistake in apportioning salmonids. In the addition, the simulated numbers of salmonid consumed was based on an estimated 29.7 salmonids found in 11 of 45 stomachs over a two-month period that were then used to extrapolate to a total of ~50,000 (corrected to ~29,000) salmonids over four seemingly independent two-week periods. Many of the inputs were simulated due to inadequate sampling or in order to minimize confidence intervals. The standard deviations, and resulting standard errors, used in the simulations were critical inputs for generating the confidence intervals, but were extremely conservative or incorrectly computed. If done correctly, standard errors were greater than the mean values in many cases and confidence intervals contain negative values. Data used to calculate proportional take of salmonid prey were incorrectly computed as noted, but in addition, the resulting values were enigmatically pooled into one two-month period for individual salmonids, two one month periods for other prey and overall salmonid proportions, then calculations were done as if these were independent data for four two-week periods. The salmonid consumption data were treated in an idiosyncratic manner in attempt to convert binary data from genetic id to quantitative data for salmonid categories. Finally there appears to be an inconsistency or a bug in the model calculations based on failure of internal checks. The BRNWM has been used extensively to guide management of piscivores in the Columbia River Estuary. Based on this overview and assuming the model as used for Tillamook Bay 2012 was representative, the entire effort needs review. Such a review should first include proofing for internal consistency of the code used to generate consumption numbers. Secondly, the assumptions regarding the input values, erroneously in many instances referred to as means, need to be clearly stated, and corrected SEs need to be incorporated in reruns of the simulations. In addition the code and input and output data should be published as appendices or workbooks such as that provided to ODFW for Tillamook Bay Finally the raw sample data should also be published and place in the public domain. These steps should be sufficient to allow a full review. In general, the Tillamook Bay 2012 calculations indicate that the BRNWM used to manage piscivores lacks statistical and scientific rigor.