A New Design for the Access Point Angler Intercept Survey 2010 North Carolina Pilot Study Office of Science and Technology Dave Van Voorhees Co-authors: Jay Breidt, James Chromy, Kelly Fitzpatrick, Han-Lin Lai, Terri Menzel, Douglas Mumford, Breda Munoz, Jean Opsomer, Ronald Salz, Kevin Sullivan, Chris Wilson, Patricia Zielinski MRIP Calibration Workshop #2 Charleston, SC September 8-10, 2014
Complemented Surveys Approach Atlantic Coast & Gulf of Mexico Coastal Household Telephone Survey Access Point Angler Intercept Survey Coastal Resident Fishing Trips Adj. Ratio Accounting for Non-Coastal Resident Trips Shore and Private Boat Fishing Modes X Total Angler Trips X Mean Catch per Angler Trip Total Catch
Complemented Surveys Approach Atlantic Coast & Gulf of Mexico For-Hire Telephone Survey Access Point Angler Intercept Survey Charter Boat Fishing Mode Angler Trips on Listed Boats X Adj. Ratio Accounting for Trips on Unlisted Boats Total Angler Trips X Mean Catch per Angler Trip Total Catch
What is an Access Point Survey? On-site survey to collect catch data Sampling of completed angler fishing trips Spatiotemporal sampling frame Matrix of fishing access sites and time intervals Multi-stage cluster sampling
Multi-Stage Cluster Sampling of Private Boat Angler Fishing Trips Fish #1 Site-Time #4 Boat #5-1 Angler #5-1-1 Angler #5-1-2 Fish #2 Fish #3 Angler #5-1-3 Fish #4 Site-Time #5 Boat #5-2 Angler #5-2-1 Angler #5-2-1 Fish #5 Fish #6 Angler #5-3-1 Fish #7 Site-Time #6 Boat #5-3 Angler #5-3-2 Angler #5-3-3 Stage 1 (PSUs) Stage 2 (SSUs) Stage 3 (TSUs)
Why a New Design for the APAIS? 2006 National Research Council Review: Determine sample inclusion probabilities and use them to weight trip data in estimation Develop more formalized probability sampling protocols with known inclusion probabilities for all PSUs Expand temporal coverage
New Weighted Estimation Method Breidt, et al. (2011) weighted estimation method Estimated inclusion probabilities for all sampled trips Mix of straight design-based weights and modeled design weights Accounts for both stratum weights and stage weights 2012 re-estimation of Atlantic and Gulf of Mexico catch statistics for 2004-2011 Calibration based on 2004-2011 comparisons used to revise statistics for earlier years
Why a New Sample Design? Breidt, et al. (2011) Need to change sampling focus: Too much attention on maximizing number of angler trips intercepted Focus should be on maximizing number of site-days sampled Need to spread out sampling: Emphasis on maximizing interviews per on-site hour and caps on number of interviews per site visit compressed samples of trips Better to spread out interviews obtained within each site-day assignment Need to eliminate alternate sites: Visits allowed to sites not pre-determined in probability sampling design Creates unnecessary difficulty in development of appropriate weights for intercepts collected at alternate sites
Why a New Sample Design? Breidt, et al. (2011) Need to get accurate counts of all completed trips on site Maybe too much emphasis on maximizing interviews Equally important to know size of cluster of trips within a sampled site-day Sampling fraction needed for inclusion probabilities used in weighting Should consider approach to cover trips throughout the day Focus on visiting sites during peak activity period of the day Nighttime and off-peak daytime trips generally not sampled and assumed to be similar to peak period trips
New Sampling Design Project Team started in 2009 2010 North Carolina pilot study: Conducted side-by-side with old design Final Report (Breidt, et al., 2012): Recommended coast-wide implementation Recommended possible further enhancements Independent peer reviews endorsed implementation
What s Different in the New Design? Maximize number of site-days observed Not the number of angler interviews! Precision of multi-stage survey estimators depends almost exclusively on number of primary sampling units (site-days) observed Improved sample frame: Spatial component consists of single sites and multi-site clusters Less active sites combined into 3-site or 2-site clusters More active sites stand alone Increased temporal stratification 6-hour time blocks Better temporal coverage Increased geographic stratification some state subregions Better geographic representation Easier staffing
What s Different in the New Design? Fully formalized probability sampling: Probability-proportional-to-size sampling of site-time units (PSUs) Size = expected fishing activity Inclusion probabilities for all PSUs known Attempt to intercept all completed angler trips on site Accurate counting of all trips within sampled site/time unit Sampling fractions at each stage must be known Important for proper weighting of data
What s Different in the New Design? Emphasis on completing all site-time assignments Controlled selection advocated (later developed by John Foster) Draws thousands of possible sets of assignments Eliminates sets that don t match staffing constraints Selects one of remaining sets at random No canceling or re-scheduling of assignments Samplers cannot decide when/where to conduct interviews Fixed time interval for each site assignment Fixed order of sites for multi-site assignments No limit on number of interviews per assignment
Pilot Study: Enhanced Temporal Coverage 40% Private Boat Angler Trips 40% Shore Angler Trips Man Made Structures 30% 30% 20% 20% 10% 10% 0% 1 3 5 7 9 11 13 15 17 19 21 23 1-hr Block 0% 1 3 5 7 9 11 13 15 17 19 21 23 1-hr Block New 2010 Old 2010 40% Charter Boat Angler Trips 40% Shore Angler Trips Natural Shorelines 30% 30% 20% 20% 10% 10% 0% 1 3 5 7 9 11 13 15 17 19 21 23 1-hr Block 0% 1 3 5 7 9 11 13 15 17 19 21 23 1-hr Block
Pilot Study: Reduced Efficiency? 100% Private Boat Assignments 100% Man-Made Shore Structure Assignments 80% 80% Success Rate 60% 40% 20% Old Design New Design Success Rate 60% 40% 20% Old Design New Design 0% 0% 100% Charter Boat Assignments 100% Natural Shoreline Assignments 80% 80% Success Rate 60% 40% 20% Old Design New Design Success Rate 60% 40% 20% Old Design New Design 0% 0%
Pilot Study: Estimation Differences? 2,000,000 1,500,000 Retained Catch 1,000,000 500,000 Old Design New Design 0 4,000,000 3,000,000 BLUEFISH SPANISH MACKEREL DOLPHIN Total Catch SOUTHERN FLOUNDER RED DRUM 2,000,000 1,000,000 Old Design New Design 0 BLUEFISH SPANISH MACKEREL DOLPHIN SOUTHERN FLOUNDER RED DRUM Bars indicate 95% confidence intervals
Adequate Statistical Precision? Pilot study: New design less productive than Old Fewer angler trip interviews per assignment Higher proportion of assignments with no interviews Sampling efficiency not optimized in Pilot Simulation studies: New design could provide more statistical precision With equal sample sizes (number of PSUs) With optimized sampling allocations among strata Precision could be further improved: Adjustments to PSU selection probabilities in PPS sampling
Conclusions New design greatly reduces potential for bias in many ways Feasible to sample nighttime and off-peak daytime fishing trips as needed to achieve full temporal coverage Performance of new design can be optimized to provide: Higher levels of productivity Higher levels of statistical precision
Thank you! Consultants: F. Jay Breidt, Colorado State University Jean D. Opsomer, Colorado State University James R. Chromy, RTI International Breda Muñoz, RTI International
References Breidt, F.J., Lai, H.L., Opsomer, J.D., Van Voorhees, D.A. 2011. A Report of the MRIP Sampling and Estimation Project: Improved Estimation Methods for the Access Point Angler Intercept Survey Component of the Marine Recreational Fishery Statistics Survey. 83 pp. https://www.st.nmfs.noaa.gov/mdms/public/finalreport.jsp?reportid=353 Breidt, F.J., Chromy, J.R., Fitzpatrick, K.E., Lai, H.L., Menzel, T., Mumford, D.G., Muñoz, B., Opsomer, J.D., Salz, R.J., Sullivan, K.M., Van Voorhees, D.A., Wilson, C., Zielinski, P.A. 2012. A Pilot Study of a New Sampling Design for the Access Point Angler Intercept Survey. 77 pp. https://www.st.nmfs.noaa.gov/mdms/public/finalreport.jsp?reportid=672 National Research Council, Committee on the Review of Recreational Fisheries Survey Methods. 2006. Review of Recreational Fisheries Survey Methods. 202 pp. http://www.nap.edu/catalog/11616.html