1 Briefing from the Science Advisory Committee: Scientific and Technical Review of the Survey Design and Methods used by California Department of Fish and Wildlife to estimate red abalone (Haliotis rufescens) density May 20, :00-4:00 PM (PST) Coordinated by Moose: Introduce myself Microphones/Lines will be muted until comment period Webinar is being recorded Explain public comment instructions – / chat feature in WEBEX To ask questions of the SAC CDFW are invited to listen in, but are not active participants of this briefing *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

2 Meeting Agenda 3:00-3:05 Introduction: Review Scope and ProcessMoose O’Donnell, Senior Scientist, Ocean Science Trust 3:05-3:35 Presentation of Review Outcomes Mark Carr, Chair, Science Advisory Committee 3:35-3:55 Public Comment Period 3:55-4:00 Wrap-Up and Next Steps *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

3 Science Advisory CommitteeDr. Mark Carr (Chair) University of CA, Santa Cruz Dr. Jeremy Prince Murdoch University, Australia Dr. Brian Tissot Humboldt State University Moose: Introduce SAC and identify members who are on call Explain SAC nomination process Constituent nominations Minimum qualifications Names / CVs made available online Dr. Pete Raimondi University of CA, Santa Cruz Dr. Karina Nielsen Sonoma State University Dr. Steve Schroeter University of CA, Santa Barbara *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

4 Introduction Initial Request from CDFW Role of Ocean Science TrustIndependent facilitator Design & maintain process Funded by the Ocean Protection Council Moose: CDFW demanded candid, independent process “We want to be sure we’re getting it right” OST advance a constructive role for science in management OST core principle need to have this authoritative and legitimate in eyes of managers and concerned constituents Allow the SAC to be candid *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

5 Review Scope Scientific and technical review of:survey design, including strengths and weaknesses of current methods for estimating red abalone density; the application of existing methods, including analysis of existing data, and interpretation of results; and uncertainty associated with existing methods for estimating red abalone density in northern California and its adequacy for informing catch limits and other management controls of the recreational red abalone fishery in northern California, as outlined by the ARMP. Before process ever began, we scoped with DFW This remained guiding principle throughout *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

6 Review Process 2013 2014 Jul Aug Sept Oct Nov Dec Jan Feb Mar Apr MayJune SAC data analyses, Develop recommendations SAC Selection, Develop scope & process Technical Workshop Public Webinar Briefing Final Report Public Webinar Data requests from CDFW OST Present to Marine Resources Committee Constituent engagement throughout – newsletters, public webinars, nominations CDFW contributions – data, background documents, staff time *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

7 Meeting Agenda 3:00-3:10 Introduction: Review Scope and ProcessMoose O’Donnell, Senior Scientist, Ocean Science Trust 3:10-3:40 Presentation of Review Outcomes Mark Carr, Chair, Science Advisory Committee 3:40-3:55 Public Comment Period 3:55-4:00 Wrap-Up and Next Steps Moose: Now Mark, chair of the SAC, will briefly summarize: Management triggers (from ARMP) CDFW methods and analyses as presented to the SAC (background doc, presentations) SAC recommended analyses SAC long term recs *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

8 Presentation of Review OutcomesDr. Mark Carr (Chair) University of CA, Santa Cruz Mark: Introduce yourself briefly, role as chair

9 Recap: previous webinar (9-16-13)…DFW summarized/presented for us their sampling design, protocols, analyses, what the analyses focused on: 1) time series of changes in ab densities in index sites 2) statistical power to detect change in time series Provided us with a written report, data (design, protocol, analyses – change in time, statistical power to detect change) Do a brief summary – 1) Cannot assume people were on the first call. Don’t know what DFW presented to the SAC – add a slide that summarizes the materials presented to SAC by DFW, recap Not criticize what they presented, gave the SAC a feel for how they approach the analyses and their ability to detect changes through time. Ability to detect changes through time is critical to forecast and detect trends in the stock over time w/ confidence. The SAC gave more emphasis to detecting where you are in relation to management thresholds. DFW did not convey that to the SAC. Pull slides out of their presentations (screen caps) – BUT NOT TECHNICAL WORKSHOP SLIDES, webinar slides are OK to use publicly

10 Recap: CDFW Density Estimation MethodSpatial and temporal sampling design and protocols as specified in ARMP: Dive surveys 8 index sites, 4 depths, 9 transects per depth Samples located randomly based on GPS coordinates Avoid 50% sand 3 years to complete survey cycle Mark: Not my intention to give their whole protocol… very quick and succinct Know you’ve heard it all before Walking through this for context to understand recommendations I’ll hit later CDFG

11 Recap: CDFW Analyses Change in time and statistical power to detect that change Average density from all transects compared to ARMP triggers; if average density falls below a trigger (25% decline), management changes may be recommended Use of ANOVA to determine if average densities differ significantly between time periods, across 8 index sites/ county / site Mark: CDFW compare average densities to triggers, a “yes or no” if above or below ANOVA and power analysis conducted to assess changes from previous sampling ‘periods’ (data aggregated by survey period) Similar analyses conducted for site and county

12 ARMP Management SummaryARMP Table 7-2. Fishery management triggers. Mark: Important component of SCOPE Walk through management triggers at fishery level Also site triggers, no county level triggers but management actions have been taken at co. level “In our review of their report and ARMP provided to us, actually mgmt decisions are being made based on these threshold densities being estimated in their 8 index sites. You can see that depending on whether they look at recruitment or densities in deep/refuge, or across all depths, they use density est. relative to these threshold levels.” Highlight the second row down Ability of sampling protocol to inform these decision actions SAC focused on how density levels compare with these thresholds

13 Threshold-based AnalysesThe ARMP does not specify anything more than a comparison of a number against a threshold Comparing average density to management trigger ignores variability in the data Cumulative Probability Functions (CPFs) are most appropriate analysis to interpret a density estimate relative to a threshold Analysis of Variance of time series is particularly useful for characterizing the trend in the index These analyses require certain assumptions (e.g., normality, equal variances) not met with transects as replicates. Using sites as replicates, analyses performed by the SAC met the required assumptions. Transects could be analyzed with other approaches (e.g., resampling) . Mark: ANOVA requires set parameters, including alpha to avoid false interpretation... those are often lost when thinking of result significant or not Lots of embedded assumptions… not clear that they are all met Can use power analysis to determine magnitude, but unit of replication matters ***CAVEAT to bring: While we present recommendations on the following slides for strengthening the current survey design and analytical methods, we do not intend this as an endorsement.

14 Threshold-based AnalysesIndex site design does not give you information outside of the 8 index sites, even though the management decisions pertain to the entire region To make inferences beyond the 8 index sites Assume that the 8 sites are representative of the entire fishery Sites are the necessary unit of replication Mark: ANOVA requires set parameters, including alpha to avoid false interpretation... those are often lost when thinking of result significant or not Lots of embedded assumptions… not clear that they are all met Can use power analysis to determine magnitude, but unit of replication matters ***CAVEAT to bring: While we present recommendations on the following slides for strengthening the current survey design and analytical methods, we do not intend this as an endorsement.

15 Measured abalone density per hectareRecommended Analysis: Cumulative Probability Functions Shape and position of curve based on sampling design and threshold Curves can be generated for different spatial scales of management (site, county, fishery) Explicit about the likelihood that actual index population is above or below threshold Transparent uncertainty Number of sites 8 12 16 Prob. that actual density of abalone is less than 5000 per hectare Mark: Explicitly lays out how confident you are in a result Run through how to read these figures To be 95% confident, you’d need a population down below 4000 Managers can make decisions fully understanding risks… can be more precautionary How to change shape of curve Use cumulative probability distribution functions to show the likelihood of certain outcomes Although there is no county trigger, same analysis can applied at county level Concerned that applying too strict a standard for statistical confidence may unwittingly place the abalone population at high risk Looking at probabilities directly allows the FGC to assess exactly how much risk is involved. 1) This cumulative probablity function indicates the probablity that the actual density id less than 5000 (vertical axis) for a measured density (horizontal axis) 2) the SHAPE of the curve is based on the actual sampling and estimates generated in 2009, using the 8 index sites as the unit of replication. 3) different samples will generate a different curve... for example, basing the estimates on different numbers of sites (8, 12, or 16) changes the shape of the curve. 4) the curve indicates that for an estamited density of abalone of 3,700, there is a 95% probability that the true desnity is less than 5000. 5) In contrast, for an estimate of 4,500, there is only an 80 percent likelihood that the true density is less than 5,000. 6) The closer the density estimate is to 5,000, the lower the probability that the true density is less than 5,000 (makes sense!). 7) Likewise... an estimate of 6,000 abs per hectare equates to only a 10% likelihood that the actual density is less than 5,000. 7) Notice how you increase that probability (increase your confidence that the actual desnity is less than 5000, by increasing the number of sites you use as replicates). An estimate of 4,500 produces an 80% likelihood the true value is less than 5000, whereas that estimate based on 16 sites give you a 90% likelihood the true value is less than 5,000. Measured abalone density per hectare based on sites as replicates (data from 2009)

16 Recommended Analysis: Means with Associated Confidence IntervalsAllows identification of temporal trends Trends are only useful if sampling is conducted at an appropriate frequency; need higher temporal resolution to make more timely management decisions Mark: Walk through how to read/use these figures To use CIs, the FGC should make an a priori decision about what level of confidence they require to decide that no management action is necessary. For instance, if the decide that they require 95% confidence that they have not crossed a threshold value, then they should examine figures with that confidence interval plotted along with the relevant management trigger In the event that the appropriate confidence interval encompasses the trigger, regardless of the mean, that indicates caution is required. The next step should be to generate a CPF to explicitly explore the likelihood that a trigger density has been met.

17 Summary of SAC Recommended AnalysesGenerate CPFs for threshold assessment: Make very clear your confidence in a density estimate relative to a management threshold Requires managers to specify an acceptable level of risk Time series with CIs are useful for characterizing trends in the index sites: Forecast the direction and future state of the index population Environmental information (climate, habitat) can help explain the state of index populations Enhanced with more frequent sampling Managers can be proactive rather than reactive to the state of the stock – change fishing mortality in a more timely manner if you know direction stock is heading Population just halved from previous sampling period – an el nino for example. Helps you better understand how populations respond Habitat data that is not used…useful for understanding dynamics, but not for statistical analyses

18 Considerations for Informing Future ManagementUsing Existing Density Metric… More rapid tracking of the resource Use habitat data to explain and forecast dynamics of the index populations Codify appropriate analysis in the ARMP Revisit sustainable fishery density (e.g., to take into account years with more data, or more biologically significant) Modify the current density survey design for a more powerful and efficient approach (e.g., negate deep transects, weight sampling by habitat suitability) Moving Beyond a Density Metric… Transition to tracking the condition of population (e.g., size structure) Exploring alternative scientifically based management reference points (e.g., Spawning Potential Ratio) General Additional collaboration with external scientific experts Data should be publicly available in a timely manner Mark: We identified potential improvements that go beyond the statistical and sampling questions. We recognize this is outside scope, but hope that this may inform future discussions as the CDFW and the FGC move towards long-term management of red abalone. In some cases, require changes to current language of the ARMP, Other suggestions may provide guidance for processes of re-shaping the ARMP or creating a separate Fisheries Management Plan (FMP) for managing the fishery going forward Because many of these suggestions were outside of the scope of this review, we have not carefully tested or scoped each of them Example: Federal fishery managers (NMFS) – does their analyses but present to a statistical committee, but not limit to REVIEW

19 Meeting Agenda 3:00-3:10 Introduction: Review Scope and ProcessMoose O’Donnell, Senior Scientist, Ocean Science Trust 3:10-3:40 Presentation of Review Outcomes Mark Carr, Chair, Science Advisory Committee 3:40-3:55 Public Comment Period 3:55-4:00 Wrap-Up and Next Steps *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

20 Public Comment Participation Instructions:Each individual will be allowed up to 3 minutes to provide comments or ask questions of the SAC (OST will give warning when you have 30 seconds left) Questions within the scope may be addressed by the SAC The SAC will consider your input as they decide on the next steps of this review process, and your comments will be captured in the meeting summary If we run out of time and don’t get to your name, we will provide you with an opportunity to submit your question/comment after the meeting via Questions and/or comments such as management of the abalone fishery are not within the scope of this review *For the public: OST will keep a running queue of public commenter names and call each name in the order received. To enter the queue, please submit your first and last name at any time during the meeting by either: ing OST at: OR Sending OST your name using the “chat” feature on WebEx

21 Meeting Agenda 3:00-3:10 Introduction: Review Scope and ProcessMoose O’Donnell, Senior Scientist, Ocean Science Trust 3:10-3:40 Presentation of Review Outcomes Mark Carr, Chair, Science Advisory Committee 3:40-3:55 Public Comment Period 3:55-4:00 Wrap-Up and Next Steps Moose: Thank SAC and public for their continued engagement in the process and supporting science informed decision making (something to that effect) OST will release meeting summary, final draft on our website (www.calost.org) and in a follow up public newsletter in the next month