1 Enhanced Predictability During Extreme Winter Flow Regimes Ryan NEnhanced Predictability During Extreme Winter Flow Regimes Ryan N. Maue (WeatherBELL Analytics - Atlanta) ECMWF UEF | 2016 Reading, UK June 6 – 9, 2016

2 Where does forecast verification occur today?Internal teams within NWP centers Media: TV broadcasters, print, web Social Media, Twitter / Facebook Who is doing model forecast verification? Trained forecasters, gov’t & commercial Your neighbor High impact events

3

4 https://www. washingtonposthttps://www.washingtonpost.com/news/capital-weather-gang/wp/2015/10/06/what-the-european-model-win-over-the-american-model-means-for-weather-forecasting/ October 6, 2015

5 January 2016

6 March 10, r2 upgrade cycle. Octahedral grid … 3600x1800x137L = 888,253,200 cells 3600x 1801y 137L = 888M

7

8 May 11, 2016

9 April 6, 2016

10 Comparison of Operational SuitesHRES Global model 0-10 days (2x) EPS Ensembles days, 46 days Seasonal 7-months (1 per mon) GFS global model days (4x daily) GEFS ensembles 0–16 days (4x daily) NAM (12-km) 0-84 hours (4x), NAM-RR NAM 4-km Nest 0-60 hours (4x) RAP 12-km (24x) HRRR 3-km (24x), HRRR-E CFSv2 climate forecast days (4x) HIRESW 4-km WRF-windows (2x) SREF ensemble 16-km (4x) RTMA/URMA analysis (24x) WW3 Wave model (4x) RTOFS global model (1x) NDFD 2.5 km (48x) National Water Model, Aerosols HWRF hurricane model (4x) WPC, SPC, NWS, NSIDC, OPC, Space … Mission creep

11 Research Objectives Framework & Goals DefinitionsUse current NWP deterministic & ensemble systems to analyze large-scale flow patterns and relate to medium-range forecast skill “dropouts” Diagnose causes of low-predictability flow regimes including dropouts: inadequate observations, large analysis uncertainty, and/or model error growth Link to skill of Teleconnection Indices such as AO/NAO, EPO, WPO and PNA Forecast skill metrics: 5 or n-day 500 hPa geopotential anomaly correlation (NH: 20°-80°N), Forecast dropout: an individual or collection of several consecutive forecasts that have significantly lower 500 hPa geopotential anomaly correlation skill – compared to monthly/seasonal mean (AC < 0.8) Low-predictability regime: particular hemispheric-scale configuration of upper-level flow that leads to below average forecast skill

12 Anomaly Correlation: Definition500-hPa geopotential height Northern Hemisphere 20°-80° N Forecast anomaly from climatology* at each grid point (m) Analysis anomaly The AC is common forecast skill metric used by operational centers Forecasts with AC > 0.6 are considered as providing potential positive skill Not perfect metric, but used in concert w/ e.g. mean squared error * ERA-Interim (reanalysis) climatology

13 Archive of Analysis and Forecast FieldsHistorical record of analysis and forecasts from current NWP deterministic and ensemble systems Key is archive and/or “real-time” access to forecast products Valuable resource includes the EPS Reforecasts / Hindcasts or any frozen forecast model for post-processing purposes e.g. EFI, M-Climate MODEL TIME PERIOD GRID/FIELDS SOURCE ECMWF HRES & EPS Oct 2006-present T799-T1279L137 Tco1279L137 ~9/18 km WeatherBELL ECMWF MARS NCEP GFS GEFS Feb 2004-present T382-T574L64 T1534L64 0.25°- 0.5° -1° NOMADS/NCDC NAVY NAVGEM Jan 2004-present T239-T319-T359 0.5° NRL MONTEREY *Forecasts verified against own analysis

14 Anomaly Correlation: Forecast skillECMWF GFS Seasonal AC scores are highly correlated NH Winter skill >> Summer Skill. Both models showed record-high skill during winter

15 Anomaly Correlation: Forecast skillECMWF Summer skill improvements GFS Sustained improvement skill jumps mainly due to major model configuration changes: Jan 26, 2010: ECMWF T799  T March 2016: 41r2 July 28, 2010: GFS T382  T574 and physics major upgrade  T1534  4D-Hybrid EnVar May 11, 2016

16 Anomaly Correlation: Forecast skill5-DAY NH Geopotential Height ECMWF vs GFS calendar year “Model Wars” in U.S.A. media YEAR GFS ECMWF 2009 0.852 0.888 2010 0.872 0.904 2011 0.862 0.898 2012 0.871 0.900 2013 0.880 2014 0.875 0.905 2015 0.885 0.910 2016 YTD 0.894 12m 0.886 YTD 0.926 12m 0.915 NCEP May Upgrade: 4D-Var Hybrid EnKF GFS gains have “leveled out” Cycle 42r1 significant Improvement “Gap widened” by 20%  0.03 On pace for 0.92

17 different models “dropout” on the same casesPredictive Skill Regimes From N Hemi AC, ECMWF & GFS: Centered means of 45-days and 7-days are calculated The 7-day minus 45-day mean represents a seasonally adjusted measure of skill GFS ECMWF Above-average skill Below-average skill Anom Corr / predictive skill is strongly dependent on atmospheric flow regime: different models “dropout” on the same cases Models tend to have low and high skill on the same forecast(s)- but ECMWF skill drops off less than GFS

18 Anomaly Correlation: GFS recent skillPeriods of significant skill dropout & success Jan March June 2016

19 Anomaly Correlation: HRES ECMWF 5-dayNov Dec Jan 2013

20 Anomaly Correlation: ECMWF 6-dayCompare to 5-day: Jan Mar Jun 2016

21 Anomaly Correlation: ECMWF 10d 2013-2014Nov Dec Jan 2014

22 Anomaly Correlation: ECMWF 10d 2015-2016Oct Jan Jun 2016

23 Example: Arctic OscillationAO is the first EOF of sea-level pressure (1000 hPa geopotential height) variations north of 20°N latitude How do NWP systems perform during the + and – phase of the AO, as well as through transitions [e.g. Archambault et al. 2010] – during the Northern Hemisphere cold season? POSITIVE PHASE NEGATIVE PHASE J. Wallace, University of Washington

24 Arctic Oscillation Winter 2009 – 2010 Sustained Negative AO

25 November – January Negative AO, followed by strong PositiveArctic Oscillation Winter 2010 – 2011 November – January Negative AO, followed by strong Positive forecast Major Transitions

26 AC AO Arctic Oscillation ECMWF – Winter 2009 – 2010500 hPa NH Anomaly Correlation Arctic Oscillation Index AC AO Transitions Dec 2009 Feb 2010

27 AC AO Arctic Oscillation ECMWF – Winter 2010 – 2011500 hPa NH Anomaly Correlation Arctic Oscillation Index ECMWF – Winter 2010 – 2011 500 hPa NH Anomaly Correlation Arctic Oscillation Index AC AO Lower Skill Very High Skill Lower Skill Dec 2010 Feb 2011

28 Visualizing Arctic Oscillation Forecast IndexJanuary 8, 2016

29 Bubbles Visualizing Arctic Oscillation Forecast IndexAnalyzed AO, 10-day forecast bubbles INIT: January 8, z HRES 41r2

30 Box & Whisker Visualizing Arctic Oscillation Forecast IndexForecast AO, 15-day EPS forecast, ensemble mean & distribution + control

31 Visualizing Arctic Oscillation Forecast Index2015 2016

32 Visualizing PNA Forecast Index

33 Summary and Future DirectionsA decade archive of high-resolution operational center deterministic forecasts has been developed to study “dropouts” in medium-range forecast skill The models tend to “dropout” during the same forecasts, in “lower-predictability” flow regimes The Arctic Oscillation (AO) index is (to some extent) anti-correlated with medium-range forecast skill, as measured by the 5-day anomaly correlation of 500mb height Value of frozen model w/many years of data (~ 20) to evaluate model performance during particular large-scale flow regimes Value of multi-model & ensemble forecast-forecast correlations for medium-range extreme events

34 Acknowledgements WeatherBELL Analytics [2012-present]Data: ECMWF, NCEP/NOMADS, NRL-Monterey NRC Postdoc at NRL Monterey [ ] advisor: Dr. Rolf H. Langland Updated research work based on Langland and Maue (Tellus A, 2012): Recent Northern Hemisphere mid-latitude medium-range deterministic forecast skill Appreciation and Thanks to Dr. Ghelli and ECMWF!

35 WeatherBELL Models