Your thoughts on conceptual models - Session Forum

One of the biggest problems in the meteorological world when it comes to 3D mental models is actually Numerical Weather Prediction (NWP) output.  Although it can be very useful to have NWP output for times beyond 24 hours, forecasters have the ability to out-forecast NWP in areas where the NWP and observations have a mismatch (in the first 24 hours).  However, because we have so much data to slice and dice with NWP, we often just want to believe the NWP because it offers us a more "elegant" solution than our own (when it is under-developed).  

In the past two weeks, we convinced forecasters to believe in their own predictions of the atmospheric evolution through their 3D mental model.  The best part (as facilitator) was when the forecasters had to try to actually build a 3D mental model (playdoh model in our case) out of NWP data.  They came to realize that they didn't get extra useful information from looking at the NWP.  In fact, sometimes they lacked information like they had from the observations.  It was a struggle for them to build a 3D mental model from the NWP in the end.  Multiple forecasters mentioned how hard it was to use NWP to make a 3D mental model even though they thought it was going to be easy with all the extra data.

An easy exercise aside from making forecasters build a 3D mental model out of NWP data is to layer a bunch of standard NWP output into a 3D grid, then ask forecasters to point to locations where they don't have any information.  Holes in the analysis process become evident whether it is from observations or from NWP.  Both have their holes when forecasters don't build it all out mentally.  

We use NWP as such a crutch (in a lot of instances) that we tend to forget its physical/dynamical meaning.  We can get away with far less NWP output than we think we need in a lot of instances.  We love the forecast funnel (for good reason), but some use it inappropriately by just thinking in isobaric surfaces from the top of the troposphere down.  It is truly a scale issue (much like mental models).  Use more appropriate output and your 3D mental model grows by leaps and bounds with far fewer maps.  Robust 3D mental models often trump NWP in the first 24 hours of a forecast. 

On a personal note, we often mention the oncoming "fire hose" of data that the new GOES-R satellite will be bringing to the western hemisphere.  I find this issue to be different than some believe it to be.  We should be working on using the basic channels better and only using the advanced products when they are necessary.  We tend to jump to the advanced products right away (similar to NWP) rather than look at the more "basic" products.  I don't find the advanced products to be useless, I just find we leap over the basics that help us make mental models before getting to the advanced products.  We could use the basic products to find areas where we need more information from different sensors or different NWP fields or other observations.  All of these products can work together to make the best possible answers.

Another thing forecasters (and I) have a tendency to forget (or lack time to do) is to make a forecast based on the observations before moving into the NWP realm.  If you have a guess (3D mental model) when you go into the NWP output, you can test your hypothesis.  If you go into the NWP without a hypothesis, then you can only pull up products and search for meaning.  It all "looks right" though because you have nothing to rate it against.  Without a 3D mental model, you have nothing but the NWP to work with.  For this reason, in labs during our courses, we always make the forecasters build their 3D mental model and make a forecast with it before letting them look at the NWP.  This past week, I had a forecaster say to me, "I should have believed my mental model and not gone with the NWP.  It lead me astray."  This was validation for the methods we are employing.

If in the end we can enable forecasters to trust their own perception, we are doing better than ever before.  Confidence without overconfidence is one of the keys to making forecasts outside of the NWP envelope.  I feel we have taught the students in our courses some form of confidence to help them make those predictions in the face of NWP.  Without 3D mental models and truly defining them with some visible shapes (playdoh), we wouldn't have ever noticed the gaps.  

Bryan

Great question Lara!

I don't have a good answer, just a few thoughts.  As more and more forecasters rely on NWP to create their short-term forecasts, their analysis, diagnosis and prognosis skills atrophy.  We are approaching a situation in which many forecasters have become so dependent on NWP that when NWP is wrong, their forecast is wrong (or vice versa).

If this becomes prevalent, decision-makers will have little choice but to dismiss the forecasters.  If they are only as good as NWP, why are forecasters needed?  No matter how much one argues that it is important to know the models, their biases, etc., if the forecasters are correct only when NWP is correct, forecasters could be working themselves out of a job.

So strengthening forecasters' abilities to create and adapt 3D mental models based on real-life observations may be a good way to help them question NWP forecasts and improve on them.  In that situation, NWP will be one tool among many, not the primary tool (crutch as Bryan called it).