02 Is training the answer?

02 Is training the answer?

par Ian Bell,
Nombre de réponses : 7

Hi All,

When our staff have a performance gap we may be asked to provide training. But sometimes we should say, "No". This session explores the situations when traing is the solution and when it is not.

Once again there is a narrated PowerPoint in .pptx format. There is also a 1 page summary as a Word document.

I hope I have the correct files in the correct location this time. (Thanks Vesa. I have a performance gap!)

Please leave any comments in the forum.

Tomorrow will be 28 C here so I'm going to the Ausytralian bush to find and photograph wildflowers. I'll be back the day after with criteria for a good competency.

cheers

Ian

En réponse à Ian Bell

Re: 02 Is training the answer?

par Erik Hagemark,

Hi Ian,

Thanks for this. My initial reaction is that the topic of this presentation is equally important as the earlier ones you have provided to us, and so easy to forget. 

Last year during a training course, on forecasting convective precipitation, there was a comment that to be able to perform better the models (NWP) need to be better (eg higher resolution). I think this view is quite common, since NWP is an important tool forecasters use.Now, I assume this would probably fit under "systems gap", and would not be considered as a training issue? However, up until now, much of our training is on how to use NWP. So, I guess we can't avoid the systems gap, since this is integrated in the way they do their tasks.

Thanks,

Erik

En réponse à Erik Hagemark

Re: 02 Is training the answer?

par Ian Bell,

Hi Erik,

Yes, you are right. We can't always avoid these things. It helps if we are aware of them. NWP models are a moving target. Improving the forecast based on them may have to wait for model improvements and is not something trainers can solve.

I can't remember how many times frecasters said that they wanted us to teach them which model to use on a particular day and how to tell whether it was going to be correct or not. I often had to say that we couldn't teach them this because nobody knew. It was a knowledge gap but not one we could help with because of a "research gap".

Given our limited training resources we need to concentrate on things where we can add value.

Thanks for your comments

Ian

En réponse à Ian Bell

Re: 02 Is training the answer?

par Patrick Parrish,

Hi Ian and everyone,

Many people think that the use of NWP is also a training gap, so I am curious how others think about this difficult decision about how and what to train about NWP. Yes, fundamentally improved model construction and new research are needed, but also intelligent use. If I know my model resolution, particularly the vertical resolution and coordinate system used, I may have a better physical understand about why the model has a particular solution in the small scale in variable terrain. The same is true in judging NWP precipitation forecasts and needing to know how a model treats moisture and decides when it becomes precipitation. 

These are obviously cases of "background knowledge" improving performance, and when teaching it, it is more difficult to couch in "authentic" activities (but not impossible).

For me, perhaps in violation of your advice (I am not sure), I would write learning objectives for these such as "Understand how the XXX NWP model produces precipitation in xxxx conditions," or "Understand how the vertical coordinate system used impacts the ability of the model to generate terrain-enhanced precipition." I would evaluate them by presenting a situational question and asking learners to explain how and why the model would perform well or poorly. (Using multiple choice questions :)

Of course, I would not think the training should stop there, but should also including learning activities that lead to analysis and decision making using NWP products, but these enabling objectives seem important still. So, a competency (not the highest level one, which should not be specific to NWP) might be something like "Use NWP products in the process of analysing meteorological conditions and generating a forecast," but lower level learning objectives might include something like those I wrote above. 

Thoughts? Concerns?

Pat

En réponse à Patrick Parrish

Re: 02 Is training the answer?

par Ian Bell,

Hi Pat,

I think this is analogous to Vesa's point about people knowing satellite theory not necessarily being able to apply it to diagnosis and prediction. My view is that knowing the theory of NWP models in no way guarantees that this can be applied to forecasting decisions. I have known many modelers who were unable to apply their knowledge to advise on forecast decisions.

I totally agree that forecasters should know as much about the physics as possible, although not the mathematical details. However, they need to put that knowledge into practice. See below.

How would I write the competencies? Here's a VERY rough draft I wrote in an email to Heleen when she was preparing a course, particularly around ensembles. The original learning outcomes (not written by Heleen) were all in terms "describe", "explain", etc. (I would want to give a lot more consideration to the wording and content so don't hold me to the details. I was trying to give an idea of a possible direction for improvement.)

Competencies
1. Improve weather forecasts by applying and adapting operational NWP guidance, including ensemble predictions, in the forecast process. This is done through identification of model reliability, errors and biases and model characteristics due to:
o    analysis errors and biases due to the data assimilation scheme
o    specific model characteristics (resolution and grid point or spectral scheme, physical parameterisations, hydrostatic or non-hydrostatic)
o     etc.
 
and their response to
o    meteorological situations and systems (eg. How they perform in a heavy rain situation, a frontal situation, cut-off, etc)
o    forecast lead time

2. Identify situations and systems where operational models will be more or less predictable

3. Assess the current forecasts, including ensembles, to determine how well their evolution matches that of the real atmosphere.


4. Use ensemble predictions as guidance for probability weather forecasts and to assess the likelihood of extreme situations

One of the best Advanced Forecasters courses Jeff Wilson and I ran was way back in 1993 or 1994. We were asked to improve the use of nwp by the forecasters.

We had a new computer lab with brand new x486 computers with the screens set to 256 colours so the satelllite images looked OK. Very  exciting!And one of our colleagues in the research centre had a numerical model that could run on a PC. (He had adapted the real-time model for use in briefing sailors in the America's Cup ocean yacht racing competition.) With his assistance we installed the model on our PCs.

We asked the forecasters coming on the course to pair up and select a  situation that occurred in their region and was significant for them, eg a heavy rain situation. It was their choice.

We (our friend) ran the analysis and initialisation schemes for each case before the course and produced boundary conditions from the global model. On the first day we taught people how to run the model on the PCs and they produced a forecast with the standard model configuration. They used McIDAS to investigate the meteorological predictions and compare them with their experience from the situation.

During the course we had a theme each day - moisture and cloud schemes, boundary layer, radiation, topography, resolution, etc, etc. A researcher came and presented an aspect of the theory with an emphasis on its role in the model. The participants then ran the model (an hour and a half or so over lunch time) with changed configurations based on advice from the researcher. For example, add levels in the boundary layer but there is no point if it is in the constant flux layer, remove the mountains, remove the moisture, increase resolution, etc.

The afternoon was spent with the participants conducting a meteorological assessment of the changes brought about by the changed model. The researcher spent the afternoon with them, helping them to interpret the results in terms of model behaviour. For example, in one case the rainfall increased when the convection was turned off. This seemed paradoxical but the researcher was able to relate this to the fact that the moisture was still there and that the layer cloud precipitation processes in this model were more efficient at converting the available misture into precipitation. As there was more moisture available to the large scale processes, due to the lack of convection, the total rainfall increased.

The course was extremely successful. The forecasters loved it and the researchers were very happy, being able to make their passions concrete for forecasters and, in turn, learning a lot from the forecasters. The participants returned to their forecast offices with the model software and some of them installed it and continued to run it locally for several years.

I would love to see a numerical model that the training community could run (maybe online) in such a manner.

Does anyone else have experiences teaching NWP that you could share? Also any NWP competencies?

cheers

Ian

---------------------------------------------------------------------------------------
This is my third try to send this. I had a message saying, "Incorrect sesskey submitted, form not accepted!" Luckily I had copied it to a Wordpad doc.

En réponse à Ian Bell

Re: 02 Is training the answer?

par Patrick Parrish,

Thanks Ian,

It sounds like you and Jeff had quite a successful approach to teaching these things in an application-oriented way. I suspect the work you did to prepare the cases was well worth the effort. 

In terms of the competencies, I like your solution to write the enabling objects in terms of "Identify..." reliability, errors, biases, situations where the model would be less accurate, etc..

Pat

En réponse à Patrick Parrish

Re: 02 Is training the answer?

par Kathy-Ann Caesar,

Hi Pat

Forgive but I answer you without having reviewed Ian latest presentation.  However I can give some insite after my experience in the CMO aeronautical Continuing Professional development course in which one of the topics in Use if Mesoscale NWP products operatioanlly.

The approach we took is first introducing the participants to the model, its make up and dynamics, cautioning then that diffirent models especially on the mesoscale have various parametization factors and that the forecaster much be aware of these in the model.

NEXT remeber MODEL are guidance NOT GOSPLE!

In the use of the model we teach that the forecasters but first develop a 'conceptual model' of the particular weather situation based on the various observations and their own knownledge and theory.  Tehn thsy should compare the NWP initial output to the their Conceptual model, and decide if the model output matches their conceptual. If it does go with it if not use other parameters to make up the forecast.  They should also repeatedly monitor the performance of the model to determine its biases. 

But I agree with you that it is a continuous learning process and the forecasters have to made mindful of that fact, especially since NWPs are continuously being updated. 

En réponse à Ian Bell

Re: 02 Is training the answer?

par Mark Higgins,

I have been thinking about Ian's first sentence

"When our staff have a performance gap we may be asked to provide training."

I am imagining a situation where a manager has said to me as a provider of training (either in house of externally)  can I have a course for X person.

so much is then implicit - how was the need for any intervention identified, who identified it the manager or the person, how was the idea of a course selected as the appropriate means of intervention. If there is a gap and the course is the right intervention what will happen when the person gets back to work.

I liked the structure in the power point it gave a helpful way of thinking about the presenting issue a bit more deeply - my thought was who is best placed to do this thinking - what if the manager and the person was able to do the analysis as much as they can and them come to the provider of training for some insight into the best intervention for their need.

And before the intervention the manager and the person plan how it fits into a wider idea of learning something and changing working practice/performance.  

To my thinking this is a standard management skill - to be able to work with individuals and teams on their development needs.

Mark