Unfortunately, I wasn't able to follow Paul's lecture live, so it's nice to have the recording available (playable in a browser...great).
The concept of competency based training is quite new to me, and I find this session and opportunity in Calmet helpful for me to get a good start in working towards building a competency based training program within our institute. I believe this is the way to go.
I have many questions, starting out with what approach is recommended in identifying the "right" competencies? It seems to me pretty essential to get this right, since training material and activities build on the competencies involved. For operational meteorologists, we have started by breaking down the forecasting process in disciplinary topics, such as observational data (radar, satellite and so forth), analysis and NWP. I feel this work is quite complete and started to look at our training material to "fill gaps".
But recently, a colleague showed me a powerpoint by Pat Parrish and Joe Lamos about defining "job functions" or "areas of responsibility" in a "professional development series", instead of using the above mentioned topics. A refreshing presentation that really started a lot of new thoughts. I started to think that "short range forecasting" or "issuing warnings" could be a type of job function that implies a lot of the disciplinary topics?
I appreciate any input here, at least to just think "out loud" in such a relevant forum.
Cheers,
Erik
Erik,
your posting helps a lot. At EUMETCAL we have a structure that is closely related to what you are talking about (see picture below).
Until now our course / training delivery has been much at "Training Tools" and "Forecasting Tools" level , which are equal to what you mention as "disciplinary topics". The top level ("Improving skills...") would be better described with your quotes from Joe/Pat ("job functions", "areas of responsibility"). This is where we need to focus on the next years - Aviation is the topic currently on the table, but I agree with you that "short range forecasting", "issuing warnings" are other important job functions that are on top of disciplinary topics. I think I will re-draw the picture above!
Would be nice if the presentation by Joe/Pat would be available.
your posting helps a lot. At EUMETCAL we have a structure that is closely related to what you are talking about (see picture below).
Until now our course / training delivery has been much at "Training Tools" and "Forecasting Tools" level , which are equal to what you mention as "disciplinary topics". The top level ("Improving skills...") would be better described with your quotes from Joe/Pat ("job functions", "areas of responsibility"). This is where we need to focus on the next years - Aviation is the topic currently on the table, but I agree with you that "short range forecasting", "issuing warnings" are other important job functions that are on top of disciplinary topics. I think I will re-draw the picture above!
Would be nice if the presentation by Joe/Pat would be available.
Hi Erik,
As you have found, it is not easy setting good competencies. I started the process in the Australian Bureau of Meteorology over 10 years ago and they are still evolving. Roger Deslandes could send you some examples.
We spent many days working on them with forecasters, researchers, people from our services policy area, trainers, etc.
I am a strong advocate of starting at the highest levels and working backwards. This means that you set authentic objectives based on what people are actually required to do. This, of course, requires a clear understanding of what is expected of your staff, eg, the forecast process, what techniques they should use, etc. This is really positive for forecasters as it states what you expect them to be able to do and how.
By starting at the top level, eg, forecast thunderstorms, you implicitly include the lower level competencies. For example, they won't be able to show they are competent in thunderstorm forecasting if they can't interpret the satellite imagery, can't access the radar data, etc, etc. Ultimately, of course, you will need competencies for all of these skills as well.
Assessment is against the competencies. This keeps it authentic and assures you that your staff are competent, not just that they can pass an exam. (And NO multichoice questions!)
I'm attaching a PPT that I use when I am teaching to show the relationship between goals, competencies, learning outcomes, assessment, etc.
Currently I am planning a thunderstorm nowcasting course, basing it on competencies, and finding this a very useful approach. I'll attach a Word doc separately, as I don't seem to be able to attach two files at once.
Good luck with it.
Ian
As you have found, it is not easy setting good competencies. I started the process in the Australian Bureau of Meteorology over 10 years ago and they are still evolving. Roger Deslandes could send you some examples.
We spent many days working on them with forecasters, researchers, people from our services policy area, trainers, etc.
I am a strong advocate of starting at the highest levels and working backwards. This means that you set authentic objectives based on what people are actually required to do. This, of course, requires a clear understanding of what is expected of your staff, eg, the forecast process, what techniques they should use, etc. This is really positive for forecasters as it states what you expect them to be able to do and how.
By starting at the top level, eg, forecast thunderstorms, you implicitly include the lower level competencies. For example, they won't be able to show they are competent in thunderstorm forecasting if they can't interpret the satellite imagery, can't access the radar data, etc, etc. Ultimately, of course, you will need competencies for all of these skills as well.
Assessment is against the competencies. This keeps it authentic and assures you that your staff are competent, not just that they can pass an exam. (And NO multichoice questions!)
I'm attaching a PPT that I use when I am teaching to show the relationship between goals, competencies, learning outcomes, assessment, etc.
Currently I am planning a thunderstorm nowcasting course, basing it on competencies, and finding this a very useful approach. I'll attach a Word doc separately, as I don't seem to be able to attach two files at once.
Good luck with it.
Ian
Here's the competency outline for the Thunderstorm Nowcasting course.
It gives the presenters a clear view of what I am wanting from them and moves us away from content to what participants will be able to do.
Ian
It gives the presenters a clear view of what I am wanting from them and moves us away from content to what participants will be able to do.
Ian
Vesa and Ian - many thanks for your postings! A lot of helpful information to digest, I wonder if I should find comfort in that I have many years ahead of me on this matter 
Also, it's great to see things in a figurative way. From what's provided to me now, I see I need to broaden my thinking a bit and make some adjustments to my list of competencies. I'm sure we could discuss further other "job functions" or topics, there are many approaches.
I couldn't agree more to the comment on being clear of what is expected from each individual. That's the main motivation for me starting on this process. I like to think this will provide bread crumbs to an individual lost in the weather forecasting jungle.
I also appreciate the presentation and comments here because it got me thinking that one seldom gets things right the first time and must draw the line somewhere with respect to identifying competencies. Then concentrate on the assessments and revising continuously.
Ian, the "High-impact weather" group in Eumetcal, of which I am part of, received some good documents from Roger on competencies and assessment with respect to severe thunderstorms.
Sincerely,
Erik
Also, it's great to see things in a figurative way. From what's provided to me now, I see I need to broaden my thinking a bit and make some adjustments to my list of competencies. I'm sure we could discuss further other "job functions" or topics, there are many approaches.
I couldn't agree more to the comment on being clear of what is expected from each individual. That's the main motivation for me starting on this process. I like to think this will provide bread crumbs to an individual lost in the weather forecasting jungle.
I also appreciate the presentation and comments here because it got me thinking that one seldom gets things right the first time and must draw the line somewhere with respect to identifying competencies. Then concentrate on the assessments and revising continuously.
Ian, the "High-impact weather" group in Eumetcal, of which I am part of, received some good documents from Roger on competencies and assessment with respect to severe thunderstorms.
Sincerely,
Erik
Hi Erik and all!
Sorry I couldn’t be with you from the beginning of this topic…
I’ll show you briefly the approach we’re using in determining the competencies and the structure of development.
We have three different starting points: WMO 258, Romanian CAA requirements and Job description file. Why? Because we decided from the very beginning that our meteorologists have to be WMO meteorologists – i.e. to be compliant to 258 and CAA regulations. Also we need to make them do their jobs properly. I just say that being compliant with 258 isn’t enough. So we start with all three requirements.
Having a good job description at this point is extremely important. It has to describe the tasks, tools, contingency, etc… in order to be able to put things together in defining the competencies.
Training, assessment, gap evaluation, and the loop goes on. The problem is when we identify too many gaps that have to be filled… then we have to identify the source: is the competence description wrong or the training delivered improperly? If it’s the competence description, then is it connected with the job description or the CAA’s requirements? In all cases some SWAT analysis has to be done and some discussions with CAA in order to redefine the training structure or the competencies if necessary.
Have a beautiful day!
Paul
Erik,
You and others have raised some great points here about competencies. Yes, we have worked hard with the NWS to get people thinking more in terms of job responsibilities versus traditional disciplinary divisions. Its a hard perspective to change, given that that is often how we are educated at school, but also how expertise is developed and recognized (we speak of radar experts, but how often do we speak of short-term forecasting experts?).
I have to admit a little concern sometimes that people can take competencies too far, and begin to feel that teaching and assessing the isolated sub-sub competencies will allow people to learn and demonstrate that they can fulfill the actual broader job responsibilities. That is why I like the approach that Ian describes, that he and Jeff are promoting, and that the CAeM group are applying in writing Aeronautical Forecaster job competencies. Starting with the highest level job responsibilities not only helps make sure that you are getting to all the important more detailed competencies, knowledge, and skills, but also reminds you that you also have to teach, provide practice in, and assess the larger competency as well. People have to be able to perform the larger task in context. This is also why I like ROMATSA's insistence on making on-the-job training a formal part of the training program, and also part of the assessment as well.
By the way, a really nice but rather long treatment of this whole-task vs. part-task issue is covered in a book by a professor from the Netherlands, van Merrienboer's "Training Complex Cognitive Skills." (I have to confess not having read the whole thing, but even the overview of the model provides a lot of good food for thought.)
Joe is the first author of that presentation and paper you mentioned. I'll see if I can find a copy to add to this session's resources as requested. Ian may have some good things to share also.
You and others have raised some great points here about competencies. Yes, we have worked hard with the NWS to get people thinking more in terms of job responsibilities versus traditional disciplinary divisions. Its a hard perspective to change, given that that is often how we are educated at school, but also how expertise is developed and recognized (we speak of radar experts, but how often do we speak of short-term forecasting experts?).
I have to admit a little concern sometimes that people can take competencies too far, and begin to feel that teaching and assessing the isolated sub-sub competencies will allow people to learn and demonstrate that they can fulfill the actual broader job responsibilities. That is why I like the approach that Ian describes, that he and Jeff are promoting, and that the CAeM group are applying in writing Aeronautical Forecaster job competencies. Starting with the highest level job responsibilities not only helps make sure that you are getting to all the important more detailed competencies, knowledge, and skills, but also reminds you that you also have to teach, provide practice in, and assess the larger competency as well. People have to be able to perform the larger task in context. This is also why I like ROMATSA's insistence on making on-the-job training a formal part of the training program, and also part of the assessment as well.
By the way, a really nice but rather long treatment of this whole-task vs. part-task issue is covered in a book by a professor from the Netherlands, van Merrienboer's "Training Complex Cognitive Skills." (I have to confess not having read the whole thing, but even the overview of the model provides a lot of good food for thought.)
Joe is the first author of that presentation and paper you mentioned. I'll see if I can find a copy to add to this session's resources as requested. Ian may have some good things to share also.