What are your experiences???

What are your experiences???

par Heleen ter Pelkwijk,
Nombre de réponses : 21

Ha all,

Sorry for the long silence! It was a bit too busy in my office lately. But now I have some time left before going to Schiphol Airport for an excursion, so I thought let’s take the advantage.

It is great to hear from all your experiences. Thank you for sharing! Until now I have read stories of case studies, real life simulators and canned data simulators. These stories make me very curious and now I am wondering what does it all look like in reality? Do you have pictures? I only have seen something from Vesa until now, but I am sure there is more to see and experience for us…..

 Lih Mei when you are doing your case study simulations with the junior forecasters do you put any form of external pressure (by for instance using a clock) on this process? If yes, how do you do that? In what way do you present the data to the participant? Electronically, paper, all data sources at the same time or do you give the data with every time stamp? Who makes the case studies? Do you collect them in a sort of database?

The real life simulators..... I assume you are using a forecasting bench like all the forecasters are using in real life playing a shift with some supervisors who are also the opponents for the forecasters and all the product made stay within the building but are checked and criticized by the supervisors……is that true for every one? Or are there different ways to do a real life simulator session? With how many forecasters at once do you such a simulator session? One, or more than one? How many people do you need as supervisors? What are their tasks? What goes well during these sessions and what goes wrong? What do we have to keep in mind organizing these kind of sessions? Kathy-Ann I love to hear your experiences, your results and Do’s and Don’ts  and Natalie I am pretty sure that you have some experiences we can learn from too. Last year I heard some experiences of the training with real life simulators in the initial forecasting course from the South African Weather Service. When the new forecasters were going to their work spot after this training they only had to do only 3 more on the job training sessions and then they could perform their job solo….this is impressive I think and that means that the real life sessions were really close to reality.

The canned data simulator session…..Natalie what does your canned data simulator look like. Do you have a simulator that looks exactly like the work bench from the forecasters and are all the production tools the same too? Or does it look different? How many people are trained at the same time? And how much staff do you need for that? Where do you get the data from used in this canned data simulator? Do you have multiple cases you can select one? How is such a tinned data simulator session prepared? Are there special thing you need to think of? Vesa…. you had  a great experience using the simulator in your training as a warming  up exercise. When you describe what happened I only see very positive points…which is great but I am sure that there are things you want to do different next time. What are they??

Lara you write that you have planned simulator sessions in October….What are you going to do during those sessions? What are your plans and how are you going to make sure that you are going to meet your goal during these sessions?

And then the dreamers Izolda en Roro…what are your dreams regarding simulators? What do they look like? What do you think you need to realize them? Do you see any problems on your way to realize this dream and if there are….are ways we can help you to make your dream to come true?

Now I probably have forgot some reactions on the posts…..sorry for that…but if you have stories we can learn from please tell us….

I have to run now to get my train, but soon I will tell some more of my experiences.

Hope to hear from you all!

Have a nice day!

Hartelijke groet,

Heleen

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Mark Higgins,
Heleen, Thank for your post with all your questions! I am looking forward to everyone's answers. I am really interested in other's experiences. Now I think about it - the last weather related simulation I took part in was some 15 years ago. The simulation was of a forecast office, and included producing a media briefing. The purpose of the simulation was to get us used to what a forecast office was like and help us to calibrate weather forecasting skill. If I recall correctly the de-breifing focused on these two areas. We were asked to reflect what we had learned about the forecast process, about how prople worked together. We were also asked to consider what we learned about what we knew about meteorology. The whole process was paper based, we had charts for the observations and NWP output which we then applied hand drawn contours and annotations. Looking back the question I would add into this experience would be what is the role of NWP in the forecast process? This simulation was not teaching a specific process or set of meteorological situation. It had quite a low fidelity (all paper based, not all products available) but it was real enough for its purposes. In the earlier posts many people mused the word authentic. Looking back I think this simulation was authentic in several ways. The learning experience outcomes fitted what we as learners needed to learn, the process was a nice way of doing it by having the experience we were able to develop an appreciation of the forecast office. The process was consistent with the desired outcomes. It was learning by doing and mistakes were welcome! In the experience we made out own sense of what what was going on, in the de-briefing discussion we would have been able to make sense of the experience together. This was part of a longer course, the simulation was formative and we continued making more sense of it during the social spaces in the course. I can't remember if we had a clock I assume we did as it is a reality of the forecast office. I look forward to hearing from others soon, And safe travels to those travelling at the moment Warm wishes Mark
En réponse à Mark Higgins

Re: What are your experiences???

par Kathy-Ann Caesar,

Hi Mark

I am happy to chime in here. THe CIMH Forecast Office Simulation (FOS) is designed to include as many experiences of that in the forecast office and is all practical. But keeping in mind that there are sitations that can not be simulated.  Thus our goal has be to have the forecasters be in a "Be prepared" mode (Yes the Girl Guides roots are showing).

In the last few years we have added. media presentation training, which includes how to deal with media briefing, TV and radio presentations and even simple telephone calls interactions.  In the simulation exercise itself the forecaster is given a 'non real time' event and needs to analyze the situation, formulate an official response.  The response is in the form of a bulletin and vocal presentaion.  While time does not allow for extensive training in this area, the feedback from the forecasters and their supervisor  is that the forecasters are more ready to handle interactions with internal and external stakeholders.

Like Mark's the FOS also include a de-briefing and it is a great experience to see the forecasters having to go through the process of re-analysing and discussing their forecasts. Since is forces use of the meteorology, their own understanding of it and reality. 

Our experience has thus been that in the majority of cases it helps and in a few cases quickly reveals those not quite ready for the real thing.

Cheers

Kathy-Ann

En réponse à Mark Higgins

Re: What are your experiences???

par Heleen ter Pelkwijk,

Ha All,

Reading Mark's post it is my idea that this is a really nice example how you can run a simulator session in a easy way. I have seen very sophisticated simulator sessions at our air traffic control training centre in Amsterdam. But this sophisticated simulator took them also 3 out of 12 sessions before they were able to keep that simulator running the during whole exercise.....that means in the end that the simulator was beautiful, but also 3 sessions with people that were not properly trained. So to be honest the simple simulator sessions are often the best! And you don’t want to have your training screwed by techniques which are not working properly

An other thing I want to highlight from Mark’s description: "It was learning by doing and mistakes were welcome!" This bold written bit is very important to me when I organise a simulator session. Of course it depends whether your have a training or assessment goal, but my feeling is that the participants should feel themselves save to start with. When I would use a simulator as assessment tool (which I am going to do in the future) I would start using it as a training tool first (and that is what I am doing right now!). In my view people only learn new things when they dare to. And when you experiment in your work mistakes are easily made...a good thing of training with simulator sessions is that participants get the opportunity to experience the consequences of their actions (and learn from it) without any damage to your institute. We all have this opportunity so please make use of it!

Hartelijke groet,

Heleen

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Heleen ter Pelkwijk,

Ha all,

 

Looks like I am not the only one very busy. Never mind……

Somewhere in the future we will get some more info from Lih Mei and Izolda I heard in the email. Also the other posts added to the forum are great. I will read and react on them soon! …it is good to know that I am not Remi in this forum ;-)

 

Okay my simulator training for the severe weather and safety forecasters. The severe weather and safety forecasters are a group of forecasters who have to act in extreme situations like severe weather, extreme high floods, chemical and nuclear accidents and other extreme situations where weather can influence the situation. During those kind of events they often work in crisis teams with people from different discipline.

 

The goal of this simulator event:

We had taken an extreme weather event (convective situation)  and we wanted to know how the individual forecasters act in a stressy situation and wanted to train this group in:

-           Doing the procedures (Erik Hagemark calls them tasks) correct at the right time… Do they take the right decisions on the right moment and do they contact the right people at the right time? This was not so much about meteorology, but more the tasks this forecasters has to do in extreme events

-           How do they communicate both internally and to the customer….

For us it was important to take the forecasters out of his convert zone AND that they were not  hiding behind meteorological content as they like to do in normal situations.

 

We had 4 forecasters working at the same time in separate rooms. We started the simulator event with a general briefing together in which we told the forecaster:

-         What  we were going to do

-         What we were expecting from them

-         That they were going to be assessed by the facilitators (on attitude, the way they were working, process knowledge, the published products, the written texts and the timeliness of their actions)

-         That the exercise was only working when they were actively joining the game

-         And that we wanted to train their colleagues with this exercise too…so they had not to give away the content (and that worked!)

Then all the forecasters got a private weather briefing and half an hour to adapt to the simulator. Then the exercise started. We started them we a separation time of  15 minutes……which appeared to be too short at the first day already...so we made 30 minutes from that in the next training

 

The exercise

ü      Eumetcal simulator was used

ü      We found an extreme convective case

ü      forecasters were doing the simulation exercise on their own (separate rooms) in 2 times normal speed.

ü       We took the warn criteria down, to make it obvious for them that they had to go out an warn

ü       they had to do an expert meeting

ü       their texts products were checked and commented by a professional afterwards

ü       their graphical products were being checked

ü      and we had someone stalking them with questions from different corners during their work.

 

The people involved as facilitators:

-         Me running the simulators and tackling the problems popping up (there is always at least one simulator going down…sometimes it is an accident and sometimes it is deliberately…please be prepared for that!)

-         The training manager checking the timeliness of the products

-         A boss from the forecasters office (our crisis manager played that role)

-         Someone from university who taught the forecasters how to write proper weather alarm texts in the year before.

-         The communication officer

-         The model expert

The latter two also helped out the training manager with the phone calls to the “customers” the forecasters had to do.

 

After the exercise we had a debriefing which took about two hours. The lady from university gave a lecture with the help of the text products made during the exercise; what was good, what was wrong and how could it be done better. And Everyone told his/hers experiences. We had great discussion on the matter and appointments for the future were made.

 

Some reactions from the forecasters:

-           For some forecasters it was the first time they had to go through the whole weather alarm procedure

-           An exercise like this is different from reality. Normally you get the possibility to grow towards a certain weather situation because of the media attention and the outside weather. Now it looks like you are thrown in a swimming pool and you have to swim directly to survive. But when adapted to the new “reality” it is great (takes a little)

-           It was fun and very dynamic

-           It was no problem for the forecasters that the weather alarm criteria were taken down for the moment.

-           Sometimes time was going very (to) fast, but the clock doesn’t need to be adapted to a slower speed. This way the exercise looked like a normal reality in forecasting office.

 

What am I going to do different next time:

-           Not more than 3 people at once for a simulator session…else it is getting too much for the facilitators

-           I will arrange sandwiches for the participants. Then we don’t have to stop the exercise at its top to have lunch when the exercise takes longer than we expected.

-           We will have the weather briefing for all participants at the same time…otherwise the participants start with different stories

-           We start the participants with a time slot of minimal half an hour in between. Otherwise you will get all the reactions from the participants at the same time and the participant has to wait because the facilitators can’t cope with it..This takes the action out of the exercise.

-           We had to adapt the phone and score tables we were using for the assessment, because they were not handy enough…think about that before you start!

-           We should not give the response cell to many different roles

-           We should add more external impulses to this exercise.

 

In the end: we corrected some problems in the procedures and adapted them in a way that they were even better workable for the forecasters. Next to that we made appointments within the group and had everyone properly trained for these circumstances.

 

Please keep in mind that you can never do it right with the data you supply in your simulator, but the experience is that in the end it doesn’t matter and has no influence on the exercise.

 

We had a lot of fun and my forecasters are looking forward to the next one which is in preparation now.

 

Erik….very sorry for this (too) long post ….and Ian very sorry for all the bullet point…. ;-)

But I hope this is helpful!

 

Hartelijke groet,

 

Heleen

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Lih Mei LIM,

Hello,

After reading experiences shared, I start to wonder if mine could be considered as a simulator training :)

In Singapore, our duration of OJT for a new forecaster lasted about 3 months which means he may have little experience or exposure to different weather systems in a year. The chance to practise writing trend statements and issuing aerodrome warnings may be limited because there is no provision when the weather is good.

To "overcome", we came up with a "simulator" training Exercise.

We collected data (METAR/SPECI, radar images, satellite pictures and so on) representative of a typical weather event in each "season" of a year in Singapore. We have 8 cases now and would be working on another 2 before this year ends.

For simplicity and no assistance from IT personnel required, the data is displayed using powerpoint presentation. Forecasters are paired in which a senior or more experienced leading a junior. They would go through each case when time permits and the junior would write appropriate trend statements, "issue" aerodrome warnings and so on, as if he is facing the severe weather in real-time. The senior would provide advice or suggestion as and when appropriate.

We didnt really have a clock that runs faster than normal. However, each case would be completed within a time frame, typical in the real-time situation as the senior is not going to allow the junior to spend a significant time to think what is the trend statement to be written. In this way, we are trying hard to mimic the real-time situation.

The benefits include:

1) new forecasters have more chances to write trend statements and "issue" aerodrome warnings on top of real-time situations.

2) they get more experience or exposure to weather systems occurring in different parts of the year.

3) they note any snytax errors in trend statements written and reinforce the criteria of issuance for aerodrome warnings.

4) the seniors and the existing forecasters use this Exercise as a revision tool, to revise their knowledge on trend statements and aerodrome warnings.

Thank you.

En réponse à Lih Mei LIM

Re: What are your experiences???

par Patrick Parrish,

Hi Lih Mei Lim,

I for one would definitely classify what you are doing as simulator training. As far as techology goes, it is at the lower end of the spectrum. But as for the training strategy, it is in the simulation category, pure and clear.

The fact that you have already 8 cases, and will have 10 before the year is done, suggests that your strategy is a cost- and time-effective approach. Exposing learners to more cases might in the end be equally as good as creating an approach that requires significant IT resources.

Please keep up the great work and share some of your cases and instructional designs with us. We all appreciate seeing cases from distant locations to help us learn. And developers, like those at COMET in the USA, can benefit greatly if they have cases from international locations to include in their work.

Patrick Parrish

WMO

En réponse à Lih Mei LIM

Re: What are your experiences???

par Heleen ter Pelkwijk,

Ha Lih Mei,

Yes, your training is a simulator training! Don't worry about that. Your training sounds a bit like the realtime simulations Natalie, Kathy Ann and the South African Weather Service are doing with their new forecasters. Of course your data is canned (many will be jealous about that) and you are using different presentation methods, but for your service it works and that is the most important thing!  

Great idea to use a PPT to display the data!

Thank you for sharing your experiences and when you really want to use a clock you could use the simple FLASH clock.

By the way I think it is very a very good idea in your case not to run the clock faster than normal....You are already using a situation with a lot of pressure on and I have the feeling that the participants have to all the normal work.....when I speed up the time the participants only have to do part of their normal duties...I only pick out a few important products in that case.

Hartelijke groet,

Heleen

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Lih Mei LIM,

Hi Mr Parrish,

Thank you for your encouragement.

Hi Heleen,

Without your presentation in Citeko, Indonesia in 2010, this training programme would not materialise at all. We would continue to explore both time and cost effective training tools which would work for new forecasters and act as revision for current forecasters.

Thank you.

En réponse à Lih Mei LIM

Re: What are your experiences???

par Erik Hagemark,

Hi all, 

Inspired by the many posts here, I would like to share something we tried for the first time last year, and are going to repeat it this year (Nov).

Twice a year, a subset of forecasters from our three regional offices undergo a week of classroom training (refresher courses). Each course focuses on a certain theme, and last year it was forecasting convective precipitation. The idea than came up that we should somehow involve flood forecasters, who don't belong to our institute, but use our forecasts as a basis for their warnings to the public. The answer: roleplay.

We then designed this using "canned data" (pre-agreed on a three situations) and set up the forecasters workstations to read the data. The flood forecasters brought their own data they normally use, so each party had what is common to them. At the start, they were briefed on the plan, and given instructions on a sheet of papir and any practical questions were resolved. The instructions included a time schedule and tasks to perform, and phone numbers of the other groups (they were allowed to interact), but no specific questions. The participants were then split into four groups, one from each regional office and one flood forecasters, and placed in four separate rooms. The group had to agree within itself who to do the particular forecasting, but we figured they would rotate since we had given them three cases. In addition, an observer from one of the other groups was placed in the room, but not allowed to interact, only note what they observed. We rotated the observers after each case. Finally, at a set time, we rejoined in the classroom for a debrief, and the observers were asked to share what they noted.

This was the first time we tried this, and it took quite a bit of preparation. At least 5 people were involved in making the roleplay schedule, and to be present during the session. Fortunately, everything worked out nicely, and there were no techical problems (!).

The feedback was overwhelming, an average score of 4.8 (out of 5) and many comments and suggestions for improvements. Obviously, we are going to try this again very soon, with a new group of forecasters.

 

Cheers!

Erik (Met.no)

En réponse à Erik Hagemark

Re: What are your experiences???

par Heleen ter Pelkwijk,

Dear Erik,

Thank you for this great example! With this case I think you have  shown how easy it is to have a great excercise for your forecasters together with your customers...which is very useful I think personally. Hope it will be an inspiration for others too....

Hartelijke groet,

Heleen

 

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Izolda Marcinonienė,

Heleen, I agree -Erik's experience is easy to transfer to other institutes.But it takes much time to prepare such kind of trainings.In this case, results should pay dividents and show the quallity of trainings and satisfy the trainer-moderator:)

Izolda

En réponse à Erik Hagemark

Re: What are your experiences???

par Patrick Parrish,

Hi Mark,

What a great exercise! Thanks for sharing it. I like the balance between structure and openess in the simulations. And having the observers as built-in reflective practioners is a helpful element.

What I would like to know is what the observers reported. Is there any record of that? Anonymous of course.

Pat

En réponse à Patrick Parrish

Re: What are your experiences???

par Erik Hagemark,

Hi Pat,

Most of the observers comments fall into one category: communication. For instance, they noticed one party strictly using UTC-time, while the other party used local time. Also, terminology needs to be well-defined.

They also noted that both parties involved had limited knowledge of the data used as a basis for their decisions. 

Cheers,

Erik

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Natalie Werbitski,

Hi Heleen,

I’ll try to give some more details on the different sort of simulators that we use in our office.

Our real-time simulators try to mimic the routine in Forecast Operations as much as possible.  We use the same deadlines for forecast products and the same tools, working on a non-operational training server.  Our new forecaster interns go through an intensive 2-week simulator before they enter Operations, in order to get used to the deadlines and desk schedules, as well as to become familiar with their own developing forecast routine.  These simulators are conducted in the training room, adjacent to Operations, and the desks are set up in a similar fashion to what is in Operations.  That way we encourage the interns to interact with each other across various forecast boundaries the same way that is expected to occur in Operations.  Each desk has a phone, and so the trainers will also call the interns from another room pretending to be various users (such as aviation dispatchers, pilots, or other special users).  This gets interns used to the idea of giving telephone briefings (which can easily fluster any new forecaster on the job).  Typically we have from 5-10 interns in the simulator at once.  These simulator sessions are organized by our Meteorologist Internship Operational Program training supervisors.  Interns have commented that they enjoy these simulators to help them get used to the product deadlines and for becoming familiar with the tools/resources used in our office.   Areas that can be improved upon would be some initial training on developing an effective forecast routine, before the simulators occur.

As for our canned simulators, they are usually quite a bit simpler.  We have some canned data for a few different volcanic ash scenarios; we’ll give the forecasters some data (observations, PIREPs, etc.) and get them to run through the routine of how to handle those scenarios (who do they need to call, where to get data, creating volcanic ash SIGMETs, etc.).  We’ve also done convective simulators in the past for our interns (when it was the dead of winter and convective season was far off).  In those simulators, the interns were provided with canned radar/satellite loops, list of observations (up to a certain point in the day), various charts (surface, 850mb, 700mb, 500mb, 250mb); they then had to do a convective analysis and come up with the forecast for later in the day.  During the latter portion of the simulator, they were provided a second set of radar/satellite loops, obs, etc. to show the development of the convection, and then the interns would compare their forecasts to what actually occurred.  In this example interns were still able to use the same production tools and tried to follow similar deadlines, though it is challenging when your obs/data feeds are not coming to you in real-time.

We’ve been trying to incorporate more simulator training into our learning plans, though it is sometimes challenging trying to find the time to organize and plan such events.  I am glad to be learning some new things from the experiences of others on this forum.

Regards,

Natalie

En réponse à Natalie Werbitski

Re: What are your experiences???

par Heleen ter Pelkwijk,

Ha Natalie, Kathy-Ann and all the others,

Your story of the real-time simulator sounds very much like the way they are working in the South African Weather Service too. I think it is great that you guys are doing this type of training in the way you are doing it….we (KNMI) really should put more effort to incorporate this way of working in the training of our new staff. Until now we are doing most of this real time simulator training for new forecasters in a “training on the job” situation with staff being trained to do this type of training. I think the training Natalie and Kathy-Ann are describing is so good because you prevent that old rusted principles are being transferred to the new people and all the people are training in the same way.

 At the end of Natalie’s post she is saying something about “sometimes challenging trying to find the time to organize and plan such events”. That is also our challenge for the canned simulator sessions!

So my questions to all of you is: How do you select your cases? What are interesting learning cases? How do you stimulate your forecasters to bring you interesting learning cases? Being outside forecasting office these events often pass very silence for me…and I am probably not the only one having this problem. And what are your suggestions to make a success of the setting up simulator training? Also think of where you will find your data needed for your simulator session….

At KNMI we have a severe weather catalogue in which we save the most important dates. The data in our MWS will be saved too as soon as the date is in the severe weather catalogue. The severe weather catalogue includes the date, some (model) data the forecasters use in forecasting office, parts of an evaluation report and when I am very lucky I have a forecaster who makes a case study out of this case. The experience learns that as soon as you are not saving a description of the case what happen in this situation the case is useless. Then it simply takes to much time to work it out when you are busy to create a simulator. Most of the time the cases which are completely described are used. I am lucky with a couple of extra people closely related to the forecasters process who are prepared to make a script for a simulator case and run it when I have managed the technical details. To help them I sometimes have forecasters from different forecasting positions who collect the data needed for them. Still a lot of work!

Hope to hear how you all are handling this problem!

Hartelijke groet,

Heleen

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Heleen ter Pelkwijk,

At the Bureau of Meteorology (BoM) in Australia they use simulators as an assessment tool. Roger Deslandes gave me the document I attached on this email some years ago….and it inspired us on the other side of the globe a lot!  It gave us a hand in how to prepare simulators for our training (what do you want, timelines, ways to score,….). Please read it.

…..until now we have hardly touched the possibility of using the simulators for assessment…..

What do you think of the possibility of using simulators as an assessment tool?

Hartelijke groet,

Heleen

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Mark Higgins,

I find it interesting that the word "authentic" is used on page two. I think this echo's soome of the discussion in the first few weeks on what is real enough for simulators.

thanks to heleen (and roger) for posting this. 

what do people think about sims and assessment?

warm wishes

Mark

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Heleen ter Pelkwijk,

Dear All,

Still hoping for dreams/ideas about your future with training an simulators from the people who are not using them yet. If there are ways we can help you to realize them. Please let us know!

I am also hoping for more stories from people who use simulators/role plays to train people at home but were silent so far. We can all learn from your experiences too!

And is there something you want to know from us contributing to the forum already?

Hope to hear from you!

Hartelijke groet,

Heleen

En réponse à Heleen ter Pelkwijk

Re: What are your experiences???

par Izolda Marcinonienė,

Good afternoon,

It seems to be that developement of  usefull and friendly-use simulator is a big chalange!I tried NOMEK simulator  suggested by Vesa-wow !

We've just finished the last Satrep workshop here in Vilnius.My colleagues from Baltic region (Poland, Latvia, Estonia and Lithuania) tried to work with WMS1 programme created at ZAMG (Austria)for special purpose-identification of Satrep conceptual models using satellite data and ECMWF model parameters. We could work with current weather and also use archives when severe weather in our region occured. My opinion is that it looked like a simulator:) We liked it very much! To add, we were lucky- very experienced teachers- Dr. Veronika Zwatz-Meise and Barbara Zeiner from ZAMG taught us during 3 days.

It means, that sometimes it is not necessary to create new tools -forecasters could practise and do exercises with programme they use. But they have to be interesed in it and want to improve their skills. I'd like to emphasize,   in any case good teachers are necessary!

Izolda