Error in the data or something else?

Error in the data or something else?

от Theodoros Gkousarov -
Количество ответов: 2

 

Hi again,

I downloaded the complete climate dataset of the SIS product from the WUI. Merged the files and created 2 big files around 1.5GB each, which due to the large size i couldnt manage to merge into 1.

However when I tried to plot the time series from one specific location, I found out that there is a gap in the data for a long period. the question is...

There is an actual error in the long term data file or i caused an error when tried to merge the multiple files?

I attach the charts. (the third chart is from a different file that i managed to merge a few weeks ago, but still same period of time missing).

В ответ на Theodoros Gkousarov

Re: Error in the data or something else?

от Jörg Trentmann -

Dear Theo,

even though there are some data gaps in the CM SAF SIS data set, the gaps in your data cannot be explained by the gaps in the original data set. For me it seems like you are missing the data from Meteosat 5, which has been operational from 1994 until 1997. This data is available from the CM SAF, maybe there was a problem with the data transfer etc. As the data comes in packages per satellite, maybe this one has not been prepared or so. I suggest you repeat your request focussing on the gap in 1994 to 1997.

Another issue I noticed when looking at your time series plot is that there are some days that have a SIS value of zero. This is obviously not realistic (even in the UK!) and has recently been updated in the CM SAF data base. These files have been replaced by files that contain missing_data instead of zero values, which should allow the users to automatically remove these data from the analysis. Alternatively you might want to set all zero-values to NA.

In addition we found a couple of days with suspicious data (weird stripes across the data) that also have been set to NA recently in the data base. These days include 15.08.1992, 14.12.1992, 13.06.2000. Please make sure to remove these data from you analysis (or update your data).

Sorry for an inconvenience the suspicious data has caused, obviously we try to minimaze these problems, but things like these cannot always been ruled out.

Hope this help, kind regards,
Jörg

P.S. You should be able to merge the two times series, once you have extracted the specified location.

В ответ на Jörg Trentmann

Re: Error in the data or something else?

от Theodoros Gkousarov -

Thank you very much Joerg, Yes I am aware of the data gaps  which are mentioned when I try to obtain the data, but as you said there should be something more than that. I will try to download it again.

It is very usefull that the database is improved continusly for improving suspicius data :) keep up the good job guys.

I will work more on that the next few days and wll write back here the resutls.

 

thanks,

Theo