Dear DRIHM community members,

I'm pleased to present here some feedbacks from the first "evaluation" of Cb-TRAM by MeteoNetwork's citizen scientists. It took us a little bit more than what we expected and I really apologize for that. Not all the questionnaires have been handed back, few are still on their way and I'm pressing our volunteers to fill them in as soon as possible.
Anyway, I hope this will provide you with some useful hints and comments about this great tool.

This first part is based on the evaluation questionnaires that were kindly provided by Tatiana Bedrina, however there will be a second part, hopefully coming soon, based on a different evaluation questionnaire that I wrote down. Even though the latter is aimed more at giving us (MeteoNetwork) some feedbacks about such an activity it might contain some interesting stuff for you as well, hence I will post it here too. I finished collecting the questionnaires so I just need to find some time to look at them and summarize their content.

Here I translated and summarized the contents of the first questionnaires in order to convey information more clearly, however in the attachment you'll find a zipped folder containing them all.

QUESTIONNAIRES REPORT:

Table of evaluation.
Rates scale: 1- very poor, 2- unsatisfactory, 3- about average, 4-good, 5- superior

Q1: In your opinion, is Cb-TRAM quite intuitive? Average rate: 4.26/5
Q2: Is Cb-TRAM interface user-friendly? Average rate: 4.10/5
Q3: How do you evaluate the execution time? Average rate: 3.05/5
Q4: How presented critical cases are interesting to you? Average rate: 4.42/5
Q5: Was the output of the Cb-TRAM web application (images, xml, kml) useful for you? Average rate: 4.68/5
Q6: How would you rate the service as a whole? Average rate: 4.10/5

Summary of answers:

Q7: Could Cb-TRAM be useful to you in the frame of your professional interests?
- Sure, especially as a tool to support short term forecasting and warning
- Yes, sure.
- Yes, sure.
- Of course, especially as a support during nowcasting
- In principle yes. I’d like to test its performance with real time data though.
- Sure, Cb-TRAM could be a very useful tool.
- Yes. It can be a first step towards a more stable cooperation and even more contributions from citizen scientists.
- Yes, very useful.
- It is a great tool.
- Sure, it could have many applications even for citizen scientists activities
- Yes, sure.
- Sure, as long as it’s free of charges
- Sure, it has many possible practical and more theoretical applications
- Sure, I think it’s a fairly innovative tool.
- Yes.
- Yes, it is really interesting to have a closer look at such events
- Sure. Very useful during severe weather events from a nowcasting perspective.
- Of course, could be very useful in order to minimize the damage associated with severe weather.

Q8: How could you use the Cb-TRAM output files?
- To support analysis of thunderstorms
- To study the tracks of convective cells during past events and if possible to identify preferential -paths of thunderstorms in a specific area.
- To create an archive of past events to be used as an educational tool.
- As an integration to other data sets coming from different sources.
- To understand the evolution of thunderstorms in the past and better understand their possible behaviour under a similar synoptic scenario.
- I could compare them to other outputs from different softwares.
- To create an historical archive of severe weather events
- First, to create an archive and also to understand the “speculate” about the possible evolution of future events based on our prior knowledge.
- Sharing and archiving them.
- To create an archive
- Real time monitoring of severe weather events; building case studies for NWP models verification
- To study the interaction of convective cells with complex orography
- To re-investigate past events.

Q9: In your opinion, how could the Cb-TRAM service be improved?
- The moving windows feature is problematic. I’d prefer them to be fixed and not one inside the other. The map plot crashes if I click on the “OpenStreetMap” link, both in domain selection stage and in the visualization one. It would be great to highlight more the legend describing the color code associated to cells.
- It would be great to speed up the thing a little bit and making the software interface a little bit more captivating
-It would be great to have the possibility of zooming over a given area even during the visualization stage; to speed it the execution time but keeping the interface intuitive. If possible it would be great to combine this information with precipitation data coming from Radar or MPE channel from Eumetsat.
- by roviding a real-time application
- Maybe by adding some more geographical details in the map and making some graphical tuning of the interface, however it is already very intuitive and user-friendly.
- By providing a very detailed user-guide.
- By increasing the number of past events in the archive
- Cb-TRAM could be improved by reducing images load-time and removing the actual multi-windows structure of the web page.
- Speeding up the execution time and having the chance to look at the .xml data on the web page.
- by combining it with radar images
- by overlapping the output with some NWP model parameters plots.


I hope this is a valuable feedback for you all, if you need more information or have any sort of question feel free to contact me.

Best wishes,

Edoardo Mazza - Meteonetwork

Dear Edoardo,

I want to thank you for this valuable information for DRIHM community!

The analysis is quite curious and provides us with some interesting ideas for the further work.
We should stay in touch with the citizen-scientists' groups for the next discussions.

To maintain the conversation, my question is
if you are able to provide us some characterization of the
respondents and their motivations: residence, why
they might have gotten interested in the citizen-science.

Thank you for this impressive effort!
With best regards,
Tatiana Bedrina

Dear Edoardo,

Thank you very much for the detailed evaluation of the Cb-TRAM web application for citizen scientists. We are pleased that the rating of our service is quite high, and we appreciate the mostly positive feedback from the users. The hints and remarks of the users are very helpful to improve the service. In the following please find DLR’s reply to critical issues and specific requirements mentioned by the users in Q9:


• Execution time: DLR is currently working on an improvement/speeding up of the plotting routine which is the bottleneck with regard to execution time. As soon as a new plotting routine is available, we will replace the one on the web application. We will make an announcement on the website then.
• Overlay of Cb-TRAM with other data sources (e.g. radar, NWP model data etc.): This is not the scope of the web application. The web application provides images, XML, and kml-files which can be downloaded for further use by the citizen scientists. These data can then be combined/overlaid with any kind of other data by the users themselves.
• Cb-TRAM as real time application: Cb-TRAM is a commercial tool. Therefore, a real time service cannot be offered. In addition, a real time Cb-TRAM service would be in conflict with severe weather warnings of the national weather agencies. All we can do, is to post more historical events. Over the summer of 2014, DLR will collect thunderstorm events over Europe and the Mediterranean and add the Cb-TRAM analyses of these events to the web application.
• Problems regarding zooming or viewing the web site: this has nothing to do with the web application or Cb-TRAM itself. The users have to verify that they have the latest browser versions available. If Microsoft Explorer does not work properly use Mozilla or Google Chrome.
• DLR will provide a user guide of the web application including a more detailed description on what can be seen on the images. We will make an announcement as soon as it is available on the web side.

Some users asked for Cb-TRAM results for specific cases of flooding events, e.g. over Great Britain which were not related to thunderstorms but to intense mid-latitude cyclones and related orographic precipitation. Currently, Cb-TRAM is not capable to detect such events properly, as it is optimized to detect thunderstorms only. We will put a note on this into the user guide of the web application.

Best regards,
Caroline Forster, DLR