Evaluation of the INADEF prototype on the basis of “historical events”
Basically, four different results of the forecast are possible (see Table 1). Since the forecast is made on a daily basis, the forecasts output “no event predicted and no event observed (n_pnon)” strongly dominate. This results in a high number of randomly correct hits, which can be eliminated by means of Cohen’s Kappa (Landis and Koch, 1977).
The analyses were done for nine different precipitation input scenarios, for all five test sites The input areas are the resultof the intersection of the selected precipitation values with the catchment area and a defined buffer. Example: M1_max would be the highest INCA pixel value within the basin+a buffer area 1km wide.
Table 2: Scenarios resulting from selected pixel values (INCA) and buffer width around the catchment.
Table 3: Cohen’s kappa for the quality of event prediction (0-1), best values highlighted in green
As shown in Table 3, there are significant differences in the quality of the simulations between the different test sites, while the choice of precipitation scenario has a moderate influence on the forecast quality, except in the Rio Moscardo. The best results are obtained in Gröbentalbach and Cancia. A similar picture emerged from the analyses of the (modified) hit rate and the probability of occurrence (probability in case of alarm that it is no false alarm).
With the exception of one event (Bettelwurfmure), the analysis of the individual events showed consistently good temporal accuracy for correctly predicted events, but moderate (Gröbentalbach) to very low qualitiy in terms of volume. The precipitation hydrographs generated by INCA mostly agree quiet well in their course with the values of precipitation measured by the rain gauges, however the absolute values may deviate significantly from each other. The loss of quality due to increasing forecast periods could not be analysed because the prediction-data sets were not available.
Across the evaluations, the INADEF prototype in Gröbentalbach and Cancia test sites was able to achieve good hit rates right from the start, while in the other test sites – probably due to insufficiently accurate input data – recalibration is required.
Recommendations for the implementation of the early warning system
The prototype developed within INADEF has the potential to provide high-quality early warnings for debris flows. It is a cost-effective and flexible alternative or supplement to technical protection measures that heavily affects the landscape. The prototype is currently being supervised by the University of Udine. However, before the INADEF early warning system is set up, the following questions have to be clarified:
- Primary event trigger: Due to its structure, the prototype can only predict debris flows triggered by (high) channel discharges (mostly in the course of convective precipitation events). Events due to landslides, snowmelt, etc. cannot be detected.
- Available information: the permanent “just in time” availability of high-quality precipitation forecasts is essential, as the accuracy of the event-prediction depends on it. These forecast models require high-quality weather radar data as well as nearby meteorological stations that can be integrated.
- On site requirements and expectations: Several model specifications have to be clarified on site according to the needs. E.g. the question of risk tolerance: should the INADEF EWS detect as many events as possible, even though a higher number of false alarms is generated, or is the highest possible number of correct forecasts desired, even if more events are not detected? A parallel calculation of two scenarios (e.g. a sensitive one that detects as many potential hazardous situations as possible (message event possible) and a less sensitive one (message event probable)) is possible with moderate approach-modifications.
- Calibration: Not least because INADEF is an EWS-prototype it lacks on empirical values. Thus calibration during operation is advised to achieve good (improved) prediction qualities. A cost-effective, optically based event documentation method (detection via camera and reflectors) was developed within the project.
- Organisation and support: Issues ranging from system maintenance to the elaboration and implementation of alarm plans must be addressed. Setting up several areas together increases the efficiency (cost/benefit) of the system.