 |
| Titel |
Metrics for expert judgement in volcanic hazard assessment: comparing the Cooke classical model with a new method based on individual performance likelihood |
| VerfasserIn |
F. Flandoli, E. Giorgi, W. A. Aspinall, A. Neri |
| Konferenz |
EGU General Assembly 2009
|
| Medientyp |
Artikel
|
| Sprache |
Englisch
|
| Digitales Dokument |
PDF |
| Erschienen |
In: GRA - Volume 11 (2009) |
| Datensatznummer |
250028273
|
|
|
|
|
|
| Zusammenfassung |
| Expert elicitation is a method to obtain estimates for variables of interest when data is sparse or ambiguous. A team of experts is created and each is asked to provide three values for each target variable (typically the 5% quantile, the median, and the 95% quantile). If some weight can be associated with each expert, then different opinions can be pooled to generate a weighted mean, thus providing an estimate of the uncertain variable. The key challenge is to assign a proper weight to each expert. To determine this weight empirically, the experts can be asked a set of ’seed’ questions, whose values are known by the analyst (facilitator). In this approach, the experts provide three separate quantile values for each question, and the expert’s capability of quantifying uncertainty can be evaluated. For instance, the Cooke classical model quantifies the collective scientific uncertainty through an expert scoring scheme by which weights are ascribed to individual experts on the basis of empirically determined calibration and informativeness scores obtained from a probability analysis of individual performances. In our work, we compare such a method to a new algorithm in which the calibration score is substituted by a one based on the likelihood of observing these expert performances. The simple idea behind this is that of rewarding more strongly those experts whose seed item median values are systematically closer to the true values. Given the three quantile values provided by every expert for each question, we fit a Beta distribution to each test item response, and compute the probability that the location parameter of that distribution corresponds to the real value, by chance. For each expert, the geometric mean of these probabilities is computed as the likelihood factor, L(e), of the expert, thus providing an alternative ‘calibration’ score. An information factor, I(e), is also computed as arithmetic mean of the relative entropies of the expert’s distributions with respect to the uniform distribution, and this is combined with L(e) to produce the expert’s new pooling weight. Advantages and drawbacks of the two methods are illustrated with specific application to the assessment of volcanic hazard at Vesuvius and Campi Flegrei. |
|
|
|
|
|
|