 |
| Titel |
Bayesian Analysis Diagnostics: Diagnosing Predictive and Parameter Uncertainty for Hydrological Models |
| VerfasserIn |
Mark Thyer, Dmitri Kavetski, Guillaume Evin, George Kuczera, Ben Renard, David McInerney |
| Konferenz |
EGU General Assembly 2015
|
| Medientyp |
Artikel
|
| Sprache |
Englisch
|
| Digitales Dokument |
PDF |
| Erschienen |
In: GRA - Volume 17 (2015) |
| Datensatznummer |
250114187
|
| Publikation (Nr.) |
 EGU/EGU2015-14750.pdf |
|
|
|
|
|
| Zusammenfassung |
| All scientific and statistical analysis, particularly in natural sciences, is based on
approximations and assumptions. For example, the calibration of hydrological models using
approaches such as Nash-Sutcliffe efficiency and/or simple least squares (SLS) objective
functions may appear to be “assumption-free”. However, this is a naïve point of view, as SLS
assumes that the model residuals (residuals=observed-predictions) are independent,
homoscedastic and Gaussian. If these assumptions are poor, parameter inference and model
predictions will be correspondingly poor. An essential step in model development
is therefore to verify the assumptions and approximations made in the modeling
process. Diagnostics play a key role in verifying modeling assumptions. An important
advantage of the formal Bayesian approach is that the modeler is required to make the
assumptions explicit. Specialized diagnostics can then be developed and applied to test and
verify their assumptions. This paper presents a suite of statistical and modeling
diagnostics that can be used by environmental modelers to test their modeling calibration
assumptions and diagnose model deficiencies. Three major types of diagnostics are
presented:
Residual Diagnostics
Residual diagnostics are used to test whether the assumptions of the residual error model
within the likelihood function are compatible with the data. This includes testing for
statistical independence, homoscedasticity, unbiasedness, Gaussianity and any distributional
assumptions.
Parameter Uncertainty and MCMC Diagnostics
An important part of Bayesian analysis is assess parameter uncertainty. Markov
Chain Monte Carlo (MCMC) methods are a powerful numerical tool for estimating
these uncertainties. Diagnostics based on posterior parameter distributions can be
used to assess parameter identifiability, interactions and correlations. This provides
a very useful tool for detecting and remedying model deficiencies. In addition,
numerical diagnostics are provided to test the convergence of the MCMC sampling
chains.
Diagnostics for Probabilistic Predictions
Quantifying predictive uncertainty is becoming a standard part of the modeling
process. However, simply providing probability limits on the predictions provides little
information on the reliability of these estimates. A series of methods are presented
to verify and quantify predictive reliability, resolution and accuracy. A series of
hydrological modeling case studies are used to demonstrate the use of these diagnostics for
testing statistical and modeling assumptions and diagnosing model deficiencies.
Guidance is given on the interpretation of these diagnostics. The practical implications
of poor modeling assumptions is highlighted. Recommendations are provided on
the general methodologies for improving the modeling assumptions and reducing
modeling deficiencies. The suite of diagnostics is available as an R package, enabling
modelers to apply them to their own model development and application endeavours. |
|
|
|
|
|
|