| Statistical hypothesis testing is ad hoc in two ways. First, setting probabilistic rejection criteria is, as Neyman (1957) put it, an act of will rather than an act of rationality. Second, physical theories like conservation laws do not inherently admit probabilistic predictions, and so we must use what are called epistemic bridge principles to connect model predictions with the actual methods of hypothesis testing. In practice, these bridge principles are likelihood functions, error functions, or performance metrics.
I propose that the reason we are faced with these problems is because we have historically failed to account for a fundamental component of basic logic – namely the portion of logic that explains how epistemic states evolve in the presence of empirical data. This component of Cox’ (1946) calculitic logic is called information theory (Knuth, 2005), and adding information theory our hypothetico-deductive account of science yields straightforward solutions to both of the above problems.
This also yields a straightforward method for dealing with Popper’s (1963) problem of verisimilitude by facilitating a quantitative approach to measuring process isomorphism. In practice, this involves data assimilation. Finally, information theory allows us to reliably bound measures of epistemic uncertainty, thereby avoiding the problem of Bayesian incoherency under misspecified priors (Grünwald, 2006).
I therefore propose solutions to four of the fundamental problems inherent in both hypothetico-deductive and/or Bayesian hypothesis testing.
- Neyman (1957) Inductive Behavior as a Basic Concept of Philosophy of Science.
- Cox (1946) Probability, Frequency and Reasonable Expectation.
- Knuth (2005) Lattice Duality: The Origin of Probability and Entropy.
- Grünwald (2006). Bayesian Inconsistency under Misspecification.
- Popper (1963) Conjectures and Refutations: The Growth of Scientific Knowledge. |