Insights on Statistical Practice and Reduction from Andrew Hartley's Presentation

Discussion of Andrew Hartleys Presentation Russ Wolfinger, PhD CSIDG, JSM August 1, 2011 1

Strong Agreement on Foundations Core beliefs Christian faith s relevance to all of life, including statistical practice The PLI Myth of religious neutrality Dangers of reduction These provide us with a very fertile common ground on which to discuss and wrestle with issues. Focus: Reduction and the Bayes versus frequentist debate 2

Slide from Andrew Creation & Some PLI principles Science Humans experience the world in various aspects Aspect = a kind of properties & laws. PLI identifies the following aspects: 1. quantitative 2. spatial 3. kinetic 4. biotic 5. emotional 6. logical 7. historical / formative 8. symbolic 9. social 10. economic 11. aesthetic 12. moral 13. legal 14. Certitudinal Each science (economics, physics, biology, ) focuses on 1 or more aspect(s) This is abstraction, lifting out a part from the whole fruit Concentration on select aspects facilitates new insights & knowledge 3

Hartleys Hypotheses 1. Frequentist methods are very prone to misinterpretation and reductive thinking, usually to the degree that they cannot be trusted 2. Bayesian methods provide protection against common errors and are more in line with PLI principles 3. When frequentist methods work well in practice it is because they approximate a valid Bayesian approach 4

Reduction Elevating one or more aspects to divine status; a form of idolatry Violates PLI principle of sphere sovereignty Explains most philosophical isms and the violent debates between them over the centuries Andrew highlights some dangerous influences of humanism 5

Slide from Andrew Reduction in Statistical Inference Examples BF Skinner: When a statistician speaks of experimental design, he means designing experiments he can analyze with his methods. RA Fisher: Statistical methods are essential to social studies, and it is principally by the aid of such methods that these studies may be raised to the rank of sciences. The statistician is the guardian of scientific rigor. These attitudes illustrate the general concept of idolatrous reduction in science. 6

Replies The Skinner and Fisher quotes are not necessarily reductionistic, but could be interpreted as such if the quantitative and logical aspects are granted divine status. From Dennis Lindley: The grand assertion is that you must see the world through probability and that probability is the only guide you need. (The Rules of Probability, p.66) This quote appears to be unambiguously reductionistic, and shows that Bayesians are not immune from its dangers. 7

Slide from Andrew Humanism & Associated Overviews PLI analyzes some non-Christian religions, including humanism, & their effects on science Humanism s central claim: Man is self-sufficient 2 overviews of reality ( Ground-motives ) are consistent with humanism; each idolizes a self-sufficient area of humanness as the path to truth / prosperity / happiness Science Ground-motive: Reduces meaning to logic / math Truth founded only on logic / math; all else is mere opinion A spirit of the Enlightenment / Modernism Personality Ground-motive: Reduces meaning to self-expression / personal freedom Universal truth may not exist; if it does, we cannot know it In any case, what matters is personal truth A spirit of the Romantic Period / Post-modernism (Rousseau s Noble Savage ) I have tried to show: Each Ground-motive sets its own bounds for statistical schools-of-thought We MAY have time for discussing 8

Slide from Andrew Humanism & Science Ground-motive Example of Reduction in Statistical Inference P=0.03, so a relationship probably exists between eating Big Macs and developing cancer. Reduces certitudinal & biological functioning to quantitative functioning, as if the data alone could tell us what to believe (without consideration of the pre-scientific, such as prior plausibility of a relationship) about a biological law We all know that stats rules do not justify such a statement; however, such statements are ubiquitous (textbooks, papers, software user manuals, ) ---BUT--- People follow what they love, not what they know (St Augustine of Hippo) A cause for celebration? (not so fast!): Supreme Court Finds Statistical Significance Is Not Necessary for Causation (JSM Session 438 on Wednes) 9

Replies Agree p-values are often misinterpreted and can be used reductively. Automobile accident analogy P-values, in spite of their vulnerabilities and somewhat awkward construction, are monotonically related to evidence and are a very powerful and simple tool for statistical communication. 10

Slide from Andrew Humanism & Personality Ground-motive Example of Reduction in Statistical Inference P=0.08>0.05, but we can conclude H is false, since a p-value is subject to interpretation. Reduces certitudinal functioning to feeling (a functioning in the sensory aspect) Before we calculated P, we knew it was subject to interpretation & would give us freedom to conclude what we wish about H Calculating P has allowed us to take advantage of that, without explaining that we could have produced other, more restrictive (prescriptive) statistical results These 2 examples (of the 2 GMs) pre-suppose idolatrous relations between quantitative, certitudinal & other aspects Part of the Christian task: Identify Biblically-consistent relations between those aspects 11

Replies Of course p-values provide freedom for interpretation, as does any summary statistic. This freedom can also be used responsibly and properly respecting all PLI aspects, not reductively. P-values should certainly be interpreted in an appropriate scientific context, which does not necessarily require construction of prior probability distributions. 12

Slide from Andrew Redemption & Some PLI Principles for Statistical Inference Satisfaction in God reduces our impulse to idolize, e.g., logic/math or our feelings/preferences Sketch of Biblically consistent statistical inference a) Immersion (to the extent possible) in field of application & b) Data collection / analysis & c) Synthesis of all relevant considerations all help to ensure Justifiable (quantitative) degrees of (certitudinal) beliefs Maxims for statistical inference in a Humanist World Positive: Statistical Inference should enhance pre-analytic experience, but not replace it. The data alone cannot indicate what to believe or what to do Negative: Inference places limitations on beliefs. We are not free to believe whatever we want 13

Replies Agree with positive maxim, but disagree with negative, because we are in fact free to believe whatever we want regardless of statistical methodology. Statistical analyses in general are prone to being overtly or tacitly twisted to match religious presuppositions. 14

Another Possible Form of Bayesian Reduction The Bayesian approach requires mapping the certitudinal sphere into a quantitative/logical probability calculus, constructed classically via betting behavior (leveraging the economic aspect). Tacit transfer from aleatory to epistemic probability sigma algebras. Do beliefs necessarily follow Kolmogorov s axioms? The mapping can certainly be approximately correct, but what if relevant beliefs are omitted (intentionally or unintentionally)? Numerous relevant beliefs can often be found by asking questions within each aspect. 15

Hampel Frequentist approach that claims to exploit a proper fiducial argument. Even Fisher got it wrong in the end. Uses betting arguments to create interpretable results without priors. 16

Kass Statistical Inference: The Big Picture Statistical Science, 26(1), 2011. Statistical Pragmatism: acknowledges usefulness of both Bayesian and frequentist methods. Real World (Data) <--> Theoretical World (Scientific Models and Statistical Models) instead of Population <--> Data 17

Conclusions The PLI is uber compelling. Neither frequentist nor Bayesian methods will save you from reductionistic religious beliefs. Frequentist methods, when used properly, may even align better to the PLI than Bayesian ones. 18