Bayesian probability theory

[Rascolnikova]

Wish I knew more to start a thread with, but here goes. . .

I just had a long conversation with a physicist in which he explained to me how Bayes theorem (http://en.wikipedia.org/wiki/Bayes%27_theorem) isn't applied, usually, in the classical statistics. It's sort of mind boggling, actually. .. I don't really get the math, thought I haven't had a close look at it yet, but just from the descriptions it really makes me wonder how the application of this would change the practice--and perception--of science.

I'm looking forward to reading this book (http://omega.math.albany.edu:8008/JaynesBook.html) on it shortly.

Does anyone here know more about this?

[jake williams]

Bayes is win.

[Sean]

This (http://yudkowsky.net/rational/bayes)is a good explanation I came across about a year or so ago, and bookmarked. Especially as someone who tinkers with code, I got really excited about it when I first discovered and understood it. Some spam filters use Bayesian reasoning. There are lots of little magic tricks like this in math that seem counterintuitive yet work astonishingly well. Rasco, I'd suggest checking out my link. :)

[MarxSchmarx]

I don't really get the math, thought I haven't had a close look at it yet, but just from the descriptions it really makes me wonder how the application of this would change the practice--and perception--of science.

It will be astounding. Bayesian statistics will allow science to be a more self-consciously learning exercise rather than a mere accumulation of curious facts. It promises to unite scientific theories with data in the process of scientific discovery in a way that frequentism cannot.

As far as the perception of science goes, Bayesianism is far more intuitive and "user-friendly" than frequentist statistics. In fact, I regard the latter as a relic of the days before computing. No doubt Bayesian approaches to teaching statistics will become dominant in an academic generation or two. Having said this, I am a frequentist in practice, because the Bayesian paradigm is probably a couple of decades from being the "industry standard".

Once it does, it will provide incredible promise to how the next generation of scientists can contribute to human understanding.

Still, Jammoe is right. Bayesianism FTW

[Rosa Lichtenstein]

Its weaknesses are explored from a Marxist angle here:

Miller, R. Fact And Method (Princeton University Press, 1987).

Incidentally, this is the single best book on the Philosophy of Science, written from a Marxist angle, you will find anywhere on the planet.

[jake williams]

Its weaknesses are explored from a Marxist angle here:

Miller, R. Fact And Method (Princeton University Press, 1987).

Incidentally, this is the single best book on the Philosophy of Science, written from a Marxist angle, you will find anywhere on the planet.
ONLY Rosa would have a Marxist critique of Bayesian probability. Could you summarize the critique? Or is there a summary somewhere I could read?

[Rosa Lichtenstein]

It's years since I read it; if I get a chance I will try to summarise it. But, don't hold your breath, since I am in the middle of finishing a 120,000 word Essay before the deadline I have set myself: the end of January.

You can find a list of some of the weaknesses of this theory, here:

http://plato.stanford.edu/entries/epistemology-bayesian/#ObjSimPriConRulInfOthObjBayConThe

And here's a discussion article that runs through several criticisms, and attempts to answer them:

http://www.stat.columbia.edu/~gelman/research/published/badbayesresponsemain.pdf

Check out the reference to Miller in this list:

http://homepages.wmich.edu/~mcgrew/bayes.htm

Here is one comment:


Philosophers of science are no longer hypnotized by the hypothetico-deductive method, and while some quarters persist in thinking inductive reasoning can and should be formalized, the prospects for this seem rather dim (See John D. Norton's essays on induction). More recently, Bayesian reasoning has been the norm. As Richard Miller explains, Bayesian reasoning represents the latest incarnation of positivist fantasy, "an excess of formalism in which truisms about likelihood (plausibility, simplicity, and so forth) are given one-sided readings and abstract results are developed at too far a remove from the problems to be solved." Miller more than plausibly avers that this falling head over heals for such formalism is a consequence of “the triumph and prestige of the physical sciences, or ingrained ways of thinking in a highly monetary society, or both…” Nicholas Rescher likewise tries to account for this "penchant for quantities," this "fetish for measurement:" "People incline to think that if something significant is to be said, then you can say it with numbers and thereby transmute it into a meaningful measurement. They endorse Lord Kelvin’s dictum that ‘When you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind.’ But when one looks at the issue more clearly and critically, one finds there is no convincing reason to think this is so on any universal and pervasive basis."

While applicable to stochastic systems (as probabilistic analysis of games of chance), Bayesianism has been stretched in application to belief, “based on the principle that belief comes in degrees, usually numerical, and is governed by a calculus modeled more or less closely on the probability calculus" (Norton). Bayesianism has all the formalist pretensions of deductive reasoning, for “if there is one account of induction that does aspire to be the universal and final account, it is the Bayesian account” (Norton). Richard Miller, John D. Norton, John Earman, and the late L. Jonathan Cohen, provide reason enough to be skeptical of this latest adventure in “scientific imperialism,” that is, “the tendency for a successful scientific idea to be applied far beyond its original home, and generally with decreasing success, the more its application is extended” (John Dupré). Recall that in philosophy, Pragmatism began as a revolt against formalism: "This revolt against formalism is not a denial of the utility of formal models in certain contexts; but it manifests itself in a sustained critique of the idea that formal models, in particular, systems of symbolic logic, rule books of inductive logic, formalizations of scientific theories, etc.—describe a condition to which rational thought can or should aspire” (Hilary Putnam). To paraphrase and quote again from Putnam, our conceptions of rationality cast a net far wider than all that can be scientized, logicized, mathematized, in short, formalized: “The horror of what cannot be methodized is nothing but method fetishism.”


http://money-law.blogspot.com/2007/08/mathematics-of-truecolor-and-what-it.html

You can find and download John Norton's objections to Bayesianism here:

http://www.pitt.edu/~jdnorton/homepage/cv.html#PSA_2008


Manuscripts

"There are No Universal Rules for Induction," Prepared for Symposium “Induction Without Rules” PSA 2008: Philosophy of Science Biennial Meeting, November 2008, Pittsburgh PA. Download latest version.

"Deductively Definable Logics of Induction." Download draft.
For a less formal development, see "What Logics of Induction are There?" in Goodies.

“Challenges to Bayesian Confirmation Theory,” Prepared for Prasanta S. Bandyopadhyay and Malcolm Forster (eds.), Philosophy of Statistics: Vol. 7 Handbook of the Philosophy of Science. Elsevier. Download draft.

"Is There an Independent Principle of Causality in Physics?" Earlier version on philsci-archive.pitt.edu; latest version.

"History of Science and the Material Theory of Induction: Einstein's Quanta, Mercury's Perihelion." Download.

"Induction without Probabilities." Download.

Norton is Professor of The History and Philosophy of Science at the University of Pittsburg.

Worth reading:

http://www.pitt.edu/~jdnorton/Goodies/I_without_P/index.html

http://www.pitt.edu/~jdnorton/papers/Challenges.pdf

And the non-Marxist book on this is John Earman's 'Bayes or Bust':

http://www.amazon.com/Critical-Examination-Bayesian-Confirmation-Theory/dp/0262050463