: What is Bayesian Statistics

I've now worked my way through a fair part of Statistics : A Bayesian Perspective by Berry, and thought it was time to step back from the mechanics of calculation and see what my understanding of Bayesian statistics actually was.

-          How does Bayesian statistics differ from Frequent statistics
-          What are the differences in practice, for a student
-          Is it possible / reasonable to combine both schools.

Some comments, not necessarily in any order:

1.       Bayesian statistics specifically acknowledges that statistical research does not exist in a vacuum : the same data can lead different researchers to different conclusions. Bayesian statistics also makes explicit the learning process in research : each statistical study builds on what has gone before.

2.       Generalised Bayes Rule : separation of prior information and current data.

3.       Is the difference more one of philosophy, which doesn't have to impact one at a practical level. Is it the case that Bayesian statistics is "just" more explicit about "how we come to know things about the world".

4.       Bayesian statistics formalises the evidentiary value of observations : people can change their beliefs in a rational way when confronted with new information. In fact, Bayesian statistics provides a method to assign numerical values to changing beliefs. If the prior probability is based on prior statistical studies, the comparing  prior and posterior probabilities shows (and measures) the effect of the data.

5.       Both Frequentist and Bayesian statistics calculate a confidence ( or Probability)  interval : to this student they seem very similar

6.       A Bayesian analysis with a flat prior seems very similar to the Frequentist model.

7.       Sample size matters in both models. But is the Bayesian model more valuable with small samples, where prior knowledge can formally inform the analysis.

8.       Bayesian statistics doesn't include the frequentist "null hypothesis significance testing"  that is becoming so outmoded (and reasonably so – for a science that is all about information, the significance test provides so much less information than a confidence interval.

9.       Most of the explanations of Bayesian statistics use very artificial  examples to compare and contrast it with Frequentist statistics. I'd like to see an example that compares a study with real world data

10.   In some ways, Bayesian analysis seems to have similarities with meta-analysis (http://en.wikipedia.org/wiki/Meta-analysis  ) : The first meta-analysis was performed by Karl Pearson in 1904, in an attempt to overcome the problem of reduced statistical power in studies with small sample sizes; analyzing the results from a group of studies can allow more accurate data analysis

11.   Need to better understand what likelihoods are.

12.   Berry has an example (example 7.11) that involves defects in the manufacture of glass. The probability of a glass pane breaking due to a manufacturing defect is 5%; however if a glass breakage is due to a manufacturing defect, then the probability of other breakages in glass from the same batch being due to manufacturing defects rises to 95%  :  I'm sure that in the Frequent real world, that sort of prior information is taken account of.

13.   Is it necessary to spend some much time explaining probability theory as an introduction ?

14.   What a pity the stock standard statistics computer software (S-Plus, SPSS) does not allow for Bayesian analysis.  The spreadsheet models used in Berry are ok, but do not allow for detailed analysis. Winbugs looks difficult (and I am still to learn how to use R)