BBC Home

Explore the BBC

Front Page

Life | The Universe | Everything | Advanced Search
 
Front PageReadTalkContributeHelp!FeedbackWho is Online

Click here to complete your registration.

 
3. Everything / Maths, Science & Technology / Mathematics

Bayes' Theorem and Bayesian Statistics

Bayes' Theorem. Bayesian Statistics. You may have heard of them. But if you're not a statistician or mathematician, you probably don't have much idea what they are or why anybody cares. So, for you, here is a summary.

Reverend Thomas Bayes

Reverend Thomas Bayes lived from 1702 to 1761. The son of a nonconformist minister, he was privately educated and went on to become a Presbyterian minister in Tunbridge Wells in 1731, where he remained until he died. After his death his papers, including mathematical works, were found by his friend Richard Price and submitted for publication. His paper An Essay Towards Solving a Problem in the Doctrine of Chances was published in 17631 and included his form of what was to become known as Bayes' Theorem.

Statement of the Theorem

Here will follow the simplest statement of the theorem. First, though, a summary of the terminology:

  • P(A) - The probability that event A will happen. The event can be anything, such as winning the lottery or finding a parking space. The probability is the chance it will happen, from 0 to 1. 0 means that it will never happen, and 1 means that it will always happen. Most events are somewhere in between.

  • P(A,B) - The probability that both events A and B occur. This is called the 'joint probability'.

  • P(A|B) - This is the probability that event A occurs given we already know that B has occurred. This is called the 'conditional probability'.

Our simple Bayes' Theorem states that:

P(A|B) = P(A,B) / P(B)

In words, it says that the probability of A given B equals the probability of A and B divided by the probability of B.

Use of the Theorem

That's all fine and dandy, but why is it important? Well, the theorem relates three probabilities, and if two of them are known it is possible to find the other. This may not seem to be much, but it underpins just about every form of prediction or forecasting. It provides a way of calculating the probability of an event it's not possible to observe directly.

If event A is defined as 'what we want to predict' and event B as 'what we have just observed', then A|B becomes 'what we want to predict, in light of what we have just observed' and Bayes' Theorem provides a way of updating the probability for A as new observations trickle in. This is why Bayes' Theorem is so powerful.

Bayesian Statistics

Traditional statistics typically estimate something by taking a big bag full of data, performing some kind of function on it and arriving at an answer. This is perfectly adequate as far as it goes. Sometimes this means that if new data arrive the calculation has to be performed again from scratch. It also means that care must be taken to ensure that the data are 'fair', because if there is some bias in them then the answer will be wrong. Even if the statistician knows that the data are biased, often the estimates of the bias will not be independent of the model and so can't be used sensibly in the model. If data are sparse, or if there are no data at all, then traditional methods don't usually give very good answers.

Step forward Bayesian Statistics. This method starts with an answer (or 'prior') and a measure of certainty regarding this answer. When new data arrive they are incorporated into the new answer (the 'posterior'2). The answer is updated using Bayes' theorem: the new probability 'given' the new data. With more data, the uncertainty regarding the prediction diminishes and following predictions improve. Not only does this incorporate new data by design, but it can give reasonable predictions from the very start. Even if the initial answer is wayward, performance quickly improves as data flow in. It is also possible to account for bias or incorporate information from outside the system in updating it. This makes the method even more powerful.

Apart from an impressive battery of statistical methods3, Bayesian statistics are also found in neural nets, other machine learning (such as radar tracking) and decision analysis.

Controversy Rages

Some statisticians have problems with Bayesian Statistics. The two main concerns are as follows:

  • How do we arrive at the starting 'answer'? - It is not always obvious what should be chosen as the starting point. If a woefully inadequate one is picked, the early predictions will be poor. It is also difficult to establish how certain to be about it. If the model is too certain then the predictions won't adapt to new data quickly enough, but if the model is not certain enough then the early predictions have too big a margin of error to be useful.

  • Bayesian probabilities are subjective - meaning that the probabilities are based on a personal belief about the universe. One person's probability that a sports team will win can vary wildly from another's. The fact that both can be considered 'correct' by Bayesian statistics is enough to put some people off4. It is more comfortable to think of an objective probability that is 'true' and definite for everyone. From a practical standpoint, however, probabilities are used to help people make decisions, and these decisions are personal and are wrapped up in subjective values anyway. Another way to look at it is to consider the results as the model's beliefs.

Disclaimer

As you can probably tell, this was written by a Bayesian statistician and so is not entirely without bias. Feel free to form your own opinions about it.

Further Reading

  • You can read the original paper in a variety of formats from this list of important essays.

  • Unless you're a maths professional, the details about the subject will probably be a little dry5. You can find out more information in primers in statistics and academic journals. These aren't the most friendly of sources, but academics have to keep their jobs safe, after all.


1 Philos Trans R Soc London 1763; 53: 370-418. If you don't know what any of this means, consider yourself lucky.
2 Stop that sniggering.
3 The most colourfully named of which is 'Markov Chain Monte Carlo'.
4 Chinese philosophers, they say, will have no problems. Good for them.
5 Unless statements such as: Yt=FTθt+vt; vt~N[0,V] really excite you.

Discuss this Entry  People have been talking about this Guide Entry. Here are the most recent Conversations:

Fuzzy logic
(Last Posting: Sep 30, 2002)

I thought...
(Last Posting: Sep 17, 2002)

Spam filtering
(Last Posting: Sep 17, 2002)

Nice one!
(Last Posting: Sep 14, 2002)




Add your Opinion!

There are tens of thousands of h2g2 Guide Entries, written by our Researchers. If you want to be able to add your own opinions to the Guide, simply become a member as an h2g2 Researcher. Tell me More!

 
Entry Data
Entry ID: A801695 (Edited)

Written and Researched by:
Queex Quimwrangler (Not Egon)

Edited by:
Danny B


Date: 13   September   2002


Text only
Like this page?
Send it to a friend


Referenced Guide Entries
Mathematics
Parallel Parking
The Lottery
Artificial Neural Networks
What do Probabilities Mean?
How to Be a Philosopher


Referenced Sites
List of Important Essays

Please note that the BBC is not responsible for the content of any external sites listed.

Most of the content on this site is created by h2g2's Researchers, who are members of the public. The views expressed are theirs and unless specifically stated are not those of the BBC. The BBC is not responsible for the content of any external sites referenced. In the event that you consider anything on this page to be in breach of the site's House Rules, please click here to alert our Moderation Team. For any other comments, please start a Conversation below.
 


Front PageReadTalkContributeHelp!FeedbackWho is Online

Most of the content on h2g2 is created by h2g2's Researchers, who are members of the public. The views expressed are theirs and unless specifically stated are not those of the BBC. The BBC is not responsible for the content of any external sites referenced. In the event that you consider anything on this page to be in breach of the site's House Rules, please click here. For any other comments, please start a Conversation above.


About the BBC | Help | Terms of Use | Privacy & Cookies Policy