Monday, June 23, 2008

The Documentary Hypothesis, Torah Min Hashamayim, and Bayes' Theorem - The Basics

There are two competing theories for the origins of the Torah: (1) Moses wrote it under God's guidance or dictation (often called "Torah from Heaven" or "Torah Min Hashamayim" or simply "TMH") and (2) multiple authors well after Moses' time wrote separate parts, and these were later compiled and edited into one document (the "Documentary Hypothesis" or "DH"). There are many arguments on all sides of this issue. (For what its worth, I think the DH is correct, and I will be examining the arguments for and against this in future posts. It is hard to beat the job that Little Foxling has done on his blog, although I will be approaching the problem from a slightly different perspective.)

But now I would like to focus on a logically prior issue: as we gather and consider evidence and arguments for and against each theory, how do we go about deciding which theory is correct? There's a lot of muddled thinking on this issue, and people recklessly throw around terms like "presumptions" and "burden of proof" here.

I think the answer lies in Bayes' Theorem, a well-known idea from probability theory. This theory tells us, in a rigorous way, how new information changes the relative probability of two separate and competing theories. And there are many implications of this theorem for the TMH / DH debate. For example, this theory explains why different people might reasonably reach different conclusions, even though they are looking at the same evidence and arguments.

The math is not that hard, although for these purposes the broader implications of Bayes' theorem are what's really important, not the technical mathematical ideas. After all, none of the TMS/DH claims can be given precise probability values. So if math isn't your "thing", don't worry. Skim the math, read the text, and you'll be fine.

Here's the basic problem Bayes' theorem addresses. Suppose you start with an initial estimate of the probability that a particular theory is true. Then you get new information that makes that theory more likely or less likely to be true. How does that effect your overall probability estimate?

The simplest example (and one used in just about every probability book) involves jars of colored marbles. So here goes. Suppose you have 100 (opaque and identical) jars filled with marbles. There are 50 "Type Y" jars (for "yellow") that have 90 yellow marbles and 10 green marbles. And there are 50 "Type "H" jars that have half-and-half: 50 green marbles and 50 yellow marbles.

Suppose you pick a jar at random. What is the probability you have a Type Y jar? That one shouldn't be too hard: 50%. (If you missed that one, you'll have to stay and clean the erasers.)

Now, here's the good part. You randomly pick one marble from your mystery jar (without looking inside), and it is a yellow marble. Great. 90% of the marbles in Type Y jars are yellow, but only 50% of the marbles in Type H jars are yellow. This new information --- your yellow marble --- is more consistent with a Type Y jar than a Type H jar. It is still possible you have an H jar, but it now seems more likely that you have a Y jar. How much did this new information shift your initial 50% chance of having a Y jar?

OK. Here comes the math. (Feel free to skim this part.) Here's Bayes' Theorem, with the explanation to follow.

P(A|B) = P(B|A) * P(A) / P(B)

P(something) means the probability of that something occurring.

A is the "prior event." In our example, it is "picking a Type Y jar" It is "prior" in the sense of not taking into account the later information.

B is the "posterior event." In our example, it is drawing out one yellow marble.

The vertical line | means "given that".

So this theorem in English says

The probability of prior even A given posterior event B is: the probability of B given A, times the probability of A, divided by the probability of B.

Lets see how this works out with our example.

P(A) is the prior or initial probability of getting a Type Y jar. That's 50% (That one is easy.)

P(B|A) is the probability of drawing out the yellow marble given that we picked a Type Y jar. That's easy too. There are 100 marbles in a Type Y jar, 90 of which are yellow. So it is 90%. Note that P(B|A) is the reverse of what we want to find, P(A|B). And that's where the power of Bayes' theorem comes in. We cannot easily figure out the latter, but we usually can figure out the former. Bayes' theorem tells us how to use the information we have to figure out what we don't have.

P(B) is the overall probability of drawing a yellow marble. Well, there are 10,000 marbles total (100 jars with 100 marbles each). Of those, 4,500 are yellow in Type Y jars (50 jars x 90 per jar), and 2,500 are yellow in Type H jars (50 jars x 50 per jar). So 7,000 of 10,000 marbles are yellow, and so P(B) = 70%.

Now, plugging all this in gives P(A|B) = (.9 * .5) / .7, which equals about 64.29%. Cool. We stated with a 50% chance of a Type Y jar, drew out one yellow marble, and now have a 64% chance of a Type Y jar.

Suppose we replaced the marble, shook the jar, drew a second marble, and it was yellow. The probability of 2 yellow marbles in a Type Y jar is 90% of 90% or 81%. But the probability of 2 yellow marbles in a Type H jar is 50% of 50%, or 25%. If we plug all this in, we get the probability of a Type Y jar given 2 yellow marbles is just over 76%. (I leave the math to the interested reader.)

So as we keep drawing marbles, the odds would continue to shift up or down, depending on whether we drew a yellow or green marble. As we draw lots of marbles, if our sample starts approaching 90% yellow, we become very likely to have picked a Type Y jar. And if our sample starts approaching 50% yellow, we become very likely to have picked a Type H jar.

Let me make one more quantitative observation. Suppose we start with a very unlikely initial probability estimate. To revise our example, suppose there are only 5 Type Y jars and 95 Type H jars. The initial odds are very small (only 5%) that we picked a Type Y jar. But if we keep drawing yellow marbles, those odds will go up dramatically and eventually get close to 100%.

I'll end this post here, and start the qualitative implications in the next post. Please comment in the next post only.

blog comments powered by Disqus