Science: Chaos and a piece of rope, Free Energy(?), Maxwell's Deamon, New periodic table, Nu periodic table, Nu periodic table, Occram's razor, Poetry in mathematics, Prince Rupert drops, Rules of science, Science of conjecture, String Theory, Directory
******
Many who have been introduced to Statistics know a little about Bayenism but few really understands what the "rule" is and how to apply it. By giving it a mathematical coat of arms, it dissuades application and worst of all, understanding. Larry Gunick has provided a simplified way of evaluating data based on Bayes' Rule.
Simply put Bayes said that probabilities should be revised when we learn more about an event. That is, if you have some initial information (data) and can find an additional relevant set of information you may be able to improve on the probability (that is likelihood) of an event occurring. The new set is a two edged sword. Typically there is an upside as to the way in which it interacts with the question at hand but also a negative side as well. To begin with, an understanding of the problem and an attempt to improve the probability that an event will occur it is first necessary to write down what it is that you understand about the problem. Then from another data set that has relevance to the problem you may be able to guess better.
[Note: this is not the same as the weatherman who says that we are due to have a tropical story because a category three or above storm hits this area of the mainland about once in twenty years, and we haven't had one lately. Or, the spokesperson for science that says that smoker's die much earlier than non smokers and their medical cost to society is greater (Without considering the additional cost of taking care of a body for twenty additional years than that of their counterpart.)]
Here are some examples before we get to the question at hand which is whether a scientific society should send out a survey to its members and expect as a result of the solicitation, that there will be an increase in meeting attendance.
a) Natural Gas Production -
Question - Should Wildcat Joe lease a section of land in Gillette, Wyoming and drill for gas?
Action - List what it is that you know, as example if you are drilling for methane gas you want to know how likely you will get production or a dry hole. To help you along, you can hire a seismic crew to come in and provide information on the likelihood of finding gas (or oil). Adding this to your own experience record will give you something to take to the bank when you go begging for money to drill.
Facts:
1) You (and the bankers) know that you have been successful 20% of the time in finding producing quantities of gas. The flip side of this is that you drill dry holes 80% of the time!
2) The seismic crew that you hire usually finds gas 90 percent of the time if it's there.
3) The crew does make errors - Some 70 % of the time they say gas isn't there and it isn't. Or said another way, 30% of the time there's gas there but they miss it.
Now we substitute information for the letters in our table.
A = Gas present (Your past luck)
B = Gas present (The seismic crews guess)
| tr>Seismic Crew | Wildcat Joe A = Gas present | Wildcat Joe Not A = Nothing here | Total |
| B = Gas here | B x A | B x Not A | |
| Not B = Gas not here | |||
| Total | 1.0 |
Now plug in some numbers:
| tr>Seismic Crew | Wildcat Joe A = Gas present | Wildcat Joe Not A = Nothing here | Total |
| B = Gas here | 0.2 X 0.9 = 0.18 | 0.8 X 0.3 = 0.24 | |
| Not B = Gas not here | |||
| Total | 0.2 | 0.8 | 1.0 |
| tr>Seismic Crew | Wildcat Joe A = Gas present | Wildcat Joe Not A = Nothing here | Total |
| B = Gas here(90%) | 0.2 X 0.9 = 0.18 | 0.8 X 0.3 = 0.24 | 0.42 |
| Not B = Gas not here (70%) | 0.02 | 0.56 | 0.58 |
| Total | 0.2 | 0.8 | 1.0 |
Voila, we now have compiled a joint probability table. And from this we can draw conclusions that we can take to the bank!
Divide the column where both agree gas is present (0.18) by the total (0.42) and we have 43%. Odds of finding gas have now improved to 43%. And by the way, that section where the seismic crew said there was little chance of finding gas; chances are only 3.4 percent that they are wrong (divide 0.02 by 0.58).
That should impress the banker.
Bayes proposed a theorem that yields the same result. It reads:
Pr[Gas Present] = (Pr[A]XPr[B]) / (Pr[A]XPr[B])+ Pr[A, not gas]X(Pr[B, not gas]
Substitute in the terms and you should get the same answer. You pays your money and takes your choice on how to determine probability [Pr], and display the results.
Note: Statisticians will be quick to point out that what you have determined is the "posterior probability" which to them means the calculation comes after, i.e., posterior, something else. Meaning that Wildcat Joe's numbers came before and it was only after he had access to the seismic crews information could he legitimately calculate the likelihood of finding gas in Gillette Wyoming. Rather than posterior which has other meanings to us common folk, let's call it the "rear-end".
2) AIDS
Caring Martha worked for a company in Maryland that developed a good but not perfect AIDS test.
Here's our standard block:
Diagnostics for AIDS
| A | Not A | Total | |
| B | |||
| Not B | |||
| Total | 1.00 |
Question: Should doctors use the test? And more important, if you test positive what should you do?
Action - List what it is that you know:
Facts:
1) In the US one of 1500 individuals has AIDS or the precursor to AIDS. (0.0007%)
[Sad to say, some recent statistics indicate the incidence may be as high as one in 300.]
2) The test correctly identifies a patient with AIDS 98% of the time.
3) The test unfortunately finds that a patient has AIDS when in fact he or she does not 5% of the time.
Let's fill in the heading blanks:
Evaluation of diagnostic for AIDS
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | P(B x A) | P(B x Not A) | |
| Not B (wrong 5%) | |||
| Total | PA | P(Not A) | 1.00 |
Now let's put in some numbers:
Diagnostics for AIDS
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 0.0007 x 0.98 | 0.9993 x (0.05) | |
| Not B (wrong 5%) | |||
| Total | 0.0007 | 0.9993 | 1.00 |
And do some calculations:
Diagnostics for AIDS
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 0.0007 x 0.98 =0.000686 | 0.9993 x (0.05) =0.049965 | |
| Not B (wrong 5%) | |||
| Total | 0.0007 | 0.9993 | 1.00 |
Then fill in the blanks:
Diagnostics for AIDS
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 0.000686 | 0.049965 | 0.050651 |
| Not B (wrong 5%) | 0.000014 | .949335 | 0.949349 |
| Total | 0.0007 | 0.9993 | 1.00 |
To analyze:
Let's find the patients that have the disease and are positive for the test:
Divide the patients with the disease by the total (0.000686/.050651) and we find that far less than one percent of those who test positive actually have the disease (0.014%). Gonick calls this a false positive paradox. This is because the majority of those who test positive actually come from the population of those who do not have the disease and are more highly represented because of the testing error.
The above table can be restated based on 1500 patients tested by multiplying all the numbers in the table by 1500:
Diagnostics for AIDS
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 1 | 75 | 76 |
| Not B (wrong 5%) | 0.02 = 0 | 1424 | 1424 |
| Total | 1 | 14991500 |
Or, of 76 positive test only one actually has AIDS. This demonstrates how poor the test was in detecting AIDS. However, If this population of 76 were subjected to a repeat of the test, the following results would be obtained.
Let's put in the new numbers:
Diagnostics for AIDS - Retesting positive group (76 individuals) with same test
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 0.0132 x 0.98 | 0.9868 x 0.05 | |
| Not B (wrong 5%) | 0.02 = 0 | 1424 | 1424 |
| Total | 0.0132 | 0.98681 |
And do some calculations:
Diagnostics for AIDS - Retesting positive group (76 individuals) with same test
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 0.0132 x 0.98 =0.012936 | 0.9868 x 0.05 =0.04934 | |
| Not B (wrong 5%) | 0.02 = 0 | 1424 | 1424 |
| Total | 0.0132 | 0.98681 |
Then fill in the blanks:
Diagnostics for AIDS - Retesting positive group (76 individuals) with same test
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 0.012936 | 0.04934 | 0.062276 |
| Not B (wrong 5%) | 0.000264 | 0.93746 | 0.949349 |
| Total | 0.0132 | 0.98681 |
Our analysis now finds when we divide those with the disease (0.012936) by the total (0.062276) and express the number as percent: the number is 21%. Again the majority of those that tested positive did not have the disease and are over represented because of the test low accuracy rate. Here's what our group of that retook the test looks like.
Diagnostics for AIDS - Retesting positive group (76 individuals) with same test
| Test | Patient
A (has disease) | Patient
Not A (doesn't have) | Total |
| B (accurate 98%) | 1 | 4 | 5 |
| Not B (wrong 5%) | 0 | 70 | 70 |
| Total | 1 | 7475 |
It's very likely that a person that has AIDS is still within the group testing positive. The good news is that of those 75 that had the scare of AIDS, 70 can rest a bit easier. Rounding accounted for the difference between the original 76 and the 75 shown here (of course death could have been the explanation.)
This particular test never made it to market.
3) Failure rate of a computer due to a faulty new chip.
Dr. Bill's advice was sought after a computer manufacturer began to have problems with a particular generic chip. They can buy the chip from a facility in California or one in Mexico. Both facilities belong to the same company with 25% of production from Mexico and 75% from California.
Engineers have determined that the new chip has a failure rate of from 3% from the Mexican facility and 5% from the California site. A batch of chips is found to be defective. If it can be demonstrated that the defective chip came from, the computer manufacturer will shift purchase to the alternative source.
Site of manufacture:
California = 75%, Mexico = 25%
A = 75% and (not A) =25%
Bcalif. = 95%, and not Bmex = 97%
Here's the box:
| A | Not A | Total | |
| B | (.75)(.95) | (.25)(.97) | |
| Not B | |||
| Total | .75 | .25 | 1.00 |
Calculate and fill in the boxes:
| A | Not A | Total | |
| B | (.75)(.95) | (.25)(.97) | .9550 |
| Not B | .0375 | .0075 | .0450 |
| Total | .75 | .25 | 1.00 |
Calculating the percentage (decimal equivalent): .7125/.9550 =0.7461
Dr. Bill concludes that the faulty chip had a 75% probability of coming from the California site. Further, the chip manufacturer uses a fluorocarbon wash in one facility and a liquid carbon dioxide rinse at the other. The fluorocarbon removes 95 percent of the impurities and the CO2 removes 60 percent. Does this add any further information which permits determination of from which site the defective chip came?
40% made in one factory(a) with a defect rate of 0.25%
60% made in other factory with a defect rate of 0.35%
Factory b uses supercritical carbon dioxide flush which removes 60% of the impurities
Other factory (a) uses fluorocarbon wash to remove 95% of impurities
From preceding analysis:
A = 0.69, not A = 0.31
fluorocarbon = 0.95
Not fluorocarbon = 0.60
| A | Not A | Total | |
| B | (.69)(0.95)= .6555 | (.31)(0.60) = .1860 | .9550 |
| Not B | .0375 | .0075 | .0450 |
| Total | .69 | .31 | 1.00 |
Or,
| A | Not A | Total | |
| B | .6555 | .1860 | .8415 |
| Not B | .0345 | .1240 | .1585 |
| Total | .69 | .31 | 1.00 |
Dr. Bill's analysis:
.6555/.8415 = 77.90 probability that defective chips came from plant A. It is likely that the cause of the defective chip manufacture is the use of the less thorough washing action of the carbon dioxide.
Should the computer manufacturer insist that plant a switch back to the fluorocarbon wash?
d) Membership and involvement in society meetings.
Surveying Sid has sold a society on the benefits of doing member surveys to increase participation in local meetings. This was an easy sell since the society has been routinely mailing out newsletters announcing meetings to its 800 members and yet only 8 show up (of which 4 are officers). Sid ran a pilot survey of fifty members and got a 20% response rate. While 7 said they would like to more actively participate, 3 asked to be removed from the mailing list.
Based on this response, should the society make a mass mailing of the survey to all its members?
What is known:
A 800 members and 1% response rate to newsletter (the number who show up at meetings).
B. 50 members of which 7 (14%) say they will participate and 3 (6%) say no. The other 80% did not respond to the survey. The positive response rate is 14% and the negative 86% (6% + 80%)
Here's Bayes' boxes:
A: 1% positive response (attend meetings)
not A: 99% response (do not attend meetings)
B: 14% say they will participate
not B: 86% negative or no response
| A | Not A | Total | |
| B | (.01)(.14)=.0014 | (.99)(.86)=.8514 | |
| Not B | |||
| Total | .01 | .99 | 1.00 |
| A | Not A | Total | |
| B | .0014 | .8514 | .8528 |
| Not B | .0086 | .1386 | .1472 |
| Total | .01 | .99 | 1.00 |
Surveying Sid concludes based on Baye's box: .0014/.8528 = 0.16%
Or, the 8 (0.1%) who have been attending meetings, will be joined by an additional 5 to make a total of 13 (0.16%). Since this more than doubles the number that will be participating (not including officers), Sid wants to go ahead with the full member survey. What do you think?
Here's an alternative analysis:
What is known:
A 800 members and 1% response rate to newsletter (the number who show up at meetings).
B. 50 members of which 7 (14%) say they will participate and 3 (6%) say no. The other 80% did not respond to the survey.
Here's Baye's boxes:
A: 1% positive response (attend meetings)
not A: 99% response (do not attend meetings)
B: 14% say they will participate
not B: 6% negative
| A | Not A | Total | |
| B | .01 x .14= .0014 | .99 x .06= .0594 | |
| Not B | |||
| Total | .01 | .99 | 1.00 |
| A | Not A | Total | |
| B | .0014 | .0594 | .0608 |
| Not B | .0086 | .9306 | .9392 |
| Total | .01 | .99 | 1.00 |
Sid's conclusion: 0.0014/.0608 = 0.230 or 2.3%. If the survey is mailed to all those in the society, attendance will improve as 2.3 percent of the members will become involved. Of 800 members, 18 may attend the meetings. Not great but that's 12 more than current.
Should the society fund a complete survey of its membership?
****
Thomas Bayes' work was published post humorously (according to some statisticians), in 1763. Those puzzled by statistics can well appreciate Larry Gonick's book, Cartoon Guide to Statistics, published by Harper Collins in 1993, which provides the box-guidelines for making a Bayes' analysis in this essay.
****
Kevin Murphy at http://www.ai.mit.edu/~murphyk/Bayes/bayesrule.html provided the following:.
Here is a simple introduction to Bayes 'rule from an article in the Economist (9/30/00).
"The essence of the Bayesian approach is to provide a mathematical rule explaining how you should change your existing beliefs in the light of new evidence. In other words, it allows scientists to combine new data with their existing knowledge or expertise. The canonical example is to imagine that a precocious newborn observes his first sunset, and wonders whether the sun will rise again or not. He assigns equal prior probabilities to both possible outcomes, and represents this by placing one white and one black marble into a bag. The following day, when the sun rises, the child places another white marble in the bag. The probability that a marble plucked randomly from the bag will be white (ie, the child's degree of belief in future sunrises) has thus gone from a half to two-thirds. After sunrise the next day, the child adds another white marble, and the probability (and thus the degree of belief) goes from two-thirds to three-quarters. And so on. Gradually, the initial belief that the sun is just as likely as not to rise each morning is modified to become a near-certainty that the sun will always rise."
Murphy concludes:
Bayes theorem and, in particular, its emphasis on prior probabilities has caused considerable controversy. The great statistician Ronald Fisher was very critical of the ``subjectivist'' aspects of priors. By contrast, a leading proponent I. J. Good argued persuasively that ``the subjectivist (i.e. Bayesian) states his judgements, whereas the objectivist sweeps them under the carpet by calling assumptions knowledge, and he basks in the glorious objectivity of science''.
To which can be added that far too many scientist consider collected data as facts, not recognizing that the relevance depends greatly on the circumstances under which they were collected and the biases that have been imposed. The biologist approach of successive approximations yielding estimates of reality, remains the best method of approach. Or perhaps Dave Barry had it right -- if it has a decimal point, it must be true.
****
Joe Wortham's Home Page, About Joe Wortham, Directory of Web Pages
Questions? Comments? [email protected]