bayes

Slide1: 

A Tale of Missing H-Bombs, Google, and the USS Scorpion Rob Huber

Slide2: 

“If something can be installed backward, it will be.” – John Craven [Chief Scientist, U.S. Navy Special Projects]

Slide3: 

The USS Scorpion Beliefs Intuition Hypothesis Theory Hunches A Hydrogen Bomb What Do These Things Have In Common? Science Probability Molecular Kinetics Google Las Vegas

Slide4: 

Bayes’ Theorem Of Subjective Probability

Slide5: 

Bayesian Inference A statistical inference in which probabilities are interpreted not as frequencies or proportions or the like, but rather as degrees of belief. A formalization of the scientific method involving collecting evidence that points towards or away from a given hypothesis.

Slide6: 

Evidence Original Belief (Subjective Probability) Modified Belief Hypothesis Theory

Slide7: 

1966 A B-52 collides with an air tanker off the coast of Palomares, Spain, causing loss of atomic payload. Three of four H-bombs recovered quickly. Fourth H-bomb remains missing. US / USSR both search for the missing H-bomb. US enlists the aid of John Craven, regarded as Navy’s deep-ocean expert. Craven organizes a group of mathematicians and designs a search for the missing H-bomb based upon the theorem of subjective probability developed by Thomas Bayes (b. 1760). All available hypothesis are pooled and assigned probability values (bets). A probability grid of the Mediterranean region of interest is constructed. The grid is searched systematically, and probabilities revised with information obtained during the search. The H-bomb was recovered using this method.

Slide8: 

1968 USS Scorpion goes missing in May. Two listening posts record a series of apparent explosions, but the data is insufficient to pinpoint the missing submarine. Analysis of the explosion pattern leads Craven to believe that the submarine was traveling East, not West as was expected. If true, the wrong area was being searched. As before, Craven devises a probability grid based on the available data. The grid is a a merger of two probability distributions: The probability of the submarine being at a given location. The probability that the submarine would be found at location X, given that it was actually there (a function of depth and topography). One week after the search turned to the area predicted by Craven, the submarine was located.

Slide9: 

1975 & Beyond Although initially greeted with considerable skepticism, the interpretation of Bayes’ Theorem developed by John Craven in the 1960’s was later adopted by other branches of the military. Theory of Optimal Search is published by the Operations Research Society of America (Spooky guys!). The U.S. Coast Guard adopts the method for search and rescue operations. The Navy uses the method to locate and clear sunken ordinance in the Suez Canal.

Slide10: 

Marbles Wal-Mart sells two kinds of bags of marbles: (1) Bags of all black marbles, and (2) Bags of mixed marbles in which 20% of the marbles are black. The bags are opaque and wrapped in plastic, and I have no idea which bag is more common. I buy a bag and figure there is a 50:50 chance that the bag I purchased contains all black marbles. A guess! I pull a marble out of the bag and see that it is black. How should this new evidence affect the 50:50 assessment I assigned to the probability of my having purchased an all black bag of marbles?

Slide11: 

Marbles Prior Belief There is a 50% chance that I have an all-black bag of marbles … a guess. 50% chance of all-black (100%) marble bag. 50% chance of 20% black marble bag. Posterior Belief Probability that my bag of marbles is all black = 83.3%.

Slide12: 

Marbles Prior Belief 83 % 83 % chance of all-black (100%) marble bag. 17 % chance of 20% black marble bag. New Belief 96 % I put the marble back, shake the bag, and draw another marble. It is black? What happens now that my new prior probability is 83%? Remember, I don’t know which type of marble bag is most popular … Wal-Mart may have 100 bags of mixed marbles on the shelf for every bag of all black marbles. Bayes’ Theorem doesn’t tell me the probability of my marble bag being all black – it only tells me how I should revise my initial best guess based on the newly obtained information.

Slide13: 

Concrete Evidence 1st Black Marble Original Belief I shrug my shoulders and guess is that there is a 50 % chance that my bag contains all black marbles. Modified Belief Increased to 83 % Probability Concrete Evidence 2nd Black Marble Modified Belief Increased to 96 % Certainty

Slide14: 

The Skeptic, Agnostic, and Zealot start with differing levels of belief, but approach consensus as new information is made available.

Slide15: 

Popper: A theory can never be absolutely confirmed. [i.e. proven for all cases to which the theory applies] A theory may be decisively disconfirmed if it makes a prediction that turns out to be false. Bayes’ Theorem proves valuable in the confirmation of positive instances. The instant a working theory is disconfirmed, by drawing a green marble out of the bag, for example, all confidences plunge to zero.

Slide16: 

The Skeptic, Agnostic, and Zealot each get a green marble on the 4th draw. All confidences immediately go to zero.

Slide17: 

Medical Tests & The False Positive Crunk … The latest drug fad! It is estimated that 0.1 % of the population is addicted to Crunk. A test for Crunk is devised. The test returns a positive 99 % of the time for Crunk users. The test returns a negative 95 % of the time for non-users. (i.e. five in a hundred non-users test positive) Bob Black tests positive for Crunk. What is the probability that Bob is a Crunk user?

Slide18: 

Medical Tests & The False Positive Crunk … The Probability of a True Positive … 5 % of the Crunk-free population test positive for the drug. 99 % of the Crunk-using population test positive for the drug. In other words, 98.1 % of all positive Crunk tests can be expected to be false positives, and so Bob is most likely drug free.

Slide19: 

Medical Tests & The False Positive Improving the Crunk test. 0.1 % of the Crunk-free population test positive for the drug. 99 % of the Crunk-using population test positive for the drug. Still, over half of the positive drug tests are false positives!!! An improved test for Crunk is devised The test returns a positive 99 % of the time for Crunk users. (Same as before.) The test returns a negative 99.9 % of the time for non-users. (Much improved from the previous 95 %.) The Probability of a True Positive is now …

Slide20: 

Medical Tests & The False Positive What’s Wrong? With 1 % of the population now Crunk users, only 9 % of those who test positive for the drug are false positives. (Still sounds kind of high to me!) Only one in a thousand drug-free people get hit with a false positive. That sounds pretty good, but Crunk use is rare (also one in a thousand), so half of all those who test positive for Crunk use are actually drug free. The easiest way to improve the efficacy of the Crunk test is to increase Crunk use in the general population. Suppose Crunk use reaches “epidemic” proportions, with one in a hundred becoming addicted to the stuff …

Slide21: 

Police Dogs Two border patrol dogs sniff Kathy Macedon’s car at the U.S. – Mexico border. One of the dogs is trained to detect explosives, while the other is trained to detect Marijuana. Both dogs “alert” on the car, indicating the presence of explosives and marijuana. Is Kathy an Al Qaeda stoner, or is she merely packing a healthy supply of beef jerky?

Slide22: 

Police Dogs Marijuana use is fairly high, so let’s say 1/50 people stopped at the checkpoint have marijuana in their cars. The number of people carrying bombs across the border must surely be very low. We’ll say 1/100,000. Various sources indicate that the accuracy of substance-sniffing dogs is between 80-90 percent. One source indicates the Department of Defense has a proficiency requirement of 5 % or less for “nonproductive” alerts (false positives). For both dogs, we’ll assume 90 % accuracy when the substances are present, and 95 % when no substances are present.

Slide23: 

Police Dogs The marijuana dog scores: The bomb dog scores: We should therefore adjust our prior beliefs about Kathy to discount the notion that she is an Al Qaeda lunatic – the chances that the bomb alert was valid are practically zero. There is a 27 % chance that she is a stoner, however, so we might just want to keep an eye on her!

Slide24: 

Police Dogs Only when the concentration of contraband-containing cars is unrealistically high (e.g. 20%) in the sampled population do correct dog alerts outweigh false alerts. Can dog alerts be reasonably considered “probable cause” to search an automobile?

Slide25: 

Other Applications Juries / Courtrooms Often recommended as means of weighing DNA evidence in light of other evidence. Computational Pattern Recognition and Artificial Intelligence I was [probably] unwittingly using Bayesian inference in my Google searches on “Bayesian inference.” Alternatively, I may have been using fuzzy logic. Fuzzy logic and Bayesian inference are competing approaches that are mathematically and semantically incompatible with one another. The Wikipedia reference notes: “You cannot, in general, understand the degree of truth in fuzzy logic as probability and vice versa.”

Slide26: 

Naive Bayes Classifier Computational Classification (Search Engine Technique) New object (white circle) will be classified as red based on pooled prior probabilities. Probabilities evaluated are: Red/Green populations within total population. Red/Green populations within vicinity of new object.

Slide27: 

Naive Bayes Classifier Four Naive Bayes Modeling Techniques

Slide28: 

References Blind Man’s Bluff: The Untold Story of American Submarine Espionage Sherry Sontag & Christopher Drew Bayes’ Theorem and the Philosophy of Science (Google Hit!) Curtis Brown; Feb. 14, 2002 Bayesian Inference (Google Hit!) Wikipedia, the free encyclopedia StatSoft / Statistica Electronic Statistics Textbook http://www.statsoft.com/

Slide29: 

Note Internet searches will lead to a variety of conspiracy theories regarding the loss of the USS Scorpion. The evidence, by and large, points towards a faulty torpedo battery component. An alert of either “Hot torpedo!” (indicating a warm torpedo on the rack) or “Hot-running torpedo!” (indicating a “hot” torpedo in the tube) would have initiated a 180-degree turn of the submarine (right full rudder) to activate the torpedo’s fail-safe system. In the case of a faulty battery, such a maneuver would not have prevented the battery from continuing to heat up, leading to “cook off” of the warhead. Torpedo battery “cook-offs” had been reported on other submarines, although in those cases the crews were able to cool the torpedoes enough (by spraying them with water) that they could load them into tubes and jettison them. A battery explosion during a vibration test at the Weapons Quality Engineering Center led to a recall of the lot. The torpedo batteries aboard the USS Scorpion would have been replaced as a result of this recall, had the submarine made it back to Norfolk.