Bayes' Theorem - Math is Fun

Bayes' Theorem is a way of finding a probability when we know certain other probabilities. The formula is: P(A|B) = P(A) P(B|A)P(B) ... Advanced ShowAds HideAds AboutAds Bayes'Theorem Bayescandomagic! Everwonderedhowcomputerslearnaboutpeople? Example: Aninternetsearchfor"movieautomaticshoelaces"bringsup"Backtothefuture" Hasthesearch enginewatchedthemovie?No,butitknowsfromlotsofothersearcheswhatpeopleareprobablylookingfor. AnditcalculatesthatprobabilityusingBayes'Theorem. Bayes'Theoremisawayoffindingaprobabilitywhenweknowcertainotherprobabilities. Theformulais: P(A|B)= P(A)P(B|A)P(B) Whichtellsus:   howoftenAhappensgiventhatBhappens,writtenP(A|B), Whenweknow:   howoftenBhappensgiventhatAhappens,writtenP(B|A)     andhowlikelyAisonitsown,writtenP(A)     andhowlikelyBisonitsown,writtenP(B)   LetussayP(Fire)meanshowoftenthereisfire,andP(Smoke)meanshowoftenweseesmoke,then: P(Fire|Smoke)meanshowoftenthereisfirewhenwecanseesmoke P(Smoke|Fire)meanshowoftenwecanseesmokewhenthereisfire Sotheformulakindoftellsus"forwards"P(Fire|Smoke)whenweknow"backwards"P(Smoke|Fire) Example: dangerousfiresarerare(1%) butsmokeisfairlycommon(10%)duetobarbecues, and90%ofdangerousfiresmakesmoke WecanthendiscovertheprobabilityofdangerousFirewhenthereisSmoke: P(Fire|Smoke)=P(Fire)P(Smoke|Fire)P(Smoke) =1%x90%10% =9% Soitisstillworthcheckingoutanysmoketobesure. Example:PicnicDay Youareplanningapicnictoday,butthemorningiscloudy Ohno!50%ofallrainydaysstartoffcloudy! Butcloudymorningsarecommon(about40%ofdaysstartcloudy) Andthisisusuallyadrymonth(only3of30daystendtoberainy,or10%) Whatisthechanceofrainduringtheday? WewilluseRaintomeanrainduringtheday,andCloudtomeancloudymorning. ThechanceofRaingivenCloudiswrittenP(Rain|Cloud) Solet'sputthatintheformula: P(Rain|Cloud)=P(Rain)P(Cloud|Rain)P(Cloud) P(Rain)isProbabilityofRain=10% P(Cloud|Rain)isProbabilityofCloud,giventhatRainhappens=50% P(Cloud)isProbabilityofCloud=40% P(Rain|Cloud)=0.1x0.50.4 =.125 Ora12.5%chanceofrain.Nottoobad,let'shaveapicnic! Just4Numbers Imagine100peopleataparty,andyoutallyhowmanywearpinkornot,andifamanornot,andgetthesenumbers: Bayes'Theoremisbasedoffjustthose4numbers! Letusdosometotals: Andcalculatesomeprobabilities: theprobabilityofbeingamanisP(Man)=40100=0.4 theprobabilityofwearingpinkisP(Pink)=25100=0.25 theprobabilitythatamanwearspinkisP(Pink|Man)=540=0.125 theprobabilitythatapersonwearingpinkisamanP(Man|Pink)=...   Andthenthepuppyarrives!Suchacutepuppy.   Butallyourdataisrippedup!Only3valuessurvive: P(Man)=0.4, P(Pink)=0.25and P(Pink|Man)=0.125 CanyoudiscoverP(Man|Pink)? Imagineapink-wearingguestleavesmoneybehind...wasitaman?WecananswerthisquestionusingBayes'Theorem: P(Man|Pink)=P(Man)P(Pink|Man)P(Pink) P(Man|Pink)=0.4×0.1250.25=0.2 Note:ifwestillhadtherawdatawecouldcalculatedirectly525=0.2 BeingGeneral Whydoesitwork? Letusreplacethenumberswithletters: Nowletuslookatprobabilities.Sowetakesomeratios: theoverallprobabilityof"A"isP(A)=s+ts+t+u+v theprobabilityof"BgivenA"isP(B|A)=ss+t Andthenmultiplythemtogetherlikethis:   NowletusdothatagainbutuseP(B)andP(A|B):   Bothwaysgetthesameresultofss+t+u+v Sowecanseethat: P(B)P(A|B)=P(A)P(B|A) Niceandsymmetricalisn'tit? Itactuallyhastobesymmetricalaswecanswaprowsandcolumnsandgetthesametop-leftcorner. AnditisalsoBayesFormula...justdividebothsidesbyP(B): P(A|B)=P(A)P(B|A)P(B) Remembering Firstthink"ABABAB"thenremembertogroupitlike:"AB=ABA/B" P(A|B)=P(A)P(B|A)P(B) CatAllergy? OneofthefamoususesforBayesTheoremisFalsePositivesandFalseNegatives. Forthosewehavetwopossiblecasesfor"A",suchasPass/Fail(orYes/Noetc) Example:AllergyorNot? Huntersayssheisitchy.ThereisatestforAllergytoCats,butthistestisnotalwaysright: Forpeoplethatreallydohavetheallergy,thetestsays"Yes"80%ofthetime Forpeoplethatdonothavetheallergy,thetestsays"Yes"10%ofthetime("falsepositive") If1%ofthepopulationhavetheallergy,andHunter'stestsays"Yes", whatarethechancesthatHunterreallyhastheallergy? Wewanttoknowthechanceofhavingtheallergywhentestsays"Yes",writtenP(Allergy|Yes) Let'sgetourformula: P(Allergy|Yes)=P(Allergy)P(Yes|Allergy)P(Yes) P(Allergy)isProbabilityofAllergy=1% P(Yes|Allergy)isProbabilityoftestsaying"Yes"forpeoplewithallergy=80% P(Yes)isProbabilityoftestsaying"Yes"(toanyone)=??% Ohno!Wedon'tknowwhatthegeneralchanceofthetestsaying"Yes"is... ...butwecancalculateit byaddingupthosewith,andthosewithouttheallergy: 1%havetheallergy,andthetestsays"Yes"to80%ofthem 99%donothavetheallergy andthetestsays"Yes"to10%ofthem Let'saddthatup: P(Yes)=1%×80%+99%×10%=10.7% Whichmeansthatabout10.7%ofthepopulationwillgeta"Yes"result. Sonowwecancompleteourformula: P(Allergy|Yes)=1%×80%10.7% =7.48% P(Allergy|Yes)=about7% ThisisthesameresultwegotonFalsePositivesandFalseNegatives. InfactwecanwriteaspecialversionoftheBayes'formulajustforthingslikethis: P(A|B)=P(A)P(B|A)P(A)P(B|A)+P(notA)P(B|notA) "A"WithThree(ormore)Cases Wejustsaw"A"withtwocases(AandnotA),whichwetookcareofinthebottomline. When"A"has3ormorecasesweincludethemallinthebottomline: P(A1|B)=P(A1)P(B|A1)P(A1)P(B|A1)+P(A2)P(B|A2)+P(A3)P(B|A3)+...etc Example:TheArtCompetitionhasentriesfromthreepainters:Pam,PiaandPablo Pamputin15paintings,4%ofherworkshavewonFirstPrize. Piaputin5paintings,6%ofherworkshavewonFirstPrize. Pabloputin10paintings,3%ofhisworkshavewonFirstPrize. WhatisthechancethatPamwillwinFirstPrize? P(Pam|First)=P(Pam)P(First|Pam)P(Pam)P(First|Pam)+P(Pia)P(First|Pia)+P(Pablo)P(First|Pablo) Putinthevalues: P(Pam|First)=(15/30)×4%(15/30)×4%+(5/30)×6%+(10/30)×3% Multiplyallby30(makescalculationeasier): P(Pam|First)=15×4%15×4%+5×6%+10×3% =0.60.6+0.3+0.3 =50% Agoodchance! Pamisn'tthemostsuccessfulartist,butshedidputinlotsofentries. Now,backtoSearchEngines. SearchEnginestakethisideaandscaleitupalot(plussomeothertricks). Itmakesthemlookliketheycanreadyourmind! Itcanalsobeusedformailfilters,musicrecommendationservicesandmore.     FalsePositivesandFalseNegatives ConditionalProbability Probability DataIndex Copyright©2020MathsIsFun.com