Bayes Theorem - Statement, Formula, Derivation, Examples

Bayes theorem is a theorem in probability and statistics, named after the Reverend Thomas Bayes, that helps in determining the probability of an event that ... LearnPracticeDownload BayesTheorem Bayestheoremisatheoreminprobabilityandstatistics,namedaftertheReverendThomasBayes,thathelpsindeterminingtheprobabilityofaneventthatisbasedonsomeeventthathasalreadyoccurred.Bayestheoremhasmanyapplicationssuchasbayesianinterference,inthehealthcaresector-todeterminethechancesofdevelopinghealthproblemswithanincreaseinageandmanyothers.Here,wewillaimatunderstandingtheuseoftheBayestheoremindeterminingtheprobabilityofevents,itsstatement,formula,andderivationwiththehelpofexamples. 1. WhatisBayesTheorem? 2. ProofofBayesTheorem 3. BayesTheoremFormula 4. DifferencebetweenConditionalProbabilityandBayesTheorem 5. TermsRelatedtoBayesTheorem 8. FAQsonBayesTheorem WhatisBayesTheorem? Bayestheorem,insimplewords,determinestheconditionalprobabilityofaneventAgiventhateventBhasalreadyoccurred.BayestheoremisalsoknownastheBayesRuleorBayesLaw.Itisamethodtodeterminetheprobabilityofaneventbasedontheoccurrencesofpriorevents.Itisusedtocalculateconditionalprobability.Bayestheoremcalculatestheprobabilitybasedonthehypothesis.Now,letusstatethetheoremanditsproof.BayestheoremstatesthattheconditionalprobabilityofaneventA,giventheoccurrenceofanothereventB,isequaltotheproductofthelikelihoodofB,givenAandtheprobabilityofA.Itisgivenas: \(P(A|B)=\dfrac{P(B|A)P(A)}{P(B)}\) Here,P(A)=howlikelyAhappens(Priorknowledge)-Theprobabilityofahypothesisistruebeforeanyevidenceispresent. P(B)=howlikelyBhappens(Marginalization)-Theprobabilityofobservingtheevidence. P(A/B)=howlikelyAhappensgiventhatBhashappened(Posterior)-Theprobabilityofahypothesisistruegiventheevidence. P(B/A)=howlikelyBhappensgiventhatAhashappened(Likelihood)-Theprobabilityofseeingtheevidenceifthehypothesisistrue. BayesTheorem-Statement ThestatementofBayesTheoremisasfollows:Let\(E_{1},E_{2},E_{3},...,E_{n}\)beasetofeventsassociatedwithasamplespaceS,whereallevents\(E_{1},E_{2},E_{3},...,E_{n}\)havenon-zeroprobabilityofoccurrenceandtheyformapartitionofS.LetAbeanyeventwhichoccurswith\(E_{1}orE_{2}orE_{3}...orE_{n}\),thenaccordingtoBayesTheorem, \(P(E_{i}|A)=\dfrac{P(E_{i})P(A|E_{i})}{\sum_{k=1}^{n}P(E_{k})P(A|E_{k})},i=1,2,3,...,n\) HereE\(_i\)∩E\(_j\)=φ,wherei≠j.(i.e)Theyaremutuallyexhaustiveevents Theunionofalltheeventsofthepartition,shouldgivethesamplespace. 0≤P(E\(_{i}\))≤1 ProofofBayesTheorem ToprovetheBayesTheorem,wewillusethetotalprobabilityandconditionalprobabilityformulas.ThetotalprobabilityofaneventAiscalculatedwhennotenoughdataisknownabouteventA,thenweuseothereventsrelatedtoeventAtodetermineitsprobability.ConditionalprobabilityistheprobabilityofeventAgiventhatotherrelatedeventshavealreadyoccurred. (E\(_{i}\)),beisapartitionofthesamplespaceS.LetAbeaneventthatoccurred.LetusexpressAintermsof(E\(_{i}\)). A=A∩S =A∩(\(E_{1},E_{2},E_{3},...,E_{n}\)) A=(A∩\(E_{1}\))∪(A∩\(E_{1}\))∪(A∩\(E_{1}\))....∪(A∩\(E_{1}\)) P(A)=P[(A∩\(E_{1}\))∪(A∩\(E_{1}\))∪(A∩\(E_{1}\))....∪(A∩\(E_{1}\))] WeknowthatwhenAandBaredisjointsets,thenP(A∪B)=P(A)+P(B) Thushere,P(A)=P(A∩\(E_{1}\))+P(A∩\(E_{1}\))+P(A∩\(E_{1}\)).....P(A∩\(E_{n}\)) Accordingtothemultiplicationtheoremofadependentevent,wehave P(A)=P(E).P(A/\(E_{1}\))+P(E).P(A/\(E_{2}\))+P(E).P(A/\(E_{3}\))......+P(A/\(E_{n}\)) ThustotalprobabilityofP(A)=\(\sum_{i=1}^{n}P(E_{i})P(A|E_{i}),i=1,2,3,...,n\)---(1) Recallingtheconditionalprobability,weget \(P(E_{i}|A)=\dfrac{P(E_{i}\capA)}{P(A)},i=1,2,3,...,n\)---(2) Usingtheformulaforconditionalprobabilityof\(P(A|E_{i})\),wehave \(P(E_{i}\capA)=P(A|E_{i})P(E_{i})\)---(3) Substitutingequations(1)and(3)inequation(2),weget \(P(E_{i}|A)=\dfrac{P(A|E_{i})P(E_{i})}{\sum_{k=1}^{n}P(E_{k})P(A|E_{k})},i=1,2,3,...,n\) Hence,BayesTheoremisproved. BayesTheoremFormula Bayestheoremformulaexistsforeventsandrandomvariables.BayesTheoremformulasarederivedfromthedefinitionofconditionalprobability.ItcanbederivedforeventsAandB,aswellascontinuousrandomvariablesXandY.Letusfirstseetheformulaforevents. BayesTheoremFormulaforEvents Theformulaforeventsderivedfromthedefinitionofconditionalprobabilityis: \(P(A|B)=\dfrac{P(B|A)P(A)}{P(B)},P(B)\neq0\) Derivation: Accordingtothedefinitionofconditionalprobability,\(P(A|B)=\dfrac{P(A\capB)}{P(B)},P(B)\neq0\)andweknowthat\(P(A\capB)=P(B\capA)=P(B|A)P(A)\),whichimplies, \(P(A|B)=\dfrac{P(B|A)P(A)}{P(B)}\) Hence,theBayestheoremformulaforeventsisderived. BayesTheoremforContinuousRandomVariables TheformulaforcontinuousrandomvariablesXandYderivedfromthedefinitionoftheconditionalprobabilityofcontinuousvariablesis: \(f_{X|Y=y}(x)=\dfrac{f_{Y|X=x}(y)f_{X}(x)}{f_{Y}(y)}\) Derivation: Accordingtothedefinitionofconditionaldensityorconditionalprobabilityofcontinuousrandomvariables,weknowthat\(f_{X|Y=y}(x)=\dfrac{f_{X,Y}(x,y)}{f_{Y}(y)}\)and\(f_{Y|X=x}(y)=\dfrac{f_{X,Y}(x,y)}{f_{X}(x)}\),whichimplies, \(f_{X|Y=y}(x)=\dfrac{f_{Y|X=x}(y)f_{X}(x)}{f_{Y}(y)}\) Hence,theBayesTheoremformulaforrandomcontinuousvariablesisderived. DifferenceBetweenConditionalProbabilityandBayesTheorem ConditionalProbability BayesTheorem ConditionalProbabilityistheprobabilityofaneventAthatisbasedontheoccurrenceofanothereventB. BayesTheoremisderivedusingthedefinitionofconditionalprobability.TheBayestheoremformulaincludestwoconditionalprobabilities. Formula:\(P(A|B)=\dfrac{P(A\capB)}{P(B)}\) Formula:\(P(A|B)=\dfrac{P(B|A)P(A)}{P(B)}\) TermsRelatedtoBayesTheorem AswehavestudiedaboutBayestheoremindetail,letusunderstandthemeaningsofafewtermsrelatedtotheconceptwhichhavebeenusedintheBayestheoremformulaandderivation: ConditionalProbability-ConditionalProbabilityistheprobabilityofaneventAbasedontheoccurrenceofanothereventB.ItisdenotedbyP(A|B)andrepresentstheprobabilityofAgiventhateventBhasalreadyhappened. JointProbability-Jointprobabilitymeasurestheprobabilityoftwomoreeventsoccurringtogetherandatthesametime.FortwoeventsAandB,itisdenotedby\(P(A\capB)\). RandomVariables-Randomvariableisareal-valuedvariablewhosepossiblevaluesaredeterminedbyarandomexperiment.Theprobabilityofsuchvariablesisalsocalledtheexperimentalprobability. PosteriorProbability-Posteriorprobabilityistheprobabilityofaneventthatiscalculatedafteralltheinformationrelatedtotheeventhasbeenaccountedfor.Itisalsoknownasconditionalprobability. PriorProbability-Priorprobabilityistheprobabilityofaneventthatiscalculatedbeforeconsideringthenewinformationobtained.Itistheprobabilityofanoutcomethatisdeterminedbasedoncurrentknowledgebeforetheexperimentisperformed. ImportantNotesonBayesTheorem Bayestheoremisusedtodetermineconditionalprobability. WhentwoeventsAandBareindependent,P(A|B)=P(A)andP(B|A)=P(B) ConditionalprobabilitycanbecalculatedusingtheBayestheoremforcontinuousrandomvariables. ☛AlsoCheck: Probabilityandstatistics   BayesTheoremExamples Example1:Amyhastwobags.BagIhas7redand2blueballsandbagIIhas5redand9blueballs.Amydrawsaballatrandomanditturnsouttobered.DeterminetheprobabilitythattheballwasfromthebagIusingtheBayestheorem. Solution:LetXandYbetheeventsthattheballisfromthebagIandbagII,respectively.AssumeAtobetheeventofdrawingaredball.Weknowthattheprobabilityofchoosingabagfordrawingaballis1/2,thatis, P(X)=P(Y)=1/2 Sincethereare7redballsoutofatotalof11ballsinthebagI,therefore,P(drawingaredballfromthebagI)=P(A|X)=7/11 Similarly,P(drawingaredballfrombagII)=P(A|Y)=5/14 WeneedtodeterminethevalueofP(theballdrawnisfromthebagIgiventhatitisaredball),thatis,P(X|A).TodeterminethiswewilluseBayesTheorem.UsingBayestheorem,wehavethefollowing: \(P(X|A)=\dfrac{P(A|X)P(X)}{P(A|X)P(X)+P(A|Y)P(Y)}\) =[((7/11)(1/2))/(7/11)(1/2)+(5/14)(1/2)] =0.64 Answer:Hence,theprobabilitythattheballisdrawnisfrombagIis0.64 Example2:Assumethatthechancesofapersonhavingaskindiseaseare40%.Assumingthatskincreamsanddrinkingenoughwaterreducestheriskofskindiseaseby30%andprescriptionofacertaindrugreducesitschanceby20%.Atatime,apatientcanchooseanyoneofthetwooptionswithequalprobabilities.Itisgiventhatafterpickingoneoftheoptions,thepatientselectedatrandomhastheskindisease.FindtheprobabilitythatthepatientpickedtheoptionofskinscreamsanddrinkingenoughwaterusingtheBayestheorem. Solution:AssumeE1:Thepatientusesskincreamsanddrinksenoughwater;E2:Thepatientusesthedrug;A:Theselectedpatienthastheskindisease P(E1)=P(E2)=1/2 Usingtheprobabilitiesknowntous,wehave P(A|E1)=0.4×(1-0.3)=0.28 P(A|E2)=0.4×(1-0.2)=0.32 UsingBayesTheorem,theprobabilitythattheselectedpatientusesskincreamsanddrinksenoughwaterisgivenby, \(P(E1|A)=\dfrac{P(A|E1)P(E1)}{P(A|E1)P(E1)+P(A|E2)P(E2)}\) =(0.28×0.5)/(0.28×0.5+0.32×0.5) =0.14/(0.14+0.16) =0.47 Answer:Theprobabilitythatthepatientpickedthefirstoptionis0.47 Example3:Amanisknowntospeakthetruth3/4times.Hedrawsacardandreportsitisking.Findtheprobabilitythatitisactuallyaking. Solution: LetEbetheeventthatthemanreportsthatkingisdrawnfromthepackofcards Abetheeventthatthekingisdrawn Bbetheeventthatthekingisnotdrawn. ThenwehaveP(A)=probabilitythatkingisdrawn=1/4 P(B)=probabilitythatkingisdrawn=3/4 P(E/A)=Probabilitythatthemansaysthetruththatkingisdrawnwhenactuallykingisdrawn=P(truth)=3/4 P(E/B)=Probabilitythatthemanliesthatkingisdrawnwhenactuallykingisdrawn=P(lie)=1/4 ThenaccordingtoBayestheorem,theprobabilitythatitisactuallyaking=P(A/E) =\(\dfrac{P(A)P(E|A)}{P(A)P(E|A)+P(B)P(E|B)}\) =[1/4×3/4]÷[(1/4×3/4)+(1/4×3/4)] =3/16÷12/16 =3/16×16/12 =1/2=0.5 Answer:Thustheprobabilitythatthedrawncardisactuallyaking=0.5 ViewAnswer> gotoslidegotoslidegotoslide Greatlearninginhighschoolusingsimplecues Indulginginrotelearning,youarelikelytoforgetconcepts.WithCuemath,youwilllearnvisuallyandbesurprisedbytheoutcomes. BookaFreeTrialClass PracticeQuestionsonBayesTheorem     CheckAnswer>   gotoslidegotoslide FAQsonBayesTheorem WhatIsBayesTheoreminStatistics? Bayestheoremisastatisticalformulatodeterminetheconditionalprobabilityofanevent.Itdescribestheprobabilityofaneventbasedonpriorknowledgeofeventsthathavealreadyhappened.BayesTheoremisnamedaftertheReverendThomasBayesanditsformulaforrandomeventsis\(P(A|B)=\dfrac{P(B|A)P(A)}{P(B)}\) Here,P(A)=howlikelyAhappens P(B)=howlikelyBhappens P(A/B)=howlikelydoesAhappengiventhatBhashappened P(B/A)=howlikelydoesBhappengiventhatAhashappened WhatDoestheBayesTheoremState? Let\(E_{1},E_{2},E_{3},...,E_{n}\)beasetofeventsassociatedwithasamplespaceS,whereallevents\(E_{1},E_{2},E_{3},...,E_{n}\)havenon-zeroprobabilityofoccurrenceandtheyformapartitionofS.LetAbeanyeventassociatedwithS,thenaccordingtoBayesTheorem, \(P(E_{i}|A)=\dfrac{P(E_{i})P(A|E_{i})}{\sum_{k=1}^{n}P(E_{k})P(A|E_{k})},i=1,2,3,...,n\) HowtoUseBayesTheorem? TodeterminetheprobabilityofaneventAgiventhattherelatedeventBhasalreadyoccurred,thatis,P(A|B)usingtheBayesTheorem,wecalculatetheprobabilityoftheeventB,thatis,P(B);theprobabilityofeventBgiventhateventAhasoccurred,thatis,P(B|A);andtheprobabilityoftheeventAindividually,thatis,P(A).Then,wesubstitutethesevaluesintotheformula\(P(A|B)=\dfrac{P(B|A)P(A)}{P(B)}\)todeterminetheprobabilityusingtheBayesTheorem. IsBayesTheoremforIndependentEvents? IftwoeventsAandBareindependent,thenP(A|B)=P(A)andP(B|A)=P(B),thereforeBayestheoremcannotbeusedheretodeterminetheconditionalprobabilityasweneedtodeterminethetotalprobabilityandthereisnodependencyofevents. IsConditionalProbabilitytheSameasBayesTheorem? ConditionalprobabilityistheprobabilityoftheoccurrenceofaneventbasedontheoccurrenceofothereventswhereastheBayestheoremisderivedfromthedefinitionofconditionalprobability.Bayestheoremincludesthetwoconditionalprobabilities. WhatIstheBayesTheoreminMachineLearning? Bayestheoremprovidesamethodtodeterminetheprobabilityofahypothesisbasedonitspriorprobability,theprobabilitiesofobservingvariousdatagiventhehypothesis,andtheobserveddataitself.Ithelpsimmenselyingettingamoreaccurateresult.Hence,wheneverthereisaconditionalprobabilityproblem,theBayesTheoreminMachineLearningisused. ExploremathprogramMathworksheetsandvisualcurriculumBookaFREEClass