Bayesian statistics - Wikipedia

Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability where probability expresses a degree of ... Bayesianstatistics FromWikipedia,thefreeencyclopedia Jumptonavigation Jumptosearch Theoryinthefieldofstatistics PartofaseriesonBayesianstatistics Theory Admissibledecisionrule Bayesianefficiency Bayesianepistemology Bayesianprobability Probabilityinterpretations Bayes'theorem Bayesfactor Bayesianinference Bayesiannetwork Prior Posterior Likelihood Conjugateprior Posteriorpredictive Hyperparameter Hyperprior Principleofindifference Principleofmaximumentropy EmpiricalBayesmethod Cromwell'srule Bernstein–vonMisestheorem Schwarzcriterion Credibleinterval Maximumaposterioriestimation Radicalprobabilism Techniques Bayesianlinearregression Bayesianestimator ApproximateBayesiancomputation MarkovchainMonteCarlo IntegratednestedLaplaceapproximations  Mathematicsportalvte BayesianstatisticsisatheoryinthefieldofstatisticsbasedontheBayesianinterpretationofprobabilitywhereprobabilityexpressesadegreeofbeliefinanevent.Thedegreeofbeliefmaybebasedonpriorknowledgeabouttheevent,suchastheresultsofpreviousexperiments,oronpersonalbeliefsabouttheevent.Thisdiffersfromanumberofotherinterpretationsofprobability,suchasthefrequentistinterpretationthatviewsprobabilityasthelimitoftherelativefrequencyofaneventaftermanytrials.[1] BayesianstatisticalmethodsuseBayes'theoremtocomputeandupdateprobabilitiesafterobtainingnewdata.Bayes'theoremdescribestheconditionalprobabilityofaneventbasedondataaswellaspriorinformationorbeliefsabouttheeventorconditionsrelatedtotheevent.[2][3]Forexample,inBayesianinference,Bayes'theoremcanbeusedtoestimatetheparametersofaprobabilitydistributionorstatisticalmodel.SinceBayesianstatisticstreatsprobabilityasadegreeofbelief,Bayes'theoremcandirectlyassignaprobabilitydistributionthatquantifiesthebelieftotheparameterorsetofparameters.[1][2] BayesianstatisticsisnamedafterThomasBayes,whoformulatedaspecificcaseofBayes'theoreminapaperpublishedin1763.Inseveralpapersspanningfromthelate18thtotheearly19thcenturies,Pierre-SimonLaplacedevelopedtheBayesianinterpretationofprobability.[4]LaplaceusedmethodsthatwouldnowbeconsideredBayesiantosolveanumberofstatisticalproblems.ManyBayesianmethodsweredevelopedbylaterauthors,butthetermwasnotcommonlyusedtodescribesuchmethodsuntilthe1950s.Duringmuchofthe20thcentury,Bayesianmethodswereviewedunfavorablybymanystatisticiansduetophilosophicalandpracticalconsiderations.ManyBayesianmethodsrequiredmuchcomputationtocomplete,andmostmethodsthatwerewidelyusedduringthecenturywerebasedonthefrequentistinterpretation.However,withtheadventofpowerfulcomputersandnewalgorithmslikeMarkovchainMonteCarlo,Bayesianmethodshaveseenincreasingusewithinstatisticsinthe21stcentury.[1][5] Contents 1Bayes'theorem 2OutlineofBayesianmethods 2.1Bayesianinference 2.2Statisticalmodeling 2.3Designofexperiments 2.4ExploratoryanalysisofBayesianmodels 3Seealso 4References 5Furtherreading 6Externallinks Bayes'theorem[edit] Mainarticle:Bayes'theorem Bayes'theoremisusedinBayesianmethodstoupdateprobabilities,whicharedegreesofbelief,afterobtainingnewdata.Giventwoevents A {\displaystyleA} and B {\displaystyleB} ,theconditionalprobabilityof A {\displaystyleA} giventhat B {\displaystyleB} istrueisexpressedasfollows:[6] P ( A ∣ B ) = P ( B ∣ A ) P ( A ) P ( B ) {\displaystyleP(A\midB)={\frac{P(B\midA)P(A)}{P(B)}}} where P ( B ) ≠ 0 {\displaystyleP(B)\neq0} .AlthoughBayes'theoremisafundamentalresultofprobabilitytheory,ithasaspecificinterpretationinBayesianstatistics.Intheaboveequation, A {\displaystyleA} usuallyrepresentsaproposition(suchasthestatementthatacoinlandsonheadsfiftypercentofthetime)and B {\displaystyleB} representstheevidence,ornewdatathatistobetakenintoaccount(suchastheresultofaseriesofcoinflips). P ( A ) {\displaystyleP(A)} isthepriorprobabilityof A {\displaystyleA} whichexpressesone'sbeliefsabout A {\displaystyleA} beforeevidenceistakenintoaccount.Thepriorprobabilitymayalsoquantifypriorknowledgeorinformationabout A {\displaystyleA} . P ( B ∣ A ) {\displaystyleP(B\midA)} isthelikelihoodfunction,whichcanbeinterpretedastheprobabilityoftheevidence B {\displaystyleB} giventhat A {\displaystyleA} istrue.Thelikelihoodquantifiestheextenttowhichtheevidence B {\displaystyleB} supportstheproposition A {\displaystyleA} . P ( A ∣ B ) {\displaystyleP(A\midB)} istheposteriorprobability,theprobabilityoftheproposition A {\displaystyleA} aftertakingtheevidence B {\displaystyleB} intoaccount.Essentially,Bayes'theoremupdatesone'spriorbeliefs P ( A ) {\displaystyleP(A)} afterconsideringthenewevidence B {\displaystyleB} .[1] Theprobabilityoftheevidence P ( B ) {\displaystyleP(B)} canbecalculatedusingthelawoftotalprobability.If { A 1 , A 2 , … , A n } {\displaystyle\{A_{1},A_{2},\dots,A_{n}\}} isapartitionofthesamplespace,whichisthesetofalloutcomesofanexperiment,then,[1][6] P ( B ) = P ( B ∣ A 1 ) P ( A 1 ) + P ( B ∣ A 2 ) P ( A 2 ) + ⋯ + P ( B ∣ A n ) P ( A n ) = ∑ i P ( B ∣ A i ) P ( A i ) {\displaystyleP(B)=P(B\midA_{1})P(A_{1})+P(B\midA_{2})P(A_{2})+\dots+P(B\midA_{n})P(A_{n})=\sum_{i}P(B\midA_{i})P(A_{i})} Whenthereareaninfinitenumberofoutcomes,itisnecessarytointegrateoveralloutcomestocalculate P ( B ) {\displaystyleP(B)} usingthelawoftotalprobability.Often, P ( B ) {\displaystyleP(B)} isdifficulttocalculateasthecalculationwouldinvolvesumsorintegralsthatwouldbetime-consumingtoevaluate,sooftenonlytheproductofthepriorandlikelihoodisconsidered,sincetheevidencedoesnotchangeinthesameanalysis.Theposteriorisproportionaltothisproduct:[1] P ( A ∣ B ) ∝ P ( B ∣ A ) P ( A ) {\displaystyleP(A\midB)\proptoP(B\midA)P(A)} Themaximumaposteriori,whichisthemodeoftheposteriorandisoftencomputedinBayesianstatisticsusingmathematicaloptimizationmethods,remainsthesame.Theposteriorcanbeapproximatedevenwithoutcomputingtheexactvalueof P ( B ) {\displaystyleP(B)} withmethodssuchasMarkovchainMonteCarloorvariationalBayesianmethods.[1] OutlineofBayesianmethods[edit] Thegeneralsetofstatisticaltechniquescanbedividedintoanumberofactivities,manyofwhichhavespecialBayesianversions. Bayesianinference[edit] Mainarticle:Bayesianinference Bayesianinferencereferstostatisticalinferencewhereuncertaintyininferencesisquantifiedusingprobability.[7]Inclassicalfrequentistinference,modelparametersandhypothesesareconsideredtobefixed.Probabilitiesarenotassignedtoparametersorhypothesesinfrequentistinference.Forexample,itwouldnotmakesenseinfrequentistinferencetodirectlyassignaprobabilitytoaneventthatcanonlyhappenonce,suchastheresultofthenextflipofafaircoin.However,itwouldmakesensetostatethattheproportionofheadsapproachesone-halfasthenumberofcoinflipsincreases.[8] Statisticalmodelsspecifyasetofstatisticalassumptionsandprocessesthatrepresenthowthesampledataaregenerated.Statisticalmodelshaveanumberofparametersthatcanbemodified.Forexample,acoincanberepresentedassamplesfromaBernoullidistribution,whichmodelstwopossibleoutcomes.TheBernoullidistributionhasasingleparameterequaltotheprobabilityofoneoutcome,whichinmostcasesistheprobabilityoflandingonheads.DevisingagoodmodelforthedataiscentralinBayesianinference.Inmostcases,modelsonlyapproximatethetrueprocess,andmaynottakeintoaccountcertainfactorsinfluencingthedata.[1]InBayesianinference,probabilitiescanbeassignedtomodelparameters.Parameterscanberepresentedasrandomvariables.BayesianinferenceusesBayes'theoremtoupdateprobabilitiesaftermoreevidenceisobtainedorknown.[1][9] Statisticalmodeling[edit] TheformulationofstatisticalmodelsusingBayesianstatisticshastheidentifyingfeatureofrequiringthespecificationofpriordistributionsforanyunknownparameters.Indeed,parametersofpriordistributionsmaythemselveshavepriordistributions,leadingtoBayesianhierarchicalmodeling,[10][11][12]alsoknownasmulti-levelmodeling.AspecialcaseisBayesiannetworks. ForconductingaBayesianstatisticalanalysis,bestpracticesarediscussedbyvandeShootetal.[13] ForreportingtheresultsofaBayesianstatisticalanalysis,Bayesiananalysisreportingguidelines(BARG)areprovidedinanopen-accessarticlebyJohnK.Kruschke.[14] Designofexperiments[edit] TheBayesiandesignofexperimentsincludesaconceptcalled'influenceofpriorbeliefs'.Thisapproachusessequentialanalysistechniquestoincludetheoutcomeofearlierexperimentsinthedesignofthenextexperiment.Thisisachievedbyupdating'beliefs'throughtheuseofpriorandposteriordistribution.Thisallowsthedesignofexperimentstomakegooduseofresourcesofalltypes.Anexampleofthisisthemulti-armedbanditproblem. ExploratoryanalysisofBayesianmodels[edit] ExploratoryanalysisofBayesianmodelsisanadaptationorextensionoftheexploratorydataanalysisapproachtotheneedsandpeculiaritiesofBayesianmodeling.InthewordsofPersiDiaconis:[15] Exploratorydataanalysisseekstorevealstructure,orsimpledescriptionsindata.Welookatnumbersorgraphsandtrytofindpatterns.Wepursueleadssuggestedbybackgroundinformation,imagination,patternsperceived,andexperiencewithotherdataanalyses Theinferenceprocessgeneratesaposteriordistribution,whichhasacentralroleinBayesianstatistics,togetherwithotherdistributionsliketheposteriorpredictivedistributionandthepriorpredictivedistribution.Thecorrectvisualization,analysis,andinterpretationofthesedistributionsiskeytoproperlyanswerthequestionsthatmotivatetheinferenceprocess.[16] WhenworkingwithBayesianmodelsthereareaseriesofrelatedtasksthatneedtobeaddressedbesidesinferenceitself: Diagnosesofthequalityoftheinference,thisisneededwhenusingnumericalmethodssuchasMarkovchainMonteCarlotechniques Modelcriticism,includingevaluationsofbothmodelassumptionsandmodelpredictions Comparisonofmodels,includingmodelselectionormodelaveraging Preparationoftheresultsforaparticularaudience AllthesetasksarepartoftheExploratoryanalysisofBayesianmodelsapproachandsuccessfullyperformingthemiscentraltotheiterativeandinteractivemodelingprocess.Thesetasksrequirebothnumericalandvisualsummaries.[17][18][19] Seealso[edit] Bayesianepistemology Foralistofmathematicallogicnotationusedinthisarticle Notationinprobabilityandstatistics Listoflogicsymbols References[edit] ^abcdefghiGelman,Andrew;Carlin,JohnB.;Stern,HalS.;Dunson,DavidB.;Vehtari,Aki;Rubin,DonaldB.(2013).BayesianDataAnalysis(Third ed.).ChapmanandHall/CRC.ISBN 978-1-4398-4095-5. ^abMcElreath,Richard(2020).StatisticalRethinking :ABayesianCoursewithExamplesinRandStan(2nd ed.).ChapmanandHall/CRC.ISBN 978-0-367-13991-9. ^Kruschke,John(2014).DoingBayesianDataAnalysis:ATutorialwithR,JAGS,andStan(2nd ed.).AcademicPress.ISBN 978-0-12-405888-0. ^McGrayne,Sharon(2012).TheTheoryThatWouldNotDie:HowBayes'RuleCrackedtheEnigmaCode,HuntedDownRussianSubmarines,andEmergedTriumphantfromTwoCenturiesofControversy(First ed.).ChapmanandHall/CRC.ISBN 978-0-3001-8822-6. ^Fienberg,StephenE.(2006)."WhenDidBayesianInferenceBecome"Bayesian"?".BayesianAnalysis.1(1):1–40.doi:10.1214/06-BA101. ^abGrinstead,CharlesM.;Snell,J.Laurie(2006).Introductiontoprobability(2nd ed.).Providence,RI:AmericanMathematicalSociety.ISBN 978-0-8218-9414-9. ^Lee,SeYoon(2021)."Gibbssamplerandcoordinateascentvariationalinference:Aset-theoreticalreview".CommunicationsinStatistics-TheoryandMethods.51(6):1549–1568.arXiv:2008.01006.doi:10.1080/03610926.2021.1921214.S2CID 220935477. ^Wakefield,Jon(2013).Bayesianandfrequentistregressionmethods.NewYork,NY:Springer.ISBN 978-1-4419-0924-4. ^Congdon,Peter(2014).AppliedBayesianmodelling(2nd ed.).Wiley.ISBN 978-1119951513. ^Kruschke,JK;Vanpaemel,W(2015)."BayesianEstimationinHierarchicalModels".InBusemeyer,JR;Wang,Z;Townsend,JT;Eidels,A(eds.).TheOxfordHandbookofComputationalandMathematicalPsychology(PDF).OxfordUniversityPress.pp. 279–299. ^Hajiramezanali,E.&Dadaneh,S.Z.&Karbalayghareh,A.&Zhou,Z.&Qian,X.Bayesianmulti-domainlearningforcancersubtypediscoveryfromnext-generationsequencingcountdata.32ndConferenceonNeuralInformationProcessingSystems(NIPS2018),Montréal,Canada.arXiv:1810.09433 ^Lee,SeYoon;Mallick,Bani(2021)."BayesianHierarchicalModeling:ApplicationTowardsProductionResultsintheEagleFordShaleofSouthTexas".SankhyaB.84:1–43.doi:10.1007/s13571-020-00245-8. ^vandeSchoot,Rens;Depaoli,Sarah;King,Ruth;Kramer,Bianca;Märtens,Kaspar;Tadesse,MahletG.;Vannucci,Marina;Gelman,Andrew;Veen,Duco;Willemsen,Joukje;Yau,Christopher(January14,2021)."Bayesianstatisticsandmodelling".NatureReviewsMethodsPrimers.1(1):1–26.doi:10.1038/s43586-020-00001-2.hdl:1874/415909.S2CID 234108684. ^Kruschke,JK(Aug16,2021)."BayesianAnalysisReportingGuidelines".NatureHumanBehaviour.5(10):1282–1291.doi:10.1038/s41562-021-01177-7.PMC 8526359.PMID 34400814. ^Diaconis,Persi(2011)TheoriesofDataAnalysis:FromMagicalThinkingThroughClassicalStatistics.JohnWiley&Sons,Ltd2:e55doi:10.1002/9781118150702.ch1 ^Kumar,Ravin;Carroll,Colin;Hartikainen,Ari;Martin,Osvaldo(2019)."ArviZaunifiedlibraryforexploratoryanalysisofBayesianmodelsinPython".JournalofOpenSourceSoftware.4(33):1143.Bibcode:2019JOSS....4.1143K.doi:10.21105/joss.01143. ^Gabry,Jonah;Simpson,Daniel;Vehtari,Aki;Betancourt,Michael;Gelman,Andrew(2019)."VisualizationinBayesianworkflow".JournaloftheRoyalStatisticalSociety,SeriesA(StatisticsinSociety).182(2):389–402.arXiv:1709.01449.doi:10.1111/rssa.12378.S2CID 26590874. ^Vehtari,Aki;Gelman,Andrew;Simpson,Daniel;Carpenter,Bob;Bürkner,Paul-Christian(2021)."Rank-Normalization,Folding,andLocalization:AnImprovedRˆforAssessingConvergenceofMCMC(WithDiscussion)".BayesianAnalysis.16(2).arXiv:1903.08008.doi:10.1214/20-BA1221.S2CID 88522683. ^Martin,Osvaldo(2018).BayesianAnalysiswithPython:IntroductiontostatisticalmodelingandprobabilisticprogrammingusingPyMC3andArviZ.PacktPublishingLtd.ISBN 9781789341652. Furtherreading[edit] Bernardo,JoséM.;Smith,AdrianF.M.(2000).BayesianTheory.NewYork:Wiley.ISBN 0-471-92416-4. Bolstad,WilliamM.;Curran,JamesM.(2016).IntroductiontoBayesianStatistics(3rd ed.).Wiley.ISBN 978-1-118-09156-2. Downey,AllenB.(2021).ThinkBayes:BayesianStatisticsinPython(2nd ed.).O'Reilly.ISBN 978-1-4920-8946-9. Hoff,PeterD.(2009).AFirstCourseinBayesianStatisticalMethods(2nd ed.).NewYork:Springer.ISBN 978-1-4419-2828-3. Lee,PeterM.(2012).BayesianStatistics:AnIntroduction(4th ed.).Wiley.ISBN 978-1-118-33257-3. Robert,ChristianP.(2007).TheBayesianChoice :FromDecision-TheoreticFoundationstoComputationalImplementation(2nd ed.).NewYork:Springer.ISBN 978-0-387-71598-8. Externallinks[edit] WikiversityhaslearningresourcesaboutBayesianstatistics EliezerS.Yudkowsky."AnIntuitiveExplanationofBayes'Theorem"(webpage).Retrieved2015-06-15. TheoKypraios."AGentleTutorialinBayesianStatistics"(PDF).Retrieved2013-11-03. JordiVallverdu.BayesiansVersusFrequentistsAPhilosophicalDebateonStatisticalReasoning. BayesianstatisticsDavidSpiegelhalter,KennethRiceScholarpedia4(8):5230.doi:10.4249/scholarpedia.5230 Bayesianmodelingbookandexamplesavailablefordownloading. RensvandeSchoot."AGentleIntroductiontoBayesianAnalysis"(PDF). BayesianA/BTestingCalculatorDynamicYield AuthoritycontrolNationallibraries Spain France(data) Other SUDOC(France) 1 Retrievedfrom"https://en.wikipedia.org/w/index.php?title=Bayesian_statistics&oldid=1096436199" Categories:BayesianstatisticsHiddencategories:ArticleswithshortdescriptionShortdescriptionisdifferentfromWikidataArticleswithBNEidentifiersArticleswithBNFidentifiersArticleswithSUDOCidentifiers Navigationmenu Personaltools NotloggedinTalkContributionsCreateaccountLogin Namespaces ArticleTalk English Views ReadEditViewhistory More Search Navigation MainpageContentsCurrenteventsRandomarticleAboutWikipediaContactusDonate Contribute HelpLearntoeditCommunityportalRecentchangesUploadfile Tools WhatlinkshereRelatedchangesUploadfileSpecialpagesPermanentlinkPageinformationCitethispageWikidataitem Print/export DownloadasPDFPrintableversion Inotherprojects Wikiversity Languages العربيةবাংলাCatalàČeštinaCymraegDeutschEspañolFrançais한국어BahasaIndonesiaItalianoעבריתMagyarNederlands日本語РусскийTürkçeУкраїнська中文 Editlinks