Bayesian statistics and modelling | Nature Reviews Methods ...

Bayesian statistics is an approach to data analysis and parameter estimation based on Bayes' theorem. Unique for Bayesian statistics is that all ... Skiptomaincontent Thankyouforvisitingnature.com.YouareusingabrowserversionwithlimitedsupportforCSS.Toobtain thebestexperience,werecommendyouuseamoreuptodatebrowser(orturnoffcompatibilitymodein InternetExplorer).Inthemeantime,toensurecontinuedsupport,wearedisplayingthesitewithoutstyles andJavaScript. Advertisement nature naturereviewsmethodsprimers primers article Subjects ScientificcommunityStatistics APublisherCorrectiontothisarticlewaspublishedon03February2021 Thisarticlehasbeenupdated AbstractBayesianstatisticsisanapproachtodataanalysisbasedonBayes’theorem,whereavailableknowledgeaboutparametersinastatisticalmodelisupdatedwiththeinformationinobserveddata.Thebackgroundknowledgeisexpressedasapriordistributionandcombinedwithobservationaldataintheformofalikelihoodfunctiontodeterminetheposteriordistribution.Theposteriorcanalsobeusedformakingpredictionsaboutfutureevents.ThisPrimerdescribesthestagesinvolvedinBayesiananalysis,fromspecifyingtheprioranddatamodelstoderivinginference,modelcheckingandrefinement.Wediscusstheimportanceofpriorandposteriorpredictivechecking,selectingapropertechniqueforsamplingfromaposteriordistribution,variationalinferenceandvariableselection.ExamplesofsuccessfulapplicationsofBayesiananalysisacrossvariousresearchfieldsareprovided,includinginsocialsciences,ecology,genetics,medicineandmore.Weproposestrategiesforreproducibilityandreportingstandards,outlininganupdatedWAMBS(whentoWorryandhowtoAvoidtheMisuseofBayesianStatistics)checklist.Finally,weoutlinetheimpactofBayesiananalysisonartificialintelligence,amajorgoalinthenextdecade. Thisisapreviewofsubscriptioncontent Accessoptions Accessthroughyourinstitution Changeinstitution Buyorsubscribe SubscribetoJournalGetfulljournalaccessfor1year92,52€only92,52€perissueSubscribeAllpricesareNETprices.VATwillbeaddedlaterinthecheckout.Taxcalculationwillbefinalisedduringcheckout.BuyarticleGettimelimitedorfullarticleaccessonReadCube.$32.00BuyAllpricesareNETprices. Additionalaccessoptions: Login Learnaboutinstitutionalsubscriptions Fig.1:TheBayesianresearchcycle.Fig.2:IllustrationofthekeycomponentsofBayes’theorem.Fig.3:PriorpredictivecheckingforthePhDdelayexample.Fig.4:PosteriorestimationusingMCMCforthePhD-delaysexample.Fig.5:ExamplesofshrinkagepriorsforBayesianvariableselection.Fig.6:Posteriorpredictivecheckingandpredictedfuturepageviewsbasedoncurrentobservations.Fig.7:Elementsofreproducibilityintheresearchworkflow. Changehistory03February2021ACorrectiontothispaperhasbeenpublished:https://doi.org/10.1038/s43586-021-00017-2.ReferencesBayes,M.&Price,M.LII.Anessaytowardssolvingaprobleminthedoctrineofchances.BythelateRev.Mr.Bayes,F.R.S.communicatedbyMr.Price,inalettertoJohnCanton,A.M.F.R.S.Philos.Trans.RSoc.Lond.BBiol.Sci.53,370–418(1997).ADS  MATH  GoogleScholar  Laplace,P.S.EssaiPhilosophiquesurlesProbabilities(Courcier,1814).König,C.&vandeSchoot,R.Bayesianstatisticsineducationalresearch:alookatthecurrentstateofaffairs.Educ.Rev.https://doi.org/10.1080/00131911.2017.1350636(2017).Article  GoogleScholar  vandeSchoot,R.,Winter,S.,Zondervan-Zwijnenburg,M.,Ryan,O.&Depaoli,S.AsystematicreviewofBayesianapplicationsinpsychology:thelast25years.Psychol.Methods22,217–239(2017). GoogleScholar  Ashby,D.Bayesianstatisticsinmedicine:a25yearreview.Stat.Med.25,3589–3631(2006).MathSciNet  GoogleScholar  Rietbergen,C.,Debray,T.P.A.,Klugkist,I.,Janssen,K.J.M.&Moons,K.G.M.ReportingofBayesiananalysisinepidemiologicresearchshouldbecomemoretransparent.J.Clin.Epidemiol.https://doi.org/10.1016/j.jclinepi.2017.04.008(2017).Article  GoogleScholar  Spiegelhalter,D.J.,Myles,J.P.,Jones,D.R.&Abrams,K.R.Bayesianmethodsinhealthtechnologyassessment:areview.HealthTechnol.Assess.https://doi.org/10.3310/hta4380(2000).Article  GoogleScholar  Kruschke,J.K.,Aguinis,H.&Joo,H.Thetimehascome:Bayesianmethodsfordataanalysisintheorganizationalsciences.Organ.Res.Methods15,722–752(2012). GoogleScholar  Smid,S.C.,McNeish,D.,Miočević,M.&vandeSchoot,R.Bayesianversusfrequentistestimationforstructuralequationmodelsinsmallsamplecontexts:asystematicreview.Struct.Equ.Modeling27,131–161(2019).MathSciNet  GoogleScholar  Rupp,A.A.,Dey,D.K.&Zumbo,B.D.ToBayesornottoBayes,fromwhethertowhen:applicationsofBayesianmethodologytomodeling.Struct.Equ.Modeling11,424–451(2004).MathSciNet  GoogleScholar  vandeSchoot,R.,Yerkes,M.A.,Mouw,J.M.&Sonneveld,H.Whattookthemsolong?ExplainingPhDdelaysamongdoctoralcandidates.PloSONE8,e68839(2013).ADS  GoogleScholar  vandeSchoot,R.Onlinestatstraining.Zenodohttps://zenodo.org/communities/stats_training(2020).Heo,I.&vandeSchoot,R.Tutorial:advancedBayesianregressioninJASP.Zenodohttps://doi.org/10.5281/zenodo.3991325(2020).Article  GoogleScholar  O’Hagan,A.etal.UncertainJudgements:ElicitingExperts’Probabilities(Wiley,2006).Thisbookpresentsagreatcollectionofinformationwithrespecttopriorelicitation,andincludeselicitationtechniques,summarizespotentialpitfallsanddescribesexamplesacrossawidevarietyofdisciplines.Howard,G.S.,Maxwell,S.E.&Fleming,K.J.Theproofofthepudding:anillustrationoftherelativestrengthsofnullhypothesis,meta-analysis,andBayesiananalysis.Psychol.Methods5,315–332(2000). GoogleScholar  Veen,D.,Stoel,D.,Zondervan-Zwijnenburg,M.&vandeSchoot,R.Proposalforafive-stepmethodtoelicitexpertjudgement.Front.Psychol.8,2110(2017). GoogleScholar  Johnson,S.R.,Tomlinson,G.A.,Hawker,G.A.,Granton,J.T.&Feldman,B.M.MethodstoelicitbeliefsforBayesianpriors:asystematicreview.J.Clin.Epidemiol.63,355–369(2010). GoogleScholar  Morris,D.E.,Oakley,J.E.&Crowe,J.A.Aweb-basedtoolforelicitingprobabilitydistributionsfromexperts.Environ.Model.Softw.https://doi.org/10.1016/j.envsoft.2013.10.010(2014).Article  GoogleScholar  Garthwaite,P.H.,Al-Awadhi,S.A.,Elfadaly,F.G.&Jenkinson,D.J.Priordistributionelicitationforgeneralizedlinearandpiecewise-linearmodels.J.Appl.Stat.40,59–75(2013).MathSciNet  MATH  GoogleScholar  Elfadaly,F.G.&Garthwaite,P.H.ElicitingDirichletandGaussiancopulapriordistributionsformultinomialmodels.Stat.Comput.27,449–467(2017).MathSciNet  MATH  GoogleScholar  Veen,D.,Egberts,M.R.,vanLoey,N.E.E.&vandeSchoot,R.Expertelicitationforlatentgrowthcurvemodels:thecaseofposttraumaticstresssymptomsdevelopmentinchildrenwithburninjuries.Front.Psychol.11,1197(2020). GoogleScholar  Runge,A.K.,Scherbaum,F.,Curtis,A.&Riggelsen,C.Aninteractivetoolfortheelicitationofsubjectiveprobabilitiesinprobabilisticseismic-hazardanalysis.Bull.Seismol.Soc.Am.103,2862–2874(2013). GoogleScholar  Zondervan-Zwijnenburg,M.,vandeSchoot-Hubeek,W.,Lek,K.,Hoijtink,H.&vandeSchoot,R.Applicationandevaluationofanexpertjudgmentelicitationprocedureforcorrelations.Front.Psychol.https://doi.org/10.3389/fpsyg.2017.00090(2017).Article  GoogleScholar  Cooke,R.M.&Goossens,L.H.J.TUDelftexpertjudgmentdatabase.Reliab.Eng.Syst.Saf.93,657–674(2008). GoogleScholar  Hanea,A.M.,Nane,G.F.,Bedford,T.&French,S.ExpertJudgmentinRiskandDecisionAnalysis(Springer,2020).Dias,L.C.,Morton,A.&Quigley,J.Elicitation(Springer,2018).Ibrahim,J.G.,Chen,M.H.,Gwon,Y.&Chen,F.Thepowerprior:theoryandapplications.Stat.Med.34,3724–3749(2015).MathSciNet  GoogleScholar  Rietbergen,C.,Klugkist,I.,Janssen,K.J.,Moons,K.G.&Hoijtink,H.J.Incorporationofhistoricaldataintheanalysisofrandomizedtherapeutictrials.Contemp.Clin.Trials32,848–855(2011). GoogleScholar  vandeSchoot,R.etal.BayesianPTSD-trajectoryanalysiswithinformedpriorsbasedonasystematicliteraturesearchandexpertelicitation.MultivariateBehav.Res.53,267–291(2018). GoogleScholar  Berger,J.ThecaseforobjectiveBayesiananalysis.BayesianAnal.1,385–402(2006).ThisdiscussionofobjectiveBayesiananalysisincludescriticismsoftheapproachandapersonalperspectiveonthedebateonthevalueofobjectiveBayesianversussubjectiveBayesiananalysis.MathSciNet  MATH  GoogleScholar  Brown,L.D.In-seasonpredictionofbattingaverages:afieldtestofempiricalBayesandBayesmethodologies.Ann.Appl.Stat.https://doi.org/10.1214/07-AOAS138(2008).MathSciNet  Article  MATH  GoogleScholar  Candel,M.J.&Winkens,B.PerformanceofempiricalBayesestimatorsoflevel-2randomparametersinmultilevelanalysis:aMonteCarlostudyforlongitudinaldesigns.J.Educ.Behav.Stat.28,169–194(2003). GoogleScholar  vanderLinden,W.J.Usingresponsetimesforitemselectioninadaptivetesting.J.Educ.Behav.Stat.33,5–20(2008). GoogleScholar  Darnieder,W.F.BayesianMethodsforData-DependentPriors(TheOhioStateUniv.,2011).Richardson,S.&Green,P.J.OnBayesiananalysisofmixtureswithanunknownnumberofcomponents(withdiscussion).J.R.Stat.Soc.SeriesB59,731–792(1997).MATH  GoogleScholar  Wasserman,L.Asymptoticinferenceformixturemodelsbyusingdata-dependentpriors.J.R.Stat.Soc.SeriesB62,159–180(2000).MathSciNet  MATH  GoogleScholar  Muthen,B.&Asparouhov,T.Bayesianstructuralequationmodeling:amoreflexiblerepresentationofsubstantivetheory.Psychol.Methods17,313–335(2012). GoogleScholar  vandeSchoot,R.etal.FacingoffwithScyllaandCharybdis:acomparisonofscalar,partial,andthenovelpossibilityofapproximatemeasurementinvariance.Front.Psychol.4,770(2013). GoogleScholar  Smeets,L.&vandeSchoot,R.CodefortheShinyApptodeterminetheplausibleparameterspaceforthePhD-delaydata(versionv1.0).Zenodohttps://doi.org/10.5281/zenodo.3999424(2020).Article  GoogleScholar  Chung,Y.,Gelman,A.,Rabe-Hesketh,S.,Liu,J.&Dorie,V.Weaklyinformativepriorforpointestimationofcovariancematricesinhierarchicalmodels.J.Educ.Behav.Stat.40,136–157(2015). GoogleScholar  Gelman,A.,Jakulin,A.,Pittau,M.G.&Su,Y.-S.Aweaklyinformativedefaultpriordistributionforlogisticandotherregressionmodels.Ann.Appl.Stat.2,1360–1383(2008).MathSciNet  MATH  GoogleScholar  Gelman,A.,Carlin,J.B.,Stern,H.S.&Rubin,D.B.BayesianDataAnalysisVol.2(Chapman&HallCRC,2004).Jeffreys,H.TheoryofProbabilityVol.3(Clarendon,1961).SeamanIII,J.W.,SeamanJr,J.W.&Stamey,J.D.Hiddendangersofspecifyingnoninformativepriors.Am.Stat.66,77–84(2012).MathSciNet  GoogleScholar  Gelman,A.Priordistributionsforvarianceparametersinhierarchicalmodels(commentonarticlebyBrowneandDraper).BayesianAnal.1,515–534(2006).MathSciNet  MATH  GoogleScholar  Lambert,P.C.,Sutton,A.J.,Burton,P.R.,Abrams,K.R.&Jones,D.R.Howvagueisvague?AsimulationstudyoftheimpactoftheuseofvaguepriordistributionsinMCMCusingWinBUGS.Stat.Med.24,2401–2428(2005).MathSciNet  GoogleScholar  Depaoli,S.MixtureclassrecoveryinGMMundervaryingdegreesofclassseparation:frequentistversusBayesianestimation.Psychol.Methods18,186–219(2013). GoogleScholar  Depaoli,S.&vandeSchoot,R.ImprovingtransparencyandreplicationinBayesianstatistics:theWAMBS-Checklist.Psychol.Methods22,240(2017).Thisarticledescribes,inastep-by-stepmanner,thevariouspointsthatneedtobecheckedwhenestimatingamodelusingBayesianstatistics.ItcanbeusedasaguideforimplementingBayesianmethods. GoogleScholar  vanErp,S.,Mulder,J.&Oberski,D.L.PriorsensitivityanalysisindefaultBayesianstructuralequationmodeling.Psychol.Methods23,363–388(2018). GoogleScholar  McNeish,D.OnusingBayesianmethodstoaddresssmallsampleproblems.Struct.Equ.Modeling23,750–773(2016).MathSciNet  GoogleScholar  vandeSchoot,R.&Miocević,M.SmallSampleSizeSolutions:AGuideforAppliedResearchersandPractitioners(Taylor&Francis,2020).Schuurman,N.K.,Grasman,R.P.&Hamaker,E.L.Acomparisonofinverse-Wishartpriorspecificationsforcovariancematricesinmultilevelautoregressivemodels.MultivariateBehav.Res.51,185–206(2016). GoogleScholar  Liu,H.,Zhang,Z.&Grimm,K.J.ComparisonofinverseWishartandseparation-strategypriorsforBayesianestimationofcovarianceparametermatrixingrowthcurveanalysis.Struct.Equ.Modeling23,354–367(2016).MathSciNet  GoogleScholar  Ranganath,R.&Blei,D.M.Populationpredictivechecks.Preprintathttps://arxiv.org/abs/1908.00882(2019).Daimon,T.PredictivecheckingforBayesianinterimanalysesinclinicaltrials.Contemp.Clin.Trials29,740–750(2008). GoogleScholar  Box,G.E.SamplingandBayes’inferenceinscientificmodellingandrobustness.J.R.Stat.Soc.Ser.A143,383–404(1980).MathSciNet  MATH  GoogleScholar  Gabry,J.,Simpson,D.,Vehtari,A.,Betancourt,M.&Gelman,A.VisualizationinBayesianworkflow.J.R.Stat.Soc.Ser.A182,389–402(2019).MathSciNet  GoogleScholar  Silverman,B.W.DensityEstimationforStatisticsandDataAnalysisVol.26(CRC,1986).Nott,D.J.,Drovandi,C.C.,Mengersen,K.&Evans,M.ApproximationofBayesianpredictivep-valueswithregressionABC.BayesianAnal.13,59–83(2018).MathSciNet  MATH  GoogleScholar  Evans,M.&Moshonov,H.inBayesianStatisticsanditsApplications145–159(Univ.ofToronto,2007).Evans,M.&Moshonov,H.Checkingforprior–dataconflict.BayesianAnal.1,893–914(2006).MathSciNet  MATH  GoogleScholar  Evans,M.&Jang,G.H.Alimitresultforthepriorpredictiveappliedtocheckingforprior–dataconflict.Stat.Probab.Lett.81,1034–1038(2011).MathSciNet  MATH  GoogleScholar  Young,K.&Pettit,L.Measuringdiscordancybetweenprioranddata.J.R.Stat.Soc.SeriesBMethodol.58,679–689(1996).MathSciNet  MATH  GoogleScholar  Kass,R.E.&Raftery,A.E.Bayesfactors.J.Am.Stat.Assoc.90,773–795(1995).ThisarticleprovidesanextensivediscussionofBayesfactorswithseveralexamples.MathSciNet  MATH  GoogleScholar  Bousquet,N.Diagnosticsofprior–dataagreementinappliedBayesiananalysis.J.Appl.Stat.35,1011–1029(2008).MathSciNet  MATH  GoogleScholar  Veen,D.,Stoel,D.,Schalken,N.,Mulder,K.&vandeSchoot,R.Usingthedataagreementcriteriontorankexperts’beliefs.Entropy20,592(2018).ADS  GoogleScholar  Nott,D.J.,Xueou,W.,Evans,M.&Englert,B.Checkingforprior–dataconflictusingpriortoposteriordivergences.Preprintathttps://arxiv.org/abs/1611.00113(2016).Lek,K.&vandeSchoot,R.Howthechoiceofdistancemeasureinfluencesthedetectionofprior–dataconflict.Entropy21,446(2019).ADS  MathSciNet  GoogleScholar  O’Hagan,A.Bayesianstatistics:principlesandbenefits.Frontis3,31–45(2004). GoogleScholar  Etz,A.Introductiontotheconceptoflikelihoodanditsapplications.Adv.MethodsPracticesPsychol.Sci.1,60–69(2018). GoogleScholar  Pawitan,Y.InAllLikelihood:StatisticalModellingandInferenceUsingLikelihood(OxfordUniv.Press,2001).Gelman,A.,Simpson,D.&Betancourt,M.Thepriorcanoftenonlybeunderstoodinthecontextofthelikelihood.Entropy19,555(2017).ADS  GoogleScholar  Aczel,B.etal.DiscussionpointsforBayesianinference.Nat.Hum.Behav.4,561–563(2020). GoogleScholar  Gelman,A.etal.BayesianDataAnalysis(CRC,2013).Greco,L.,Racugno,W.&Ventura,L.RobustlikelihoodfunctionsinBayesianinference.J.Stat.Plan.Inference138,1258–1270(2008).MathSciNet  MATH  GoogleScholar  Shyamalkumar,N.D.inRobustBayesianAnalysisLectureNotesinStatisticsCh.7,127–143(Springer,2000).Agostinelli,C.&Greco,L.AweightedstrategytohandlelikelihooduncertaintyinBayesianinference.Comput.Stat.28,319–339(2013).MathSciNet  MATH  GoogleScholar  Rubin,D.B.Bayesianlyjustifiableandrelevantfrequencycalculationsfortheappliedstatistician.Ann.Stat.12,1151–1172(1984).MathSciNet  MATH  GoogleScholar  Gelfand,A.E.&Smith,A.F.M.Sampling-basedapproachestocalculatingmarginaldensities.J.Am.Stat.Assoc.85,398–409(1990).ThisseminalarticleidentifiesMCMCasapracticalapproachforBayesianinference.MathSciNet  MATH  GoogleScholar  Geyer,C.J.MarkovchainMonteCarlomaximumlikelihood.IFNAhttp://hdl.handle.net/11299/58440(1991).vandeSchoot,R.,Veen,D.,Smeets,L.,Winter,S.D.&Depaoli,S.inSmallSampleSizeSolutions:AGuideforAppliedResearchersandPractitionersCh.3(edsvandeSchoot,R.&Miocevic,M.)30–49(Routledge,2020).Veen,D.&Egberts,M.inSmallSampleSizeSolutions:AGuideforAppliedResearchersandPractitionersCh.4(edsvandeSchoot,R.&Miocevic,M.)50–70(Routledge,2020).Robert,C.&Casella,G.MonteCarloStatisticalMethods(SpringerScience&BusinessMedia,2013).Geman,S.&Geman,D.Stochasticrelaxation,Gibbsdistributions,andtheBayesianrestorationofimages.IEEETrans.PatternAnal.Mach.Intell.6,721–741(1984).MATH  GoogleScholar  Metropolis,N.,Rosenbluth,A.W.,Rosenbluth,M.N.,Teller,A.H.&Teller,E.Equationofstatecalculationsbyfastcomputingmachines.J.Chem.Phys.21,1087–1092(1953).ADS  MATH  GoogleScholar  Hastings,W.K.MonteCarlosamplingmethodsusingMarkovchainsandtheirapplications.Biometrika57,97–109(1970).MathSciNet  MATH  GoogleScholar  Duane,S.,Kennedy,A.D.,Pendleton,B.J.&Roweth,D.HybridMonteCarlo.Phys.Lett.B195,216–222(1987).ADS  MathSciNet  GoogleScholar  Tanner,M.A.&Wong,W.H.Thecalculationofposteriordistributionsbydataaugmentation.J.Am.Stat.Assoc.82,528–540(1987).Thisarticleexplainshowtousedataaugmentationwhendirectcomputationoftheposteriordensityoftheparametersofinterestisnotpossible.MathSciNet  MATH  GoogleScholar  Gamerman,D.&Lopes,H.F.MarkovChainMonteCarlo:StochasticSimulationforBayesianInference(CRC,2006).Brooks,S.P.,Gelman,A.,Jones,G.&Meng,X.-L.HandbookofMarkovChainMonteCarlo(CRC,2011).ThisbookpresentsacomprehensivereviewofMCMCanditsuseinmanydifferentapplications.Gelman,A.Burn-inforMCMC,whywepreferthetermwarm-up.SatisticalModeling,CausalInference,andSocialSciencehttps://statmodeling.stat.columbia.edu/2017/12/15/burn-vs-warm-iterative-simulation-algorithms/(2017).Gelman,A.&Rubin,D.B.Inferencefromiterativesimulationusingmultiplesequences.Stat.Sci.7,457–511(1992).MATH  GoogleScholar  Brooks,S.P.&Gelman,A.Generalmethodsformonitoringconvergenceofiterativesimulations.J.Comput.Graph.Stat.7,434–455(1998).MathSciNet  GoogleScholar  Roberts,G.O.Markovchainconceptsrelatedtosamplingalgorithms.MarkovChainMonteCarloinPractice57,45–58(1996).MathSciNet  MATH  GoogleScholar  Vehtari,A.,Gelman,A.,Simpson,D.,Carpenter,B.&Bürkner,P.Rank-normalization,folding,andlocalization:animproved\(\hat{R}\)forassessingconvergenceofMCMC.Preprintathttps://arxiv.org/abs/1903.08008(2020).Bürkner,P.-C.AdvancedBayesianmultilevelmodelingwiththeRpackagebrms.Preprintathttps://arxiv.org/abs/1705.11123(2017).Merkle,E.C.&Rosseel,Y.blavaan:Bayesianstructuralequationmodelsviaparameterexpansion.Preprintathttps://arxiv.org/abs/1511.05604(2015).Carpenter,B.etal.Stan:aprobabilisticprogramminglanguage.J.Stat.Softw.https://doi.org/10.18637/jss.v076.i01(2017).Article  GoogleScholar  Blei,D.M.,Kucukelbir,A.&McAuliffe,J.D.Variationalinference:areviewforstatisticians.J.Am.Stat.Assoc.112,859–877(2017).ThisrecentreviewofvariationalinferencemethodsincludesstochasticvariantsthatunderpinpopularapproximateBayesianinferencemethodsforlargedataorcomplexmodellingproblems.MathSciNet  GoogleScholar  Minka,T.P.ExpectationpropagationforapproximateBayesianinference.Preprintathttps://arxiv.org/abs/1301.2294(2013).Hoffman,M.D.,Blei,D.M.,Wang,C.&Paisley,J.Stochasticvariationalinference.J.Mach.Learn.Res.14,1303–1347(2013).MathSciNet  MATH  GoogleScholar  Kingma,D.P.&Ba,J.Adam:amethodforstochasticoptimization.Preprintathttps://arxiv.org/abs/1412.6980(2014).Li,Y.,Hernández-Lobato,J.M.&Turner,R.E.Stochasticexpectationpropagation.Adv.NeuralInf.Process.Syst.28,2323–2331(2015). GoogleScholar  Liang,F.,Paulo,R.,Molina,G.,Clyde,M.A.&Berger,J.O.MixturesofgpriorsforBayesianvariableselection.J.Am.Stat.Assoc.103,410–423(2008).MathSciNet  MATH  GoogleScholar  Forte,A.,Garcia-Donato,G.&Steel,M.MethodsandtoolsforBayesianvariableselectionandmodelaveraginginnormallinearregression.Int.Stat.Rev.86,237–258(2018).MathSciNet  GoogleScholar  Mitchell,T.J.&Beauchamp,J.J.Bayesianvariableselectioninlinearregression.J.Am.Stat.Assoc.83,1023–1032(1988).MathSciNet  MATH  GoogleScholar  George,E.J.&McCulloch,R.E.VariableselectionviaGibbssampling.J.Am.Stat.Assoc.88,881–889(1993).Thisarticlepopularizestheuseofspike-and-slabpriorsforBayesianvariableselectionandintroducesMCMCtechniquestoexplorethemodelspace. GoogleScholar  Ishwaran,H.&Rao,J.S.Spikeandslabvariableselection:frequentistandBayesianstrategies.Ann.Stat.33,730–773(2005).MathSciNet  MATH  GoogleScholar  Bottolo,L.&Richardson,S.EvolutionarystochasticsearchforBayesianmodelexploration.BayesianAnal.5,583–618(2010).MathSciNet  MATH  GoogleScholar  Ročková,V.&George,E.I.EMVS:theEMapproachtoBayesianvariableselection.J.Am.Stat.Assoc.109,828–846(2014).MathSciNet  MATH  GoogleScholar  Park,T.&Casella,G.TheBayesianlasso.J.Am.Stat.Assoc.103,681–686(2008).MathSciNet  MATH  GoogleScholar  Carvalho,C.M.,Polson,N.G.&Scott,J.G.Thehorseshoeestimatorforsparsesignals.Biometrika97,465–480(2010).MathSciNet  MATH  GoogleScholar  Polson,N.G.&Scott,J.G.Shrinkglobally,actlocally:sparseBayesianregularizationandprediction.BayesianStat.9,105(2010).Thisarticleprovidesaunifiedframeworkforcontinuousshrinkagepriors,whichallowglobalsparsitywhilecontrollingtheamountofregularizationforeachregressioncoefficient. GoogleScholar  Tibshirani,R.Regressionshrinkageandselectionviathelasso.J.R.Stat.Soc.SeriesB58,267–288(1996).MathSciNet  MATH  GoogleScholar  VanErp,S.,Oberski,D.L.&Mulder,J.ShrinkagepriorsforBayesianpenalizedregression.J.Math.Psychol.89,31–50(2019).MathSciNet  MATH  GoogleScholar  Brown,P.J.,Vannucci,M.&Fearn,T.MultivariateBayesianvariableselectionandprediction.J.R.Stat.Soc.SeriesB60,627–641(1998).MathSciNet  MATH  GoogleScholar  Lee,K.H.,Tadesse,M.G.,Baccarelli,A.A.,Schwartz,J.&Coull,B.A.MultivariateBayesianvariableselectionexploitingdependencestructureamongoutcomes:applicationtoairpollutioneffectsonDNAmethylation.Biometrics73,232–241(2017).MathSciNet  MATH  GoogleScholar  Frühwirth-Schnatter,S.&Wagner,H.StochasticmodelspecificationsearchforGaussianandpartiallynon-Gaussianstatespacemodels.J.Econom.154,85–100(2010).MATH  GoogleScholar  Scheipl,F.,Fahrmeir,L.&Kneib,T.Spike-and-slabpriorsforfunctionselectioninstructuredadditiveregressionmodels.J.Am.Stat.Assoc.107,1518–1532(2012).MathSciNet  MATH  GoogleScholar  Tadesse,M.G.,Sha,N.&Vannucci,M.Bayesianvariableselectioninclusteringhighdimensionaldata.J.Am.Stat.Assoc.https://doi.org/10.1198/016214504000001565(2005).MathSciNet  Article  MATH  GoogleScholar  Wang,H.Scalingitup:stochasticsearchstructurelearningingraphicalmodels.BayesianAnal.10,351–377(2015).MathSciNet  MATH  GoogleScholar  Peterson,C.B.,Stingo,F.C.&Vannucci,M.BayesianinferenceofmultipleGaussiangraphicalmodels.J.Am.Stat.Assoc.110,159–174(2015).MathSciNet  MATH  GoogleScholar  Li,F.&Zhang,N.R.Bayesianvariableselectioninstructuredhigh-dimensionalcovariatespaceswithapplicationsingenomics.J.Am.Stat.Assoc.105,1978–2002(2010).MathSciNet  GoogleScholar  Stingo,F.,Chen,Y.,Tadesse,M.G.&Vannucci,M.Incorporatingbiologicalinformationintolinearmodels:aBayesianapproachtotheselectionofpathwaysandgenes.Ann.Appl.Stat.5,1202–1214(2011).MathSciNet  MATH  GoogleScholar  Guan,Y.&Stephens,M.Bayesianvariableselectionregressionforgenome-wideassociationstudiesandotherlarge-scaleproblems.Ann.Appl.Stat.5,1780–1815(2011).MathSciNet  MATH  GoogleScholar  Bottolo,L.etal.GUESS-ingpolygenicassociationswithmultiplephenotypesusingaGPU-basedevolutionarystochasticsearchalgorithm.PLoSGenetics9,e1003657–e1003657(2013). GoogleScholar  Banerjee,S.,Carlin,B.P.&Gelfand,A.E.HierarchicalModelingandAnalysisforSpatialData(CRC,2014).Vock,L.F.B.,Reich,B.J.,Fuentes,M.&Dominici,F.Spatialvariableselectionmethodsforinvestigatingacutehealtheffectsoffineparticulatemattercomponents.Biometrics71,167–177(2015).MathSciNet  MATH  GoogleScholar  Penny,W.D.,Trujillo-Barreto,N.J.&Friston,K.J.BayesianfMRItimeseriesanalysiswithspatialpriors.Neuroimage24,350–362(2005). GoogleScholar  Smith,M.,Pütz,B.,Auer,D.&Fahrmeir,L.AssessingbrainactivitythroughspatialBayesianvariableselection.Neuroimage20,802–815(2003). GoogleScholar  Zhang,L.,Guindani,M.,Versace,F.&Vannucci,M.Aspatio-temporalnonparametricBayesianvariableselectionmodeloffMRIdataforclusteringcorrelatedtimecourses.Neuroimage95,162–175(2014). GoogleScholar  Gorrostieta,C.,Fiecas,M.,Ombao,H.,Burke,E.&Cramer,S.Hierarchicalvectorauto-regressivemodelsandtheirapplicationstomulti-subjecteffectiveconnectivity.Front.Computat.Neurosci.7,159–159(2013). GoogleScholar  Chiang,S.etal.Bayesianvectorautoregressivemodelformulti-subjecteffectiveconnectivityinferenceusingmulti-modalneuroimagingdata.HumanBrainMapping38,1311–1332(2017). GoogleScholar  Schad,D.J.,Betancourt,M.&Vasishth,S.TowardaprincipledBayesianworkflowincognitivescience.Preprintathttps://arxiv.org/abs/1904.12765(2019).Gelman,A.,Meng,X.-L.&Stern,H.Posteriorpredictiveassessmentofmodelfitnessviarealizeddiscrepancies.Stat.Sinica6,733–760(1996).MathSciNet  MATH  GoogleScholar  Meng,X.-L.Posteriorpredictivep-values.Ann.Stat.22,1142–1160(1994).MathSciNet  MATH  GoogleScholar  Asparouhov,T.,Hamaker,E.L.&Muthén,B.Dynamicstructuralequationmodels.Struct.Equ.Modeling25,359–388(2018).MathSciNet  GoogleScholar  Zhang,Z.,Hamaker,E.L.&Nesselroade,J.R.Comparisonsoffourmethodsforestimatingadynamicfactormodel.Struct.Equ.Modeling15,377–402(2008).MathSciNet  GoogleScholar  Hamaker,E.,Ceulemans,E.,Grasman,R.&Tuerlinckx,F.Modelingaffectdynamics:stateoftheartandfuturechallenges.Emot.Rev.7,316–322(2015). GoogleScholar  Meissner,P.wikipediatrend:PublicSubjectAttentionviaWikipediaPageViewStatistics.Rpackageversion2.1.6.PeterMeissnerhttps://CRAN.R-project.org/package=wikipediatrend(2020).Veen,D.&vandeSchoot,R.BayesiananalysisforPhD-delaydataset.OSFhttps://doi.org/10.17605/OSF.IO/JA859(2020).Article  GoogleScholar  Harvey,A.C.&Peters,S.Estimationproceduresforstructuraltimeseriesmodels.J.Forecast.9,89–108(1990). GoogleScholar  Taylor,S.J.&Letham,B.Forecastingatscale.Am.Stat.72,37–45(2018).MathSciNet  GoogleScholar  Gopnik,A.&Bonawitz,E.Bayesianmodelsofchilddevelopment.WileyInterdiscip.Rev.Cogn.Sci.6,75–86(2015). GoogleScholar  Gigerenzer,G.&Hoffrage,U.HowtoimproveBayesianreasoningwithoutinstruction:frequencyformats.Psychol.Rev.102,684(1995). GoogleScholar  Slovic,P.&Lichtenstein,S.ComparisonofBayesianandregressionapproachestothestudyofinformationprocessinginjudgment.Organ.Behav.Hum.Perform.6,649–744(1971). GoogleScholar  Bolt,D.M.,Piper,M.E.,Theobald,W.E.&Baker,T.B.Whytwosmokingcessationagentsworkbetterthanone:roleofcravingsuppression.J.Consult.Clin.Psychol.80,54–65(2012). GoogleScholar  Billari,F.C.,Graziani,R.&Melilli,E.StochasticpopulationforecastingbasedoncombinationsofexpertevaluationswithintheBayesianparadigm.Demography51,1933–1954(2014). GoogleScholar  Fallesen,P.&Breen,R.Temporarylifechangesandthetimingofdivorce.Demography53,1377–1398(2016). GoogleScholar  Hansford,T.G.,Depaoli,S.&Canelo,K.S.LocatingU.S.SolicitorsGeneralintheSupremeCourt’spolicyspace.Pres.Stud.Q.49,855–869(2019). GoogleScholar  Phipps,D.J.,Hagger,M.S.&Hamilton,K.Predictinglimiting‘freesugar’consumptionusinganintegratedmodelofhealthbehavior.Appetite150,104668(2020). GoogleScholar  Depaoli,S.,Rus,H.M.,Clifton,J.P.,vandeSchoot,R.&Tiemensma,J.AnintroductiontoBayesianstatisticsinhealthpsychology.HealthPsychol.Rev.11,248–264(2017). GoogleScholar  Kruschke,J.K.Bayesianestimationsupersedesthettest.J.Exp.Psychol.Gen.142,573–603(2013). GoogleScholar  Lee,M.D.HowcognitivemodelingcanbenefitfromhierarchicalBayesianmodels.J.Math.Psychol.55,1–7(2011).MathSciNet  MATH  GoogleScholar  Royle,J.&Dorazio,R.HierarchicalModelingandInferenceinEcology(Academic,2008).Gimenez,O.etal.inModelingDemographicProcessesinMarkedPopulationsVol.3(edsThomsonD.L.,CoochE.G.&ConroyM.J.)883–915(Springer,2009).King,R.,Morgan,B.,Gimenez,O.&Brooks,S.P.BayesianAnalysisforPopulationEcology(CRC,2009).Kéry,M.&Schaub,M.BayesianPopulationAnalysisusingWinBUGS:AHierarchicalPerspective(Academic,2011).McCarthy,M.BayesianMethodsofEcology5thedn(CambridgeUniv.Press,2012).Korner-Nievergelt,F.etal.BayesianDataAnalysisinEcologyUsingLinearModelswithR,BUGS,andStan(Academic,2015).Monnahan,C.C.,Thorson,J.T.&Branch,T.A.FasterestimationofBayesianmodelsinecologyusingHamiltonianMonteCarlo.MethodsEcol.Evol.8,339–348(2017). GoogleScholar  Ellison,A.M.Bayesianinferenceinecology.Ecol.Lett.7,509–520(2004). GoogleScholar  Choy,S.L.,O’Leary,R.&Mengersen,K.Elicitationbydesigninecology:usingexpertopiniontoinformpriorsforBayesianstatisticalmodels.Ecology90,265–277(2009). GoogleScholar  Kuhnert,P.M.,Martin,T.G.&Griffiths,S.P.AguidetoelicitingandusingexpertknowledgeinBayesianecologicalmodels.Ecol.Lett.13,900–914(2010). GoogleScholar  King,R.,Brooks,S.P.,Mazzetta,C.,Freeman,S.N.&Morgan,B.J.Identifyinganddiagnosingpopulationdeclines:aBayesianassessmentoflapwingsintheUK.J.R.Stat.Soc.SeriesC57,609–632(2008).MathSciNet  GoogleScholar  Newman,K.etal.ModellingPopulationDynamics(Springer,2014).Bachl,F.E.,Lindgren,F.,Borchers,D.L.&Illian,J.B.inlabru:anRpackageforBayesianspatialmodellingfromecologicalsurveydata.MethodsEcol.Evol.10,760–766(2019). GoogleScholar  King,R.&Brooks,S.P.OntheBayesianestimationofaclosedpopulationsizeinthepresenceofheterogeneityandmodeluncertainty.Biometrics64,816–824(2008).MathSciNet  MATH  GoogleScholar  Saunders,S.P.,Cuthbert,F.J.&Zipkin,E.F.Evaluatingpopulationviabilityandefficacyofconservationmanagementusingintegratedpopulationmodels.J.Appl.Ecol.55,1380–1392(2018). GoogleScholar  McClintock,B.T.etal.Ageneraldiscrete-timemodelingframeworkforanimalmovementusingmultistaterandomwalks.Ecol.Monog.82,335–349(2012). GoogleScholar  Dennis,B.,Ponciano,J.M.,Lele,S.R.,Taper,M.L.&Staples,D.F.Estimatingdensitydependence,processnoise,andobservationerror.Ecol.Monog.76,323–341(2006). GoogleScholar  Aeberhard,W.H.,MillsFlemming,J.&Nielsen,A.Reviewofstate-spacemodelsforfisheriesscience.Ann.Rev.Stat.Appl.5,215–235(2018). GoogleScholar  Isaac,N.J.B.etal.Dataintegrationforlarge-scalemodelsofspeciesdistributions.TrendsEcolEvol35,56–67(2020). GoogleScholar  McClintock,B.T.etal.UncoveringecologicalstatedynamicswithhiddenMarkovmodels.Preprintathttps://arxiv.org/abs/2002.10497(2020).King,R.Statisticalecology.Ann.Rev.Stat.Appl.1,401–426(2014). GoogleScholar  Fearnhead,P.inHandbookofMarkovChainMonteCarloCh.21(edsBrooks,S.,Gelman,A.,Jones,G.L.&Meng,X.L.)513–529(Chapman&Hall/CRC,2011).Andrieu,C.,Doucet,A.&Holenstein,R.ParticleMarkovchainMonteCarlomethods.J.R.Stat.Soc.SeriesB72,269–342(2010).MathSciNet  MATH  GoogleScholar  Knape,J.&deValpine,P.FittingcomplexpopulationmodelsbycombiningparticlefilterswithMarkovchainMonteCarlo.Ecology93,256–263(2012). GoogleScholar  Finke,A.,King,R.,Beskos,A.&Dellaportas,P.EfficientsequentialMonteCarloalgorithmsforintegratedpopulationmodels.J.Agric.Biol.Environ.Stat.24,204–224(2019).MathSciNet  MATH  GoogleScholar  Stephens,M.&Balding,D.J.Bayesianstatisticalmethodsforgeneticassociationstudies.Nat.Rev.Genet10,681–690(2009). GoogleScholar  Mimno,D.,Blei,D.M.&Engelhardt,B.E.Posteriorpredictivecheckstoquantifylack-of-fitinadmixturemodelsoflatentpopulationstructure.Proc.NatlAcad.Sci.USA112,E3441–3450(2015).ADS  GoogleScholar  Schaid,D.J.,Chen,W.&Larson,N.B.Fromgenome-wideassociationstocandidatecausalvariantsbystatisticalfine-mapping.Nat.Rev.Genet.19,491–504(2018). GoogleScholar  Marchini,J.&Howie,B.Genotypeimputationforgenome-wideassociationstudies.Nat.Rev.Genet.11,499–511(2010). GoogleScholar  Allen,N.E.,Sudlow,C.,Peakman,T.,Collins,R.&Biobank,U.K.UKBiobankdata:comeandgetit.Sci.Transl.Med.6,224ed224(2014). GoogleScholar  Cortes,A.etal.Bayesiananalysisofgeneticassociationacrosstree-structuredroutinehealthcaredataintheUKBiobank.Nat.Genet.49,1311–1318(2017). GoogleScholar  Argelaguet,R.etal.Multi-omicsfactoranalysis—aframeworkforunsupervisedintegrationofmulti-omicsdatasets.Mol.Syst.Biol.14,e8124(2018). GoogleScholar  Stuart,T.&Satija,R.Integrativesingle-cellanalysis.Nat.Rev.Genet.20,257–272(2019). GoogleScholar  Yau,C.&Campbell,K.Bayesianstatisticallearningforbigdatabiology.Biophys.Rev.11,95–102(2019). GoogleScholar  Vallejos,C.A.,Marioni,J.C.&Richardson,S.BASiCS:Bayesiananalysisofsingle-cellsequencingdata.PLoSComput.Biol.11,e1004333(2015).ADS  GoogleScholar  Wang,J.etal.Datadenoisingwithtransferlearninginsingle-celltranscriptomics.Nat.Methods16,875–878(2019). GoogleScholar  Lopez,R.,Regier,J.,Cole,M.B.,Jordan,M.I.&Yosef,N.Deepgenerativemodelingforsingle-celltranscriptomics.Nat.Methods15,1053–1058(2018). GoogleScholar  NationalCancerInstitute.TheCancerGenomeAtlas.Qeioshttps://doi.org/10.32388/e1plqh(2020).Kuipers,J.etal.Mutationalinteractionsdefinenovelcancersubgroups.Nat.Commun.9,4353(2018).ADS  GoogleScholar  Schwartz,R.&Schaffer,A.A.Theevolutionoftumourphylogenetics:principlesandpractice.Nat.Rev.Genet.18,213–229(2017). GoogleScholar  Munafò,M.R.etal.Amanifestoforreproduciblescience.Nat.Hum.Behav.https://doi.org/10.1038/s41562-016-0021(2017).Article  GoogleScholar  Wilkinson,M.D.etal.TheFAIRguidingprinciplesforscientificdatamanagementandstewardship.Sci.Data3,160018(2016). GoogleScholar  Lamprecht,A.-L.etal.TowardsFAIRprinciplesforresearchsoftware.DataSci.3,37–59(2020). GoogleScholar  Smith,A.M.,Katz,D.S.&Niemeyer,K.E.Softwarecitationprinciples.PeerJComput.Sci.2,e86(2016). GoogleScholar  Clyburne-Sherin,A.,Fei,X.&Green,S.A.Computationalreproducibilityviacontainersinpsychology.MetaPsychol.https://doi.org/10.15626/MP.2018.892(2019).Article  GoogleScholar  Lowenberg,D.Dryad&Zenodo:ourpathahead.WordPresshttps://blog.datadryad.org/2020/03/10/dryad-zenodo-our-path-ahead/(2020).Nosek,B.A.etal.Promotinganopenresearchculture.Science348,1422–1425(2015).ADS  GoogleScholar  Vehtari,A.&Ojanen,J.AsurveyofBayesianpredictivemethodsformodelassessment,selectionandcomparison.Stat.Surv.6,142–228(2012).MathSciNet  MATH  GoogleScholar  Abadi,M.etal.inUSENIXSymposiumonOperatingSystemsDesignandImplementation(OSDI'16)265–283(USENIXAssociation,2016).Paszke,A.etal.inAdvancesinNeuralInformationProcessingSystems(edsWallach,H.etal.)8026–8037(UrranAssociates,2019).Kingma,D.P.&Welling,M.Anintroductiontovariationalautoencoders.Preprintathttps://arxiv.org/abs/1906.02691(2019).Thisrecentreviewofvariationalautoencodersencompassesdeepgenerativemodels,there-parameterizationtrickandcurrentinferencemethods.Higgins,I.etal.beta-VAE:learningbasicvisualconceptswithaconstrainedvariationalframework.ICLR2017https://openreview.net/forum?id=Sy2fzU9gl(2017).Märtens,K.&Yau,C.BasisVAE:tTranslation-invariantfeature-levelclusteringwithvariationalautoencoders.Preprintathttps://arxiv.org/abs/2003.03462(2020).Liu,Q.,Allamanis,M.,Brockschmidt,M.&Gaunt,A.inAdvancesinNeuralInformationProcessingSystems31(edsBengio,S.etal.)7795–7804(CurranAssociates,2018).Louizos,C.,Shi,X.,Schutte,K.&Welling,M.inAdvancesinNeuralInformationProcessingSystems8743-8754(MITPress,2019).Garnelo,M.etal.inProceedingsofthe35thInternationalConferenceonMachineLearningVol.80(edsDy,J.&Krause,A.)1704–1713(PMLR,2018).Kim,H.etal.Attentiveneuralprocesses.Preprintathttps://arxiv.org/abs/1901.05761(2019).Rezende,D.&Mohamed,S.inProceedingsofthe32ndInternationalConferenceonMachineLearningVol.37(edsBach,F.&Blei,D.)1530–1538(PMLR,2015).Papamakarios,G.,Nalisnick,E.,Rezende,D.J.,Mohamed,S.&Lakshminarayanan,B.Normalizingflowsforprobabilisticmodelingandinference.Preprintathttps://arxiv.org/abs/1912.02762(2019).Korshunova,I.etal.inAdvancesinNeuralInformationProcessingSystems31(edsBengio,S.etal.)7190–7198(CurranAssociates,2018).Zhang,R.,Li,C.,Zhang,J.,Chen,C.&Wilson,A.G.CyclicalstochasticgradientMCMCforBayesiandeeplearning.Preprintathttps://arxiv.org/abs/1902.03932(2019).Neal,R.M.BayesianLearningforNeuralNetworks(SpringerScience&BusinessMedia,2012).Neal,R.M.inBayesianLearningforNeuralNetworksLectureNotesinStatisticsCh2(edNea,R.M.)29–53(Springer,1996).ThisclassictexthighlightstheconnectionbetweenneuralnetworksandGaussianprocessesandtheapplicationofBayesianapproachesforfittingneuralnetworks.Williams,C.K.I.inAdvancesinNeuralInformationProcessingSystems295–301(MITPress,1997).MacKayDavid,J.C.ApracticalBayesianframeworkforbackpropnetworks.Neural.Comput.https://doi.org/10.1162/neco.1992.4.3.448(1992).Article  GoogleScholar  Sun,S.,Zhang,G.,Shi,J.&Grosse,R.FunctionalvariationalBayesianneuralnetworks.Preprintathttps://arxiv.org/abs/1903.05779(2019).Lakshminarayanan,B.,Pritzel,A.&Blundell,C.Simpleandscalablepredictiveuncertaintyestimationusingdeepensembles.AdvancesinNeuralInformationProcessingSystems30,6402–6413(2017). GoogleScholar  Wilson,A.G.ThecaseforBayesiandeeplearning.Preprintathttps://arxiv.org/abs/2001.10995(2020).Srivastava,N.,Hinton,G.,Krizhevsky,A.,Sutskever,I.&Salakhutdinov,R.Dropout:asimplewaytopreventneuralnetworksfromoverfitting.J.Mach.Learn.Res.15,1929–1958(2014).MathSciNet  MATH  GoogleScholar  Gal,Y.&Ghahramani,Z.inInternationalConferenceonMachineLearning1050–1059(JMLR,2016).Green,P.J.ReversiblejumpMarkovchainMonteCarlocomputationandBayesianmodeldetermination.Biometrika82,711–732(1995).MathSciNet  MATH  GoogleScholar  Hoffman,M.D.&Gelman,A.TheNo-U-TurnSampler:adaptivelysettingpathlengthsinHamiltonianMonteCarlo.J.Mach.Learn.Res.15,1593–1623(2014).MathSciNet  MATH  GoogleScholar  Liang,F.&Wong,W.H.EvolutionaryMonteCarlo:applicationstoCpmodelsamplingandchangepointproblem.Stat.Sinica317-342(2000).Liu,J.S.&Chen,R.SequentialMonteCarlomethodsfordynamicsystems.J.Am.Stat.Assoc.93,1032–1044(1998).MathSciNet  MATH  GoogleScholar  Sisson,S.,Fan,Y.&Beaumont,M.HandbookofApproximateBayesianComputation(ChapmanandHall/CRC2018).Rue,H.,Martino,S.&Chopin,N.ApproximateBayesianinferenceforlatentGaussianmodelsbyusingintegratednestedLaplaceapproximations.J.R.Stat.Soc.SeriesB71,319–392(2009).MathSciNet  MATH  GoogleScholar  Lunn,D.J.,Thomas,A.,Best,N.&Spiegelhalter,D.WinBUGS—aBayesianmodellingframework:concepts,structure,andextensibility.Stat.Comput.10,325–337(2000). GoogleScholar  Ntzoufras,I.BayesianModelingUsingWinBUGSVol.698(Wiley,2011).Lunn,D.J.,Thomas,A.,Best,N.&Spiegelhalter,D.WinBUGS—aBayesianmodellingframework:concepts,structure,andextensibility.Stat.Comput.10,325–337(2000).Thispaperprovidesanearlyuser-friendlyandfreelyavailableblack-boxMCMCsampler,openingupBayesianinferencetothewiderscientificcommunity. GoogleScholar  Spiegelhalter,D.,Thomas,A.,Best,N.&Lunn,D.OpenBUGSUserManualversion3.2.3.Openbugshttp://www.openbugs.net/w/Manuals?action=AttachFile&do=view&target=OpenBUGS_Manual.pdf(2014).Plummer,M.JAGS:aprogramforanalysisofBayesiangraphicalmodelsusingGibbssampling.Proc.3rdInternationalWorkshoponDistributedStatisticalComputing124,1–10(2003). GoogleScholar  Plummer,M.rjags:BayesiangraphicalmodelsusingMCMC.Rpackageversion,4(6)(2016).Salvatier,J.,Wiecki,T.V.&Fonnesbeck,C.ProbabilisticprogramminginPythonusingPyMC3.PeerJComput.Sci.2,e55(2016). GoogleScholar  deValpine,P.etal.Programmingwithmodels:writingstatisticalalgorithmsforgeneralmodelstructureswithNIMBLE.J.Comput.Graph.Stat.s26,403–413(2017).MathSciNet  GoogleScholar  Dillon,J.V.etal.Tensorflowdistributions.Preprintathttps://arxiv.org/abs/1711.10604(2017).Keydana,S.tfprobability:RinterfacetoTensorFlowprobability.githubhttps://rstudio.github.io/tfprobability/index.html(2020).Bingham,E.etal.Pyro:deepuniversalprobabilisticprogramming.J.Mach.Learn.Res.20,973–978(2019).MATH  GoogleScholar  Bezanson,J.,Karpinski,S.,Shah,V.B.&Edelman,A.Julia:afastdynamiclanguagefortechnicalcomputing.Preprintathttps://arxiv.org/abs/1209.5145(2012).Ge,H.,Xu,K.&Ghahramani,Z.Turing:alanguageforflexibleprobabilisticinference.ProceedingsofMachineLearningResearch84,1682–1690(2018). GoogleScholar  Smith,B.J.etal.brian-j-smith/Mamba.jl:v0.12.4.Zenodohttps://doi.org/10.5281/zenodo.3740216(2020).JASPTeam.JASP(version0.14)[computersoftware](2020).Lindgren,F.&Rue,H.BayesianspatialmodellingwithR-INLA.J.Stat.Soft.63,1–25(2015). GoogleScholar  Vanhatalo,J.etal.GPstuff:BayesianmodelingwithGaussianprocesses.J.Mach.Learn.Res.14,1175–1179(2013).MathSciNet  MATH  GoogleScholar  Blaxter,L.HowtoResearch(McGraw-HillEducation,2010).Neuman,W.L.UnderstandingResearch(Pearson,2016).Betancourt,M.TowardsaprincipledBayesianworkflow.githubhttps://betanalpha.github.io/assets/case_studies/principled_bayesian_workflow.html(2020).Veen,D.&vandeSchoot,R.PosteriorpredictivechecksforthePremierLeague.OSFhttps://doi.org/10.17605/OSF.IO/7YRUD(2020).Article  GoogleScholar  Kramer,B.&Bosman,J.Summerschoolopenscienceandscholarship2019—UtrechtUniversity.ZENODOhttps://doi.org/10.5281/ZENODO.3925004(2020).Article  GoogleScholar  Rényi,A.Onanewaxiomatictheoryofprobability.ActaMath.Hung.6,285–335(1955).MathSciNet  MATH  GoogleScholar  Lesaffre,E.&Lawson,A.B.BayesianBiostatistics(Wiley,2012).Hoijtink,H.,Beland,S.&Vermeulen,J.A.CognitivediagnosticassessmentviaBayesianevaluationofinformativediagnostichypotheses.PsycholMethods19,21–38(2014). GoogleScholar  DownloadreferencesAcknowledgementsR.v.d.S.wassupportedbygrantNWO-VIDI-452-14-006fromtheNetherlandsOrganizationforScientificResearch.R.K.wassupportedbyLeverhulmeresearchfellowshipgrantreferenceRF-2019-299andbyTheAlanTuringInstituteundertheEPSRCgrantEP/N510129/1.K.M.wassupportedbyaUKEngineeringandPhysicalSciencesResearchCouncilDoctoralStudentship.C.Y.issupportedbyaUKMedicalResearchCouncilResearchGrant(Ref.MR/P02646X/1)andbyTheAlanTuringInstituteundertheEPSRCgrantEP/N510129/1AuthorinformationAuthorsandAffiliationsDepartmentofMethodsandStatistics,UtrechtUniversity,Utrecht,NetherlandsRensvandeSchoot, DucoVeen & JoukjeWillemsenDepartmentofQuantitativePsychology,UniversityofCaliforniaMerced,Merced,CA,USASarahDepaoliSchoolofMathematics,UniversityofEdinburgh,Edinburgh,UKRuthKingTheAlanTuringInstitute,BritishLibrary,London,UKRuthKing & ChristopherYauUtrechtUniversityLibrary,UtrechtUniversity,Utrecht,NetherlandsBiancaKramerDepartmentofStatistics,UniversityofOxford,Oxford,UKKasparMärtensDepartmentofMathematicsandStatistics,GeorgetownUniversity,Washington,DC,USAMahletG.TadesseDepartmentofStatistics,RiceUniversity,Houston,TX,USAMarinaVannucciDepartmentofStatistics,ColumbiaUniversity,NewYork,NY,USAAndrewGelmanDivisionofInformatics,Imaging&DataSciences,UniversityofManchester,Manchester,UKChristopherYauAuthorsRensvandeSchootViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarSarahDepaoliViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarRuthKingViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarBiancaKramerViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarKasparMärtensViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarMahletG.TadesseViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarMarinaVannucciViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarAndrewGelmanViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarDucoVeenViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarJoukjeWillemsenViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarChristopherYauViewauthorpublicationsYoucanalsosearchforthisauthorin PubMed GoogleScholarContributionsIntroduction(R.v.d.S.);Experimentation(S.D.,D.V.,R.v.d.S.andJ.W.);Results(R.K.,M.G.T.,M.V.,D.V.,K.M.,C.Y.andR.v.d.S.);Applications(S.D.,R.K.,K.M.andC.Y.);Reproducibilityanddatadeposition(B.K.,D.V.,S.D.andR.v.d.S.);Limitationsandoptimizations(A.G.);Outlook(K.M.andC.Y.);OverviewofthePrimer(R.v.d.S.).CorrespondingauthorCorrespondenceto RensvandeSchoot.Ethicsdeclarations Competinginterests Theauthorsdeclarenocompetinginterests. AdditionalinformationPeerreviewinformationNatureReviewsMethodsPrimersthanksD.Ashby,J.Doll,D.Dunson,F.Feinberg,J.Liu,B.Rosenbaumandtheother,anonymous,reviewer(s)fortheircontributiontothepeerreviewofthiswork.Publisher’snoteSpringerNatureremainsneutralwithregardtojurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations.Relatedlinks Dryad: https://datadryad.org/ RegistryofResearchDataRepositories: https://www.re3data.org/ ScientificData listofrepositories: https://www.nature.com/sdata/policies/repositories Zenodo: https://zenodo.org/ GlossaryPriordistribution Beliefsheldbyresearchersabouttheparametersinastatisticalmodelbeforeseeingthedata,expressedasprobabilitydistributions. Likelihoodfunction Theconditionalprobabilitydistributionofthegivenparametersofthedata,defineduptoaconstant. Posteriordistribution Awaytosummarizeone’supdatedknowledge,balancingpriorknowledgewithobserveddata. Informativeness Priorscanhavedifferentlevelsofinformativenessandcanbeanywhereonacontinuumfromcompleteuncertaintytorelativecertainty,butwedistinguishbetweendiffuse,weaklyandinformativepriors. Hyperparameters Parametersthatdefinethepriordistribution,suchasmeanandvarianceforanormalprior. Priorelicitation Theprocessbywhichbackgroundinformationistranslatedintoasuitablepriordistribution. Informativeprior Areflectionofahighdegreeofcertaintyorknowledgesurroundingthepopulationparameters.Hyperparametersarespecifiedtoexpressparticularinformationreflectingagreaterdegreeofcertaintyaboutthemodelparametersbeingestimated. Weaklyinformativeprior Apriorincorporatingsomeinformationaboutthepopulationparameterbutthatislesscertainthananinformativeprior. Diffusepriors Reflectionsofcompleteuncertaintyaboutpopulationparameters. Improperpriors Priordistributionsthatintegratetoinfinity. Priorpredictivechecking Theprocessofcheckingwhetherthepriorsmakesensebygeneratingdataaccordingtothepriorinordertoassesswhethertheresultsarewithintheplausibleparameterspace. Priorpredictivedistribution Allpossiblesamplesthatcouldoccurifthemodelistruebasedonthepriors. Kerneldensityestimation Anon-parametricapproachusedtoestimateaprobabilitydensityfunctionfortheobserveddata. Priorpredictivep-value Anestimatetoindicatehowunlikelytheobserveddataaretobegeneratedbythemodelbasedonthepriorpredictivedistribution Bayesfactor Theratiooftheposterioroddstotheprioroddsoftwocompetinghypotheses,alsocalculatedastheratioofthemarginallikelihoodsunderthetwohypotheses.Itcanbeused,forexample,tocomparecandidatemodels,whereeachmodelwouldcorrespondtoahypothesis. Credibleinterval Anintervalthatcontainsaparameterwithaspecifiedprobability.Theboundsoftheintervalaretheupperandlowerpercentilesoftheparameter’sposteriordistribution.Forexample,a95%credibleintervalhastheupperandlower2.5%percentilesoftheposteriordistributionasitsbounds. Closedform Amathematicalexpressionthatcanbewrittenusingafinitenumberofstandardoperations. Marginalposteriordistribution Probabilitydistributionofaparameterorsubsetofparameterswithintheposteriordistribution,irrespectiveofthevaluesofothermodelparameters.Itisobtainedbyintegratingouttheothermodelparametersfromthejointposteriordistribution. MarkovchainMonteCarlo (MCMC).Amethodtoindirectlyobtaininferenceontheposteriordistributionbysimulation.TheMarkovchainisconstructedsuchthatitscorrespondingstationarydistributionistheposterior distributionofinterest.Oncethechainhasreachedthestationarydistribution,realizationscanberegardedasadependentsetofsampledparametervaluesfromtheposteriordistribution.Thesesampledparametervaluescanthenbeusedtoobtainempiricalestimatesoftheposteriordistribution,andassociatedsummarystatisticsofinterest,usingMonteCarlointegration. Markovchain AniterativeprocesswherebythevaluesoftheMarkovchainattimet + 1areonlydependentonthevaluesofthechainattimet. MonteCarlo Astochasticalgorithmforapproximatingintegralsusingthesimulationofrandomnumbersfromagivendistribution.Inparticular,forsampledvaluesfromadistribution,theassociatedempiricalvalueofagivenstatisticisanestimateofthecorrespondingsummarystatisticofthedistribution. Transitionkernel TheupdatingprocedureoftheparametervalueswithinaMarkovchain. Auxiliaryvariables Additionalvariablesenteredinamodelsuchthatthejointdistributionisavailableinclosedformandquicktoevaluate. Traceplots PlotsdescribingtheposteriorparametervalueateachiterationoftheMarkovchain(ontheyaxis)againsttheiterationnumber(onthexaxis). \(\hat{R}\)statistic Theratioofwithin-chainandbetween-chainvariability.ValuesclosetooneforallparametersandquantitiesofinterestsuggesttheMarkovchainMonteCarloalgorithmhassufficientlyconvergedtothestationarydistribution. Variationalinference AtechniquetobuildapproximationstothetrueBayesianposteriordistributionusingcombinationsofsimplerdistributionswhoseparametersareoptimizedtomaketheapproximationascloseaspossibletotheactualposterior. Approximatingdistribution Inthecontextofposteriorinference,replacingapotentiallycomplicatedposteriordistributionwithasimplerdistributionthatiseasytoevaluateandsamplefrom.Forexample,invariationalinference,itiscommontoapproximatethetrueposteriorwithaGaussiandistribution. Stochasticgradientdescent Analgorithmthatusesarandomlychosensubsetofdatapointstoestimatethegradientofalossfunctionwithrespecttoparameters,providingcomputationalsavingsinoptimizationproblemsinvolvingmanydatapoints. Multicollinearity Asituationthatarisesinaregressionmodelwhenapredictorcanbelinearlypredictedwithhighaccuracyfromtheotherpredictorsinthemodel.Thiscausesnumericalinstabilityintheestimationofparameters. Shrinkagepriors Priordistributionsforaparameterthatshrinkitsposteriorestimatetowardsaparticularvalue. Sparsity Asituationwheremostparametervaluesarezeroandonlyafewarenon-zero. Spike-and-slabprior Ashrinkagepriordistributionusedforvariableselectionspecifiedasamixtureoftwodistributions,onepeakedaroundzero(spike)andtheotherwithalargevariance(slab). Continuousshrinkageprior Aunimodalpriordistributionforaparameterthatpromotesshrinkageofitsposteriorestimatetowardszero. Global–localshrinkageprior Acontinuousshrinkagepriordistributioncharacterizedbyahighconcentrationaroundzerotoshrinksmallparametervaluestozeroandheavytailstopreventexcessiveshrinkageoflargeparametervalues. Horseshoeprior Anexampleofaglobal–localshrinkagepriorforvariableselectionthatusesahalf-Cauchyscalemixtureofnormaldistributions. Autoencoder Aparticulartypeofmultilayerneuralnetworkusedforunsupervisedlearningconsistingoftwocomponents:anencoderandadecoder.Theencodercompressestheinputinformationintolow-dimensionalsummariesoftheinputs.Thedecodertakesthesesummariesandattemptstorecreatetheinputsfromthese.Bytrainingtheencoderanddecodersimultaneously,thehopeisthattheautoencoderlearnslow-dimensional,buthighlyinformative,representationsofthedata. Split-\(\hat{R}\) Todetectnon-stationaritywithinindividualMarkovchainMonteCarlochains(forexample,ifthefirstpartshowsgraduallyincreasingvalueswhereasthesecondpartinvolvesgraduallydecreasingvalues),eachchainissplitintotwopartsforwhichthe\(\hat{R}\)statisticiscomputedandcompared. Amortization Atechniqueusedinvariationalinferencetoreducethenumberoffreeparameterstobeestimatedinavariationalposteriorapproximationbyreplacingthefreeparameterswithatrainablepredictionfunctionthatcaninsteadpredictthevaluesoftheseparameters. RightsandpermissionsReprintsandPermissionsAboutthisarticleCitethisarticlevandeSchoot,R.,Depaoli,S.,King,R.etal.Bayesianstatisticsandmodelling. NatRevMethodsPrimers1,1(2021).https://doi.org/10.1038/s43586-020-00001-2DownloadcitationAccepted:21October2020Published:14January2021DOI:https://doi.org/10.1038/s43586-020-00001-2SharethisarticleAnyoneyousharethefollowinglinkwithwillbeabletoreadthiscontent:GetshareablelinkSorry,ashareablelinkisnotcurrentlyavailableforthisarticle.Copytoclipboard ProvidedbytheSpringerNatureSharedItcontent-sharinginitiative Furtherreading Intelligentmetasurfaces:control,communicationandcomputing LianlinLi HantingZhao TieJunCui eLight(2022) Remote,real-timeexpertelicitationtodeterminethepriorprobabilitydistributionforBayesiansamplesizedeterminationininternationalrandomisedcontrolledtrials:BronchiolitisinInfantsPlaceboVersusEpinephrineandDexamethasone(BIPED)study JingxianLan AmyC.Plint AnnaHeath Trials(2022) 4polar-STORMpolarizedsuper-resolutionimagingofactinfilamentorganizationincells CaioVazRimoli CesarAugustoValades-Cruz SophieBrasselet NatureCommunications(2022) OneyearofMethodsPrimers NatureReviewsMethodsPrimers(2022) Fielddeployableimpedance-basedcorrosionsensor JiajunLi XiaoxueJiang JieChen ScientificReports(2022) Accessthroughyourinstitution Changeinstitution Buyorsubscribe AssociatedContent Collection FirstAnniversaryCollection Bayesianstatisticsandmodelling NatureReviewsMethodsPrimers PrimeView 14Jan2021 Advertisement Explorecontent Primers&PrimeViews News&Comment Collections FollowusonTwitter Subscribe Signupforalerts RSSfeed Aboutthejournal Aims&Scope JournalInformation AbouttheEditors JournalCredits ContentTypes Editorialpolicies Editorialinputandchecks EditorialValuesStatement JournalMetrics Publishingmodel Contact Publishwithus ForAuthors ForReferees Submitmanuscript Search Searcharticlesbysubject,keywordorauthor Showresultsfrom Alljournals Thisjournal Search Advancedsearch Quicklinks Explorearticlesbysubject Findajob Guidetoauthors Editorialpolicies Closebanner Close SignupfortheNatureBriefingnewsletter—whatmattersinscience,freetoyourinboxdaily. Emailaddress Signup IagreemyinformationwillbeprocessedinaccordancewiththeNatureandSpringerNatureLimitedPrivacyPolicy. Closebanner Close Getthemostimportantsciencestoriesoftheday,freeinyourinbox. SignupforNatureBriefing