What is a Learning Curve in machine learning? - Stack Overflow

An ROC curve is a graphical depiction of classifier performance that shows the trade-off between increasing true positive rates (on the vertical axis) and ... Home Public Questions Tags Users Companies Collectives ExploreCollectives Teams StackOverflowforTeams –Startcollaboratingandsharingorganizationalknowledge. CreateafreeTeam WhyTeams? Teams CreatefreeTeam Collectives™onStackOverflow Findcentralized,trustedcontentandcollaboratearoundthetechnologiesyouusemost. Learnmore Teams Q&Aforwork Connectandshareknowledgewithinasinglelocationthatisstructuredandeasytosearch. Learnmore WhatisaLearningCurveinmachinelearning? AskQuestion Asked 11years,6monthsago Modified 1year,8monthsago Viewed 37ktimes 58 21 Iwanttoknowwhatalearningcurveinmachinelearningis.Whatisthestandardwayofplottingit?Imeanwhatshouldbethexandyaxisofmyplot? machine-learning Share Follow editedNov9,2016at5:16 MartinThoma 110k145145goldbadges557557silverbadges859859bronzebadges askedJan6,2011at16:48 HosseinHossein 37.8k5555goldbadges135135silverbadges174174bronzebadges 4 1 Neverheardofalearningcurve.DoyoumeanaROCcurve?en.wikipedia.org/wiki/Receiver_operating_characteristic – Stompchicken Jan6,2011at17:07 7 No,learningcurveandROCcurvearenotsynonymous,asIattempttodescribebelow. – MattBagg Dec5,2012at2:16 @MattBagg:youareabsolutelyright,Irolledbacktobeforetheedit. – Amro Jan20,2013at1:18 SeeAnalysisandOptimizationofConvolutionalNeuralNetworkArchitectures – MartinThoma Aug1,2017at5:42 Addacomment  |  10Answers 10 Sortedby: Resettodefault Highestscore(default) Trending(recentvotescountmore) Datemodified(newestfirst) Datecreated(oldestfirst) 58 Itusuallyreferstoaplotofthepredictionaccuracy/errorvs.thetrainingsetsize(i.e:howbetterdoesthemodelgetatpredictingthetargetasyoutheincreasenumberofinstancesusedtotrainit) Usuallyboththetrainingandtest/validationperformanceareplottedtogethersowecandiagnosethebias-variancetradeoff(i.edetermineifwebenefitfromaddingmoretrainingdata,andassessthemodelcomplexitybycontrollingregularizationornumberoffeatures). Share Follow editedJun25,2019at13:19 answeredJan7,2011at2:45 AmroAmro 123k2525goldbadges236236silverbadges442442bronzebadges 5 2 There'salsoamorecurrentarticle:scikit-learn.org/stable/modules/learning_curve.html – TheDiscoSpider Mar28,2015at20:16 TheWikipediaentrymentionedanalternativetypeoflearningcurvethatistheperformancev.s.numberofiterations.Igetabitconfusedaboutthese2definitions.Fortheperformance-sizedefinition,foreachtrainingsizex,they-axisvalueisobtainedfromthemodelthathasbeentrainedasmuchaspossible(e.g.byfeedinginthexsamplesmultipletimestillconvergence),ortrainedusingonlyonepassofthexsamples? – Jason May12,2021at5:58 ...continuingfromprevious:Fortheperformance-iterationdefinition,itmustbequitecomputationallyheavyforstochastictraining,isn'tit?Becauseforeachinputsample,onehastopredictalltrainingsamplesandgettheaveragescore,thenitwouldbescalingwithn^2. – Jason May12,2021at5:59 performance-iterations:youtrainyourmodelovertheentiretrainingsetandyouplotthelossfunctiononeachiterationofthecurrentmodelmeasuredonthefulltrain/validationset.Themodeloptimizationisiterativesothelongeryouletthealgorithmrunthemorelikelyitistoimprove,andweusesuchaplottodecidewhentostoplearningasthemodelconvergesorbecomestoosensitivetotrainingdatalosinggeneralizationoverthevalidationset. – Amro May14,2021at5:31 performance-samples:youtrainyourmodeloveranincreasingsubsetsizeofthetrainingdataandyouplotthelossfunctionofthecurrentmodelmeasuredonthefulltrain/validationset.You'dnormallytrainthemodeluntilconvergenceeachtime(usingthesamefixedcriteriatodetermineconvergence).Itcanbeusedtofindoutifthemodelisunderfitting(wecouldusemoredata)oroverfitting(weneedtotweakregularizationtoimprovegeneralizationandbelesssensitivetonoisytrainingdata). – Amro May14,2021at5:31 Addacomment  |  33 IjustwanttoleaveabriefnoteonthisoldquestiontopointoutthatlearningcurveandROCcurvearenotsynonymous. Asindicatedintheotheranswerstothisquestion,alearningcurveconventionallydepictsimprovementinperformanceontheverticalaxiswhentherearechangesinanotherparameter(onthehorizontalaxis),suchastrainingsetsize(inmachinelearning)oriteration/time(inbothmachineandbiologicallearning).Onesalientpointisthatmanyparametersofthemodelarechangingatdifferentpointsontheplot.Otheranswersherehavedoneagreatjobofillustratinglearningcurves. (Thereisalsoanothermeaningoflearningcurveinindustrialmanufacturing,originatinginanobservationinthe1930sthatthenumberoflaborhoursneededtoproduceanindividualunitdecreasesatauniformrateasthequantityofunitsmanufactureddoubles.Itisn'treallyrelevantbutisworthnotingforcompletenessandtoavoidconfusioninwebsearches.) Incontrast,ReceiverOperatingCharacteristiccurve,orROCcurve,doesnotshowlearning;itshowsperformance.AnROCcurveisagraphicaldepictionofclassifierperformancethatshowsthetrade-offbetweenincreasingtruepositiverates(ontheverticalaxis)andincreasingfalsepositiverates(onthehorizontalaxis)asthediscriminationthresholdoftheclassifierisvaried.Thus,onlyasingleparameter(thedecision/discriminationthreshold)associatedwiththemodelischangingatdifferentpointsontheplot.ThisROCcurve(fromWikipedia)showsperformanceofthreedifferentclassifiers. Thereisnolearningbeingdepictedhere,butratherperformancewithrespecttotwodifferentclassesofsuccess/errorastheclassifier'sdecisionthresholdismademorelenient/strict.Bylookingattheareaunderthecurve,wecanseeanoverallindicationoftheabilityoftheclassifiertodistinguishtheclasses.Thisarea-under-the-curvemetricisinsensitivetothenumberofmembersinthetwoclasses,soitmaynotreflectactualperformanceifclassmembershipisunbalanced.TheROCcurvehasmanysubtitlesandinterestedreadersmightcheckout: Fawcett,Tom."ROCgraphs:Notesandpracticalconsiderationsforresearchers."MachineLearning31(2004):1-38. Swets,JohnA.,RobynM.Dawes,andJohnMonahan."BetterdecisionsthroughScience."ScientificAmerican(2000):83. Share Follow answeredDec5,2012at2:14 MattBaggMattBagg 9,87833goldbadges3939silverbadges4747bronzebadges 0 Addacomment  |  17 Somepeopleuse"learningcurve"torefertotheerrorofaniterativeprocedureasafunctionoftheiterationnumber,i.e.,itillustratesconvergenceofsomeutilityfunction.Intheexamplebelow,Iplotmean-squareerror(MSE)oftheleast-mean-square(LMS)algorithmasafunctionoftheiterationnumber.ThatillustrateshowquicklyLMS"learns",inthiscase,thechannelimpulseresponse. Share Follow answeredJan7,2011at0:21 SteveTjoaSteveTjoa 56.2k1616goldbadges8989silverbadges9898bronzebadges Addacomment  |  10 Basically,amachinelearningcurveallowsyoutofindthepointfromwhichthealgorithmstartstolearn.Ifyoutakeacurveandthensliceaslopetangentforderivativeatthepointthatitstartstoreachconstantiswhenitstartstobuilditslearningability. Dependingonhowyourxandyaxisaremapped,oneofyouraxiswillstarttoapproachaconstantvaluewhiletheotheraxis'svalueswillkeepincreasing.Thisiswhenyoustartseeingsomelearning.Thewholecurveprettymuchallowsyoutomeasuretherateatwhichyouralgorithmisabletolearn.Themaximumpointisusuallywhentheslopestartstorecede.Youcantakeanumberofderivativemeasurestothemaximum/minimumpoint. Sofromtheaboveexamplesyoucanseethatthecurveisgraduallytendingtowardsaconstantvalue.Itinitiallystartstoharnessitslearningthroughthetrainingexamplesandtheslopewidensatmaximum/mimimumpointwhereittendstoapproachcloserandclosertowardstheconstantstate.Atthispointitisabletopickupnewexamplesfromtestdataandfindnewanduniqueresultsfromdata. Youwouldhavesuchx/yaxismeasuresforepochsvserror. Share Follow editedMay29,2011at22:43 ScottGottreu 3,52644goldbadges2626silverbadges3333bronzebadges answeredMay29,2011at3:49 memememe 14911silverbadge33bronzebadges Addacomment  |  4 InAndrew'smachinelearningclass,alearningcurveistheplotofthetraining/cross-validationerrorversusthesamplesize.Thelearningcurvecanbeusedtodetectwhetherthemodelhasthehighbiasorhighvariance.Ifthemodelsuffersfromhighbiasproblem,asthesamplesizeincreases,trainingerrorwillincreaseandthecrossvalidationerrorwilldecreaseandatlasttheywillbeveryclosetoeachotherbutstillatahigherrorrateforbothtrainingandclassificationerror.Andincreasingthesamplesizewillnothelpmuchforhighbiasproblem. Ifthemodelsuffersfromhighvariance,asthekeepincreasingthesamplesize,thetrainingerrorwillkeepincreasingandcross-validationerrorwillkeepdecreasingandtheywillendupatalowtrainingandcross-validationerrorrate.Somoresampleswillhelptoimprovethemodelpredictionperformanceifthemodelsufferfromhighvariance. Share Follow answeredJun22,2018at18:30 EmmaZhangEmmaZhang 9111goldbadge22silverbadges44bronzebadges Addacomment  |  4 Howcanyoudetermineforagivenmodelwhethermoretrainingpointswillbehelpful?Ausefuldiagnosticforthisarelearningcurves. •Plotofthepredictionaccuracy/errorvs.thetrainingsetsize(i.e.:howbetterdoesthemodelgetatpredictingthetargetasyoutheincreasenumberofinstancesusedtotrainit) •Learningcurveconventionallydepictsimprovementinperformanceontheverticalaxiswhentherearechangesinanotherparameter(onthehorizontalaxis),suchastrainingsetsize(inmachinelearning)oriteration/time •Alearningcurveisoftenusefultoplotforalgorithmicsanitycheckingorimprovingperformance •Learningcurveplottingcanhelpdiagnosetheproblemsyouralgorithmwillbesufferingfrom Personally,thebelowtwolinkshelpedmetounderstandbetteraboutthisconcept LearningCurve SklearnLearningCurve Share Follow editedJun23,2020at12:02 erotavlas 3,85433goldbadges4141silverbadges9090bronzebadges answeredJun14,2016at21:19 AravindKrishnakumarAravindKrishnakumar 2,56111goldbadge2626silverbadges2323bronzebadges Addacomment  |  1 usethiscodetoplot: #LossCurves plt.figure(figsize=[8,6]) plt.plot(history.history['loss'],'r',linewidth=3.0) plt.plot(history.history['val_loss'],'b',linewidth=3.0) plt.legend(['Trainingloss','ValidationLoss'],fontsize=18) plt.xlabel('Epochs',fontsize=16) plt.ylabel('Loss',fontsize=16) plt.title('LossCurves',fontsize=16) #AccuracyCurves plt.figure(figsize=[8,6]) plt.plot(history.history['acc'],'r',linewidth=3.0) plt.plot(history.history['val_acc'],'b',linewidth=3.0) plt.legend(['TrainingAccuracy','ValidationAccuracy'],fontsize=18) plt.xlabel('Epochs',fontsize=16) plt.ylabel('Accuracy',fontsize=16) plt.title('AccuracyCurves',fontsize=16) notethathistory=model.fit(...) Share Follow answeredAug25,2019at8:54 ElieSokhonElieSokhon 1133bronzebadges Addacomment  |  1 ItisaGraphthatcomparestheperformanceofamodelonpreparingandtestingdataoverachangingnumberoftraininginstancesandtheseareagenerallyutilizedasanalyticinstrumentinmachinelearningforcalculationsthatlearnfromatrainingdatasetincrementally.Itallowsustoverifywhenamodelhaslearningasmuchasitcanaboutthedata. TherearethreekindsofexpectationstoLearningcurvesabsorbinformation BadLearningCurve:HighBias BadLearningCurve:HighVariance IdealLearningCurve Share Follow answeredJan14,2020at11:12 TidyquantTidyquant 1111bronzebadge Addacomment  |  1 Insimpleterms,thelearningcurveisaplotbetweenthenumberofinstancesandametricsuchaslossoraccuracy.Thisplotshowsthejourneylearningwiththegainofexperienceandhenceisnamedlearningcurve. Learningcurvesarewidelyusedinmachinelearningforalgorithmsthatlearn(optimizetheirinternalparameters)incrementallyovertime,suchasdeeplearningneuralnetworks. Share Follow answeredApr16,2020at10:13 user13320096user13320096 Addacomment  |  -3 Example X=Level y=salary XY 02000 24000 46000 68000 Regressiongivesaccuracy75%itisastateline polynomialgivesaccuracy85%becauseofthecurve Share Follow answeredJun14,2018at5:29 ParitoshYadavParitoshYadav 3122bronzebadges Addacomment  |  Highlyactivequestion.Earn10reputation(notcountingtheassociationbonus)inordertoanswerthisquestion.Thereputationrequirementhelpsprotectthisquestionfromspamandnon-answeractivity. Nottheansweryou'relookingfor?Browseotherquestionstaggedmachine-learningoraskyourownquestion. TheOverflowBlog StackExchangesitesaregettingprettierfaster:IntroducingThemes Moneythatmovesatthespeedofinformation(Ep.462) FeaturedonMeta Duplicatedvotesarebeingcleanedup AnnouncingtheStacksEditorBetarelease! Trending:Anewanswersortingoption Shouldweburninatethe[options]tag? Linked 2 HowtoimprovetheperfomanceofSVM? 1 Isthereatheorytotestthemaximumtheoreticalaccuracyforadataset? Related 82 Extractinganinformationfromwebpagebymachinelearning 2 Interpretationofalearningcurveinmachinelearning 1 HowdoyouplotlearningcurvesforRandomForestmodels? HotNetworkQuestions Amigamultitasking-differentresolutionsforeachprogramonscreendragging WhyareresidualsNormallydistributed? ExtractStringsfromText UniversalPhoneticAlphabet HowdoesWPgeneratethedefault$queryinWP_QuerybasedontheURL? StrangeusesofmovzxbyClangandGCC Howtousedriedonionflakesasasubstituteforfreshonion? Do"unlimitedammo"or"bottomlessmagazine"tropesruinstorytelling? Whyaretherepre-compiledpackagesinrepositories? IsitlegalforBMWownerstoenableheatedcarseatswithoutpayingthesubscriptionfee? DeepLearningwithBest-so-farinsteadofWhere-you-are Doesduskreallyremainforashorterperiodoftimeattheequator? ShouldItellmyprospectiveemployerthatIdraftedmyownreferenceletter? Isthereatoolorscriptthatcanveryquicklyfindduplicatesbyonlycomparingfilesizeandasmallfractionofthefilecontents? Isitpossibletohaveeveryblastershothit? WhyisthisLatintextblackbuttheLatinnumeralsarered? WhendidUrsulaliveatthepalace,andwhywassheexiled? Doesachargedparticlerespondtoachargedblackholebothgravitationallyandelectromagnetically? WhyistheUSlistedasa"flaweddemocracy"intheDemocracyIndex? What'sthedifferencebetweenCharging,OfficialCharger,andUSBChargerprofilesinsys-clkmanager? lilypond-hidetupletnumbers MOONLIGHTPHYSICS—Whitefullmoonreflectedorange-redlightoffofAtlanticOceansurface.Whyisthis? Whenacreatureattacksyouversustappedandattackingrules Isitokaytoadmittothepolicethatyouwerespeeding?Arethereanybenefits? morehotquestions Questionfeed SubscribetoRSS Questionfeed TosubscribetothisRSSfeed,copyandpastethisURLintoyourRSSreader. Yourprivacy Byclicking“Acceptallcookies”,youagreeStackExchangecanstorecookiesonyourdeviceanddiscloseinformationinaccordancewithourCookiePolicy. Acceptallcookies Customizesettings