User's Manual: Elements of a Test Plan - Apache JMeter

The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 ... Maincontent Twitter github 3.ElementsofaTestPlan Thissectiondescribesthedifferentpartsofatestplan. AminimaltestwillconsistoftheTestPlan,aThreadGroupandoneormoreSamplers. 3.0TestPlan¶ TheTestPlanobjecthasacheckboxcalled"FunctionalTesting".Ifselected,it willcauseJMetertorecordthedatareturnedfromtheserverforeachsample.Ifyouhave selectedafileinyourtestlisteners,thisdatawillbewrittentofile.Thiscanbeusefulif youaredoingasmallruntoensurethatJMeterisconfiguredcorrectly,andthatyourserver isreturningtheexpectedresults.Theconsequenceisthatthefilewillgrowhugequickly,and JMeter'sperformancewillsuffer.Thisoptionshouldbeoffifyouaredoingstress-testing(it isoffbydefault). Ifyouarenotrecordingthedatatofile,thisoptionmakesnodifference. YoucanalsousetheConfigurationbuttononalistenertodecidewhatfieldstosave. 3.1ThreadGroup¶ Threadgroupelementsarethebeginningpointsofanytestplan. Allcontrollersandsamplersmustbeunderathreadgroup. Otherelements,e.g.Listeners,maybeplaceddirectlyunderthetestplan, inwhichcasetheywillapplytoallthethreadgroups. Asthenameimplies,thethreadgroup elementcontrolsthenumberofthreadsJMeterwillusetoexecuteyourtest.The controlsforathreadgroupallowyouto: Setthenumberofthreads Settheramp-upperiod Setthenumberoftimestoexecutethetest Eachthreadwillexecutethetestplaninitsentiretyandcompletelyindependently ofothertestthreads.Multiplethreadsareusedtosimulateconcurrentconnections toyourserverapplication. Theramp-upperiodtellsJMeterhowlongtotaketo"ramp-up"tothefullnumberof threadschosen.If10threadsareused,andtheramp-upperiodis100seconds,then JMeterwilltake100secondstogetall10threadsupandrunning.Eachthreadwill start10(100/10)secondsafterthepreviousthreadwasbegun.Ifthereare30threads andaramp-upperiodof120seconds,theneachsuccessivethreadwillbedelayedby4seconds. Ramp-upneedstobelongenoughtoavoidtoolargeawork-loadatthestart ofatest,andshortenoughthatthelastthreadsstartrunningbefore thefirstonesfinish(unlessonewantsthattohappen). StartwithRamp-up=numberofthreadsandadjustupordownasneeded. Bydefault,thethreadgroupisconfiguredtolooponcethroughitselements. ThreadGroupalsoallowstospecifyThreadlifetime. ClickthecheckboxatthebottomoftheThreadGrouppaneltoenable/disableextrafields inwhichyoucanenterthedurationoftestandthestartupdelay YoucanconfigureDuration(seconds)andStartupDelay(seconds)tocontrol thedurationofeachthreadgroupandtheafterhowmuchsecondsitstarts. Whenthetestisstarted,JMeterwillwaitStartupDelay(seconds)beforestartingtheThreads oftheThreadGroupandrunfortheconfiguredDuration(seconds)time. 3.2Controllers¶ JMeterhastwotypesofControllers:SamplersandLogicalControllers. Thesedrivetheprocessingofatest. SamplerstellJMetertosendrequeststoaserver.For example,addanHTTPRequestSamplerifyouwantJMeter tosendanHTTPrequest.Youcanalsocustomizearequestbyaddingone ormoreConfigurationElementstoaSampler.Formore information,see Samplers. LogicalControllersletyoucustomizethelogicthatJMeterusesto decidewhentosendrequests.Forexample,youcanaddanInterleave LogicControllertoalternatebetweentwoHTTPRequestSamplers. Formoreinformation,seeLogicalControllers. 3.2.1Samplers¶ SamplerstellJMetertosendrequeststoaserverandwaitforaresponse. Theyareprocessedintheordertheyappearinthetree. Controllerscanbeusedtomodifythenumberofrepetitionsofasampler. JMetersamplersinclude: FTPRequest HTTPRequest(canbeusedforSOAPorRESTWebservicealso) JDBCRequest Javaobjectrequest JMSrequest JUnitTestrequest LDAPRequest Mailrequest OSProcessrequest TCPrequest Eachsamplerhasseveralpropertiesyoucanset. YoucanfurthercustomizeasamplerbyaddingoneormoreConfigurationElementstotheTestPlan. Ifyouaregoingtosendmultiplerequestsofthesametype(forexample, HTTPRequest)tothesameserver,considerusingaDefaultsConfiguration Element.EachcontrollerhasoneormoreDefaultselements(seebelow). RemembertoaddaListenertoyourtestplantoviewand/orstorethe resultsofyourrequeststodisk. IfyouareinterestedinhavingJMeterperformbasicvalidationon theresponseofyourrequest,addanAssertionto thesampler.Forexample,instresstestingawebapplication,theserver mayreturnasuccessful"HTTPResponse"code,butthepagemayhaveerrorsonitor maybemissingsections.YoucouldaddassertionstocheckforcertainHTMLtags, commonerrorstrings,andsoon.JMeterletsyoucreatetheseassertionsusingregular expressions. JMeter'sbuilt-insamplers 3.2.2LogicControllers¶ LogicControllersletyoucustomizethelogicthatJMeterusesto decidewhentosendrequests. LogicControllerscanchangetheorderofrequestscomingfromtheir childelements.Theycanmodifytherequeststhemselves,causeJMetertorepeat requests,etc. TounderstandtheeffectofLogicControllersonatestplan,considerthe followingtesttree: TestPlan ThreadGroup OnceOnlyController LoginRequest(anHTTPRequest) LoadSearchPage(HTTPSampler) InterleaveController Search"A"(HTTPSampler) Search"B"(HTTPSampler) HTTPdefaultrequest(ConfigurationElement) HTTPdefaultrequest(ConfigurationElement) CookieManager(ConfigurationElement) Thefirstthingaboutthistestisthattheloginrequestwillbeexecutedonly thefirsttimethrough.Subsequentiterationswillskipit.Thisisduetothe effectsoftheOnceOnlyController. Afterthelogin,thenextSamplerloadsthesearchpage(imaginea webapplicationwheretheuserlogsin,andthengoestoasearchpagetodoasearch).This isjustasimplerequest,notfilteredthroughanyLogicController. Afterloadingthesearchpage,wewanttodoasearch.Actually,wewanttodo twodifferentsearches.However,wewanttore-loadthesearchpageitselfbetween eachsearch.Wecoulddothisbyhaving4simpleHTTPrequestelements(loadsearch, search"A",loadsearch,search"B").Instead,weusetheInterleaveControllerwhichpassesononechildrequesteachtimethroughthetest.Itkeepsthe ordering(i.e.itdoesn'tpassoneonatrandom,but"remembers"itsplace)ofits childelements.Interleaving2childrequestsmaybeoverkill,buttherecouldeasilyhave been8,or20childrequests. NotetheHTTPRequestDefaultsthat belongstotheInterleaveController.Imaginethat"SearchA"and"SearchB"share thesamePATHinfo(anHTTPrequestspecificationincludesdomain,port,method,protocol, path,andarguments,plusotheroptionalitems).Thismakessense-botharesearchrequests, hittingthesameback-endsearchengine(aservletorcgi-script,let'ssay).Ratherthan configurebothHTTPSamplerswiththesameinformationintheirPATHfield,we canabstractthatinformationouttoasingleConfigurationElement.WhentheInterleave Controller"passeson"requestsfrom"SearchA"or"SearchB",itwillfillintheblankswith valuesfromtheHTTPdefaultrequestConfigurationElement.So,weleavethePATHfield blankforthoserequests,andputthatinformationintotheConfigurationElement.Inthis case,thisisaminorbenefitatbest,butitdemonstratesthefeature. ThenextelementinthetreeisanotherHTTPdefaultrequest,thistimeaddedtothe ThreadGroupitself.TheThreadGrouphasabuilt-inLogicController,andthus,ituses thisConfigurationElementexactlyasdescribedabove.Itfillsintheblanksofany Requestthatpassesthrough.ItisextremelyusefulinwebtestingtoleavetheDOMAIN fieldblankinallyourHTTPSamplerelements,andinstead,putthatinformation intoanHTTPdefaultrequestelement,addedtotheThreadGroup.Bydoingso,youcan testyourapplicationonadifferentserversimplybychangingonefieldinyourTestPlan. Otherwise,you'dhavetoediteachandeverySampler. ThelastelementisaHTTPCookieManager.ACookieManagershouldbeaddedtoallwebtests-otherwiseJMeterwill ignorecookies.ByaddingitattheThreadGrouplevel,weensurethatallHTTPrequests willsharethesamecookies. LogicControllerscanbecombinedtoachievevariousresults.Seethelistofbuilt-in LogicControllers. 3.2.3TestFragments¶ TheTestFragmentelementisaspecialtypeofcontrollerthat existsontheTestPlantreeatthesamelevelastheThreadGroupelement.Itisdistinguished fromaThreadGroupinthatitisnotexecutedunlessitis referencedbyeitheraModuleControlleroranInclude_Controller. Thiselementispurelyforcodere-usewithinTestPlans 3.3Listeners¶ ListenersprovideaccesstotheinformationJMetergathersaboutthetestcaseswhile JMeterruns.TheGraphResultslistenerplotstheresponsetimesonagraph. The"ViewResultsTree"Listenershowsdetailsofsamplerrequestsandresponses,andcandisplaybasicHTMLandXMLrepresentationsoftheresponse. Otherlistenersprovidesummaryoraggregationinformation. Additionally,listenerscandirectthedatatoafileforlateruse. EverylistenerinJMeterprovidesafieldtoindicatethefiletostoredatato. ThereisalsoaConfigurationbuttonwhichcanbeusedtochoosewhichfieldstosave,andwhethertouseCSVorXMLformat. NotethatallListenerssavethesamedata;theonlydifferenceisinthewaythedataispresentedonthescreen. Listenerscanbeaddedanywhereinthetest,includingdirectlyunderthetestplan. Theywillcollectdataonlyfromelementsatorbelowtheirlevel. Thereareseverallisteners thatcomewithJMeter. 3.4Timers¶ Bydefault,aJMeterthreadexecutessamplersinsequencewithoutpausing. Werecommendthatyouspecifyadelaybyaddingoneoftheavailabletimersto yourThreadGroup.Ifyoudonotaddadelay,JMetercouldoverwhelmyourserverby makingtoomanyrequestsinaveryshortamountoftime. AtimerwillcauseJMetertodelayacertainamountoftimebeforeeach samplerwhichisinitsscope. IfyouchoosetoaddmorethanonetimertoaThreadGroup,JMetertakesthesumof thetimersandpausesforthatamountoftimebeforeexecutingthesamplerstowhichthetimersapply. Timerscanbeaddedaschildrenofsamplersorcontrollersinordertorestrictthesamplerstowhichtheyareapplied. Toprovideapauseatasingleplaceinatestplan,onecanusetheFlowControlActionSampler. 3.5Assertions¶ Assertionsallowyoutoassertfactsaboutresponsesreceivedfromthe serverbeingtested.Usinganassertion,youcanessentially"test"thatyour applicationisreturningtheresultsyouexpectitto. Forinstance,youcanassertthattheresponsetoaquerywillcontainsome particulartext.ThetextyouspecifycanbeaPerl-styleregularexpression,and youcanindicatethattheresponseistocontainthetext,orthatitshouldmatch thewholeresponse. YoucanaddanassertiontoanySampler.Forexample,youcan addanassertiontoaHTTPRequestthatchecksforthetext,"

".JMeter willthencheckthatthetextispresentintheHTTPresponse.IfJMetercannotfindthe text,thenitwillmarkthisasafailedrequest. Notethatassertionsapplytoallsamplerswhichareintheirscope. Torestrictanassertiontoasinglesampler,addtheassertionasachildofthesampler. Toviewassertionresults,addanAssertionListenertotheThreadGroup. FailedAssertionswillalsoshowupintheTreeViewandTableListeners, andwillcounttowardstheerror%ageforexampleintheAggregateandSummaryreports. 3.6ConfigurationElements¶ AconfigurationelementworkscloselywithaSampler.Althoughitdoesnotsendrequests (exceptforHTTP(S)TestScriptRecorder),itcanaddtoormodifyrequests. Aconfigurationelementisaccessiblefromonlyinsidethetreebranchwhereyouplacetheelement. Forexample,ifyouplaceanHTTPCookieManagerinsideaSimpleLogicController,theCookieManagerwill onlybeaccessibletoHTTPRequestControllersyouplaceinsidetheSimpleLogicController(seefigure1). TheCookieManagerisaccessibletotheHTTPrequests"WebPage1"and"WebPage2",butnot"WebPage3". Also,aconfigurationelementinsideatreebranchhashigherprecedencethanthesameelementina"parent" branch.Forexample,wedefinedtwoHTTPRequestDefaultselements,"WebDefaults1"and"WebDefaults2". Sinceweplaced"WebDefaults1"insideaLoopController,only"WebPage2"canaccessit.TheotherHTTP requestswilluse"WebDefaults2",sinceweplaceditintheThreadGroup(the"parent"ofallotherbranches). Figure1- TestPlanShowingAccessibilityofConfigurationElements TheUserDefinedVariablesConfigurationelementisdifferent. Itisprocessedatthestartofatest,nomatterwhereitisplaced. Forsimplicity,itissuggestedthattheelementisplacedonlyatthestartofaThreadGroup. 3.7Pre-ProcessorElements¶ APre-ProcessorexecutessomeactionpriortoaSamplerRequestbeingmade. IfaPre-ProcessorisattachedtoaSamplerelement,thenitwillexecutejustpriortothatsamplerelementrunning. APre-ProcessorismostoftenusedtomodifythesettingsofaSampleRequestjustbeforeitruns,ortoupdatevariablesthataren'textractedfromresponsetext. SeethescopingrulesformoredetailsonwhenPre-Processorsareexecuted. 3.8Post-ProcessorElements¶ APost-ProcessorexecutessomeactionafteraSamplerRequesthasbeenmade. IfaPost-ProcessorisattachedtoaSamplerelement,thenitwillexecutejustafterthatsamplerelementruns. APost-Processorismostoftenusedtoprocesstheresponsedata,oftentoextractvaluesfromit. SeethescopingrulesformoredetailsonwhenPost-Processorsareexecuted. 3.9Executionorder¶ Configurationelements Pre-Processors Timers Sampler Post-Processors(unlessSampleResultisnull) Assertions(unlessSampleResultisnull) Listeners(unlessSampleResultisnull) PleasenotethatTimers,Assertions,Pre-andPost-Processorsareonlyprocessedifthereisasamplertowhichtheyapply. LogicControllersandSamplersareprocessedintheorderinwhichtheyappearinthetree. Othertestelementsareprocessedaccordingtothescopeinwhichtheyarefound,andthetypeoftestelement. [Withinatype,elementsareprocessedintheorderinwhichtheyappearinthetree]. Forexample,inthefollowingtestplan: Controller Post-Processor1 Sampler1 Sampler2 Timer1 Assertion1 Pre-Processor1 Timer2 Post-Processor2 Theorderofexecutionwouldbe: Pre-Processor1 Timer1 Timer2 Sampler1 Post-Processor1 Post-Processor2 Assertion1 Pre-Processor1 Timer1 Timer2 Sampler2 Post-Processor1 Post-Processor2 Assertion1 3.10ScopingRules¶ TheJMetertesttreecontainselementsthatarebothhierarchicalandordered.Someelementsinthetesttreesarestrictlyhierarchical(Listeners,ConfigElements,Post-Processors,Pre-Processors,Assertions,Timers),andsomeareprimarilyordered(controllers,samplers).Whenyoucreateyourtestplan,youwillcreateanorderedlistofsamplerequest(viaSamplers)thatrepresentasetofstepstobeexecuted.Theserequestsareoftenorganizedwithincontrollersthatarealsoordered.Giventhefollowingtesttree: Exampletesttree Theorderofrequestswillbe,One,Two,Three,Four. Somecontrollersaffecttheorderoftheirsubelements,andyoucanreadaboutthesespecificcontrollersinthecomponentreference. Otherelementsarehierarchical.AnAssertion,forinstance,ishierarchicalinthetesttree. Ifitsparentisarequest,thenitisappliedtothatrequest.Ifits parentisaController,thenitaffectsallrequeststhataredescendantsof thatController.Inthefollowingtesttree: Hierarchyexample Assertion#1isappliedonlytoRequestOne,whileAssertion#2isappliedtoRequestsTwoandThree. Anotherexample,thistimeusingTimers: complexexample Inthisexample,therequestsarenamedtoreflecttheorderinwhichtheywillbeexecuted.Timer#1willapplytoRequestsTwo,Three,andFour(noticehoworderisirrelevantforhierarchicalelements).Assertion#1willapplyonlytoRequestThree.Timer#2willaffectalltherequests. Hopefullytheseexamplesmakeitclearhowconfiguration(hierarchical)elementsareapplied.IfyouimagineeachRequestbeingpassedupthetreebranches,toitsparent,thentoitsparent'sparent,etc.,andeachtimecollectingalltheconfigurationelementsofthatparent,thenyouwillseehowitworks. TheConfigurationelementsHeaderManager,CookieManagerandAuthorizationmanagerare treateddifferentlyfromtheConfigurationDefaultelements. ThesettingsfromtheConfigurationDefaultelementsaremergedintoasetofvaluesthattheSamplerhasaccessto. However,thesettingsfromtheManagersarenotmerged. IfmorethanoneManagerisinthescopeofaSampler, onlyoneManagerisused,butthereiscurrentlynowaytospecifywhichisused. 3.11PropertiesandVariables¶ JMeterpropertiesaredefinedinjmeter.properties(seeGettingStarted-ConfiguringJMeterformoredetails). Propertiesareglobaltojmeter,andaremostlyusedtodefinesomeofthedefaultsJMeteruses. Forexamplethepropertyremote_hostsdefinestheserversthatJMeterwilltrytorunremotely. Propertiescanbereferencedintestplans -seeFunctions-readaproperty- butcannotbeusedforthread-specificvalues. JMetervariablesarelocaltoeachthread.Thevaluesmaybethesameforeachthread,ortheymaybedifferent. Ifavariableisupdatedbyathread,onlythethreadcopyofthevariableischanged. ForexampletheRegularExpressionExtractorPost-Processor willsetitsvariablesaccordingtothesamplethatitsthreadhasread,andthesecanbeusedlater bythesamethread. Fordetailsofhowtoreferencevariablesandfunctions,seeFunctionsandVariables NotethatthevaluesdefinedbytheTestPlanandtheUserDefinedVariablesconfigurationelement aremadeavailabletothewholetestplanatstartup. IfthesamevariableisdefinedbymultipleUDVelements,thenthelastonetakeseffect. Onceathreadhasstarted,theinitialsetofvariablesiscopiedtoeachthread. Otherelementssuchasthe UserParametersPre-ProcessororRegularExpressionExtractorPost-Processor maybeusedtoredefinethesamevariables(orcreatenewones).Theseredefinitionsonlyapplytothecurrentthread. ThesetPropertyfunctioncanbeusedtodefineaJMeterproperty. Theseareglobaltothetestplan,socanbeusedtopassinformationbetweenthreads-shouldthatbeneeded. Bothvariablesandpropertiesarecase-sensitive. 3.12UsingVariablestoparameterisetests¶ Variablesdon'thavetovary-theycanbedefinedonce,andifleftalone,willnotchangevalue. Soyoucanusethemasshort-handforexpressionsthatappearfrequentlyinatestplan. Orforitemswhichareconstantduringarun,butwhichmayvarybetweenruns. Forexample,thenameofahost,orthenumberofthreadsinathreadgroup. WhendecidinghowtostructureaTestPlan, makeanoteofwhichitemsareconstantfortherun,butwhichmaychangebetweenruns. Decideonsomevariablenamesforthese- perhapsuseanamingconventionsuchasprefixingthemwithC_orK_orusinguppercaseonly todistinguishthemfromvariablesthatneedtochangeduringthetest. Alsoconsiderwhichitemsneedtobelocaltoathread- forexamplecountersorvaluesextractedwiththeRegularExpressionPost-Processor. Youmaywishtouseadifferentnamingconventionforthese. Forexample,youmightdefinethefollowingontheTestPlan: HOSTwww.example.com THREADS10 LOOPS20 Youcanrefertotheseinthetestplanas${HOST}${THREADS}etc. Ifyoulaterwanttochangethehost,justchangethevalueoftheHOSTvariable. Thisworksfineforsmallnumbersoftests,butbecomestediouswhentestinglotsofdifferentcombinations. Onesolutionistouseapropertytodefinethevalueofthevariables,forexample: HOST${__P(host,www.example.com)} THREADS${__P(threads,10)} LOOPS${__P(loops,20)} Youcanthenchangesomeorallofthevaluesonthecommand-lineasfollows: jmeter…-Jhost=www3.example.org-Jloops=13 Sharethispage: share tweet Gototop