Depth and stencils - OpenGL

The stencil buffer is an optional extension of the depth buffer that gives you more control over the question of which fragments should be drawn and which ... Introduction Contextcreation Drawingpolygons Textures Transformations Depthandstencils Framebuffers Geometryshaders TransformFeedback Links OpenGLboilerplatecode Easy-to-buildcode Matrixmathtutorials OpenGLreference Extrabuffers Upuntilnowthereisonlyonetypeofoutputbufferyou'vemadeuseof,thecolorbuffer.Thischapterwilldiscusstwoadditionaltypes,thedepthbufferandthestencilbuffer.Foreachoftheseaproblemwillbepresentedandsubsequentlysolvedwiththatspecificbuffer. Preparations Tobestdemonstratetheuseofthesebuffers,let'sdrawacubeinsteadofaflatshape.Thevertexshaderneedstobemodifiedtoacceptathirdcoordinate: invec3position; ... gl_Position=proj*view*model*vec4(position,1.0); We'realsogoingtoneedtoalterthecoloragainlaterinthischapter,somakesurethefragmentshadermultipliesthetexturecolorbythecolorattribute: vec4texColor=mix(texture(texKitten,Texcoord), texture(texPuppy,Texcoord),0.5); outColor=vec4(Color,1.0)*texColor; Verticesarenow8floatsinsize,soyou'llhavetoupdatethevertexattributeoffsetsandstridesaswell.Finally,addtheextracoordinatetothevertexarray: floatvertices[]={ //XYZRGBUV -0.5f,0.5f,0.0f,1.0f,0.0f,0.0f,0.0f,0.0f, 0.5f,0.5f,0.0f,0.0f,1.0f,0.0f,1.0f,0.0f, 0.5f,-0.5f,0.0f,0.0f,0.0f,1.0f,1.0f,1.0f, -0.5f,-0.5f,0.0f,1.0f,1.0f,1.0f,0.0f,1.0f }; Confirmthatyou'vemadealltherequiredchangesbyrunningyourprogramandcheckingifitstilldrawsaflatspinningimageofakittenblendedwithapuppy.Asinglecubeconsistsof36vertices(6sides*2triangles*3vertices),soIwilleaseyourlifebyprovidingthearrayhere. glDrawArrays(GL_TRIANGLES,0,36); Wewillnotmakeuseofelementbuffersfordrawingthiscube,soyoucanuseglDrawArraystodrawit.Ifyouwereconfusedbythisexplanation,youcancompareyourprogramtothisreferencecode. Itimmediatelybecomesclearthatthecubeisnotrenderedasexpectedwhenseeingtheoutput.Thesidesofthecubearebeingdrawn,buttheyoverlapeachotherinstrangeways!TheproblemhereisthatwhenOpenGLdrawsyourcubetriangle-by-triangle,itwillsimplywriteoverpixelseventhoughsomethingelsemayhavebeendrawntherebefore.InthiscaseOpenGLwillhappilydrawtrianglesinthebackovertrianglesatthefront. LuckilyOpenGLofferswaysoftellingitwhentodrawoverapixelandwhennotto.I'llgooverthetwomostimportantwaysofdoingthat,depthtestingandstencilling,inthischapter. Depthbuffer Z-bufferingisawayofkeepingtrackofthedepthofeverypixelonthescreen.Thedepthisanincreasingfunctionofthedistancebetweenthescreenplaneandafragmentthathasbeendrawn.Thatmeansthatthefragmentsonthesidesofthecubefurtherawayfromtheviewerhaveahigherdepthvalue,whereasfragmentscloserhavealowerdepthvalue. Ifthisdepthisstoredalongwiththecolorwhenafragmentiswritten,fragmentsdrawnlatercancomparetheirdepthtotheexistingdepthtodetermineifthenewfragmentisclosertotheviewerthantheoldfragment.Ifthatisthecase,itshouldbedrawnoverandotherwiseitcansimplybediscarded.Thisisknownasdepthtesting. OpenGLoffersawaytostorethesedepthvaluesinanextrabuffer,calledthedepthbuffer,andperformtherequiredcheckforfragmentsautomatically.Thefragmentshaderwillnotrunforfragmentsthatareinvisible,whichcanhaveasignificantimpactonperformance.ThisfunctionalitycanbeenabledbycallingglEnable. glEnable(GL_DEPTH_TEST); Ifyouenablethisfunctionalitynowandrunyourapplication,you'llnoticethatyougetablackscreen.Thathappensbecausethedepthbufferisfilledwith0depthforeachpixelbydefault.Sincenofragmentswilleverbecloserthanthattheyarealldiscarded. ThedepthbuffercanbeclearedalongwiththecolorbufferbyextendingtheglClearcall: glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT); Thedefaultclearvalueforthedepthis1.0f,whichisequaltothedepthofyourfarclippingplaneandthusthefurthestdepththatcanberepresented.Allfragmentswillbecloserthanthat,sotheywillnolongerbediscarded. Withthedepthtestcapabilityenabled,thecubeisnowrenderedcorrectly.Justlikethecolorbuffer,thedepthbufferhasacertainamountofbitsofprecisionwhichcanbespecifiedbyyou.Lessbitsofprecisionreducetheextramemoryuse,butcanintroducerenderingerrorsinmorecomplexscenes. Stencilbuffer Thestencilbufferisanoptionalextensionofthedepthbufferthatgivesyoumorecontroloverthequestionofwhichfragmentsshouldbedrawnandwhichshouldn't.Likethedepthbuffer,avalueisstoredforeverypixel,butthistimeyougettocontrolwhenandhowthisvaluechangesandwhenafragmentshouldbedrawndependingonthisvalue.Notethatifthedepthtestfails,thestenciltestnolongerdetermineswhetherafragmentisdrawnornot,butthesefragmentscanstillaffectvaluesinthestencilbuffer! Togetabitmoreacquaintedwiththestencilbufferbeforeusingit,let'sstartbyanalyzingasimpleexample. Inthiscasethestencilbufferwasfirstclearedwithzeroesandthenarectangleofoneswasdrawntoit.Thedrawingoperationofthecubeusesthevaluesfromthestencilbuffertoonlydrawfragmentswithastencilvalueof1. Nowthatyouhaveanunderstandingofwhatthestencilbufferdoes,we'lllookattherelevantOpenGLcalls. glEnable(GL_STENCIL_TEST); StenciltestingisenabledwithacalltoglEnable,justlikedepthtesting.Youdon'thavetoaddthiscalltoyourcodejustyet.I'llfirstgoovertheAPIdetailsinthenexttwosectionsandthenwe'llmakeacooldemo. Settingvalues Regulardrawingoperationsareusedtodeterminewhichvaluesinthestencilbufferareaffectedbyanystenciloperation.Ifyouwanttoaffectarectangleofvalueslikeinthesampleabove,simplydrawa2Dquadinthatarea.WhathappenstothosevaluescanbecontrolledbyyouusingtheglStencilFunc,glStencilOpandglStencilMaskfunctions. TheglStencilFunccallisusedtospecifytheconditionsunderwhichafragmentpassesthestenciltest.Itsparametersarediscussedbelow. func:Thetestfunction,canbeGL_NEVER,GL_LESS,GL_LEQUAL,GL_GREATER,GL_GEQUAL,GL_EQUAL,GL_NOTEQUAL,andGL_ALWAYS. ref:Avaluetocomparethestencilvaluetousingthetestfunction. mask:AbitwiseANDoperationisperformedonthestencilvalueandreferencevaluewiththismaskvaluebeforecomparingthem. Ifyoudon'twantstencilswithavaluelowerthan2tobeaffected,youwoulduse: glStencilFunc(GL_GEQUAL,2,0xFF); Themaskvalueissettoallones(incaseofan8bitstencilbuffer),soitwillnotaffectthetest. TheglStencilOpcallspecifieswhatshouldhappentostencilvaluesdependingontheoutcomeofthestencilanddepthtests.Theparametersare: sfail:Actiontotakeifthestenciltestfails. dpfail:Actiontotakeifthestenciltestissuccessful,butthedepthtestfailed. dppass:Actiontotakeifboththestenciltestanddepthtestspass. Stencilvaluescanbemodifiedinthefollowingways: GL_KEEP:Thecurrentvalueiskept. GL_ZERO:Thestencilvalueissetto0. GL_REPLACE:ThestencilvalueissettothereferencevalueintheglStencilFunccall. GL_INCR:Thestencilvalueisincreasedby1ifitislowerthanthemaximumvalue. GL_INCR_WRAP:SameasGL_INCR,withtheexceptionthatthevalueissetto0ifthemaximumvalueisexceeded. GL_DECR:Thestencilvalueisdecreasedby1ifitishigherthan0. GL_DECR_WRAP:SameasGL_DECR,withtheexceptionthatthevalueissettothemaximumvalueifthecurrentvalueis0(thestencilbufferstoresunsignedintegers). GL_INVERT:Abitwiseinvertisappliedtothevalue. Finally,glStencilMaskcanbeusedtocontrolthebitsthatarewrittentothestencilbufferwhenanoperationisrun.Thedefaultvalueisallones,whichmeansthattheoutcomeofanyoperationisunaffected. If,likeintheexample,youwanttosetallstencilvaluesinarectangularareato1,youwouldusethefollowingcalls: glStencilFunc(GL_ALWAYS,1,0xFF); glStencilOp(GL_KEEP,GL_KEEP,GL_REPLACE); glStencilMask(0xFF); Inthiscasetherectangleshouldn'tactuallybedrawntothecolorbuffer,sinceitisonlyusedtodeterminewhichstencilvaluesshouldbeaffected. glColorMask(GL_FALSE,GL_FALSE,GL_FALSE,GL_FALSE); glDepthMask(GL_FALSE); TheglColorMaskfunctionallowsyoutospecifywhichdataiswrittentothecolorbufferduringadrawingoperation.Inthiscaseyouwouldwanttodisableallcolorchannels(red,green,blue,alpha).WritingtothedepthbufferneedstobedisabledseparatelyaswellwithglDepthMask,sothatcubedrawingoperationwon'tbeaffectedbyleftoverdepthvaluesoftherectangle.Thisiscleanerthansimplyclearingthedepthbufferagainlater. Usingvaluesindrawingoperations Withtheknowledgeaboutsettingvalues,usingthemfortestingfragmentsindrawingoperationsbecomesverysimple.Allyouneedtodonowisre-enablecoloranddepthwritingifyouhaddisabledthoseearlierandsettingthetestfunctiontodeterminewhichfragmentsaredrawnbasedonthevaluesinthestencilbuffer. glStencilFunc(GL_EQUAL,1,0xFF); Ifyouusethiscalltosetthetestfunction,thestenciltestwillonlypassforpixelswithastencilvalueequalto1.Afragmentwillonlybedrawnifitpassesboththestencilanddepthtest,sosettingtheglStencilOpisnotnecessary.Inthecaseoftheexampleaboveonlythestencilvaluesintherectangularareaweresetto1,soonlythecubefragmentsinthatareawillbedrawn. glStencilMask(0x00); Onesmalldetailthatiseasytooverlookisthatthecubedrawcallcouldstillaffectvaluesinthestencilbuffer.Thisproblemcanbesolvedbysettingthestencilbitmasktoallzeroes,whicheffectivelydisablesstencilwriting. Planarreflections Let'sspiceupthedemowehaverightnowabitbyaddingafloorwithareflectionunderthecube.I'lladdtheverticesforthefloortothesamevertexbufferthecubeiscurrentlyusingtokeepthingssimple: floatvertices[]={ ... -1.0f,-1.0f,-0.5f,0.0f,0.0f,0.0f,0.0f,0.0f, 1.0f,-1.0f,-0.5f,0.0f,0.0f,0.0f,1.0f,0.0f, 1.0f,1.0f,-0.5f,0.0f,0.0f,0.0f,1.0f,1.0f, 1.0f,1.0f,-0.5f,0.0f,0.0f,0.0f,1.0f,1.0f, -1.0f,1.0f,-0.5f,0.0f,0.0f,0.0f,0.0f,1.0f, -1.0f,-1.0f,-0.5f,0.0f,0.0f,0.0f,0.0f,0.0f } Nowaddtheextradrawcalltoyourmainloop: glDrawArrays(GL_TRIANGLES,36,6); Tocreatethereflectionofthecubeitself,itissufficienttodrawitagainbutinvertedontheZ-axis: model=glm::scale( glm::translate(model,glm::vec3(0,0,-1)), glm::vec3(1,1,-1) ); glUniformMatrix4fv(uniModel,1,GL_FALSE,glm::value_ptr(model)); glDrawArrays(GL_TRIANGLES,0,36); I'vesetthecolorofthefloorverticestoblacksothatthefloordoesnotdisplaythetextureimage,soyou'llwanttochangetheclearcolortowhitetobeabletoseeit.I'vealsochangedthecameraparametersabittogetagoodviewofthescene. Twoissuesarenoticeableintherenderedimage: Theflooroccludesthereflectionbecauseofdepthtesting. Thereflectionisvisibleoutsideofthefloor. Thefirstproblemiseasytosolvebytemporarilydisablingwritingtothedepthbufferwhendrawingthefloor: glDepthMask(GL_FALSE); glDrawArrays(GL_TRIANGLES,36,6); glDepthMask(GL_TRUE); Tofixthesecondproblem,itisnecessarytodiscardfragmentsthatfalloutsideofthefloor.Soundslikeit'stimetoseewhatstenciltestingisreallyworth! Itcanbegreatlybeneficialattimeslikethesetomakealittlelistoftherenderingstagesofthescenetogetaproperideaofwhatisgoingon. Drawregularcube. Enablestenciltestingandsettestfunctionandoperationstowriteonestoallselectedstencils. Drawfloor. Setstencilfunctiontopassifstencilvalueequals1. Drawinvertedcube. Disablestenciltesting. Thenewdrawingcodelookslikethis: glEnable(GL_STENCIL_TEST); //Drawfloor glStencilFunc(GL_ALWAYS,1,0xFF);//Setanystencilto1 glStencilOp(GL_KEEP,GL_KEEP,GL_REPLACE); glStencilMask(0xFF);//Writetostencilbuffer glDepthMask(GL_FALSE);//Don'twritetodepthbuffer glClear(GL_STENCIL_BUFFER_BIT);//Clearstencilbuffer(0bydefault) glDrawArrays(GL_TRIANGLES,36,6); //Drawcubereflection glStencilFunc(GL_EQUAL,1,0xFF);//Passtestifstencilvalueis1 glStencilMask(0x00);//Don'twriteanythingtostencilbuffer glDepthMask(GL_TRUE);//Writetodepthbuffer model=glm::scale( glm::translate(model,glm::vec3(0,0,-1)), glm::vec3(1,1,-1) ); glUniformMatrix4fv(uniModel,1,GL_FALSE,glm::value_ptr(model)); glDrawArrays(GL_TRIANGLES,0,36); glDisable(GL_STENCIL_TEST); I'veannotatedthecodeabovewithcomments,butthestepsshouldbemostlyclearfromthestencilbuffersection. Nowjustonefinaltouchisrequired,todarkenthereflectedcubealittletomakethefloorlookalittlelesslikeaperfectmirror.I'vechosentocreateauniformforthiscalledoverrideColorinthevertexshader: uniformvec3overrideColor; ... Color=overrideColor*color; Andinthedrawingcodeforthereflectedcube glUniform3f(uniColor,0.3f,0.3f,0.3f); glDrawArrays(GL_TRIANGLES,0,36); glUniform3f(uniColor,1.0f,1.0f,1.0f); whereuniColoristhereturnvalueofaglGetUniformLocationcall. Awesome!Ihopethat,especiallyinchapterslikethese,yougettheideathatworkingwithanAPIaslow-levelasOpenGLcanbealotoffunandposeinterestingchallenges!Asusual,thefinalcodeisavailablehere. Exercises Therearenorealexercisesforthischapter,buttherearealotmoreinterestingeffectsyoucancreatewiththestencilbuffer.I'llleaveresearchingtheimplementationofothereffects,suchasstencilshadowsandobjectoutliningasanexercisetoyou.