Inspect and test web pages for Structured Data. Includes both a Command Line Interface for easy ad-hoc testing of URLs and library with extendable API for use ...
Skiptocontent
{{message}}
iaincollins
/
structured-data-testing-tool
Public
Notifications
Fork
12
Star
50
AlibraryandcommandlinetooltohelpinspectandtestforStructuredData.
www.npmjs.com/package/structured-data-testing-tool
License
ISClicense
50
stars
12
forks
Star
Notifications
Code
Issues
12
Pullrequests
12
Actions
Projects
0
Wiki
Security
Insights
More
Code
Issues
Pullrequests
Actions
Projects
Wiki
Security
Insights
iaincollins/structured-data-testing-tool
Thiscommitdoesnotbelongtoanybranchonthisrepository,andmaybelongtoaforkoutsideoftherepository.
master
Branches
Tags
Couldnotloadbranches
Nothingtoshow
{{refName}}
default
Couldnotloadtags
Nothingtoshow
{{refName}}
default
14
branches
11
tags
Code
Latestcommit
Gitstats
101
commits
Files
Permalink
Failedtoloadlatestcommitinformation.
Type
Name
Latestcommitmessage
Committime
.github
__tests__
bin
examples
lib
presets
schema.org
.gitignore
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE.txt
README.md
index.js
jest.setup.js
package-lock.json
package.json
presets.js
schemas.json
Viewcode
StructuredDataTestingTool
Install
Features
Usage
CommandLineInterface
ExampleoutputfromCLI
API
HowtotestaURL
HowtotestalocalHTMLfile
Howtodefineyourowntests
Howtodefineyourownpresets
PresetExample1
PresetExample2
Testoptions
test
type
expect
warning
optional
conditional
group
groups
schema
Testingwithclientsiderendering
Contributing
README.md
StructuredDataTestingTool
InspectandtestwebpagesforStructuredData.
IncludesbothaCommandLineInterfaceforeasyad-hoctestingofURLsandlibrarywithextendableAPIforusewhenwritingtestsorbuildingothertools.
Install
Toinstallthecommandlinetool(sdtt),includethe-g(global)flagwheninstalling:
npmistructured-data-testing-tool-g
Features
CommandLineInterface(sdtt)andAPIthatcanbeusedwithanytestframework.
AcceptsaURL,fileorstring,bufferorstreamcontainingHTMLorJSON.
AutomaticallydetectsallSchema.orgschemas,inHTML(microdata),JSON-LDandRDFa.
Cantesttags(andcustomschemas)forspecifictags/fields/values.
Built-inpresetsfortestingforTwitter,FacebookandGooglestructureddata.
Supportcreationofcustompresetstotestanyschemaortestsspecifictoyoursite.
UsewithaheadlessbrowsertotestStructuredDatainjectedbyclientsideJavaScript(e.g.GoogleTagManager).
Usage
CommandLineInterface
Usage:sdtt--url[--presets][--schemas[--presets][--schemas--presetsSocialMediaTestaURLforsocialmediametatags
sdtt--url--presetsGoogleTestaURLformarkupinspectedbyGoogle
sdtt--url--presets"Twitter,Facebook"TestaURLwithmultiplepresets
sdtt--url-pTwitter-pFacebookTestaURLwithmultiplepresets(alternative)
sdtt--url--schemasArticleTestaURLfortheArticleschema
sdtt--url--schemas"jsonld:Article"TestaURLfortheArticleschemainJSON-LD
sdtt--url--schemas"microdata:Article"TestaURLfortheArticleschemainmicrodata/HTML
sdtt--url--schemas"rdfa:Article"TestaURLfortheArticleschemainRDFa
sdtt--url--schemas"Article,WPHeader,WPFooter"TestaURLformultipleschemas
sdtt--url-sArticle-sWPHeader-sWPFooterTestaURLformultipleschemas(alternative)
sdtt--url--outputresults.jsonOutputtestresultstoaJSONfile
sdtt--file.htmlTestfilecontainingHTML
sdtt--file.jsonTestfilecontainingJSON-LD
sdtt--presetsListallbuilt-inpresets
sdtt--schemasListallsupportedschemas
InspectaURLtoseewhatmarkupisfound:
sdtt--url
Inspectafiletoseewhatmarkupisfound:
sdtt--file
TestaURLcontainsspecificmarkup:
sdtt--url--presets"Twitter,Facebook"
TestaURLcontainsspecificschema:
sdtt--url--schemas"Article"
TestaURLcontainsspecificschemainbothJSON-LDandinmicrodata/HTML:
sdtt--url--schemas"jsonld:Article,microdata:Article"
Runsdtt--presetstolistthebuilt-in-presets:
NAMEDESCRIPTION
GoogleCheckforcommonmarkupusedbyGoogle
TwitterSuggestedmetatagsforTwitter
FacebookSuggestedmetatagsforFacebook
SocialMediaSuggestedmarkupforintegrationwithsocialmediasites
ExampleoutputfromCLI
$sdtt--urlhttps://www.bbc.co.uk/news/world-us-canada-49060410--presetsGoogle,SocialMedia
Tests
Schema.org>ReportageNewsArticle-100%(1passed,0failed)
✓schemainjsonld
•@context
•@type
•url
•publisher.@type
•publisher.name
•publisher.publishingPrinciples
•publisher.logo.@type
•publisher.logo.url
•datePublished
•dateModified
•headline
•image.@type
•image.width
•image.height
•image.url
•thumbnailUrl
•author.@type
•author.name
•author.logo.@type
•author.logo.url
•author.noBylinesPolicy
•mainEntityOfPage
•video.@list[0].@type
•video.@list[0].name
•video.@list[0].description
•video.@list[0].duration
•video.@list[0].thumbnailUrl
•video.@list[0].uploadDate
•video.@list[1].@type
•video.@list[1].name
•video.@list[1].description
•video.@list[1].duration
•video.@list[1].thumbnailUrl
•video.@list[1].uploadDate
Google>ReportageNewsArticle>#0(jsonld)-100%(12passed,0failed)
✓ReportageNewsArticle
✓@type
✓author
✓datePublished
✓headline
✓image
✓publisher.@type
✓publisher.name
✓publisher.logo
✓publisher.logo.url
✓dateModified
✓mainEntityOfPage
Facebook-100%(8passed,0failed)
✓musthavepagetitle
✓musthavepagetype
✓musthaveurl
✓musthaveimageurl
✓musthaveimagealttext
✓shouldhavepagedescription
✓shouldhaveaccountusername
✓shouldhavelocale
Twitter-100%(7passed,0failed)
✓musthavecardtype
✓musthavetitle
✓musthavedescription
✓musthaveimageurl
✓musthaveimagealttext
✓shouldhaveaccountusername
✓shouldhaveusernameofcontentcreator
Statistics
NumberofMetatags:38
SchemasinJSON-LD:1
SchemasinHTML:0
SchemainRDFa:0
Schema.orgschemas:ReportageNewsArticle
Otherschemas:0
Testgroupsrun:5
Optionaltestsrun:71
Pass/Failtestsrun:28
Results
Passed:28 (100%)
Warnings:0 (0%)
Failed:0 (0%)
✓28testspassedwith0warnings.
Usetheoption'-i'todisplayadditionaldetail.
API
HowtotestaURL
YoucanintegrateStructuredDataTestingToolwithaCD/CIpipelinebyusingtheAPI.
const{structuredDataTest}=require('structured-data-testing-tool')
const{Google,Twitter,Facebook}=require('structured-data-testing-tool/presets')
consturl='https://www.bbc.co.uk/news/world-us-canada-49060410'
letresult
structuredDataTest(url,{
//CheckforcompliancewithGoogle,TwitterandFacebookrecommendations
presets:[Google,Twitter,Facebook],
//CheckthepageincludesaspecificSchema(seehttps://schema.org/docs/full.htmlforalist)
schemas:['ReportageNewsArticle']
})
.then(res=>{
console.log('✅Alltestspassed!')
result=res
})
.catch(err=>{
if(err.type==='VALIDATION_FAILED'){
console.log('❌Sometestsfailed.')
result=err.res
}else{
console.log(err)//Handleothererrorshere(e.g.anerrorfetchingaURL)
}
})
.finally(()=>{
if(result){
console.log(
`Passed:${result.passed.length},`,
`Failed:${result.failed.length},`,
`Warnings:${result.warnings.length}`,
)
console.log(`Schemasfound:${result.schemas.join(',')}`)
//Loopovervalidationerrors
if(result.failed.length>0)
console.log("⚠️Errors:\n",result.failed.map(test=>test))
}
})
HowtotestalocalHTMLfile
YoucanalsotestHTMLinafilebypassingitasastring,astreamorareadablebuffer.
consthtml=fs.readFileSync('./example.html')
structuredDataTest(html)
.then(response=>{/*…*/})
.catch(err=>{/*…*/})
Howtodefineyourowntests
Thebuilt-inpresetsonlycoversomeusecasesandareonlyabletocheckifvaluesaredefined(notwhattheycontain).
WiththeAPIyoucanuseJMESPathquerysyntaxtodefineyourownteststocheckforadditionalpropertiesandspecificvalues.Youcanmixandmatchtestswithpresets.
consttestUrl='https://www.bbc.co.uk/news/world-us-canada-49060410'
constoptions={
tests:[
//Check'NewsArticle'schemaexistsinJSON-LD
{test:'NewsArticle',expect:true,type:'jsonld'},
//Checka'NewsArticle'schemaexistswith'url'propertysettothevalueofthevariable'url'
{test:'NewsArticle[*].url',expect:testUrl},
//Asimilarcheckasabove,butwon'tfail(onlywarn)ifthetestdoesn'tpass
{test:'NewsArticle[*].mainEntityOfPage',expect:testUrl,warning:true},
//TestforaTwittermetatagwithspecificvalue
{test:'"twitter:domain"'expect:'www.bbc.co.uk',type:'metatag'}
]
}
structuredDataTest(testUrl,options)
.then(response=>{/*…*/})
.catch(err=>{/*…*/})
Howtodefineyourownpresets
Apresetisacollectionoftests.
Therearebuilt-inpresetsyoucanuse,youcanlistthemwith--presetsoptionusingtheCLI.YoucanalsoeasilydefineyourowncustompresetswhenusingtheAPI.TheCommandLineInterfaceonlysupportsbuilt-inpresets.
Presetsmusthaveaname(whichshouldideallybeunique,butdoesnothavetobe)anddescriptionandanarrayoftestobjectsintests.Bothnameanddescriptionbearbitrarystrings,testsshouldbeanarrayofvalidtestobjects.
Youcanoptionallygrouptestsbyspecifyingavalueforgroupandsetadefaultschematouseforalltestsinschema.Thesecanbearbitrarystrings,thoughit'srecommendedschemasreflectSchema.orgschemanames.
Ifatestexplicitlydefinesit'sowngrouporschema,thatwilloverridethedefaultvalueforthepresetforthatspecifictest(whichmayimpacthowresultsaregrouped).
Presetscancontainotherpresetsusingthepresetsproperty(anarray).
Presetscanhaveconditionalproperty,whichcontainsatestobject,inwhichcasethetestsinthepresetwillonlyonlyberuniftheconditionaltestpasses.
PresetExample1
consturl='https://www.bbc.co.uk/news/world-us-canada-49060410'
//Thistestshowshowyoucanusedifferenttypesoftestsinonepreset.
constMyCustomPreset={
name:'MyCustomPreset',//Required
description:'TestReportageNewsArticleJSON-LDdataisdefinedandtwittermetadatawasfound',//Required
tests:[//Required(unless'presets'isspecified)
{test:'ReportageNewsArticle',type:'jsonld'},
{test:'"twitter:card"',type:'metatag'},
{test:'"twitter:domain"',expect:'www.bbc.co.uk',type:'metatag',}
],
//Additionaloptionsyoucanuseinapreset:
//group:'MyGroupName',//Agroupnamecanbeusedtogrouptestsinapreset(defaultstopresetname)
//schema:'NewsArticle',//Specifyaschemaatthetoplevelifallthetestsinthepresetapplytothesameschema
//presets:[]//Apresetcanalsoinvokeotherpresets,makingiteasytore-usecustomtests
//conditional:{}//Defineaconditional`test`,whichisevaluatedtodetermineifthepresetshouldrun
}
constoptions={
//The'presets'argumentshouldbeanarrayofpresetobjects
presets:[MyCustomPreset],
//Ifyoujustwanttodetectaschemaexists,youcanpopulatethe
//the'schemas'optionwithalistofschemanames(asstrings).
schemas:['ReportageNewsArticle'],
//Bydefault,anystructureddatadetectedwillautomaticallybetested.
//Set'auto'to'false'ifyouwanttodisablethis(defaultsto'true').
//Thismaymeanyoumisssomeerrors,butmakemakedebuggingeasier.
auto:false
}
structuredDataTest(url,options)
.then(response=>{/*…*/})
.catch(err=>{/*…*/})
PresetExample2
Thisisthecodeforoneofthebuilt-inpresets,ittestsfortheClaimReviewschema.
Itshowshowtowriteapresetthatwillautomaticallyrunagainstallinstancesofagivenschemafound.
Thisisusefultobeabletodowhenyouhavemultipleinstancesofthesameschemaonpage.
NB:Thisexampleisquitesimpleanddoesn'ttryandvalidatethecontentsofthepropertiesintheschemaorcheckforinvalidpropertiesontheschema.
constClaimReview={
name:'ClaimReview',
description:'Afact-checkingreviewofclaimsmade(orreported)insomecreativework(referencedviaitemReviewed).',
//Ifyouadd'schema'propertytoapreset**and**writeteststhatstartwithaselectorlike`ClaimReview[*]`
//(i.e.withtheschemanamefollowedbyanasteriskintheselector)thenthosetestswillautomatically
//berunagainsteveryinstanceofthatschemafound,soyoucaneasilyfindwhereanerrorisifthereare
//multipleinstancesofthesameschemaonapage.
schema:'ClaimReview',
//A'conditional'onapresetortestisjustanormaltestobject.Ifitfailstopass,thetestsinthe
//preset(ortheindividualtest,ifitisusedonatest)willnotberun.
conditional:{
test:'ClaimReview'
},
tests:[
//ExpectedbyGoogle
{test:`ClaimReview`},
{test:`ClaimReview[*]."@type"`,expect:'ClaimReview'},
{test:`ClaimReview[*].url`},
{test:`ClaimReview[*].reviewRating`},
{test:`ClaimReview[*].claimReviewed`},
//Warnings
{test:`ClaimReview[*].author`,warning:true},
{test:`ClaimReview[*].datePublished`,warning:true},
{test:`ClaimReview[*].itemReviewed`,warning:true},
],
}
module.exports={
ClaimReview
}
Testoptions
test
Type:string
Required:true
ThevaluefortestshouldbeavalidJMESPathquery.
ExamplesofJMESPathqueries:
Article
TestArticleschemafound.
Article[*].url
TesturlpropertyofanyArticleschemafound.
Article[0].headline
TestheadlinepropertyoffirstArticleschemafound.
Article[1].headline
TestheadlinepropertyofsecondArticleschemafound.
Article[*].publisher.name
TestnamevalueofpublisheronanyArticleschemafound.
Article[*].publisher."@type"
Test@typevalueofpublisheronanyArticleschemafound.
"twitter:image"||"twitter:image:src"
Checkforametatagnamedeithertwitter:image-or-twitter:image:src
Tips:
Usedoublequotestoescapespecialcharactersinpropertynames.
Youcanconsole.log()thestructuredDatapropertyoftheresponseobjectfromstructuredDataTest()toseewhatsortofmetatagsandstructureddatawasfoundtohelpwithwritingyourowntests.
type
Type:string('json'|'rdfa'|'microdata'|'any')
Required:false
Default:'any'
Youcanspecifyatypetoindicateifmarkupshouldbeinjsonld,rdfaormicrodata(HTML)format.
Youcanalsospecifyavalueofmetatagtochecktags.
Ifyoudonotspecifyatypeforatest,adefaultofanywillbeassumedandalltypeswillbechecked(andifanysourcematches,thetestwillpass).
IfyouspecificallywanttotestforavalueandyouknowifitisJSON-LD,RDFaormicrodatayoushouldspecifytheexplicittypeforthetesttocheck.
expect
Type:boolean|string|RexExp
Required:false
Default:true
Youcanspecifyavalueforexpectthatiseitheraboolean,astringoraRegularExpressionobject(defaultstotrue).
Avalueoftrueindicatesthepropertymustexist(butdoesnotcheckit'svalue).
Avalueoffalsethatindicatesthevaluemustnotexist.
ARegularExpressionisevaluatedagainstthetestquery(thetestpassesifatestforexpressionpasses).
Anyothervalueistreatedasastringandthevalueofthepropertyshouldexactlymatchit.
WhenusingaRegularExpressionifthequerypointstoanarraythenthetestwillpassifanyiteminthearraymatchestheRegularExpression.
ExamplesofhowtouseRegularExpressionswiththeexpectoption:
expect:/^[0-9]+$/g//Valuebeingtestedshouldonlycontainnumbers
expect:/^[A-z]+$/g//Valuebeingtestedshouldonlycontainletters
expect:/^[A-z0-9]+$/g//Valueshouldonlycontainletters,numbersandspaces
Youcanuseregularexpressionstovalidatedates,specificvalues,URLs,etc.
warning
Type:boolean
Required:false
Default:false
Whenwarningissettotrue,ifthetestdoesnotpassitwillonlyresultinawarning.
Thedefaultisfalse,meaningifthetestfailsitwillbecountedasafailure.
optional
Type:boolean
Required:false
Default:false
Whentheoptionalpropertyissettotrueonatest,atestwillnotcountaseitherpassedorfailed,butthetestwillstillberunandtheresultabletobeinspected.
Optionaltestsdonotcounttowardsthetotalnumberoftestsrun,testpassedortestsfailed.TheywillshowupinresultsintheCommandLineInterfaceiftheypass,butnotiftheyfail;howeverpassingoptionaltestsappeardifferentlytoothertestsintheresultstomakeitcleartheyareoptionalchecks.
Youcanuse--info/-iontheCLIorinspecttheoptionalpropertyontheresponsefromtheAPItoseetheresultofanytestthathasoptionalpropertysetonit.However,ifanoptionaltestfailsbecausethepropertyitwastestingdoesnotexist,itwillnotbedisplayedintheCLI.Ifapropertyisoptionalbutrecommended,usethewarningoptioninstead.
Note:Strictlyspeaking,inprinciplenospecificpropertiesonSchema.orgobjectsare"required"butinpracticeimplementationsbyvendorslikeGooglehavesome"required"orexpectedpropertiesandalsorespectsome"optional"properties;thisoptionisusefulforwritingteststhatdon'tfailifavalid,butnotnecessarilyrequired,propertyisnotfound.
Thedefaultisfalse.
conditional
Type:object
Required:false
Default:undefined
Aconditionalobjectcancontainaconditionaltesttoberun,todetermineifthetestitselfshouldberun.
Iftheconditionaltestfails,thetestwillnotberun(anditwillnotbeincludedinthetestresults).Iftheconditionaltestpasses,thetestwillberunasitotherwisewouldbeiftheconditionwasn'tspecified.
Thisisconsideredadvancedusage,tohelpavoidhavingtowriteoverlycomplexteststatements.Conditionaltestobjectsusethesamesyntaxasregulartestobjects,butconditionaltestsarenotincludedintheresults.
Itisparticularlyusefulforcheckingifitisappropriatetorunagroupoftests.Forexample,itisusedbyinternalpresetstocheckifaschemaexists;ifitdoesthenallthetestsforthatschemaarerun(andrequiredtestsmustpass),butifaschemadoesnotexistthennoneofthetestsforthatschemaarerun.
group
Type:string
Required:false
Default:undefined
Youcanpassastringforthegroupvaluetoindicatehowtestsshouldbegroupedwhendisplayingresults.Youdonotneedtospecifyagroupiftestsareinapreset,bydefaultthepresetnamewillbeused.
groups
Type:arrayofstrings
Required:false
Default:undefined
Youcanpassanarrayofstringstobeusedtogrouptests.Thisusedinternallytogrouptestsbythestructureddatatestingtoolandisconsideredadvancedusageforedgecasesituationslikecreatingtestsdynamically.
schema
Type:string
Required:false
Default:undefined
Youcanpassaschemavaluethatindicateswhatschemaatestisfor.Testsindifferentpresetscantestthesameschema,testsinthesamepresetcanalsotestmultipleschemas.
Thisisintendedasanoptiontocontrolhowtestsaregroupedwhendisplayingresults,thevalueisnotcheckedforvalidityandisconsideredadvancedusageforedgecasesituations.
Testingwithclientsiderendering
IfapageusesJavaScriptwithclientsiderenderingtogenerateStructuredData,youcanuseatoollikePuppeteer(aheadlessChromeAPI)tofetchtheHTMLandallowanyclientsideJavaScripttorunandthentesttherenderedpagewiththeStructuredDataTestingTool.
ThiscanbeusedtotestpagesthatrelyonclientsideinjectionwithtoolslikeGoogleTagManagertoaddStructuredDatatopages.
Notes:
Puppeteerisalargepackage(~272MB)andmustbeinstalledseparately.
YoucanonlyusePuppeteerwiththeAPI,nottheCommandLineInterface.
Exampleofhowtousepuppeteerwithstructured-data-testing-tooltowriteatestthatreliesonclientsideJavaScript:
const{structuredDataTest}=require('structured-data-testing-tool')
constpuppeteer=require('puppeteer');
(async()=>{
consturl='https://www.bbc.co.uk/news/world-us-canada-49060410'
constbrowser=awaitpuppeteer.launch();
constpage=awaitbrowser.newPage();
awaitpage.goto(url,{waitUntil:'networkidle2'});
consthtml=awaitpage.evaluate(()=>document.body.innerHTML);
awaitbrowser.close();
awaitstructuredDataTest(html)
.then(response=>{console.log("Alltestspassed.")})
.catch(err=>{console.log("Sometestsfailed.")})
})();
Contributing
Contributionsarewelcome-especiallyadditionsandimprovementstothebuilt-inpresets.
Thiscanincludebugreports,featurerequests,ideas,pullrequests,examplesofhowyouhaveusedthistool(etc).
PleaseseetheCodeofConductandcompletetheissueand/orPullRequesttemplateswhenreportingbugs,requestingenhancementsorcontributingcode.
FeedbackandinsightonhowyouuseStructuredDataTestingToolisalsoveryhelpful.
About
AlibraryandcommandlinetooltohelpinspectandtestforStructuredData.
www.npmjs.com/package/structured-data-testing-tool
Topics
schema-org
json-ld
structured-data
Resources
Readme
License
ISClicense
Codeofconduct
Codeofconduct
Stars
50
stars
Watchers
4
watching
Forks
12
forks
Releases
11
v4.5-AddoptiontoAPItodisableauto-insertionoftests
Latest
Mar11,2020
+10releases
Packages0
Nopackagespublished
Contributors3
Languages
JavaScript
93.3%
HTML
6.7%
Youcan’tperformthatactionatthistime.
Yousignedinwithanothertaborwindow.Reloadtorefreshyoursession.
Yousignedoutinanothertaborwindow.Reloadtorefreshyoursession.
Schema Testing tool