Cortex chunks storage stores indexes and chunks in table-based data storages. When such a storage type is used, multiple tables are created over the time: each ...
DocumentationGettingStartedArchitectureBlocksStorageQuerier
Store-gateway
Compactor
Productiontips
Binaryindex-header
BucketIndex
MigrateCortexclusterfromchunkstoblocks
Convertlong-termstoragefromchunkstoblocks
MigratethestoragefromThanosandPrometheus
LearnmoreChunksStorage(deprecated)Gettingstartedwithchunksstorage
RunningCortexchunksstorageinProduction
RunningCortexchunksstoragewithCassandra
SchemaConfiguration
Table-manager
Caching
IngesterswithWAL
AWStipsConfigurationConfigurationfile
CortexArgumentsExplained
PrometheusFrontend
v1.xGuaranteesGuidesRunningCortexonKubernetes
Gettingstartedwithagossipringcluster
Alertmanagerconfiguration
AuthenticationandAuthorisation
CapacityPlanning
ConfigforhorizontallyscalingtheRuler
ConfigforsendingHAPairsdatatoCortex
DeletingSeries
EncryptionatRest
gRPCstorageplugin
Ingestersrollingupdates
Ingestersscalingupanddown
OverridesExporter
SecuringcommunicationbetweenCortexcomponentswithTLS
Security
ShuffleSharding
Tracing
ZoneAwareReplication
Limitations
GlossaryHTTPAPIOperationsQueryAuditor(tool)
QueryTee(service)
Requestsmirroringtosecondarycluster
ScalingtheQueryFrontendCaseStudiesGojek
REWEdigital
BuoyantCloudProposalsBlocksstoragebucketindex
Blocksstoragesharding
Cross-TenantQueryFederation
DeletionofTenantDatafromBlocksStorage
DocumentationVersioning
GeneralizeModulesServicetomakeitextensible
HTTPAPIDesign
Migratingingestersfromchunkstoblocksandback.
ParallelCompactionbyTimeInterval
RetentionofTenantDatafromBlocksStorage
RulerTenantFederation
ScalableAlertmanager
ScalableQueryFrontend
Shuffleshardingandzoneawareness
Shuffleshardingonthereadpath
SupportmetadataAPI
TimeSeriesDeletionfromBlocksStorageContributingGovernance
DesignpatternsandCodeconventions
Howtorunthewebsitelocally
HowtoupgradeGolangversion
Howintegrationtestswork
Howtoupdatethebuildimage
HowtoaddamaintainerRoadmap
Changelog
CodeofConductEditthispage
Createdocumentationissue
CreateprojectissueDesignPeriodictablesSchemaversioningSchemaconfigperiodic_table_configSchemaconfigexampleDocumentationChunksStorage(deprecated)SchemaConfigurationSchemaConfigurationWarning:thechunksstorageisdeprecated.You’reencouragedtousetheblocksstorage.Cortexchunksstoragestoresindexesandchunksintable-baseddatastorages.Whensuchastoragetypeisused,multipletablesarecreatedoverthetime:eachtable-alsocalledperiodictable-containsthedataforaspecifictimerange.Thetable-basedstoragelayoutisconfiguredthroughaconfigurationfilecalledschemaconfig.Theschemaconfigisusedonlybythechunksstorage,whileit’snotusedbytheblocksstorageengine.DesignThetablebaseddesignbringstwomainbenefits:SchemaconfigchangesEachtableisboundedtoaschemaconfigandversion,sothatchangescanbeintroducedoverthetimeandmultipleschemaconfigscancoexist.RetentionTheretentionisimplementeddeletinganentiretable,whichallowstohavefastdeleteoperations.Thetable-manageristheCortexserviceresponsibleforcreatingaperiodictablebeforeitstimeperiodbegins,anddeletingitonceitsdatatimerangeexceedstheretentionperiod.PeriodictablesAperiodictablestorestheindexorchunksrelativetoaspecificperiodoftime.Thedurationofthetimerangeofthedatastoredinasingletableanditsstoragetypeisconfiguredintheconfigsblockoftheschemaconfigfile.Theconfigsblockcancontainmultipleentries.Eachconfigdefinesthestorageusedbetweenthedaysetinfrom(intheformatyyyy-mm-dd)andthenextconfig,or“now”inthecaseofthelastschemaconfigentry.Thisallowstohavemultiplenon-overlappingschemaconfigsoverthetime,inordertoperformschemaversionupgradesorchangestoragesettings(includingchangingthestoragetype).Thewritepathhitsthetablewherethesampletimestampfallsinto(usuallythelasttable,exceptshortperiodsclosetotheendofatableandthebeginningofthenextone),whilethereadpathhitsthetablescontainingdataforthequerytimerange.SchemaversioningCortexsupportsmultipleschemaversion(currentlythereare11)butwerecommendrunningwiththev9schemaformostusecasesandv10schemaifyouexpecttohaveveryhighcardinalitymetrics.Youcanmovefromoneschematoanotherifanewschemafitsyourpurposebetter,butyoustillneedtoconfigureCortextomakesureitcanreadtheolddataintheoldschemas.SchemaconfigThepathtotheschemaconfigYAMLfilecanbespecifiedtoCortexviatheCLIflag-schema-config-fileandhasthefollowingstructure.configs:[]
Theperiod_configconfiguresasingleperiodduringwhichthestorageisusingaspecificschemaversionandbackendstorage.#ThestartingdateinYYYY-MM-DDformat(eg.2020-03-01).
from:
#Thekey-valuestoretousefortheindex.Supportedvaluesare:
#aws-dynamo,bigtable,bigtable-hashed,cassandra,boltdb.
store:
#Theobjectstoretouseforthechunks.Supportedvaluesare:
#s3,aws-dynamo,bigtable,bigtable-hashed,gcs,cassandra,filesystem.
#Ifnoneisspecified,"store"isusedforstoringchunksaswell.
[object_store:]
#Theschemaversiontouse.Supportedversionsare:v1,v2,v3,v4,v5,
#v6,v9,v10,v11.Werecommendedv9formostusecases,alternatively
#v10ifyouexpecttohaveveryhighcardinalitymetrics.
schema:
index:
chunks:
periodic_table_configTheperiodic_table_configconfiguresthetablesforasingleperiod.#Theprefixtouseforthetablenames.
prefix:
#Thedurationforeachtable.Anewtableiscreatedevery"period",whichalso
#representsthegranularitywithwhichretentionisenforced.Typicallythisvalue
#issetto1w(1week).Mustbeamultipleof24h.
period:
#Thetagstobesetonthecreatedtable.
tags:
