Code

Use TSDB's WAL for writes.#1103

Code

Merged

gouthamve merged 79 commits intocortexproject:mastercortexproject/cortex:masterfrom

grafana:walgrafana/cortex:walCopy head branch name to clipboard

Jan 21, 2020

Commits

Commits on Nov 6, 2018

• Update vendored libraries:

  Show description for 7228c1c
  tomwilkie

  committed
  7228c1c

  Copy full SHA for 7228c1c

• Use TSDB's WAL for writes.

  Show description for 831080f
  tomwilkie

  committed
  831080f

  Copy full SHA for 831080f

Commits on Jul 10, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 8710a22
  codesome

  committed
  8710a22

  Copy full SHA for 8710a22

• Merge pull request #23 from codesome/wal

  Show description for 0802aa6
  codesome

  authored
  0802aa6

  Copy full SHA for 0802aa6

• Fix merge conflicts

  Show description for 2a6fc2d
  codesome

  committed
  2a6fc2d

  Copy full SHA for 2a6fc2d

• Merge remote-tracking branch 'grafana/wal' into wal

  codesome

  committed
  bc25534

  Copy full SHA for bc25534

Commits on Jul 11, 2019

• Fix creation of user states in WAL recover

  Show description for f023ddb
  codesome

  committed
  f023ddb

  Copy full SHA for f023ddb

Commits on Jul 12, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 5c022da
  codesome

  committed
  5c022da

  Copy full SHA for 5c022da

Commits on Aug 6, 2019

• Remove WAL recovery on start-up and allow nil Record

  Show description for ac98b13
  codesome

  committed
  ac98b13

  Copy full SHA for ac98b13

• Fix types

  Show description for b84b02e
  codesome

  committed
  b84b02e

  Copy full SHA for b84b02e

Commits on Aug 12, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  codesome

  committed
  587b8ad

  Copy full SHA for 587b8ad

• WAL compression always enabled

  Show description for 05127b0
  codesome

  committed
  05127b0

  Copy full SHA for 05127b0

Commits on Aug 13, 2019

• Change checkpoint logic to be more like prometheus's tsdb

  Show description for 787fbb8
  codesome

  committed
  787fbb8

  Copy full SHA for 787fbb8

• Add metrics for checkpoint and name changes

  Show description for 9845a5e
  codesome

  committed
  9845a5e

  Copy full SHA for 9845a5e

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 5bb950f
  codesome

  committed
  5bb950f

  Copy full SHA for 5bb950f

Commits on Aug 14, 2019

• Initial attempt for flushing chunks from WAL [WIP]

  Show description for f21f814
  codesome

  committed
  f21f814

  Copy full SHA for f21f814

• Combine checkpoint and WAL chunks before flushing

  Show description for 46eaf36
  codesome

  committed
  46eaf36

  Copy full SHA for 46eaf36

• Bring back recovery and tests

  Show description for 2b17103
  codesome

  committed
  2b17103

  Copy full SHA for 2b17103

• Fix race in the test

  Show description for 935b73e
  codesome

  committed
  935b73e

  Copy full SHA for 935b73e

Commits on Aug 16, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for a7844c7
  codesome

  committed
  a7844c7

  Copy full SHA for a7844c7

Commits on Aug 19, 2019

• Recover on startup

  Show description for 538f407
  codesome

  committed
  538f407

  Copy full SHA for 538f407

Commits on Aug 30, 2019

• Dont remove the last segment in truncation

  Show description for ff35948
  codesome

  committed
  ff35948

  Copy full SHA for ff35948

Commits on Sep 3, 2019

• Always read WAL and remove recover-only mode

  Show description for 2913b99
  codesome

  committed
  2913b99

  Copy full SHA for 2913b99

Commits on Sep 6, 2019

• Timer for WAL recovery time

  Show description for e691e01
  codesome

  committed
  e691e01

  Copy full SHA for e691e01

Commits on Sep 11, 2019

• More rigorous test for WAL

  Show description for dbd336e
  codesome

  committed
  dbd336e

  Copy full SHA for dbd336e

• More profiling in debug mode

  Show description for 0e1577a
  codesome

  committed
  0e1577a

  Copy full SHA for 0e1577a

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for be2ccbf
  codesome

  committed
  be2ccbf

  Copy full SHA for be2ccbf

• Fix race in test

  Show description for eeaae15
  codesome

  committed
  eeaae15

  Copy full SHA for eeaae15

Commits on Sep 12, 2019

• No limits on number of series during recovery

  Show description for 214b32e
  codesome

  committed
  214b32e

  Copy full SHA for 214b32e

Commits on Sep 13, 2019

• No rate limiting of checkpoint

  Show description for 43dbc9e
  codesome

  committed
  43dbc9e

  Copy full SHA for 43dbc9e

Commits on Sep 14, 2019

• Change segment deletion logic

  Show description for 7a9fec0
  codesome

  committed
  7a9fec0

  Copy full SHA for 7a9fec0

Commits on Sep 17, 2019

• Process WAL records in parallel.

  Show description for 2d85a98
  codesome

  committed
  2d85a98

  Copy full SHA for 2d85a98

• Added comments and some refactoring and not returning on no-series for samples

  Show description for 50d22c8
  codesome

  committed
  50d22c8

  Copy full SHA for 50d22c8

Commits on Sep 18, 2019

• Process checkpoint series in parallel

  Show description for 7282ee8
  codesome

  committed
  7282ee8

  Copy full SHA for 7282ee8

• Merge remote-tracking branch 'upstream/master' into wal

  codesome

  committed
  6cd9940

  Copy full SHA for 6cd9940

Commits on Sep 19, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for b8a19d2
  codesome

  committed
  b8a19d2

  Copy full SHA for b8a19d2

Commits on Sep 20, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 370dcea
  codesome

  committed
  370dcea

  Copy full SHA for 370dcea

Commits on Sep 24, 2019

• Fix race in processing WAL

  Show description for 959cf20
  codesome

  committed
  959cf20

  Copy full SHA for 959cf20

• Merge remote-tracking branch 'upstream/master' into wal

  codesome

  committed
  8eb90c9

  Copy full SHA for 8eb90c9

Commits on Sep 27, 2019

• Small enhancements

  Show description for d954865
  codesome

  committed
  d954865

  Copy full SHA for d954865

Commits on Sep 30, 2019

• Cache the user states and series when processing samples

  Show description for fb75c9c
  codesome

  committed
  fb75c9c

  Copy full SHA for fb75c9c

• Enhancement in the user state cache and fix in samples buffer

  Show description for 51beb93
  codesome

  committed
  51beb93

  Copy full SHA for 51beb93

Commits on Oct 1, 2019

• Cache user states and series right from the checkpoint

  Show description for 47a8434
  codesome

  committed
  47a8434

  Copy full SHA for 47a8434

Commits on Oct 2, 2019

• Small enhancements

  Show description for 248566a
  codesome

  committed
  248566a

  Copy full SHA for 248566a

Commits on Oct 3, 2019

• Fix the flag

  Show description for f147570
  codesome

  committed
  f147570

  Copy full SHA for f147570

Commits on Oct 10, 2019

• Remove test files

  Show description for bc8a194
  codesome

  committed
  bc8a194

  Copy full SHA for bc8a194

• Use tsdb from prometheus/prometheus repo

  Show description for f6ddbdf
  codesome

  committed
  f6ddbdf

  Copy full SHA for f6ddbdf

Commits on Oct 25, 2019

• Avoid flushing on shutdown

  Show description for ad3476e
  codesome

  committed
  ad3476e

  Copy full SHA for ad3476e

Commits on Nov 10, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 737c415
  codesome

  committed
  737c415

  Copy full SHA for 737c415

Commits on Nov 12, 2019

• Fix after rebase

  Show description for 251bde5
  codesome

  committed
  251bde5

  Copy full SHA for 251bde5

Commits on Nov 13, 2019

• Fix bug of resetting userStates

  Show description for 88009b9
  codesome

  committed
  88009b9

  Copy full SHA for 88009b9

Commits on Nov 14, 2019

• Fix review comments

  Show description for 5dfe853
  codesome

  committed
  5dfe853

  Copy full SHA for 5dfe853

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for d661843
  codesome

  committed
  d661843

  Copy full SHA for d661843

Commits on Nov 18, 2019

• Small enhancements

  Show description for c55cedc
  codesome

  committed
  c55cedc

  Copy full SHA for c55cedc

Commits on Dec 5, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  codesome

  committed
  02c3fd6

  Copy full SHA for 02c3fd6

• Update comments

  Show description for 902917f
  codesome

  committed
  902917f

  Copy full SHA for 902917f

• Remove ingester<->WAL circular dependancy

  Show description for 4b0d578
  codesome

  committed
  4b0d578

  Copy full SHA for 4b0d578

Commits on Dec 6, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for d2fb739
  codesome

  committed
  d2fb739

  Copy full SHA for d2fb739

• Change segment size of the WAL

  Show description for d799951
  codesome

  committed
  d799951

  Copy full SHA for d799951

• Use same directory for temporary tokens file

  Show description for 69e0a48
  codesome

  committed
  69e0a48

  Copy full SHA for 69e0a48

• Merge branch 'fix-tokens-on-file' into wal

  codesome

  committed
  9758838

  Copy full SHA for 9758838

Commits on Dec 11, 2019

• Disble transfer out when WAL is enabled

  Show description for 14218bf
  codesome

  committed
  14218bf

  Copy full SHA for 14218bf

• Flush on shutdown endpoint irrespective of WAL

  Show description for 3f31526
  codesome

  committed
  3f31526

  Copy full SHA for 3f31526

Commits on Dec 16, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for db679b1
  codesome

  committed
  db679b1

  Copy full SHA for db679b1

• Use sync.Pool for the records

  Show description for 7e6f5de
  codesome

  committed
  7e6f5de

  Copy full SHA for 7e6f5de

Commits on Dec 17, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  codesome

  committed
  60637c1

  Copy full SHA for 60637c1

Commits on Dec 18, 2019

• Fix Goutham's comments

  Show description for f2d0b29
  codesome

  committed
  f2d0b29

  Copy full SHA for f2d0b29

Commits on Dec 19, 2019

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 4dd4ce6
  codesome

  committed
  4dd4ce6

  Copy full SHA for 4dd4ce6

Commits on Dec 20, 2019

• Fix Goutham's comments

  Show description for ce6a244
  codesome

  committed
  ce6a244

  Copy full SHA for ce6a244

• Fix possible data corruption, goroutine deadlock and memory leak

  Show description for 3fc8e31
  codesome

  committed
  3fc8e31

  Copy full SHA for 3fc8e31

Commits on Jan 3, 2020

• Fix review comments

  Show description for e618463
  codesome

  committed
  e618463

  Copy full SHA for e618463

Commits on Jan 15, 2020

• memoryChunks counter fix, metics updated, small cleanup

  Show description for 037d4b3
  codesome

  committed
  037d4b3

  Copy full SHA for 037d4b3

Commits on Jan 20, 2020

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 27c6f85
  codesome

  committed
  27c6f85

  Copy full SHA for 27c6f85

• Update config file and argument doc

  Show description for 3fb38ca
  codesome

  committed
  3fb38ca

  Copy full SHA for 3fb38ca

• Add guide to run/migrate-to WAL in ingesters

  Show description for 92c1149
  codesome

  committed
  92c1149

  Copy full SHA for 92c1149

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for 30c71a1
  codesome

  committed
  30c71a1

  Copy full SHA for 30c71a1

• Fix review comments

  Show description for 8b55cdb
  codesome

  committed
  8b55cdb

  Copy full SHA for 8b55cdb

• Merge remote-tracking branch 'upstream/master' into wal

  Show description for aaa2aa2
  codesome

  committed
  aaa2aa2

  Copy full SHA for aaa2aa2

Commits on Jan 21, 2020

• Fix review comments

  Show description for fe3cc08
  codesome

  committed
  fe3cc08

  Copy full SHA for fe3cc08