Skip to content

Make max chunk size 1hr #118

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 7, 2016
Merged

Make max chunk size 1hr #118

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
27a4a0f
Schedule series for flushing if there is more than one chunk
tomwilkie Nov 7, 2016
bb468e4
Increase max chunk size to 1hr
tomwilkie Nov 7, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/cortex/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ func main() {
flag.DurationVar(&cfg.remoteTimeout, "remote.timeout", 5*time.Second, "Timeout for downstream ingesters.")
flag.DurationVar(&cfg.ingesterConfig.FlushCheckPeriod, "ingester.flush-period", 1*time.Minute, "Period with which to attempt to flush chunks.")
flag.DurationVar(&cfg.ingesterConfig.RateUpdatePeriod, "ingester.rate-update-period", 15*time.Second, "Period with which to update the per-user ingestion rates.")
flag.DurationVar(&cfg.ingesterConfig.MaxChunkAge, "ingester.max-chunk-age", 10*time.Minute, "Maximum chunk age before flushing.")
flag.DurationVar(&cfg.ingesterConfig.MaxChunkAge, "ingester.max-chunk-age", 1*time.Hour, "Maximum chunk age before flushing.")
flag.IntVar(&cfg.numTokens, "ingester.num-tokens", 128, "Number of tokens for each ingester.")
flag.IntVar(&cfg.distributorConfig.ReplicationFactor, "distributor.replication-factor", 3, "The number of ingesters to write to and read from.")
flag.IntVar(&cfg.distributorConfig.MinReadSuccesses, "distributor.min-read-successes", 2, "The minimum number of ingesters from which a read must succeed.")
Expand Down
2 changes: 1 addition & 1 deletion ingester/ingester.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -515,7 +515,7 @@ func (i *Ingester) flushSeries(u *userState, fp model.Fingerprint, series *memor
}

firstTime := series.chunkDescs[0].FirstTime()
flush := immediate || model.Now().Sub(firstTime) > i.cfg.MaxChunkAge
flush := immediate || len(series.chunkDescs) > 1 || model.Now().Sub(firstTime) > i.cfg.MaxChunkAge
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why strictly greater than one? I could understand > 0, but > 1 seems a bit odd. I realise at this point we know that it's non-empty, but why do we not want to flush if it has only 1 thing?

Somewhat out of scope for this PR, but it's difficult for me to reason locally about fpLocker. Any reader of this method needs to know about its call-site to understand why locking is necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why strictly greater than one?

We need to flush something in the series if:

  • we are existing (immedidate)
  • a chunk has be filled by the incoming data and a new one created (len > 1)
  • the series (and by implication, the one and only chunk) is older than the threshold

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. The logic is that we always flush closed chunks, or open old chunks.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More of less. Series.Add will 'close' chunks when they get full:

https://github.com/weaveworks/cortex/blob/master/ingester/series.go#L141

(chunks don't actually have a closed concept, we just track it for the head chunk specially)

u.fpLocker.Unlock(fp)

if flush {
Expand Down