Make max chunk size 1hr #118
Conversation
|
Will need post-merge review #97 |
| @@ -515,7 +515,7 @@ func (i *Ingester) flushSeries(u *userState, fp model.Fingerprint, series *memor | |||
| } | |||
|
|
|||
| firstTime := series.chunkDescs[0].FirstTime() | |||
| flush := immediate || model.Now().Sub(firstTime) > i.cfg.MaxChunkAge | |||
| flush := immediate || len(series.chunkDescs) > 1 || model.Now().Sub(firstTime) > i.cfg.MaxChunkAge | |||
There was a problem hiding this comment.
Why strictly greater than one? I could understand > 0, but > 1 seems a bit odd. I realise at this point we know that it's non-empty, but why do we not want to flush if it has only 1 thing?
Somewhat out of scope for this PR, but it's difficult for me to reason locally about fpLocker. Any reader of this method needs to know about its call-site to understand why locking is necessary.
There was a problem hiding this comment.
Why strictly greater than one?
We need to flush something in the series if:
- we are existing (
immedidate) - a chunk has be filled by the incoming data and a new one created (
len > 1) - the series (and by implication, the one and only chunk) is older than the threshold
There was a problem hiding this comment.
Ah, I see. The logic is that we always flush closed chunks, or open old chunks.
There was a problem hiding this comment.
More of less. Series.Add will 'close' chunks when they get full:
https://github.com/weaveworks/cortex/blob/master/ingester/series.go#L141
(chunks don't actually have a closed concept, we just track it for the head chunk specially)
Also, schedule series for flushing if it has >1 chunk.