Skip to content

feat: basic promql planner for single arg function call#828

Merged
waynexia merged 12 commits intoGreptimeTeam:developfrom
waynexia:basic-prom-planner
Jan 5, 2023
Merged

feat: basic promql planner for single arg function call#828
waynexia merged 12 commits intoGreptimeTeam:developfrom
waynexia:basic-prom-planner

Conversation

@waynexia
Copy link
Copy Markdown
Member

@waynexia waynexia commented Jan 4, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

Implement a basic PromQL planner. It only supports prom functions that require one argument. You can find the list by searching (v instant-vector) at https://prometheus.io/docs/prometheus/latest/querying/functions/, or check the unit tests.

For example, give a simple promql query

abs(some_metric{foo!="bar"})

This planner can generate plan like this (I skip the parser phase in unit test):

Projection: some_metric.timestamp, abs(some_metric.field_0) [timestamp:Timestamp(Millisecond, None), abs(some_metric.field_0):Float64;N]
  PromInstantManipulate: range=[0..100000000], lookback=[1000], interval=[5000], time index=[timestamp] [tag_0:Utf8, timestamp:Timestamp(Millisecond, None), field_0:Float64;N]
    PromSeriesNormalize: offset=[0], time index=[timestamp] [tag_0:Utf8, timestamp:Timestamp(Millisecond, None), field_0:Float64;N]
      TableScan: some_metric, unsupported_filters=[tag_0 != Utf8(\"bar\")] [tag_0:Utf8, timestamp:Timestamp(Millisecond, None), field_0:Float64;N]

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 4, 2023

Codecov Report

Merging #828 (3f240c4) into develop (7762873) will increase coverage by 0.65%.
The diff coverage is 86.56%.

@@             Coverage Diff             @@
##           develop     #828      +/-   ##
===========================================
+ Coverage    85.26%   85.92%   +0.65%     
===========================================
  Files          418      422       +4     
  Lines        55285    56411    +1126     
===========================================
+ Hits         47140    48469    +1329     
+ Misses        8145     7942     -203     
Flag Coverage Δ
rust 85.92% <86.56%> (+0.65%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/promql/src/error.rs 7.69% <0.00%> (+7.69%) ⬆️
src/promql/src/lib.rs 100.00% <ø> (ø)
src/query/src/datafusion.rs 95.30% <ø> (ø)
src/query/src/lib.rs 100.00% <ø> (ø)
src/promql/src/planner.rs 86.74% <86.74%> (ø)
src/query/src/datafusion/planner.rs 65.45% <100.00%> (ø)
src/query/src/query_engine/state.rs 85.05% <100.00%> (ø)
src/table/src/test_util/empty_table.rs 93.61% <100.00%> (+0.75%) ⬆️
src/object-store/src/test_util.rs 0.00% <0.00%> (-100.00%) ⬇️
tests-integration/src/test_util.rs 86.89% <0.00%> (-12.63%) ⬇️
... and 48 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Comment thread src/promql/src/error.rs Outdated
Comment thread src/promql/src/planner.rs Outdated
Comment thread src/promql/src/error.rs Outdated
Comment thread src/promql/src/planner.rs
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Comment thread src/promql/src/planner.rs
Comment thread src/promql/src/planner.rs Outdated
Comment thread src/promql/src/planner.rs Outdated
Comment thread src/promql/src/planner.rs Outdated
Comment thread src/promql/src/planner.rs Outdated
Comment thread src/promql/src/planner.rs Outdated
Comment thread src/promql/src/planner.rs
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@waynexia waynexia requested review from evenyag and killme2008 January 5, 2023 06:46
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
@waynexia waynexia merged commit afd9866 into GreptimeTeam:develop Jan 5, 2023
@waynexia waynexia deleted the basic-prom-planner branch January 5, 2023 08:30
@waynexia waynexia mentioned this pull request Jan 13, 2023
26 tasks
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
…#828)

* wip: draft planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle function args

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* a simple test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* list all operators that accept 1 instant vector as input

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* simplify test cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update cargo lock

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* CR suggessions

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* CR suggessions

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change the way to handle metric name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants