Skip to content

feat(inverted_index): Add applier builder to convert Expr to Predicates (Part 1)#3034

Merged
killme2008 merged 7 commits intoGreptimeTeam:mainfrom
zhongzc:zhongzc/inverted-index-integration-applier-builder-part-1
Dec 30, 2023
Merged

feat(inverted_index): Add applier builder to convert Expr to Predicates (Part 1)#3034
killme2008 merged 7 commits intoGreptimeTeam:mainfrom
zhongzc:zhongzc/inverted-index-integration-applier-builder-part-1

Conversation

@zhongzc
Copy link
Copy Markdown
Collaborator

@zhongzc zhongzc commented Dec 28, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This PR accomplishes the following:

  1. Primarily, adds preliminary inverted index integration code to the mito2 crate, currently dealing with the read aspect.

    • Introduces SstIndexApplier, which will be embedded in ParquetReaderBuilder in the future to offer indexing application services.

    • Introduces SstIndexApplierBuilder, which, as implied by its name, is responsible for constructing SstIndexApplier. It does a significant amount of work internally to recognize and transform Expr into Predicates understandable by the index crate.

      Given the substantial amount of code for the transformation, I plan to submit it in two separate PRs:

      • This PR is focused on the framework part and includes the Between expression as one of the examples.
      • The subsequent PR will introduce And, Or, Eq, Lt, LtEq, Gt, GtEq, RegexMatch, InList.
  2. Upgrade regex-automata:

    • This enhances the Regex Match Predicate to support anchors, i.e. ^, $.

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR does not require documentation updates.

Refer to a related PR or issue link (optional)

#2705

… to Predicates (Part 1)

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
@github-actions github-actions Bot added docs-not-required This change does not impact docs. Size: L labels Dec 28, 2023
@zhongzc zhongzc changed the title feat(inverted_index.integration): Add applier builder to convert Expr to Predicates (Part 1) feat(inverted_index): Add applier builder to convert Expr to Predicates (Part 1) Dec 28, 2023
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
@zhongzc zhongzc self-assigned this Dec 28, 2023
@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 28, 2023

Codecov Report

Attention: 40 lines in your changes are missing coverage. Please review.

Comparison is base (b58296d) 85.54% compared to head (c56f4ab) 84.91%.
Report is 8 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3034      +/-   ##
==========================================
- Coverage   85.54%   84.91%   -0.64%     
==========================================
  Files         793      799       +6     
  Lines      128447   129030     +583     
==========================================
- Hits       109880   109565     -315     
- Misses      18567    19465     +898     

Comment thread Cargo.toml
Comment thread src/mito2/src/sst/index/applier.rs Outdated
Comment thread src/mito2/src/sst/index/applier/builder.rs
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Comment thread src/mito2/src/sst/index/applier/builder.rs Outdated
zhongzc and others added 3 commits December 29, 2023 14:54
Co-authored-by: Yingwen <realevenyag@gmail.com>
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Copy link
Copy Markdown
Member

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Comment thread src/mito2/src/error.rs
@killme2008 killme2008 added this pull request to the merge queue Dec 30, 2023
Merged via the queue into GreptimeTeam:main with commit 69a5313 Dec 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs.

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants