feat: fill*() binop modifiers

[sunng87]

Add our parser implementation for Prometheus 3.10 fill modifiers prometheus/prometheus#17644

pending tasks:

• figure out if it's really necessary to expand tokenId to u16, and the conflicts currently suppressed by %expect 5
• support identifier called fill, fill_left and fill_right

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces comprehensive support for Prometheus 3.10's fill binary operation modifiers. It enhances the parser to correctly interpret fill, fill_left, and fill_right clauses, allowing users to specify default values for unmatched series in vector operations. The changes involve extending the AST with a new structure for fill values, updating the BinModifier to include these values, and modifying the parsing grammar and token definitions to recognize the new syntax. A minor but significant change was also made to increase the token ID capacity.

Highlights

• Prometheus 3.10 Fill Modifiers Support: Implemented support for Prometheus 3.10 fill, fill_left, and fill_right binary operation modifiers, allowing specification of default values for unmatched series in vector operations.
• Abstract Syntax Tree (AST) Enhancements: Introduced a new VectorMatchFillValues struct to represent fill values and integrated it into the BinModifier struct, along with updated display and default implementations.
• Parser and Tokenization Updates: Modified the parsing grammar (promql.y) to recognize the new fill keywords and their associated values, and expanded the TokenId type from u8 to u16 to accommodate additional tokens.
• Serialization and Testing: Updated serialization logic to correctly include fillValues in the JSON representation of BinModifier and added new test cases to validate the parsing and serialization of fill modifiers.

Changelog

• build.rs
  • Updated the CTParserBuilder and ct_token_map to use u16 for DefaultLexerTypes instead of u8.
• src/parser/ast.rs
  • Added the VectorMatchFillValues struct to hold optional lhs and rhs fill values.
  • Modified the BinModifier struct to include a fill_values field of type VectorMatchFillValues.
  • Implemented Display trait for BinModifier to correctly format fill, fill_left, and fill_right clauses.
  • Updated the Default implementation for BinModifier to initialize fill_values with VectorMatchFillValues::default().
  • Added a with_fill_values builder method to BinModifier.
  • Removed the Eq trait derivation from BinModifier, Offset, AtModifier, AggregateExpr, UnaryExpr, BinaryExpr, ParenExpr, SubqueryExpr, NumberLiteral, StringLiteral, VectorSelector, MatrixSelector, Call, and Expr due to the inclusion of f64 (which does not implement Eq) in VectorMatchFillValues.
  • Modified serialization logic for BinModifier to include fillValues.
• src/parser/function.rs
  • Removed the Eq trait derivation from FunctionArgs.
• src/parser/mod.rs
  • Exported the newly added VectorMatchFillValues struct.
• src/parser/promql.y
  • Defined new tokens: FILL, FILL_LEFT, FILL_RIGHT.
  • Introduced a new grammar rule fill_modifiers to parse fill clauses, which can appear after group_modifiers.
  • Added a fill_value rule to parse numeric or duration literals for fill values.
  • Updated metric_identifier and maybe_label rules to recognize the new FILL, FILL_LEFT, and FILL_RIGHT keywords as potential identifiers.
  • Added %expect 5 to suppress parser conflicts.
• src/parser/token.rs
  • Changed the TokenId type alias from u8 to u16.
  • Added T_FILL, T_FILL_LEFT, T_FILL_RIGHT to the KEYWORDS map.
  • Updated the token_display function to provide string representations for the new fill tokens.
  • Adjusted the test ranges for unused and unknown tokens to reflect the increased TokenId range and new tokens.
• tests/ser.rs
  • Updated existing serialization tests for BinaryExpr to include the new fillValues field with null for lhs and rhs by default.
  • Added a new test case to verify the correct serialization of a BinaryExpr with a fill (42) modifier.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[codecov]

Codecov Report

❌ Patch coverage is 99.27007% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.93%. Comparing base (345cd67) to head (0eb8135).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/parser/lex.rs	98.76%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main     #135    +/-   ##
========================================
  Coverage   98.93%   98.93%            
========================================
  Files          15       15            
  Lines        6560     6693   +133     
========================================
+ Hits         6490     6622   +132     
- Misses         70       71     +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gemini-code-assist]

Code Review

This pull request introduces support for fill*() binary operation modifiers, aligning with recent additions to Prometheus, with comprehensive changes across the AST, parser grammar, token definitions, and serialization logic, including new tests. However, a critical Denial of Service (DoS) vulnerability has been identified in the parser grammar. The number_duration_literal rule in src/parser/promql.y uses .unwrap() on user-controlled input, which can lead to a panic and application crash with malformed input. This requires refactoring to use proper error propagation with Result and the ? operator to ensure parser stability and prevent potential DoS attacks.

[evenyag]

@codex review

[sunng87]

Just added some more check to allow fill/fill_left/fill_right being used as metric identifiers

[waynexia]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b4cb570a75

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".