Skip to content

Add Parameter to Ignore Delete Messages in Flink StarRocks Connector (Default: false) #392

Open
@bugbubug

Description

@bugbubug

Issue Description:

Background:
The Flink StarRocks Connector currently processes delete messages, which may not be desired in certain use cases. To enhance flexibility and usability, we propose adding a parameter that allows users to ignore delete messages during data ingestion.

Proposed Feature:
We suggest introducing a new parameter, ignoreDeleteMessages, which will be included in both Flink SQL and the Flink API. This parameter will default to false, meaning delete messages will be processed unless explicitly set to true.

Benefits:

  • Provides users with greater control over data ingestion.
  • Enhances performance by reducing unnecessary processing of delete messages in scenarios where they are not needed.

Implementation Considerations:

  • Ensure backward compatibility by maintaining the default behavior of processing delete messages.
  • Update relevant documentation to reflect the new parameter.

We believe this enhancement will significantly improve the user experience and make the connector more versatile.


Issue 描述:

背景:
目前,Flink StarRocks Connector 会处理删除消息,而在某些用例中,这可能并不是所期望的。为了提高灵活性和可用性,我们建议增加一个参数,使用户能够在数据摄取过程中忽略删除消息。

建议功能:
我们建议在 Flink SQL 和 Flink API 中引入一个新参数 ignoreDeleteMessages。该参数的默认值为 false,这意味着删除消息将被处理,除非显式设置为 true

好处:

  • 提供用户更大的数据摄取控制。
  • 通过减少在不需要的情况下对删除消息的处理,提高性能。

实施考虑事项:

  • 确保通过维护处理删除消息的默认行为来实现向后兼容性。
  • 更新相关文档以反映新参数。

我们相信这一增强功能将显著改善用户体验,使连接器更具多样性。

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions