Skip to content

[Feature][Audit] Add alert evaluation and periodic audit check task #11999

@ZhenyuePan

Description

@ZhenyuePan

Description

This PR introduces a new alert evaluation mechanism and periodic audit check task for the InLong Audit system. The main features include:

1.Alert Evaluator:

  • Implements AlertEvaluator to evaluate audit metrics against alert rules
  • Compares source and sink metrics to detect anomalies based on configurable thresholds
  • Supports multiple comparison operators (>, >=, <, <=, ==, !=)
  • Reports alerts to Prometheus and logs when conditions are met

2.​Periodic Audit Check Task:

  • Implements AuditCheckTask a scheduled background job
  • Periodically fetches audit metrics from storage
  • Evaluates all configured alert rules against the metrics
  • Configurable execution interval and time window parameters

3.​Configuration Support:

  • Supports configurable parameters through AppConfig:
  • Execution interval time
  • Time window for metrics query
  • Source audit ID
  • Provides default values when configuration is missing

4.​Error Handling:

  • Graceful handling of missing or invalid metrics
  • Proper shutdown procedure for the scheduled task
  • Logging for important operations and errors
    The implementation helps monitor data consistency between source and sink components, providing early warning when discrepancies exceed configured thresholds.

Use case

This PR introduces a new alert evaluation mechanism and periodic audit check task for the InLong Audit system. The main features include:

1.Alert Evaluator:

  • Implements AlertEvaluator to evaluate audit metrics against alert rules
  • Compares source and sink metrics to detect anomalies based on configurable thresholds
  • Supports multiple comparison operators (>, >=, <, <=, ==, !=)
  • Reports alerts to Prometheus and logs when conditions are met

2.​Periodic Audit Check Task:

  • Implements AuditCheckTask a scheduled background job
  • Periodically fetches audit metrics from storage
  • Evaluates all configured alert rules against the metrics
  • Configurable execution interval and time window parameters

3.​Configuration Support:

  • Supports configurable parameters through AppConfig:
  • Execution interval time
  • Time window for metrics query
  • Source audit ID
  • Provides default values when configuration is missing

4.​Error Handling:

  • Graceful handling of missing or invalid metrics
  • Proper shutdown procedure for the scheduled task
  • Logging for important operations and errors
    The implementation helps monitor data consistency between source and sink components, providing early warning when discrepancies exceed configured thresholds.

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions