feat(bedrock-agentcore-alpha): add OnlineEvaluationConfig and Evaluator L2 constructs

[rezabekf]

Issue # (if applicable)

Closes #37614.

Reason for this change

Amazon Bedrock AgentCore Online Evaluation enables continuous monitoring and assessment of agent performance using live traffic. This PR adds L2 constructs for the evaluation module to the @aws-cdk/aws-bedrock-agentcore-alpha package.

CDK users can now:

• Configure continuous evaluation of agent traces using built-in and custom evaluators
• Control evaluation execution status (ENABLED/DISABLED) via executionStatus prop
• Sample and filter traces for targeted evaluation
• Integrate seamlessly with AgentCore Runtime constructs via DataSourceConfig.fromAgentRuntimeEndpoint()

Description of changes

OnlineEvaluationConfig — L2 construct backed by CfnOnlineEvaluationConfig

• Auto-creates IAM execution role with required permissions (CloudWatch Logs read/write, Bedrock model invocation, index policies)
• Supports executionStatus prop (ExecutionStatus.ENABLED / ExecutionStatus.DISABLED) to control whether evaluation actively processes traces
• Accepts a mix of built-in and custom evaluators via EvaluatorReference
• Provides fromOnlineEvaluationConfigId(), fromOnlineEvaluationConfigArn(), and fromOnlineEvaluationConfigAttributes() import methods
• Implements IGrantable for IAM permission grants and ITaggableV2 for CDK tag propagation
• Input validation for config name, description, evaluators count, sampling percentage, filters count, and session timeout

EvaluatorReference — Unified entry point for referencing evaluators

• EvaluatorReference.builtin() — References one of the 13 pre-defined evaluators (e.g., HELPFULNESS, CORRECTNESS)
• EvaluatorReference.custom() — References a user-created Evaluator construct

Evaluator — L2 construct backed by CfnEvaluator for custom evaluation logic

• EvaluatorConfig.llmAsAJudge() — Foundation model-based evaluation with custom instructions and rating scales (categorical or numerical)
• EvaluatorConfig.codeBased() — Lambda function-based evaluation; automatically grants scoped lambda:InvokeFunction permission with aws:SourceAccount and
  aws:SourceArn conditions (confused deputy prevention)
• Provides fromEvaluatorId(), fromEvaluatorArn(), and fromEvaluatorAttributes() import methods
• Input validation for evaluator name, description, rating scale options, and instructions

EvaluatorRatingScale — Factory class for custom evaluator rating scales

• EvaluatorRatingScale.categorical() — Discrete label-based scoring (e.g., Good/Bad)
• EvaluatorRatingScale.numerical() — Labeled numeric scoring (e.g., 1-5)

DataSourceConfig — Configuration for evaluation data sources

• fromCloudWatchLogs() — For external agents or custom log groups
• fromAgentRuntimeEndpoint() — Seamless integration with AgentCore Runtime (derives log group and service names automatically)

Design decisions:

• Factory classes (EvaluatorConfig, EvaluatorRatingScale, DataSourceConfig) used instead of union types for jsii compatibility
• modelId accepted as plain string — supports standard model IDs and cross-region inference profile IDs (e.g., us.anthropic.claude-sonnet-4-6)
• Instructions placeholder validation delegated to the service — placeholders vary by evaluation level and may change
• Follows existing agentcore patterns (Runtime, Memory, Gateway) with interface + base class + concrete class

Describe any new or updated permissions being added

The auto-created OnlineEvaluationConfig execution role includes:

• CloudWatch Logs Describe (logs:DescribeLogGroups) — unscoped (*), as this action does not support resource-level restrictions
• CloudWatch Logs Query (logs:GetQueryResults, logs:StartQuery) — scoped to user-specified log groups and the aws/spans log group
• CloudWatch Logs Write (logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents) — scoped to
  arn:aws:logs:*:*:log-group:/aws/bedrock-agentcore/evaluations/*
• CloudWatch Index Policy (logs:DescribeIndexPolicies, logs:PutIndexPolicy) — scoped to aws/spans log group
• Bedrock Model Invocation (bedrock:InvokeModel, bedrock:InvokeModelWithResponseStream) — for LLM-as-a-Judge evaluators

Code-based Evaluator construct:

• Lambda Invoke (lambda:InvokeFunction) — granted to bedrock-agentcore.amazonaws.com service principal, scoped with aws:SourceAccount and
  aws:SourceArn conditions to the specific evaluator resource

Description of how you validated changes

Unit Tests (online-evaluation.test.ts + custom-evaluator.test.ts) — 69 evaluation tests covering:

• Creation with minimal and full props for both constructs
• Built-in evaluators: all 13 evaluators, custom sampling and filter configurations
• Custom evaluators: LLM-as-a-Judge (categorical/numerical scales, inference config), code-based (Lambda with timeout, scoped invoke permission)
• EvaluatorReference.builtin() and EvaluatorReference.custom() produce correct evaluator references
• Mixed evaluator usage in OnlineEvaluationConfig
• executionStatus prop (ENABLED, DISABLED, omitted)
• Input validation, token passthrough, grant and import methods for both constructs

Integration Test (integ.online-evaluation.ts)

• Deploys OnlineEvaluationConfig with HELPFULNESS and CORRECTNESS built-in evaluators alongside a custom LLM-as-a-Judge evaluator
• executionStatus: ENABLED
• Deploy verified via integ-runner --update-on-failed

Rosetta — yarn rosetta:extract --strict passes (README and @example docstring snippets compile)

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[github-actions]

⚠️ Experimental Feature: This security report is currently in experimental phase. Results may include false positives and the rules are being actively refined.
This security report is NOT a review blocker. Please try merge from main to avoid findings unrelated to the PR.
To suppress a specific rule, see Suppressing Rules.

---

	Tests	Passed ✅	Skipped	Failed
Security Guardian Results	48 ran	48 passed

Test	Result
No test annotations available

• View Security Guardian Results

[github-actions]

⚠️ Experimental Feature: This security report is currently in experimental phase. Results may include false positives and the rules are being actively refined.
This security report is NOT a review blocker. Please try merge from main to avoid findings unrelated to the PR.
To suppress a specific rule, see Suppressing Rules.

---

	Tests	Passed ✅	Skipped	Failed
Security Guardian Results with resolved templates	48 ran	48 passed

Test	Result
No test annotations available

• View Security Guardian Results with resolved templates