rpk: print why we avoid consuming

[daisukebe]

When a requested offset range is out of available range in a partition, we silently quit. It doesn't send a fetch request to the broker therefore no out-of-range exception is thrown.

For a better o11y, this commit exposes why we quit when requested end offset is lower than the partition's available start offset.

Added a dt test as the filterEmptyPartitions() is a bit complicated.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v25.3.x
• v25.2.x
• v25.1.x

Release Notes

Improvements

• rpk: consume warns when requested end offset is lower than available start offset

[vbotbuildovich]

CI test results

test results on build#77852

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
AuditLogUpgradeTest	test_audit_log_upgrade_all_nodes	null	integration	https://buildkite.com/redpanda/redpanda/builds/77852#019b20f0-f0c3-49fe-a94a-e75838af021f	FAIL	0/1		https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=AuditLogUpgradeTest&test_method=test_audit_log_upgrade_all_nodes
PartitionReassignmentsTest	test_reassignments_cancel	null	integration	https://buildkite.com/redpanda/redpanda/builds/77852#019b20f0-f0c2-4779-a4dd-13d9b9bcfeac	FLAKY	10/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0451, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1293, p1=0.2506, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=PartitionReassignmentsTest&test_method=test_reassignments_cancel

test results on build#77873

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
ShadowLinkingReplicationTests	test_with_restart	null	integration	https://buildkite.com/redpanda/redpanda/builds/77873#019b22d4-a04e-4b97-ae49-fd0df7c4c68e	FLAKY	50/51	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0052, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.0154, p1=0.4606, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=ShadowLinkingReplicationTests&test_method=test_with_restart
PartitionReassignmentsTest	test_reassignments_cancel	null	integration	https://buildkite.com/redpanda/redpanda/builds/77873#019b22d9-ee3d-451f-8dc5-79f476665f64	FLAKY	10/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0811, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.2241, p1=0.0790, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=PartitionReassignmentsTest&test_method=test_reassignments_cancel
RpkConsumeEmptyPartitionWarningTest	test_warning_on_empty_partition_consume	null	integration	https://buildkite.com/redpanda/redpanda/builds/77873#019b22d9-ee38-435f-8863-31f007496e3a	FLAKY	3/11	The test was found to be new, and no failures are allowed	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=RpkConsumeEmptyPartitionWarningTest&test_method=test_warning_on_empty_partition_consume

test results on build#77917

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
CrashLoopChecksTest	test_cloud_storage_client_misconfiguration	null	integration	https://buildkite.com/redpanda/redpanda/builds/77917#019b250a-0d1c-4d2d-9c9f-4da29fc8b0aa	FAIL	0/51	Test PASSES after retries.Inconclusive result after max retries(baseline=0.9265, p0=0.0220, reject_threshold=0.0100. adj_baseline=0.9996, p1=1.0000, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=CrashLoopChecksTest&test_method=test_cloud_storage_client_misconfiguration
CrashLoopChecksTest	test_cloud_storage_client_misconfiguration	null	integration	https://buildkite.com/redpanda/redpanda/builds/77917#019b2511-61de-4912-82bb-c8dbf30cbd52	FLAKY	3/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.9265, p0=0.9957, reject_threshold=0.0100. adj_baseline=0.9996, p1=0.0000, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=CrashLoopChecksTest&test_method=test_cloud_storage_client_misconfiguration
WriteCachingFailureInjectionE2ETest	test_crash_all	{"use_transactions": false}	integration	https://buildkite.com/redpanda/redpanda/builds/77917#019b250a-0d1d-4474-b025-dd7b2e3f1a8a	FLAKY	9/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0795, p0=0.5632, reject_threshold=0.0100. adj_baseline=0.2200, p1=0.3184, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=WriteCachingFailureInjectionE2ETest&test_method=test_crash_all

test results on build#77969

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
DatalakeDLQTest	test_dlq_table_for_mixed_records	{"catalog_type": "rest_jdbc", "cloud_storage_type": 1, "query_engine": "spark"}	integration	https://buildkite.com/redpanda/redpanda/builds/77969#019b27ca-13dc-4646-8478-94de115f1592	FLAKY	49/51	Test PASSES after retries.Inconclusive result after max retries(baseline=0.0013, p0=0.0636, reject_threshold=0.0100. adj_baseline=0.0039, p1=0.9833, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=DatalakeDLQTest&test_method=test_dlq_table_for_mixed_records

test results on build#78012

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
MountUnmountIcebergTest	test_simple_remount	{"cloud_storage_type": 1}	integration	https://buildkite.com/redpanda/redpanda/builds/78012#019b2a57-9832-461c-9478-d87800e371be	FLAKY	9/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.1955, p0=0.8864, reject_threshold=0.0100. adj_baseline=0.4792, p1=0.0150, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=MountUnmountIcebergTest&test_method=test_simple_remount

[vbotbuildovich]

Retry command for Build#77873

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/rpk_consume_empty_partition_warning_test.py::RpkConsumeEmptyPartitionWarningTest.test_warning_on_empty_partition_consume

[vbotbuildovich]

Retry command for Build#77917

please wait until all jobs are finished before running the slash command

/ci-repeat 1
skip-redpanda-build
skip-units
skip-rebase
tests/rptest/tests/crash_loop_checks_test.py::CrashLoopChecksTest.test_cloud_storage_client_misconfiguration

[daisukebe]

The CI failure is tracked in https://redpandadata.atlassian.net/browse/CORE-15016

[daisukebe]

Changed as follows per conversation with @r-vasquez:

When no data to consume, we print it in stderr instead of the debug log via zap and users can notice what's happening. Since it's in stderr, it doesn't break the formatting stuff.

[vbotbuildovich]

/backport v25.3.x