Add submission_count to flow runs #19162
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
enhancement
CodSpeed Performance ReportMerging #19162 will not alter performanceComparing Summary
|
Closes #19143 Add submission_count field to track the number of times a flow run was submitted (entered Pending state), separate from run_count which tracks executions (Running state). This enables automations to retry flow runs that crash before reaching the Running state, addressing a pain point for users like Humana who need to retry flows that fail during submission. <details> <summary>Changes</summary> - Add `submission_count` field to FlowRun and ORMFlowRun models - Add `IncrementFlowRunSubmissionCount` orchestration policy that increments the count when entering Pending state - Expose `prefect.submission-count` in flow run state change events - Add database migrations for SQLite and PostgreSQL - Add tests including specific case for submission without running </details> **Future work**: While this PR adds tracking, automatic retry logic for crashed submissions could be added via orchestration rules or empirical policy (see Slack discussion with @alex). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
- Add submission_count to FlowRun in server/schemas/core.py (was missing) - Add comprehensive documentation on using submission_count with automations - Document the retry pattern for submission failures - Include examples and best practices
- Fix FlowRunSort.ID_DESC instead of non-existent CREATED_DESC - Fix timedelta import and usage in exponential backoff example - Add new doc to Automations section in docs.json navigation
The Automation class from prefect.automations has an optional id field that defaults to None, allowing .create() to work properly. The prefect.events.schemas.automations.Automation requires an id.
.serve() is a method on flow objects, not imported from prefect.deployments
Fixes TypeScript compilation errors where submission_count was missing from mock FlowRun objects
Remove incorrect example that would actually run the flow (run_count > 0). Submission failures happen during infrastructure setup, not flow execution.
zzstoatzz
force-pushed
the
add-submission-count-19143
branch
from
f3d25a8 to
7d525f5
Compare
|
This pull request is stale because it has been open 14 days with no activity. To keep this pull request open remove stale label or comment. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #19143
Summary
Add
submission_countfield to track the number of times a flow run was submitted (entered Pending state), separate fromrun_countwhich tracks executions (Running state).This enables automations to retry flow runs that crash before reaching the Running state, addressing a pain point for users who need to retry flows that fail during submission.
Key Changes
submission_countfield to FlowRun, ORMFlowRun, and FlowRunResponse modelsIncrementFlowRunSubmissionCountorchestration transform that increments the count when entering Pending stateprefect.submission-countin flow run state change eventsTest Coverage
Added test
test_submission_count_increments_without_runningthat verifies the key use case: a flow run that is submitted (enters Pending) but crashes before reaching Running, resulting insubmission_count=1andrun_count=0.Also smoke tested manually against a running server with all scenarios passing.
Future Work
While this PR adds tracking capability, automatic retry logic for crashed submissions could be added via orchestration rules or empirical policy as a follow-up enhancement.
Migration Notes
Migrations add a new
submission_countcolumn to bothflow_runandtask_runtables withserver_default='0'andnullable=False.🤖 Generated with Claude Code
Co-Authored-By: Claude [email protected]