-
Notifications
You must be signed in to change notification settings - Fork 1.2k
batch: add activity reset and update-options #8061
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…poral into spk/batch-reset-update-op
bergundy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Didn't take a close look at the tests, left some comments about the implementation.
go.mod
Outdated
| go.opentelemetry.io/otel/sdk/metric v1.34.0 | ||
| go.opentelemetry.io/otel/trace v1.34.0 | ||
| go.temporal.io/api v1.50.1-0.20250715164317-6157f960f13b | ||
| go.temporal.io/api v1.50.1-0.20250717214050-28ae23500a33 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please merge the API change before the server PR.
service/worker/batcher/activities.go
Outdated
| WorkflowId: workflowID, | ||
| RunId: runID, | ||
| }, | ||
| // QUESTION: do we want to plumb through the identity from cli/ui/sdk ? @yuri/@chetan |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question, we should be consistent across all batch jobs with our use of identity.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added for these two, the rest of the options do not have an identity in the proto definition. I will add that in a later PR
service/worker/batcher/workflow.go
Outdated
| ActivityOptions *activitypb.ActivityOptions | ||
| UpdateMask *fieldmaskpb.FieldMask |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is fine but you should know that using a nested proto struct in workflow input doesn't use proto JSON serialization, instead it uses plain JSON serialization which may yield unexpected results.
If we were writing this from scratch, the right thing to do here is to define a proto struct for the workflow input and reuse the request structs for each of these actions.
You may want to serialize these proto messages yourself and bypass the SDK's serialization.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Chatted w/ Yuri on this. We already use the proto struct for FieldMask without custom serialization in other batch commands so we should be fine here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would test that every field in these message are properly serialized.
Generally, what we've done here is bad practice and should be avoided in our codebase. It's not just FieldMask, it's also ActivityOptions. Even if we know that all fields are properly serialized today (which I'm not convinced of), there's no guarantee that future fields will.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fyi: There are known issues with the default protobuf json converter when the message contains oneoff fields - https://protobuf.dev/programming-guides/json/. It's recommended to use protojson to explicitly serialize/deserialize in such cases.
In this case you're fine since UpdateActivitiesOptionsParams doesn't contain any protobufs with oneoff fields. But that may change in the future if someone adds additional fields to this input.
bergundy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would like the input serialization issues to be addressed before merging but otherwise LGTM. Please do not merge before addressing.
service/worker/batcher/workflow.go
Outdated
| ActivityOptions *activitypb.ActivityOptions | ||
| UpdateMask *fieldmaskpb.FieldMask |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fyi: There are known issues with the default protobuf json converter when the message contains oneoff fields - https://protobuf.dev/programming-guides/json/. It's recommended to use protojson to explicitly serialize/deserialize in such cases.
In this case you're fine since UpdateActivitiesOptionsParams doesn't contain any protobufs with oneoff fields. But that may change in the future if someone adds additional fields to this input.
…nt proto serialization issue
Created structs for these options for JSON serialization purposes. |
| RestoreOriginal bool | ||
| } | ||
|
|
||
| ActivityOptions struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is suboptimal solution. Instead it would have been better to serialize the proto structs manually or to define the workflow inputs with protos.
What changed?
BatchWorkflow now supports resetting activities and updating options.
Why?
Support future cli work to support
temporal activity {batch,update-options} --query ...How did you test it?
Potential risks
This does not change previous behavior