Delete workflow execution from visibility store even if it was deleted from main store #3962
Conversation
| if nextPageToken == nil { | ||
| // If nextPageToken is nil then there are no more workflow executions to delete. | ||
| // If SuccessCount == 0 then no progress were made with this run: exit here and let caller retry and eventually fail. | ||
| if nextPageToken == nil || result.SuccessCount == 0 { |
There was a problem hiding this comment.
If success count is 0 and error count is not 0, we should continue?
There was a problem hiding this comment.
If both counters are 0, it means nothing was deleted. And if nextPageToken is not nil it means there is still something in visibility.
There was a problem hiding this comment.
I think my confusion is what happens if nextPageToken != nil && SuccessCount == 0 && ErrorCount != 0, like deletions failed for all workflows in the page case.
There was a problem hiding this comment.
if nextPageToken is not nil (there is something else left) it won't fall inside this if, and will do "continue as new". It will fall inside this if (and exit) only if there is nothing else left (nextPageToken is nil) or no progress were made (SuccessCount is 0). And this is what I explicitly said there in a comment.
There was a problem hiding this comment.
After second thought I think it is unneeded pre-caution. I removed SuccessCount check for now.
alexshtin
force-pushed
the
fix/delete-execution-inconsisten-db
branch
3 times, most recently
from
82b397b to
f8473bb
Compare
…d from main store
alexshtin
force-pushed
the
fix/delete-execution-inconsisten-db
branch
from
f8473bb to
c108f8c
Compare
What changed?
Delete workflow execution from visibility store even if it was deleted from main store.
Why?
DeleteExecutionsworkflow uses visibility store to list workflow executions to delete and if workflow execution was manually deleted from main store but not for visibility store, it gets stuck. With this change, in case ofNotFounderror,DeleteExecutionsworkflow will try to delete execution directly from visibility. If even this operation fails, workflow will exit with error and let caller (ReclaimResourcesworkflow) to retry according to retry policy.How did you test it?
Added functional test.
Potential risks
No risks.
Is hotfix candidate?
No.