Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix occasional cleartask failures #18859

Merged
merged 1 commit into from
Oct 11, 2021

Conversation

potiuk
Copy link
Member

@potiuk potiuk commented Oct 9, 2021

The cleartask tests occasionally failed due to not consistent
sequence in which task clearing was performed.

The query did not have ordering and sometimes the tasks were
returned in different order than expected.


^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@potiuk
Copy link
Member Author

potiuk commented Oct 9, 2021

@uranusjr - here is another case where sorting will fix the occasional "flakiness" - i saw those test failing in at least 4-5 cases during last 24 hours. and if we sort it, they wont (the error was that clearing tasks failed because randomly a "dependent" task was cleared first before the task it depended on - so the tests failed).

If we sort it according to task_id (which guarantees that we will never clear a dependent task before the one it depends on) - those "clear" tests will always succeed. I think this is really simple solution to a difficult problme that we are struggling for weeks.

@potiuk potiuk force-pushed the fix-occasional-test-cleartasks-failure branch from 60c5077 to 0266275 Compare October 9, 2021 22:05
@potiuk
Copy link
Member Author

potiuk commented Oct 9, 2021

@uranusjr - I've added comments to explain why order_by is equivalent to topological_sort - just to make sure this discussion is captured. But I really think trying to do it "right" is not the best approach, better to do it "simple".

The cleartask tests occasionally failed due to not consistent
sequence in which task clearing was performed.

The query did not have ordering and sometimes the tasks were
returned in different order than expected.
@potiuk potiuk force-pushed the fix-occasional-test-cleartasks-failure branch from 0266275 to 69ac9c0 Compare October 10, 2021 08:11
@potiuk potiuk merged commit 6f03529 into apache:main Oct 11, 2021
@potiuk potiuk deleted the fix-occasional-test-cleartasks-failure branch October 11, 2021 22:17
@potiuk potiuk restored the fix-occasional-test-cleartasks-failure branch April 26, 2022 20:50
@potiuk potiuk deleted the fix-occasional-test-cleartasks-failure branch July 29, 2022 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants