Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry on 504 errors (and others?) #2174

Open
benoit74 opened this issue Feb 20, 2025 · 0 comments
Open

Retry on 504 errors (and others?) #2174

benoit74 opened this issue Feb 20, 2025 · 0 comments

Comments

@benoit74
Copy link
Contributor

We have some fandom pages which are failing with a 504 error and stopping the run. See e.g. https://farm.openzim.org/pipeline/7dd8a1d4-7759-4f0a-8252-58c2a33ced68/debug

Trying these URLs manually, it looks like they sometimes work on second or third try, i.e. the problem is only that upstream was still processing, but when retrying the data is probably fresh inside the DB and hence achieves to make it before the timeout. But this is only "sometimes".

We should be cautious with that (to not spend too much time retrying) but retrying these pages could maybe help.

Somewhat related to #2152 (handling of 429) and #1617 as maybe kinda a duplicate of #1596

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant