Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added completeness checking store #397

Conversation

blakehatch
Copy link
Contributor

@blakehatch blakehatch commented Nov 16, 2023

Description

The proto promises that all results will exist in the CAS when requested using the associated actions. However we currently allow stores to prune themselves which violates this condition, because the action cache and CAS are different. Therefore we need this completeness checking store to check that all results exist in the CAS before we allow the the action result to be returned to the client.

Fixes #358 #362

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Added test where all files are present in the cas causing completeness to succeed and another where the cas does not have all of the files causing completeness checking to fail.

Checklist

  • Updated documentation if needed
  • Tests added/amended
  • bazel test //... passes locally
  • PR is contained in a single commit, using git amend see some docs

This change is Reviewable

@blakehatch
Copy link
Contributor Author

blakehatch commented Nov 16, 2023

The algorithm is has_with_results does parallelize but may be sub-optimal based on how has checks of file digests in cas is done.

@blakehatch blakehatch force-pushed the completeness-checking-store-restructuring branch from 7a6e0bf to 73d8ee7 Compare November 16, 2023 15:50
@blakehatch blakehatch force-pushed the completeness-checking-store-restructuring branch from 8b47982 to 8540019 Compare November 16, 2023 15:53
@blakehatch blakehatch closed this Nov 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement Completeness checking store
1 participant