Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log compaction behavior with or without bootstrap is not consistent #389

Closed
vordimous opened this issue Sep 1, 2023 · 0 comments
Closed

Comments

@vordimous
Copy link
Contributor

Describe the bug

The below data shows how many versions of a keyed object get returned from the same topic. If no_updates were coming into the topic then the cache was correct and showed only the latest message. if the topic was updating with multiple new versions of the same keyed object being submitted within a few seconds the result would include multiples of the same keyed object.

Individual fetch requests with object keys and the count of objects returned for each key:

fetch_no_updates_1", {
  air_quality: 1,
  freezer: 1,
  lamp/1: 1,
  lamp/2: 1,
  temperature/basement: 1,
  temperature/roof: 1
}
"fetch_updating_1", {
  air_quality: 4,
  freezer: 5,
  lamp/1: 6,
  lamp/2: 6,
  temperature/basement: 4,
  temperature/roof: 4
}
"fetch_updating_2", {
  air_quality: 2,
  freezer: 2,
  lamp/1: 3,
  lamp/2: 4,
  temperature/basement: 2,
  temperature/roof: 2
}
"fetch_updating_3", {
  air_quality: 3,
  freezer: 2,
  lamp/1: 3,
  lamp/2: 3,
  temperature/basement: 2,
  temperature/roof: 2
}
"fetch_no_updates_2", {
  air_quality: 1,
  freezer: 1,
  lamp/1: 1,
  lamp/2: 1,
  temperature/basement: 1,
  temperature/roof: 1
}

To Reproduce

  • Actively send updates to multiple keyed objects on a log-compacted topic.
  • fetch the full list of messages from that topic over HTTP

Expected behavior
a consistent return of only one message per key when a topic is log-compacted or an indication of when the race condition is possible.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant