Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Advanced Visibility with SQL] Changing visibility schema for SQLite #3819

Merged

Conversation

rodrigozhou
Copy link
Contributor

@rodrigozhou rodrigozhou commented Jan 20, 2023

What changed?
Changed SQLite visibility schema to support advanced visibility:

  • Added search attributes JSON map.
  • Added generated columns to extract the search attributes fields from JSON.
  • Changed indices.

Why?
Advanced visibility with SQL databases.

How did you test it?
Existing tests.
Started temporal using SQLite, and ran some samples.

Potential risks
No risks in production, only affects when using with SQLite.

Is hotfix candidate?
No.

@rodrigozhou rodrigozhou requested a review from alexshtin January 20, 2023 02:34
@rodrigozhou rodrigozhou requested a review from a team as a code owner January 20, 2023 02:34
@rodrigozhou rodrigozhou force-pushed the advanced-visibility-sqlite-schema branch 2 times, most recently from b9ee3ac to 7807eb9 Compare January 25, 2023 18:22
@rodrigozhou rodrigozhou changed the title [Advanced Visibility with SQL] Changing SQLite schema [Advanced Visibility with SQL] Changing visibility schema for SQLite Jan 25, 2023
@rodrigozhou rodrigozhou force-pushed the advanced-visibility-sqlite-schema branch 4 times, most recently from a3540cf to 967904b Compare January 31, 2023 01:19
TemporalScheduledStartTime TIMESTAMP GENERATED ALWAYS AS (STRFTIME('%Y-%m-%d %H:%M%:%f+00:00', JSON_EXTRACT(search_attributes, "$.TemporalScheduledStartTime"))),
TemporalScheduledById VARCHAR(255) GENERATED ALWAYS AS (JSON_EXTRACT(search_attributes, "$.TemporalScheduledById")),
TemporalSchedulePaused BOOLEAN GENERATED ALWAYS AS (JSON_EXTRACT(search_attributes, "$.TemporalSchedulePaused")),
TemporalNamespaceDivision VARCHAR(255) GENERATED ALWAYS AS (JSON_EXTRACT(search_attributes, "$.TemporalNamespaceDivision")),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it going to be easy to add new generated fields here, e.g. if we add a new feature like schedules that uses a handful of new predefined search attributes? Will we need to add v4 of the schema or just write a small migration?

Copy link
Contributor Author

@rodrigozhou rodrigozhou Jan 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The v3 means sqlite3, not the schema version. We don't track schema version for sqlite, only for MySQL and PostgreSQL, in which case, you'll need to write scripts to update the schema.

@rodrigozhou rodrigozhou force-pushed the advanced-visibility-sqlite-schema branch from 967904b to 2f9a5a3 Compare January 31, 2023 19:38
@rodrigozhou rodrigozhou merged commit f3cf504 into temporalio:master Jan 31, 2023
@rodrigozhou rodrigozhou deleted the advanced-visibility-sqlite-schema branch January 31, 2023 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants