Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: improving robustness of unpublish.spec e2e #9052

Merged
merged 2 commits into from
Mar 27, 2025

Conversation

jordanl17
Copy link
Member

@jordanl17 jordanl17 commented Mar 25, 2025

Description

Sometimes there can be 2 'Published' elements available after unpublishing is run in unpublish.spec - this is because briefly the Publish document action button shows 'Published' as the document state is reconciled.

This PR places a data-testid around the document perspective list, and uses this to target the correct 'Published'

What to review

Testing

Repeat running of this particular test (locally and in CI) shows improved robustness

Notes for release

N/A

Copy link

vercel bot commented Mar 25, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 26, 2025 2:09pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 26, 2025 2:09pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Mar 26, 2025 2:09pm
2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Mar 26, 2025 2:09pm
test-next-studio ⬜️ Ignored (Inspect) Mar 26, 2025 2:09pm

Copy link
Contributor

No changes to documentation

Copy link
Contributor

github-actions bot commented Mar 25, 2025

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 43.29% 56164 / 129715
🔵 Statements 43.29% 56164 / 129715
🔵 Functions 47.3% 2778 / 5873
🔵 Branches 79.7% 10460 / 13124
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
packages/sanity/src/structure/panes/document/documentPanel/header/DocumentPanelHeader.tsx 10.95% 100% 0% 10.95% 43-68, 72-282, 292-308, 316-350
Generated in workflow #32889 for commit 4b228c1 by the Vitest Coverage Report Action

Copy link
Contributor

github-actions bot commented Mar 25, 2025

📊 Playwright Test Report

Download Full E2E Report

This report contains test results, including videos of failing tests.

Copy link
Contributor

github-actions bot commented Mar 25, 2025

⚡️ Editor Performance Report

Updated Wed, 26 Mar 2025 14:14:20 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 18.5 efps (54ms) 18.5 efps (54ms) +0ms (-/-%)
article (body) 67.6 efps (15ms) 68.0 efps (15ms) -0ms (-/-%)
article (string inside object) 19.6 efps (51ms) 19.0 efps (53ms) +2ms (+2.9%)
article (string inside array) 17.5 efps (57ms) 17.4 efps (58ms) +1ms (+0.9%)
recipe (name) 47.6 efps (21ms) 50.0 efps (20ms) -1ms (-4.8%)
recipe (description) 52.6 efps (19ms) 52.6 efps (19ms) +0ms (-/-%)
recipe (instructions) 99.9+ efps (5ms) 99.9+ efps (5ms) +0ms (-/-%)
synthetic (title) 17.5 efps (57ms) 15.6 efps (64ms) +7ms (+12.3%)
synthetic (string inside object) 17.9 efps (56ms) 15.4 efps (65ms) +9ms (+16.1%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 54ms 60ms 67ms 184ms 1376ms 12.9s
article (body) 15ms 18ms 41ms 137ms 381ms 5.8s
article (string inside object) 51ms 58ms 68ms 326ms 1416ms 9.4s
article (string inside array) 57ms 60ms 66ms 210ms 1457ms 9.2s
recipe (name) 21ms 23ms 32ms 42ms 9ms 7.1s
recipe (description) 19ms 20ms 21ms 29ms 0ms 4.7s
recipe (instructions) 5ms 7ms 7ms 12ms 3ms 3.2s
synthetic (title) 57ms 61ms 66ms 466ms 1568ms 13.4s
synthetic (string inside object) 56ms 58ms 64ms 221ms 1035ms 9.0s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 54ms 60ms 76ms 186ms 1216ms 13.1s
article (body) 15ms 17ms 26ms 148ms 229ms 5.9s
article (string inside object) 53ms 56ms 64ms 102ms 1056ms 8.9s
article (string inside array) 58ms 63ms 73ms 245ms 1497ms 9.3s
recipe (name) 20ms 22ms 26ms 47ms 4ms 7.3s
recipe (description) 19ms 20ms 22ms 31ms 0ms 4.7s
recipe (instructions) 5ms 6ms 9ms 21ms 0ms 3.4s
synthetic (title) 64ms 69ms 84ms 331ms 2395ms 15.3s
synthetic (string inside object) 65ms 73ms 86ms 498ms 2420ms 10.6s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

@jordanl17 jordanl17 marked this pull request as ready for review March 26, 2025 13:58
@jordanl17 jordanl17 requested a review from a team as a code owner March 26, 2025 13:58
@jordanl17 jordanl17 requested review from pedrobonamin and removed request for a team March 26, 2025 13:58
Copy link
Contributor

@pedrobonamin pedrobonamin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! One flaky test at a time 👊🏻

@jordanl17 jordanl17 merged commit 37b3b6c into next Mar 27, 2025
59 of 60 checks passed
@jordanl17 jordanl17 deleted the test/robustness-of-unpublish.spec branch March 27, 2025 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants