Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gno test): implement failfast flag #3866

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

Villaquiranm
Copy link
Contributor

@Villaquiranm Villaquiranm commented Mar 2, 2025

Overview

This PR introduces support for the -failfast flag in gno test. When enabled, test execution stops as soon as any test fails. However, tests that were already running in parallel will be allowed to complete.

Motivation

In large-scale test executions, test failures can get lost in a flood of output, making it difficult to identify the faulty test. By using -failfast, we can quickly pinpoint failing tests and iterate faster, improving debugging efficiency.

Changes

Implemented -failfast behavior in test execution.

Sources

From Go 1.10 documentation:

The new go test -failfast flag disables running additional tests after any test fails. Note that tests running in parallel with the failing test are allowed to complete.

@github-actions github-actions bot added 🧾 package/realm Tag used for new Realms or Packages. 📦 🤖 gnovm Issues or PRs gnovm related labels Mar 2, 2025
@Gno2D2 Gno2D2 requested a review from a team March 2, 2025 08:05
@Gno2D2
Copy link
Collaborator

Gno2D2 commented Mar 2, 2025

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)
🟢 Pending initial approval by a review team member, or review from tech-staff

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 The pull request was created from a fork (head branch repo: Villaquiranm/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Pending initial approval by a review team member, or review from tech-staff

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 Not (🔴 Pull request author is a member of the team: tech-staff)

Then

🟢 Requirement satisfied
└── 🟢 If
    ├── 🟢 Condition
    │   └── 🟢 Or
    │       ├── 🟢 At least 1 user(s) of the organization reviewed the pull request (with state "APPROVED")
    │       ├── 🟢 At least 1 user(s) of the team tech-staff reviewed pull request
    │       └── 🔴 This pull request is a draft
    └── 🟢 Then
        └── 🟢 Not (🔴 This label is applied to pull request: review/triage-pending)

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission

Copy link

codecov bot commented Mar 2, 2025

Codecov Report

Attention: Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
gnovm/cmd/gno/test.go 88.88% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@Villaquiranm Villaquiranm marked this pull request as ready for review March 6, 2025 17:11
@Gno2D2 Gno2D2 added the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Mar 6, 2025
Copy link
Member

@notJoon notJoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be helpful for gno-base project CI pipeline. LGTM 👍

remove: review/triage-pending flag

@notJoon notJoon removed the review/triage-pending PRs opened by external contributors that are waiting for the 1st review label Mar 7, 2025
Comment on lines +257 to +260
if cfg.failfast {
io.ErrPrintfln("FAIL")
return fmt.Errorf("FAIL: %d build errors, %d test errors", buildErrCount, testErrCount)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-failfast does not fail just on the first failing package, but on the first failing test, so this implementation is incomplete

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough, good catch :)
fixed here I think 2be439e

Comment on lines +420 to +422
if opts.FailfastFlag {
return errs
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, needs another round...

This won't stop execution for a subtest, ie. if I run a test using t.Run inside a test.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also this should be named flag_failfast

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 🤖 gnovm Issues or PRs gnovm related 🧾 package/realm Tag used for new Realms or Packages.
Projects
Status: Triage
Development

Successfully merging this pull request may close these issues.

4 participants