Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onboard OpenSearch plugins to Smoke tests framework #3541

Closed
zelinh opened this issue Feb 13, 2025 · 6 comments
Closed

Onboard OpenSearch plugins to Smoke tests framework #3541

zelinh opened this issue Feb 13, 2025 · 6 comments
Assignees
Labels

Comments

@zelinh
Copy link
Member

zelinh commented Feb 13, 2025

Sub-issue from opensearch-project/opensearch-build#5317

Background

With the smoke test framework (#5223) now established, we have successfully onboarded multiple OpenSearch plugins—including Index Management, Security, SQL, and K-NN—to the framework (PR #5255).

To maximize the effectiveness of smoke testing, we aim to onboard all OpenSearch plugins into the framework. This effort will enhance overall testing coverage, ensuring greater reliability and functionality validation for OpenSearch and its plugins.

Overview of Smoke Tests

Smoke tests verify the basic functionality of an OpenSearch distribution bundle with all plugins installed. Each plugin has its own test specification YAML file, stored in the OpenSearch Build repository and organized by major version.

These YAML files list API test cases, which the smoke test workflow executes. The results are then validated against the OpenSearch API Specification to ensure API correctness and expected behavior.

How to Onboard a Plugin

To onboard your plugin to the smoke test framework, follow the Onboarding Guide.

Steps to Onboard:

  1. Add a YAML test file

  2. Define API tests in the YAML file

    • Specify:
      • API path
      • HTTP method
      • Request body (if applicable)
      • Custom Content-Type header if different from application/json.
    • See an example test definition here.
  3. Ensure the APIs are onboarded to the OpenSearch API Specification repository

  4. Reference the test file in the test manifest

    • Ensure the newly added YAML file is referenced in the test manifest for your plugin’s component.

Collaboration with Plugin Owners:

We are reaching out to all OpenSearch plugin owners to facilitate onboarding. We will provide:
✅ Documentation and guidelines for onboarding.
✅ Support in setting up test cases.
✅ Assistance in troubleshooting issues.

Expected Outcome:

  • Stronger OpenSearch releases with better validation of plugins.
  • Standardized, reusable testing framework for ongoing plugin validation.
  • Faster detection of integration issues before release.

📢 Plugin owners: We encourage you to onboard your plugin to the smoke test framework! If you have any questions or need assistance, feel free to comment on this issue or reach out to EE team.

Let’s work together to strengthen OpenSearch’s testing and release process! 🚀

@zelinh zelinh changed the title Onboard OpenSearch plugis to Smoke tests framework Onboard OpenSearch plugins to Smoke tests framework Feb 19, 2025
@pyek-bot
Copy link
Contributor

@zelinh is there a timeline we are targeting for onboarding?

@ylwu-amzn
Copy link
Collaborator

Sounds a good way to automate our manual sanity test

@zelinh
Copy link
Member Author

zelinh commented Feb 25, 2025

@zelinh is there a timeline we are targeting for onboarding?

We are targeting this to be onboarded by 3.0.0 release.

@rithin-pullela-aws
Copy link
Contributor

Hi @mingshl can you please assign this to me.

@rithin-pullela-aws
Copy link
Contributor

@mingshl @dhrubo-os, can we close this issue.
Resolved by this PR: opensearch-project/opensearch-build#5369

@mingshl
Copy link
Collaborator

mingshl commented Mar 12, 2025

Thanks @rithin-pullela-aws for taking this up! closed this issue and marked as done

@mingshl mingshl closed this as completed Mar 12, 2025
@github-project-automation github-project-automation bot moved this from On-deck to Done in ml-commons projects Mar 12, 2025
@mingshl mingshl added the v3.0.0 label Mar 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Development

No branches or pull requests

6 participants