Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onboard OpenSearch plugins to Smoke tests framework #77

Open
zelinh opened this issue Feb 13, 2025 · 6 comments
Open

Onboard OpenSearch plugins to Smoke tests framework #77

zelinh opened this issue Feb 13, 2025 · 6 comments
Assignees
Labels

Comments

@zelinh
Copy link
Member

zelinh commented Feb 13, 2025

Sub-issue from opensearch-project/opensearch-build#5317

Background

With the smoke test framework (#5223) now established, we have successfully onboarded multiple OpenSearch plugins—including Index Management, Security, SQL, and K-NN—to the framework (PR #5255).

To maximize the effectiveness of smoke testing, we aim to onboard all OpenSearch plugins into the framework. This effort will enhance overall testing coverage, ensuring greater reliability and functionality validation for OpenSearch and its plugins.

Overview of Smoke Tests

Smoke tests verify the basic functionality of an OpenSearch distribution bundle with all plugins installed. Each plugin has its own test specification YAML file, stored in the OpenSearch Build repository and organized by major version.

These YAML files list API test cases, which the smoke test workflow executes. The results are then validated against the OpenSearch API Specification to ensure API correctness and expected behavior.

How to Onboard a Plugin

To onboard your plugin to the smoke test framework, follow the Onboarding Guide.

Steps to Onboard:

  1. Add a YAML test file

  2. Define API tests in the YAML file

    • Specify:
      • API path
      • HTTP method
      • Request body (if applicable)
      • Custom Content-Type header if different from application/json.
    • See an example test definition here.
  3. Ensure the APIs are onboarded to the OpenSearch API Specification repository

  4. Reference the test file in the test manifest

    • Ensure the newly added YAML file is referenced in the test manifest for your plugin’s component.

Collaboration with Plugin Owners:

We are reaching out to all OpenSearch plugin owners to facilitate onboarding. We will provide:
✅ Documentation and guidelines for onboarding.
✅ Support in setting up test cases.
✅ Assistance in troubleshooting issues.

Expected Outcome:

  • Stronger OpenSearch releases with better validation of plugins.
  • Standardized, reusable testing framework for ongoing plugin validation.
  • Faster detection of integration issues before release.

📢 Plugin owners: We encourage you to onboard your plugin to the smoke test framework! If you have any questions or need assistance, feel free to comment on this issue or reach out to EE team.

Let’s work together to strengthen OpenSearch’s testing and release process! 🚀

@jzonthemtn jzonthemtn self-assigned this Feb 19, 2025
@jzonthemtn jzonthemtn changed the title Onboard OpenSearch plugis to Smoke tests framework Onboard OpenSearch plugins to Smoke tests framework Feb 19, 2025
@jzonthemtn
Copy link
Collaborator

@zelinh Thanks for taking this on. Ideally this would be done prior to release 3.0.0?

@zelinh
Copy link
Member Author

zelinh commented Feb 19, 2025

@zelinh Thanks for taking this on. Ideally this would be done prior to release 3.0.0?

Yes. This should be targeting to be onboarded before 3.0.0.

@jzonthemtn
Copy link
Collaborator

jzonthemtn commented Mar 12, 2025

@zelinh I want to confirm these steps are necessary for the UBI plugin since it is not included in the base OpenSearch distribution.

Can you please confirm that a plugin that requires manual installation into OpenSearch needs to have the smoke test scripts and have its API specification added to opensearch-api-specification?

@zelinh
Copy link
Member Author

zelinh commented Mar 13, 2025

@zelinh I want to confirm these steps are necessary for the UBI plugin since it is not included in the base OpenSearch distribution.

Can you please confirm that a plugin that requires manual installation into OpenSearch needs to have the smoke test scripts and have its API specification added to opensearch-api-specification?

Hey @jzonthemtn, just to clarify, I saw UBI plugin is currently in opensearch 3.0.0 input manifest. https://github.com/opensearch-project/opensearch-build/blob/9c6ebfde2ea2c6bf215be2df3d77e0f3c3cd1979/manifests/3.0.0/opensearch-3.0.0.yml#L251-L256
so I assume it would be part of the OpenSearch distribution. Does it require additional manual installation?

@jzonthemtn
Copy link
Collaborator

@zelinh Thanks for clearing that up for me. I don't think I realized that having it in the build meant it would be distributed with the build. I think I thought it just tied the build together. So I was way off! I will get these smoke tests in ASAP.

@jzonthemtn
Copy link
Collaborator

jzonthemtn added a commit to jzonthemtn/opensearch-build that referenced this issue Mar 19, 2025
…default directory.

Signed-off-by: jzonthemtn <jeff.zemerick@mtnfog.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: 🆕 New
Development

No branches or pull requests

3 participants