E2E tests: Increase timeout, update python version, and add LLM scan test #325
Conversation
ccronca
changed the title
ccronca
changed the title
ccronca
changed the title
| @@ -20,7 +20,9 @@ data: | |||
| # if it is "*stdout" or unspecified, the command's standard output will be selected | |||
| results: "/tmp/oobtkube.sarif.json" # if None or "*stdout", the command's standard output is selected | |||
| # toolDir: scanners/generic/tools | |||
| inline: "python3 oobtkube.py --log-level debug -d 60 -p 6000 -i rapidast-oobtkube -f /opt/rapidast/config/cr_example.yaml | tee /tmp/oobtkube.sarif.json" | |||
| inline: | | |||
| PYTHON_VERSION="${PYTHON_VERSION:-python3}" | |||
There was a problem hiding this comment.
Not for this PR, as an aside, should we bump to python3.12 for vanilla rapidast as well, so that both images use the same version?
|
|
||
| self.create_from_yaml(f"{self.tempdir}/rapidast-nessus-configmap.yaml") | ||
| self.create_from_yaml(f"{self.tempdir}/rapidast-nessus-pod.yaml") | ||
| wait_until_ready(field_selector="metadata.name=rapidast-nessus") | ||
| # @FIX: We don't assert the container's successful completion because it |
There was a problem hiding this comment.
:O Does this mean the nessus test should have been failing for some time already?
There was a problem hiding this comment.
Well, this test doesn’t assert much, it only retrieves logs from the RapiDAST container. While the container does generate logs, it ends with an error. It would be better to update the test to at least assert that the container completes successfully, it should have a simple solution, just remove the 'authentication' part of the config file.
I'll take care of it in a future PR (#327)
- Add Tekton configuration for LLM integration tests - Include `requirements-llm-dev` for running the tests
- Ensure that containers run successfully during LLM integration tests
|
Just to clarify so I understand right, with this change the following e2e tests will run?
|
Correct |
This PR improves the e2e testing by increasing the timeout when waiting for Rapidast or Nessus to run, since the images are quite large and take a while to download.
Also, the
oobkubetest has been updated to run with Python 3.12 when using the rapidast-llm image. It wasn’t working before with Python 3 because the dependencies were only available for Python 3.12.Lastly, @jeremychoi added an additional e2e test to validate Garak scan functionality.