Cypress on GitHub Actions #378

himdel · 2021-04-30T04:23:17Z

Fixes https://issues.redhat.com/browse/AAH-529

This adds a Cypress workflow to github actions, configured to:

allow to be triggered manually on a specific branch
run on push to master, stable-4.2 and stable-4.3 (stable-* really)
run on PRs against those same branches
run daily on master

The workflow is using the Pulp in One Container images, big thanks to @fao89 for getting these running. (More docs.)

we build our own version of https://github.com/pulp/pulp-oci-images/blob/latest/pulp_galaxy_ng/Containerfile changed to download galaxy_ng master (or other branch, see below) and remove conflicting versions of dependencies.
then we build ansible-hub-ui using build-standalone, and move those files to replace the original static UI files in the container
Cypress is running outside the container, connecting to localhost:8002/api/galaxy

Branches:

for a PR from pr_branch against ansible-hub-ui base:
checking out ansible-hub-ui@pr_branch
BRANCH=refs/heads/base
SHORT_BRANCH=base
pip install ...galaxy_ng@base
for a branch push on ansible-hub-ui:
checking out ansible-hub-ui@branch
BRANCH=refs/heads/branch
SHORT_BRANCH=branch
pip install ...galaxy_ng@branch
for manual builds, branch is whatever the user chooses, for cron builds it's master

Caching:

~/.npm and ~/.cache/Cypress get cached with a package-lock.json hash key, with fallback to the last cache on the same base branch
pulp_galaxy_ng/image gets cached with a key that is the sha of the top commit on the relevant galaxy_ng branch, no fallback, if the cache doesn't exist, we build the image (~4 minutes) and save it (~1 minute), if it does, we just load it (~1 minute).
- example of a build run: https://github.com/himdel/ansible-hub-ui/runs/2472425653?check_suite_focus=true
- example of a cached run: https://github.com/himdel/ansible-hub-ui/runs/2472476282?check_suite_focus=true

Artifacts:

on failure, we upload cypress/screenshots and cypress/videos
(example: https://github.com/himdel/ansible-hub-ui/actions/runs/797934949)

Timing:

~8 minutes total when using cache, ~12 when rebuilding container
~3 minutes of that is webpack
Cypress runs under 3 minutes

Cc @newswangerd

to be able to install master galaxy_ng without conflicts

replaces https://github.com/pulp/pulp-oci-images/blob/latest/pulp_galaxy_ng/Containerfile

… changed, else load and use podman repo prefixes

newswangerd · 2021-04-30T13:19:06Z

.github/workflows/cypress.yml

+  push:
+    branches: [ $default-branch stable-* ]
+  # daily on $default-branch
+  schedule:


is a daily job necessary? We should be pretty well covered with branch and push builds

Definitely not necessary :)

Pretty much the only use for a scheduled job in our circumstances is to check that the latest galaxy_ng changes don't break our UI.

So, I think it has some value, but maybe not quite daily?
What about weekly?

newswangerd

Love it

newswangerd · 2021-04-30T13:22:48Z

I assume this can be backported to 4.2 and 4.3, right?

.github/workflows/cypress.yml

fao89 · 2021-04-30T14:11:21Z

.github/workflows/cypress.yml

+        
+        podman run \
+             --detach \
+             --publish 8002:80 \


FYI: we are planning to change the container to use https:
pulp/pulp-oci-images#73
In the future you may need to:

Suggested change

--publish 8002:80 \

--publish 8002:443 \

@fao89 could that be implemented as an optional setting?

I don't really see any value in generating a self-signed certificate, using it on the webserver, and running cypress in "ignore SSL certificates" mode or having to copy the certificate there, just so https works in a way that http already does.

We can probably work around that on our side, but if that feature could be made optional, it may be preferrable :). Thanks :).

yes, we are discussing two options now:

tag latest: https; tag nossl: http

tag latest: http; tag https: https

so if we make latest defaults to https, it is just a matter of using nossl tag
I'll keep you informed, but if you want to follow the discussion, it is happening on pulp-dev:
https://listman.redhat.com/archives/pulp-dev/2021-May/msg00008.html

.github/workflows/cypress.yml

fao89 · 2021-04-30T14:16:55Z

.github/workflows/cypress.yml

+      working-directory: 'pulp_galaxy_ng'
+      run: |
+        echo '\
+          FROM docker.io/pulp/pulp-ci-centos:latest


you can test the https container version by using

Suggested change

FROM docker.io/pulp/pulp-ci-centos:latest

FROM docker.io/pulp/pulp-ci-centos:https

unlikelyzero · 2021-05-03T16:47:17Z

@himdel @newswangerd @calvin hello from the other side! I went snooping for an example of the github codecov integration and stumbled into this PR :)

Please ping @tiago, @apurva, and @brum on the Ansible team. We had been using Cypress with Github Actions for a few months and I think it would be great to collaborate on some common tooling or at least get contextual PR review. Also, Apurva on the AA team may be taking over aap-visual which has some examples here https://github.com/unlikelyzero/aap-visual/blob/main/.github/workflows/nightly.yml

Specifically, I'd get feedback on the utility of adhoc job execution and inline Cypress Dashboard reporting

himdel · 2021-05-10T14:25:25Z

I assume this can be backported to 4.2 and 4.3, right?

Definitely, all the branch logic should be using the base branch of a PR, or the branch it was pushed to, so this should work for all the versions. 👍

(Not sure the non-master branches are currently passing, but we can fix any tests/failures when backporting.)

himdel · 2021-05-10T14:44:07Z

@unlikelyzero Welcome, well, any reviews are certainly appreciated :)

I'm not sure there's much overlap with other projects not based on pulp, or projects not using a split repo ui-api approach, as the bulk is about building and configuring a container, but if you have any ideas, feel free to share :).

cypress-io/github-action@v2 looks interesting, though it seems we'd have to get rid of the ansible-hub-ui/test/ split in favor of having cypress+dependencies in the repo root, so definitely a separate PR.

Inline reporting defintiely looks useful, I'll look into that in a separate PR, at this point I'm not ever sure how the reporting will look without it. (The config doesn't seem to do anything related to inline reporting, so it looks like maybe the cypress action is trying to do too many things.)

(I'm not sure what you mean by ad-hoc execution, but we mostly just need this to run on PRs.)

unlikelyzero · 2021-05-10T14:57:16Z

@himdel nothing to add in terms of approach. This is awesome.
The ansible/tower-qa is using a split repo approach as it is a private repo testing the ansible/awx repo, so it is similar.

The main reason that I mentioned inline reporting is that you can avoid storing test results and videos as CI artifacts. Definitely not specific to this PR; it's something which must be configured by a repo admin on github, itself.

The tower-qa repo needed to have an "ad-hoc" workflow file which was separate from the "per PR" workflow file. I'd check in with that team to learn more.

Anyways, glad to see GHA + Cypress being used together :)

patchback · 2021-05-12T14:12:40Z

Backport to stable-4.2: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-4.2/6a5688eb96f9f889e93b9e1b55dee8ada480de06/pr-378

Backported as #430

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

* workflows: manual cypress test, initial version * Switch to pulp-oci-images * Override pulp-galaxy-ng pip versions to be able to install master galaxy_ng without conflicts * Configure and run pulp/pulp * Own Containerfile, reset password replaces https://github.com/pulp/pulp-oci-images/blob/latest/pulp_galaxy_ng/Containerfile * switch to /api/galaxy, 8002 * use buildah & podman, no sudo needed for UI replace * upload on failure * cache ~/.npm, Cypress; cache container image, only build if galaxy_ng changed, else load and use podman repo prefixes * cleanup: kill container * cleanup, rename workflow to cypress (cherry picked from commit 6a5688e)

patchback · 2021-05-12T14:12:51Z

Backport to stable-4.3: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-4.3/6a5688eb96f9f889e93b9e1b55dee8ada480de06/pr-378

Backported as #431

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

* workflows: manual cypress test, initial version * Switch to pulp-oci-images * Override pulp-galaxy-ng pip versions to be able to install master galaxy_ng without conflicts * Configure and run pulp/pulp * Own Containerfile, reset password replaces https://github.com/pulp/pulp-oci-images/blob/latest/pulp_galaxy_ng/Containerfile * switch to /api/galaxy, 8002 * use buildah & podman, no sudo needed for UI replace * upload on failure * cache ~/.npm, Cypress; cache container image, only build if galaxy_ng changed, else load and use podman repo prefixes * cleanup: kill container * cleanup, rename workflow to cypress (cherry picked from commit 6a5688e)

himdel · 2021-05-12T14:57:06Z

(Removed backport labels because there's nothing to backport, github uses the master branch to get the workflow, regardless of the branch being tested)

himdel added 11 commits April 30, 2021 03:37

workflows: manual cypress test, initial version

3a4d215

Switch to pulp-oci-images

a0bbd0f

Override pulp-galaxy-ng pip versions

ca23604

to be able to install master galaxy_ng without conflicts

Configure and run pulp/pulp

ea3a409

Own Containerfile, reset password

1cb7ab6

replaces https://github.com/pulp/pulp-oci-images/blob/latest/pulp_galaxy_ng/Containerfile

switch to /api/galaxy, 8002

70525dc

use buildah & podman, no sudo needed for UI replace

35aef45

upload on failure

3da0dc8

cache ~/.npm, Cypress; cache container image, only build if galaxy_ng…

4c16bd5

… changed, else load and use podman repo prefixes

cleanup: kill container

37de850

cleanup, rename workflow to cypress

718f0a2

newswangerd reviewed Apr 30, 2021

View reviewed changes

newswangerd approved these changes Apr 30, 2021

View reviewed changes

newswangerd added backport-4.2 backport-4.3 labels Apr 30, 2021

fao89 approved these changes Apr 30, 2021

View reviewed changes

himdel mentioned this pull request May 10, 2021

Run pulp container on https pulp/pulp-oci-images#73

Closed

himdel merged commit 6a5688e into ansible:master May 12, 2021

himdel deleted the cypress-gha branch May 12, 2021 14:12

patchback bot mentioned this pull request May 12, 2021

[PR #378/6a5688eb backport][stable-4.2] Cypress on GitHub Actions #430

Closed

patchback bot mentioned this pull request May 12, 2021

[PR #378/6a5688eb backport][stable-4.3] Cypress on GitHub Actions #431

Closed

This was referenced May 12, 2021

workflows/cypress - fix $default-branch -> master; the macro works in templates, not workflows #433

Merged

workflows/cypress - YAML syntax - have an array of 2 items, not an array of one string #434

Merged

himdel removed backport-4.2 backport-4.3 labels May 12, 2021

This was referenced May 18, 2021

Cypress - fix mkdir bug #454

Merged

Cypress - add debugging info to workflow #452

Merged

Adding container logs on the CI to facilitate debugging #453

Merged

Cypress - restart api after updating UI #457

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cypress on GitHub Actions #378

Cypress on GitHub Actions #378

himdel commented Apr 30, 2021

newswangerd Apr 30, 2021

himdel May 10, 2021

newswangerd left a comment

newswangerd commented Apr 30, 2021

fao89 Apr 30, 2021

himdel May 10, 2021

fao89 May 10, 2021 •

edited

Loading

fao89 Apr 30, 2021

unlikelyzero commented May 3, 2021 •

edited

Loading

himdel commented May 10, 2021

himdel commented May 10, 2021

unlikelyzero commented May 10, 2021

patchback bot commented May 12, 2021 •

edited

Loading

patchback bot commented May 12, 2021 •

edited

Loading

himdel commented May 12, 2021

	FROM docker.io/pulp/pulp-ci-centos:latest
	FROM docker.io/pulp/pulp-ci-centos:https

Cypress on GitHub Actions #378

Cypress on GitHub Actions #378

Conversation

himdel commented Apr 30, 2021

newswangerd Apr 30, 2021

Choose a reason for hiding this comment

himdel May 10, 2021

Choose a reason for hiding this comment

newswangerd left a comment

Choose a reason for hiding this comment

newswangerd commented Apr 30, 2021

fao89 Apr 30, 2021

Choose a reason for hiding this comment

himdel May 10, 2021

Choose a reason for hiding this comment

fao89 May 10, 2021 • edited Loading

Choose a reason for hiding this comment

fao89 Apr 30, 2021

Choose a reason for hiding this comment

unlikelyzero commented May 3, 2021 • edited Loading

himdel commented May 10, 2021

himdel commented May 10, 2021

unlikelyzero commented May 10, 2021

patchback bot commented May 12, 2021 • edited Loading

Backport to stable-4.2: 💚 backport PR created

patchback bot commented May 12, 2021 • edited Loading

Backport to stable-4.3: 💚 backport PR created

himdel commented May 12, 2021

fao89 May 10, 2021 •

edited

Loading

unlikelyzero commented May 3, 2021 •

edited

Loading

patchback bot commented May 12, 2021 •

edited

Loading

patchback bot commented May 12, 2021 •

edited

Loading