feat: weekly build #12668
feat: weekly build #12668
Conversation
…ch will actually trigger on:
release:
types: [published]
which will create a complete release cycle
amitmiran137
changed the title
Codecov Report
@@ Coverage Diff @@
## master #12668 +/- ##
==========================================
- Coverage 66.92% 66.89% -0.04%
==========================================
Files 1018 1020 +2
Lines 49786 49905 +119
Branches 4874 4888 +14
==========================================
+ Hits 33321 33385 +64
- Misses 16342 16395 +53
- Partials 123 125 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
| tag_name: weekly-tag-${{ steps.date.outputs.date }} | ||
| release_name: Reekly-release-${{ steps.date.outputs.date }} | ||
| tag_name: superset-1.0.0-${{ outputs.todays_date.value }} | ||
| release_name: superset-1.0.0-${{ outputs.todays_date.value }} |
There was a problem hiding this comment.
I wish we could read this version number from a shared place such as setup.py, then realized we don't have a version number tracked anywhere in Git. Every release we just manually set a version when preparing the release.
I think there are merits in putting the canonical current (or next) version number in setup.py or a VERSION file, like Airflow does. It helps with auto-release and makes it easier to see which version current branch is on.
@villebro @mistercrunch @eschutho what do you think?
There was a problem hiding this comment.
we could use pybump this can help us manage the release version bumps automatically
we can do it in a follow-up PR
There was a problem hiding this comment.
Catching up on the context here, so this is a weekly release, but not the "official" release, correct? More like an unstable weekly build? How will people be using this for example?
There was a problem hiding this comment.
Same as the nightly/weekly builds of any other softwares---to give developers/users early access to features that aren't officially released (or fully tested) yet. The benefit of checking out a periodic build instead of the master branch is that you don't have to run build (npm install && npm run build, etc) yourself.
We should probably call the release script here, too, to make tarballs similar to the official releases.
There was a problem hiding this comment.
You can try getting the latest release version from the tag. Something like git show-ref latest && git show --pretty=tformat:%d -s latest | grep tag: will give you a list of all tags on that sha, and you can parse through them to get the one you want. There's an example of this in https://github.com/apache/superset/blob/master/scripts/tag_latest_release.sh
I might also recommend adding something like "weekly-build" or "edge" to the tag, or something so that people don't expect that this release is just a patch on the last one.. there could be breaking changes for example, and we wouldn't be bumping the semantic version on it at all to reflect that.
There was a problem hiding this comment.
we might even be able to extract just that logic of finding the latest tag version from the script if you want to reuse it.
There was a problem hiding this comment.
I've added "weekly-build" and even changed the naming of the workflow
in addition, I added find_latest_tag_version.sh which is kind of a subset of tag_latest_release.sh.
at first, I tried extracting the logic into a "common" script but it was difficult due to bash limitations
…sed in the scheduled_weekly_build.yml
| tag_name: superset-${{outputs.latest_tag_version.value}}-weekly-build-${{ outputs.todays_date.value }} | ||
| release_name: superset-${{outputs.latest_tag_version.value}}-weekly-build-${{ outputs.todays_date.value }} | ||
| body: | | ||
| ##todo: list of commits |
There was a problem hiding this comment.
lgtm. Do we need to fill this part in before accepting?
| echo "::set-output name=latest_tag_version::${latest_tag_version_value}" | ||
| - name: Create Release | ||
| id: create_release | ||
| uses: actions/create-release@v1 |
There was a problem hiding this comment.
Personally, I try to limit the use of actions because they are not portable to another CI and very often they only add extra complexity to the scripts. If you want to understand now how this workflowsI works, you need to know bash and advanced features of Github Action.
Instead, you can use github CLI, which will allow these scripts to run locally as well, if you have the required permissions. For example, see:
https://github.com/apache/airflow-site/blob/1b26f4fcd3a2eb5aa22effa41aaa65a075583d92/.github/workflows/build.yml#L99-L119
|
I would like to add that we should not publish information about these releases on superset.apache.org.
http://www.apache.org/legal/release-policy.html This sentence also means that we should delete guides from the websitte that recommend using git clone to run the project.
https://superset.apache.org/docs/installation/installing-superset-using-docker-compose
|
It probably is, but not every developer is a contributor. There is an additional explanation in the next sentence.
http://www.apache.org/legal/release-policy.html#publication In my opinion, any documentation published on superset.apache.org should be considered as intended for the end-user. Documentation for contributors is in the repository. At Apache Airflow, we were in a similar situation and we decided not to publish information about Helm Chart in the documentation until it is officially released. |
I must agree with @mik-laj - IMHO this is really against the release policy of ASF to point to master versions. The only releases you should advertise in public website should indeed be the released ones following the official release process, which involves signing the sources, voting by PMC members and publishing them via Apache SVN. You can still publish convenience packages (such as released PyPI packages) but they should be prepared using the sources officially published and voted on by PMC members. Encouraging end-users to use master version by cloning the repo is very risky for the project and put the PMCs of the project in a difficult situation because they are responsible to check if the end users gets the software that follows all the licensing policies (this is one of the most important effect of the formal voting). Following the releas policy has the effect that it makes the PMC members idemnified of any potential litigations and ASF takes over any potential litigation costs etc. It has rather serious legal implication, so I suggest you discuss it within the PMC members (cc: @mistercrunch since you are PMC here :) I know you have experience from Airflow as well so maybe you can also state your opinion here). |
|
And since SuperSet has just been announced as TLP, maybe it's the right time to update the docs/process of installation ? |
|
On second thought, I agree with the sentiments expressed by @mik-laj and @potiuk here. If directing users to use Docker is unavoidable(since Superset is a full-fledged web app), we should probably provide them a docker image that points to a fixed version (can point to either pypi or a tagged Git snapshot). For reference, Redash has a setup.sh script for this purpose that they also link in their end user document. Seems like a good solution to me. |
|
At Apache Airflow, we are working intensively on documentation for Airflow installation on the Docker cluster. We provide docker image via Docker Hub: https://hub.docker.com/r/apache/airflow |
|
Jupyter also has extensive documentation for the Docker image. |
|
sounds like a weekly build such as this one is not the right move |
@potiuk the docs currently point users to the |
|
@eschutho Documentation should only be published on the website for the released and voted versions. In the case of Airflow, this means that we publish documentation for each version and the user can choose the version of the documentation. We also have an alias that always points to the latest released version. We also provide documentation for different packages and each package has separate versions. There are also packages that do not have a version because they contain information that relates to information from multiple packages, such as a list of all integrations available in all packages. We do not delete the integration, so there is no need to version it, and even if it is such a serious reason that it is better to delete references to this integration from all documentation. |
|
Yeah - @mik-laj described our approach. I am not sure how it was agreed for SuperSet, so if your PMC members agree between themselves that this is fine to use the repo and tags by the users and that 'latest' is good for them, i have no problems with it. It's literally whatever your project PMCs decides is actually OK with the he policy (unless it is totally against it). if latest points to the sources which have been voted on and are already released with Apache's SVN, you can probably treat is a "convenience" way of accessing the "released" version. It was not entirely clear what "latest" mean from the description. |
|
Great, thanks for that info. Our
And that's what we encourage people to use if they want the latest github release. |
SUMMARY
Weekly build will be triggered by a periodic schedule
which will actually trigger
on:
release:
types: [published]
that does a complete release cycle
the solution is based on the following examples:
https://futurestud.io/tutorials/github-actions-trigger-builds-on-schedule-cron
https://stackoverflow.com/questions/60942067/get-current-date-and-time-in-github-workflows
https://github.com/actions/create-release
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
TEST PLAN
ADDITIONAL INFORMATION