Request: blog posts / release notes for Beta releases

[nicoburns]

Suggestion / Request

Write announcement blog posts for Beta releases similar to those currently written for Stable releases.

Motivation

It is my understanding that Beta releases are currently underused / undertested. For example, this comment on Reddit suggests that lack of use of Beta builds was a factor in incremental compilation having to be disabled in the upcoming 1.59 release.

From my perspective as a Rust user, one of the main reasons I don't use Beta releases is because I don't know what they include. I am often excited to upgrade to new Stable releases so that I can try out the new features, but the same is not true of Beta releases because even if they contain particularly exciting features, I won't know about it!

To add a quote from another Rust user:

Before this post I wasn't even aware there was a beta release and I looked at this subreddit at least weekly for the last 2 years.

Other concerns

As far as I can see, it wouldn't be any more work to write up the release notes / introduction blog post for the Beta release (and then reuse the same notes for the Stable release). The work would have to happen sooner. If that's an issue then perhaps the post could be made 2-3 weeks into the release cycle (which would also have the benefit of being offset from Stable announcements).

Tagging everyone who has recently created a PR for a release blog post @tmandry @cuviper @Mark-Simulacrum @pietroalbini

[Mark-Simulacrum]

Even if we post the same exact content for beta + stable, that means that the content needs to get generated earlier - typically the release blog post is not fully ready until the ~week of the release. If we wanted to push that earlier, I suspect the main blocker is that generating the blog post and release notes takes time, and doing so on a potentially shifting baseline (i.e., if we develop as nightly is developed) "risks" needing to adjust as features land which are important and which are not. I'm not sure that the release team currently has the resources to truly pull this off well.

That said, I do think there's a significantly lighter weight task that does some readily achievable, and may be something we can do in the interim: update our "standard" release post to include some mention of beta and nightly. For example, where we say:

If you have a previous version of Rust installed via rustup, you can get 1.59.0 with:

rustup update stable

If you don't have it already, you can get rustup from the appropriate page on our website, and check out the detailed release notes for 1.59.0 on GitHub.

It seems easy enough for us to add a snippet to that, along the lines of:

If you'd like to help us out by testing future releases, you might consider updating locally to use the beta channel (rustup default beta) or the nightly channel (rustup default nightly). Please report any bugs you might come across!

It'd also be easy to add in a link from there to some longer-form page that describes the different channels and how to best test things. I think this would definitely address the use case of "Before this post I wasn't even aware there was a beta release and I looked at this subreddit at least weekly for the last 2 years."

However, for the use case of "what's coming down the pipeline", I think it'll be difficult to pull off a blog post unless someone steps up to write that material on a relatively fast turn around after beta is cut. We could try to promote the release notes earlier -- an alpha version is typically ready in the first 1-2 weeks after the release -- but that's much drier material than the blog post typically has (no examples of usage, etc.)

@nicoburns Do you think the release notes would address your desire to know what's coming well enough? Maybe with the added context you have some other ideas on what we could to help generate interest.

In general, my intuition is that the vast majority of "quite bad" bugs -- like the ones that have caused us to disable incremental a few times -- are not related to new features, so if we can get the vast majority of Rust users locally defaulting to beta and falling back to stable if they run into issues, that would work out well. Obviously, when I say "a vast majority", that sort of creates a problem where if beta isn't stable that affects lots of people. I don't know what kind of tolerance exists in the wild for the possible downgrades that would be necessary or whether this is a viable strategy.

I tried to briefly look -- might spend some more time later -- but I'm curious on how other projects with similar train models approach this problem. What are the usage statistics for Firefox across stable/beta/nightly, for example? Chrome?

I think most popular applications like browsers etc typically stage their rollouts -- i.e., not everyone upgrading to 1.59 today, but rather some users getting it today, some in a few weeks. But this seems unlikely to really help our use case or be all that feasible for a compiler :)