v4 upgrade doc, changelog for 4.0

[minrk]

adds 3-to-4 upgrade doc, which summarizes the relevant changes in dependencies and upgrade strategies in the chart config. I could find no changes in the chart itself that really needed upgrade docs, does anyone disagree, are there any new features we want to highlight in the upgrade doc in addition to possible breakages?

To prepare for final release, this collapses beta changelog entries into a single entry for 4.0, since I don't think we should keep changelog entries for prereleases.

I think theres some redundancy in the "version upgrade table" between the changelog and the upgrade doc. I think some redundancy is fine, but I don't have a great guide in my head for what level of info belongs where.

[minrk]

also needs kubespawner 7 before final, but that's ready, I think: jupyterhub/kubespawner#866

[manics]

I think we should get this ldapauthenticator fix jupyterhub/ldapauthenticator#289 into a patch release that we can include in Z2JH.

[consideRatio]

Thoughts about what to document where etc

I think theres some redundancy in the "version upgrade table" between the changelog and the upgrade doc. I think some redundancy is fine, but I don't have a great guide in my head for what level of info belongs where.

I've thought about this, and have a model in my head where I think things should go:

• I generally think of reference documentation (diataxis) as the lowest level building block of documentation to be referenced by higher level building blocks.
  If something fits in reference docs or generally in a lower level building block, it should be written there and repetition should be avoided to improve maintainability and to enable high level docs to be briefer which in turn improves readability.
• For a major release, the following documentation pieces are relevant
  • Changelog (lowest level building block)
  • General upgrade docs
  • Version specific upgrade docs (highest level building block)

For distributions like TLJH and Z2JH, with default configuration of various dependencies and ways of configuring those dependencies, what should its changelog / general upgrade docs / version specific upgrade docs mention if a dependency has a major version bump?

I think at least:

• A link to the dependecy's changelog
• Evaluation if the dependency bump is a breaking change at all in the context of the distribution. If its not, a comment about its not breaking in the context of the distribution should be added.
• Evaluation if the dependency bump's breaking change documentation in the upstream project needs to be augmented with distribution specific context/details.
  As practical examples:
  • I think LDAPAuthenticator mentioning deprecation of c.LDAPAuthenticator.use_ssl doesn't need to be augmented in z2jh with a mention of hub.config.LDAPAuthenticator.use_ssl, because use of hub.config can be assumed or documented under "General upgrade docs"
  • I think KubeSpawner's changes with escape are relevant to mention in version specific upgrade docs, where for example singleuser.storage.static.subPath = "{username}" is a default value making users of singleuser.storage.type = "static" directly impacted in a way motivating that z2jh augments documentation in KubeSpawner.

We have had examples of documentations in z2jh / tljh that caused unsustainable maintenance, because they became outdated even though the dependency project documented things clearly. That makes more docs possibly worse than less. Due to that, my mental model is to opt to lean as heavily as possible on dependent project docs and avoiding repetition as much as possible.

[consideRatio]

I think the following are essential pieces for this docs:

• a link to z2jh general upgrade docs
• mention z2jhs default of singleuser.storage.static.subPath="{username}" for users of singleuser.storage.static.type="static" could need to re-configured to "{escaped_username}"

I figure its fine to retain JupyterHub and KubeSpawner headings, but I think it can be an improvement to cut out OAuthenticator and Other package upgrades headings as it otherwise overlaps with the changelog docs without fully capturing the changelogs docs either.

[minrk]

Thanks for thinking so much about this! I think I've updated with what you suggested.

[manics]

There's a regression in LDAPauthenticator which should be fixed by jupyterhub/ldapauthenticator#294
We can either wait for confirmation from the bug reporter, or merge and release ldapauthenticator now so as not to further delay the Z2JH release.

[minrk]

No problem waiting a bit more. We can publish once that PR lands and the hub image gets its bump.

[consideRatio]

ldapauthenticator 2.0.2 is out now, bumped in z2jh's hub image, and I updated the changelog entries in this PR to reflect some recent PRs etc.

Release date updated to 2024-11-07 -- go for release tomorrow thursday?

[minrk]

Great, let's do it!

[minrk]

I'll merge this and publish 4.0 shortly.

[minrk]

tag published, drafting announcements