feat: Onboarding

[Nerivec]

Motivation

• zigbee2mqtt-frontend is not well suited for this since it is heavily depending on "proper state". Would require a heavy refactoring also in Z2M to change the startup order.
• A simple-enough HTML page suffices to quickly onboard with the most important settings
• Users of complex setups are expected to know how to configure via yaml instead if necessary
• Fixes: [Feature request]: Startup without adapter #26429

Features

• Provide a simple page to set the basic settings:
  • mqtt.base_topic, mqtt.server, mqtt.user, mqtt.password, serial.port, serial.adapter, serial.baudrate, serial.rtscts, advanced.log_level, advanced.channel, advanced.network_key, advanced.pan_id, advanced.ext_pan_id, frontend.enabled, frontend.port, homeassistant.enabled
• Provide error pages (available at default URL without needing to check logs):
  • if saved settings from onboard fail => configuration.yaml has invalid value(s) for setting(s) not present in onboard page (or somehow client-side failed to validate a setting properly)
  • if settings do no pass final validation
  • Close button allows exiting the Z2M (nodejs) process
  • Closing via page should prevent containerized environments from just stopping on failure (nodejs process does not exit until then)
• Triggers before anything else, to allow full control
• Ability to start without a configuration.yaml file present (minimal default created automatically in background)
• Triggers onboarding based on the presence of database.db to consider the install as "brand new" (also if configuration.yaml is missing)
  • If zigbee-herdsman fails on first start (invalid coordinator settings), database.db is not created, hence the onboarding will trigger again, allowing correction
• Redirect to frontend (if enabled) upon completion (after delay, with link for manual click)
• Ability to disable onboard server via env Z2M_ONBOARD_NO_SERVER
• Ability to set onboard page URL via env Z2M_ONBOARD_URL
• Ability to disable failure pages (exit Z2M process automatically - as previous) via env Z2M_ONBOARD_NO_FAILURE_PAGE
• Ability to re-run onboard for existing install (e.g. pick new coordinator after changing it) via env Z2M_ONBOARD_FORCE_RUN
• Auto detection of serial and mDNS devices for quick selection (port/adapter), updated on page refresh
• Ability to set GENERATE for network config
• Ability to derive Network Channel by picking closest WiFi channel (simple mapping, like Configurator in docs)
• Link to docs for each section
• No extra dependency, minimal code
• Easy enough to add new supported settings if required

Other changes

• Move settings migration/validation from index.js to TS (more typing!)

TODO:

• requires feat: add util to discover all serial/mDNS devices zigbee-herdsman#1333
• pick default URL (currently: http://localhost:8080)
• confirm triggers
• identify setups where some envs may have to be set (required bypass of some logic, etc.)?
• https://github.com/zigbee2mqtt/hassio-zigbee2mqtt
  • remove data path mkdir
  • remove configuration.yaml creation
  • add ZIGBEE2MQTT_CONFIG_HOMEASSISTANT_ENABLED ENV
  • add setting to allow use of ENV Z2M_ONBOARD_FORCE_RUN
• https://github.com/Koenkk/zigbee2mqtt-chart ?
• tests
• docs

[sjorge]

Interesting.

Wasn't the frontend made to be pretty modeler from the z2m side? Or was I imagining that?

I wonder if we can have multiple 'extensions' register particular URI. To this could become an /onboarding or something. And we indeed just ships in in z2m. The existing frontend could also be potentially broken up into separate components to make them easier to manage like split of the dashboard, configuration, device manager, into separate extensions.

[Nerivec]

I've taken a stab at this a few times in the past, this is the only design that appears to work on all fronts (without a massive refactoring).
To properly onboard, you have to be able to start from nothing, before anything is ready. This starts before the Z2M controller is even created (first step after building TS). You get a default page (with a couple of envs if need extra configuring) similar to onboarding that can be seen with some WiFi devices and the likes, with minimal code/involvement, that doesn't impact anything after it's done.
Using anything that depends on configuration.yaml wouldn't work. And Z2M extensions also require the full state to be loaded (zigbee, mqtt, etc.).