Breaking changes for KDLv2

[YaLTeR]

We're getting KDLv2: #1139

And it works in such a way that niri will accept both a KDLv2 config and a KDLv1 config, preserving full backwards compatibility for old configs. This gives us a unique opportunity to do some breaking changes, but only in the new KDLv2 config, so that KDLv1 configs remain working exactly as they did before.

I listed some ideas before in #447. However, thinking about it now, some of these ideas (renames) might be more appropriate for a full backwards compat break rather than a "soft" break like here.

So, here are my current ideas for what to change for KDLv2. Smaller changes from the issue above:

1. Remove deprecated four-number color format, i.e. active-color 255 0 0 0. These are all representable in the CSS syntax.
2. Change linear animation settings to be set as properties similarly to spring animation settings, i.e. linear duration-ms=200 curve="ease-out-quad".

Now, for what I mainly wanted to get feedback on.

There's a lot of confusion about spawn and spawn-at-startup: they don't use a shell, so you have to do weird argument quoting, and can't use shell variables and expansions. The reason for this is to avoid the (small, but several ms long) overhead of starting a shell for every bind.

binds {
    // Correct: every argument is in its own quotes.
    Mod+T { spawn "alacritty" "-e" "/usr/bin/fish"; }

    // Wrong: will interpret the whole `alacritty -e /usr/bin/fish` string as the binary path.
    Mod+D { spawn "alacritty -e /usr/bin/fish"; }

    // Wrong: will pass `-e /usr/bin/fish` as one argument, which alacritty won't understand.
    Mod+Q { spawn "alacritty" "-e /usr/bin/fish"; }
}

KDLv2 makes this a bit nicer on the one hand by removing most of the quotes, on the other hand some problems remain:

binds {
    // This works now: KDL splits arguments by space.
    // You still need to quote the argument starting from "/", but it will complain at you about it so it's fine.
    Mod+T { spawn alacritty -e "/usr/bin/fish" }

    // Still wrong: ~ is not expanded because we're not in a shell.
    Mod+D { spawn alacritty --working-directory "~/source" }

    // Still wrong: $() is not expanded because we're not in a shell.
    Mod+Q { spawn notify-send clipboard "$(wl-paste)" }
}

So I wanted to add a new spawn-sh (spawn-sh-at-startup) command that accepts a single argument and passes it to the shell.

The reason for accepting a single argument is that shells accept their command as a single argument, and do their own parsing and word splitting. While we could accept multiple arguments and join them back with whitespace, I don't think it's a good idea because it would appear as though you can pass one argument "like this", but you actually cannot (it will all be re-split by the shell).

binds {
    // These work:
    Mod+D { spawn-sh "alacritty --working-directory ~/source" }
    Mod+Q { spawn-sh "notify-send clipboard $(wl-paste)" }
}

Now, this is not a breaking change by itself. The breaking change part is that I want to also rename spawn to spawn-raw (spawn-raw-at-startup). This is to remove the bias towards spawn: I don't want to guide people to unknowingly using (raw) spawn and having it not work. But I still want it to remain easily accessible if you don't want the shell overhead. And I don't want to straight up replace spawn itself with a shell because it will make the migration more difficult.

So, what do you think? Both about the "spawn" ideas, and about maybe doing some other changes for KDLv2, since we have this opportunity.

[JohnDowson]

I don't think it's a good idea to use scary words like "raw" for the thing you actually want to use most of the time.

[YaLTeR]

I guess some mandatory context here is that other compositors (sway, hyprland) use a shell by default for their spawn and exec.

I don't think raw is too scary; it'll all be documented in the default config and on the wiki anyway.

[foulmerchant]

With v2 making the string look nicer and additional erroring, I think already defaulting towards the avoidance of the overhead and leaning towards the use of 'spawn' is correct as is.

Doesn't break anything and using 'spawn-sh' is a simple enough toggle for anyone needing parameter expansion to remember.

[zkat]

my contribution to the bikeshed:

binds {
  Mod+T { sh "rm -rf /**/{*.rlib}" }
  Mod+Q { exec ls -l foo }
}

and tbh, I see there's a ton of noise in the niri config because you're exclusively putting binding values in children. Consider perhaps:

binds {
  Mod+T sh "rm -rf /**/{*.rlib}"
  Mod+Q exec ls -l foo
  // And you can still use children as "additional commands" if you want.
  Mod+X {
    sh "echo hello"
    exec bash -c "echo hello"
  }
}

'cause you really can just keep giving args to a node. When you want bind options, you can always just use props for them, which will be parsed separately from args:

binds {
  Mod+T allow-when-locked=#true sh "rm -rf /**/{*.rlib}"
}

As far as syntax highlighting goes, nodes get highlighted differently, so things will look like this:

Finally remember that multiline strings are supported and they're very very nice in KDLv2 (for example, allowing dedenting), so this is possible:

[sodiboo]

I like sh/exec as the duo. They are a better pair than what i suggested.

[YaLTeR]

I kinda like sh/exec, however I can also see how exec can be confusing to less technical users (it makes more sense when you know that exec() is a syscall). The way I see it is: the word "raw" indicates that something is going on here even to people who might not yet know the difference.

Interesting idea about optionally skipping the {} for single-action binds. It works this way mainly because knuffel makes it very easy to parse with an enum derive. However, we could possibly automatically rewrite it to add {} before passing it to knuffel?

Another potential problem is that explicit {} provides clearer separation of the bind properties (e.g. allow-when-locked) from the action properties. We have a few of those: quit skip-confirmation=true, do-screen-transition delay-ms=500, screenshot-window write-to-disk=false. You could make it order-dependent, but this makes it somewhat less clear what goes where.

[zkat]

another one to chuck over the fence: exec -> run, or just keep spawn for that.

[zkat]

also knuffel can accept both children and arguments at the same time. I think your particular types may need some slight rearranging, but you should be able to just accept both, and then just concatenate your "immediate" bind to the list of "child block" binds.

[sodiboo]

I'm very much in favor of spawn-sh, because that is what a lot of people want when initially configuring niri (ideally, you'd want to avoid it as it brings overhead, but most simply don't care, and it's more important to not be confused at "why not working").

This is precisely why niri-flake's documentation for binds suggests setting sh = spawn "sh" "-c"; (scroll down a little in the link to just before the listing of all actions), and there are dozens of occurrences in the wild.

And i agree with breaking spawn so as not to make that one seem "the default". But it should still be parsed and throw a relevant error suggesting both of the others.

But i think spawn-raw is slightly confusing, if not even scary like @JohnDowson said. I think a better name would be spawn-args to indicate that the arguments passed to this action are exactly the command line arguments that will be invoked. a "raw" action feels like it has the wrong connotation/vibes for what spawn-args actually is.

I also think that spawn-{sh,raw,args}-at-startup feels weird/verbose. This is a top-level node, so there should be a single name to unify both of them, to indicate they are part of the "same section" of the config. To this end, i'd like to suggest reworking spawn-at-startup to look more like switch-events, which would accept spawn-sh or spawn-args children. So, i think that section of the config should look more like:

startup {
    spawn-sh "swaybg -i $(~/random-wallpaper.sh)"
    spawn-args gammastep -l 59:11
}

(on a more technical note, this representation is a bit friendlier to "merge", which niri doesn't currently do but other tools like niri-flake's settings do. to preserve relative ordering between spawn-sh/spawn-args invocations at startup, i will need to use an internal representation similar to this anyways. i can render them in the other format just fine, but it's just more pleasing to use the same representation in both places)

---

wacky exploration:

This also naturally lends itself to move hotkey-overlay { /-skip-at-startup; } -> startup { show-hotkey-overlay; }. It does, however, invert the "default value" that occurs when no value is present. (but we can still make it show by default if absolutely no config was present, and include it in the default config that is generated in that case).

We don't need to conform to a subset (strict or otherwise) of the bind actions, though. We can e.g. make the hotkey overlay the default, and have startup { skip-hotkey-overlay; }. (or just keep it as it was; i'm just exploring what can be done). But if we're gonna have anything that isn't a sequential series of actions here (e.g. skip-hotkey-overlay is really a boolean), then at least in niri-flake i'd basically need to split off the spawns into startup.actions = [ ]; so i can have startup.skip-hotkey-overlay = false;. I don't think the same logical split is necessary for niri's own config representation, as it still makes sense as a config if they're at the same level, and i'm at least fairly certain that knuffel can handle this just fine.

[YaLTeR]

spawn-args

I think I prefer spawn-raw to spawn-args. While you could argue that args is an important difference from the config perspective, the actual important difference (why you would mainly want to use spawn-raw over spawn-sh) is that it does vs. does not spawn a shell, which I believe "raw" conveys better.

startup {}

Interesting idea, though I'm not sure it pulls its weight. If anything, the current separate spawn-at-startup lines are already merge-able (e.g. they support a simple cat part1 part2 part3 > config.kdl), whereas a startup {} block needs specific merging code (like all other nodes in the config currently do).

to preserve relative ordering between spawn-sh/spawn-args invocations at startup

I'm not sure preserving the ordering does much good for them. Yes they are invoked sequentially, but it's a fork-exec, so realistically there's no useful determinism you can rely on here, is there?

startup { skip-hotkey-overlay; }

This is a bit far-fetched tbh, I don't think there's anything else (other than spawn and this) you might want to put into the startup block, so eh?

[bbb651]

I don't like that spawn-sh doesn't specify the shell, if it uses the default shell it makes configs less portable, if it always uses sh syntax you expect to work will fail, which can be surprising either way, and if specifically want a different shell you'll still have to resort back to the normal spawn.
I also think it's fine that non-shell spawn is biased towards, in addition to being more efficient I would argue having quoting work at the config level and not the shell level is more intuitive, and it also matches the behavior of desktop file's Exec.
I also think spawn-sh and spawn-raw fail at their goal of making it more intuitive for new users because it makes forces you to think about and understand the difference between them when it doesn't matter for 90% of cases, having good error messages and documentation makes you learn about it once if you need to, and then it's no longer in your face.

I found a couple of examples of approaching for specifying shell/languages in different projects that I liked:

• hyperfine uses a shell by default, and has a --shell=<shell> argument that can be set to the shell executable or none (-N alias) to disable and default to explicitly set it to the default.
• justfile's shabang recipes let you set the interpreter used for the recipe with a shabang line. This doesn't work for single line
• typst raw blocks work similarly to markdown codeblocks, but have a single line variant e.g. ```rust fn main()``` will only

How these will look in niri:

spawn with optional shell/sh property, defaulting to no shell

binds {
    Mod+D { spawn shell=sh "notify-send clipboard $(wl-paste)" }
}

In hyperfine the shell argument can also be a string with more arguments (this unfortunately requires shell parsing if we want to support it since kdl doesn't have lists in property value position) and implicitly adds -c which works for most cases (not all of them though, e.g. I think env makes sense to use as a shell but it errors on -c).

If we want to make e.g. sh the default, it would only work if kdl had optional property values, something like spawn sh= "notify-send clipboard $(wl-paste)" but it's not valid syntax. (This almost looks like spawn <shell>="<script>" but that is horrible on many levels xD)

spawn with shabang

This only works for multiline which is a non starter and it's also a little verbose, but otherwise is quite nice, especially with the braceless option:

binds {
    Mod+D spawn #"""
        #!/bin/sh
        notify-send clipboard $(wl-paste)
        """#
}

I guess making this single line can work if it's quoted but at this point it's a weird variant of the shell= option:

binds {
    Mod+D spawn "#!/bin/sh" "notify-send clipboard $(wl-paste)"
}

Prefix shell in raw block

This again requires syntax kdl doesn't have (and I don't necessarily think is a good idea since kdl is very general purpose) like a separate kind of string that semantics for a language/interpreter, like:

binds {
   Mod+D spawn `sh notify-send clipboard $(wl-paste)`
}

Edit: A benefit of this is that since it's on the language level, it'll work really well with tooling, e.g. you can have tree-sitter language injection

I like the shell= property a lot, it's explicit, and especially if we ever want to add more properties to spawn, they feel like they should be the same action. This also doesn't break the current spawn at all.

[YaLTeR]

I feel this may be overcomplicating it. Like, this is a compositor config, not a jupyter notebook; we don't need the whole shebang and shell parsing. If sway and hyprland can hardcode sh, I'm sure we'll be fine.

if it uses the default shell it makes configs less portable

I would argue it makes configs more portable because they don't depend on your login shell. And yeah, nobody is taking away the raw spawn if you need that.

I also think spawn-sh and spawn-raw fail at their goal of making it more intuitive for new users because it makes forces you to think about and understand the difference between them when it doesn't matter for 90% of cases, having good error messages and documentation makes you learn about it once if you need to, and then it's no longer in your face.

We will have KDLv2, better error messages and documentation in addition to an easy spawn-sh, those are not mutually exclusive. I don't think it forces you to think about it much more than the alternatives?

[bbb651]

I meant that if I was a new user and I see both spawn-sh and spawn-raw, I would not know what's the difference between them and which one I should use. I feel like that defeats the whole purpose of making things work out of the box with the less efficient option, because you have to care enough about it to know that you don't need to care about it.

But I think this is the more important part:

in addition to being more efficient I would argue having quoting work at the config level and not the shell level is more intuitive

I think this really hurts usability (also true for the current spawn "sh" "-c" "..."), you can't simply join it back like you said:

The reason for accepting a single argument is that shells accept their command as a single argument, and do their own parsing and word splitting. While we could accept multiple arguments and join them back with whitespace, I don't think it's a good idea because it would appear as though you can pass one argument "like this", but you actually cannot (it will all be re-split by the shell).

What is we encode each argument back to a literal (quote it if it was quoted in kdl, turn common escapes like newlines to "\n", etc.) and then join them back? It a bit fragile but it'll make the majority of cases work, e.g. spawn notify-send clipboard $(wl-paste) and spawn notify-send "clipboard $(wl-paste)" will just work even though the quotes are kdl quotes and not shell quotes.

[YaLTeR]

I meant that if I was a new user and I see both spawn-sh and spawn-raw, I would not know what's the difference between them and which one I should use.

I was going to write a comment in the default config explaining it. I honestly don't think it will be complicated or long? Something like:

// You can run programs with spawn-raw.
Mod+T { spawn-raw alacritty -e /usr/bin/fish }

// If you need pipes, multiple commands, variables, ~,
// you can use spawn-sh which runs the command through a shell.
// This is a tiny bit slower than spawn-raw.
// Note: the entire command is in quotes.
// Mod+T { spawn-sh "notify-send hello && exec alacritty" }

What is we encode each argument back to a literal (quote it if it was quoted in kdl, turn common escapes like newlines to "\n", etc.) and then join them back? It a bit fragile but it'll make the majority of cases work,

I really don't want anything like this. It will surely break in ways that are even harder to debug than the current ones.

When dealing with anything shell-related, I'm aiming for the least possible magic and logic on top. Which is why I am suggesting that spawn-sh should only accept a single argument, which is passed literally to the shell. While the raw spawn command should keep accepting multiple args that are literally passed as multiple args with minimal processing (only expanding a single ~ in the binary name).

[phisch]

Couldn't you introduce config migrations, so that for example 25.01 -> 25.02 migrates the otherwise breaking config changes. This way you can just change things however you like, and don't even have to consider them breaking changes. But you need to keep these migrations forever, so that you can upgrade from any version to any larger version (I would assume downgrade migrations are not necessary).

[JohnDowson]

I think the config has got some goofy aah naming going in places, that I personally would love to see fixed before Niri got more popular. Stuff like consume-or-expel-window-left. Not only is the name super long, it doesn't even describe what the action actually does. You'd think invoking it twice on the same window would get you back into the same state, but it actually expels on the opposite side.
And then there's another level of confusion, because "consume or expel" sounds as if the target of the action is the column doing consuming or expelling, but you actually want to focus the window you want consumed or expelled.

I'm actually getting more confused as I'm trying to explain my confusion.

Then there's the "why is this a separate command" thing, like with focus-column-right-or-first. I at first completely missed that this was possible and went to actually implement scroll wrapping as a "wrapping behaviour" setting, and used that fork for a few days.

[YaLTeR]

Action names are also exposed in the IPC and the CLI so we can't change them. We can add aliases though. Either way it's orthogonal to KDLv2.

Stuff like consume-or-expel-window-left. Not only is the name super long, it doesn't even describe what the action actually does. You'd think invoking it twice on the same window would get you back into the same state, but it actually expels on the opposite side.
And then there's another level of confusion, because "consume or expel" sounds as if the target of the action is the column doing consuming or expelling, but you actually want to focus the window you want consumed or expelled.

You're not the first person to get confused about this, but I don't really get where the confusion comes from. The action name is spelled as a sentence here, like in other actions: "consume or expel (the focused) window left". You take the focused window and consume/expel it to the left side. Do you have any concrete ideas on how to improve this? Because I don't know.

Then there's the "why is this a separate command" thing, like with focus-column-right-or-first. I at first completely missed that this was possible and went to actually implement scroll wrapping as a "wrapping behaviour" setting, and used that fork for a few days.

Individual contributors wanted specific actions like this and implemented them as separate actions. With time it became clear that some more generic way would be better, but it hasn't yet materialized (not even as a concrete design proposal).

[JohnDowson]

Action names are also exposed in the IPC and the CLI so we can't change them.

I disagree. Not only can, but should break everything that's currently in non-ideal state, at once, simultaneously with transition to kdl2.

With time it became clear that some more generic way would be better, but it hasn't yet materialized (not even as a concrete design proposal).

So what you're saying is that there's a space for my patch to become a PR? I personally would be somewhat against adding new orthogonal ways to achieve the same thing, without deprecating old ones.

[YaLTeR]

So what you're saying is that there's a space for my patch to become a PR? I personally would be somewhat against adding new orthogonal ways to achieve the same thing, without deprecating old ones.

I am open to discussing and afterwards implementing a generic focus command. It should support jumping between columns/workspaces/monitors and looping. It is not a problem to leave old actions hanging around.

But before a PR, please open a discussion with a design proposal on what it should look like in the config.

[phisch]

I think the amount of actions we have right now is very overwhelming, and it would be worth exploring a more programmatic approach instead. For example only having a single focus command that takes some arguments to achieve all the different outcomes we currently have in 32 (thirty-two) different actions.

And we have similar amounts for other actions. Stuff would become a lot cleaner and maintainable that way, I think.

[sodiboo]

How about removing all distinction of proportion/fixed, and instead representing them as a single property with a unit, as well as all other lengths and dimensional quantities in the config?

For the sake of not making most things unnecessarily more verbose, we probably should allow a plain float-or-int in most places that currently accept a fixed size in logical pixels. This allows existing configs to not need to add units everywhere.

But we should encourage specifying a unit of px; since KDL v2 allows strings to be unquoted in many cases, we can now say e.g. gaps 5px without the quotemarks needed in KDLv1. This makes dimensional quantities a lot more ergonomic to specify

We could also require the unit. This has a utility from the documentation perspective: Some users who are not familiar with layouts, but who know what a pixel is, might think that dimensionless lengths are just in pixels. "oh this is a low level program that deals with pixels, of course it wants pixels". As such, a lot of documentation repeats that whatever dimensionless quantity is actually in logical pixels, and the current default-config.kdl explains what that means in not just one but two places. While this isn't inherently a major source of confusion unless you're entirely unfamiliar with the concept, it's somewhat easier to understand if we define at the very top in a comment that explains that "anything ending in a unit of px represents a distance in logical pixels, which means..."; then, we wouldn't have to repeat it everywhere, and it's one of the first things you see opening the config. It also sticks out more in the only place that uses physical pixels. It also helps distinguish more clearly an actually dimensionless quantity from one that simply doesn't have a comment (yes, that one is documented in the wiki but then you have to go to a web browser)

Anything that is named proportion is of course putting the unit in the name; it would be much more ergonomic to replace that with a unit of %. For instance, default-column-width doesn't need a child node; it becomes a leaf argument taking a single argument of either default-column-width 1280px or default-column-width 33%. It also accepts default-column-width #null (such that the initial configure receives a zero-sized width, meaning "there is no default width, pick your own"), defaulting to default-column-width 50%. Explicitly naming this value #null (note to those not familiar with KDLv2: this is how null is spelled now) is more intuitive than having its omission be meaningful and distinct from the default value (i.e. currently layout {} has a default width 50%, and layout { default-column-width } sets it to null. wtf?).

preset-column-widths is also a prime candidate for the exact same transformation. Instead of being a node with children, it can just take dimensional arguments preset-column-widths 33.333% 50% 66.667% 1920px. Likewise, for preset-window-heights. These of course do not accept #null as an argument, as that only makes sense in the initial configure of a window.

The set-{column-width,window-height} actions already use this schema. But they don't specify units for the fixed size variants. I think they should be changed to require a px suffix when you're e.g. adding +5px, simply because it is easy to confuse a proportion otherwise.

Most proportions in niri represent a window size, where the proportion is relative to the output size
<highly-subjective>I think this creates a connotation of proportion meaning "proportion of the output size". Therefore i think e.g. tab-indicator { length total-proportion=1.0; } feels like the total-proportion refers to the output size.</highly-subjective>
Of course, it is a candidate for replacing with a dimensional quantity. As discussed in the matrix room prior to me making this comment, i think the existing names (width/length) for the tab indicator measures are unintuitive (because width is vertical when the tabs are aligned horizontally, and length doesn't feel like an appropriate 2D pairing with width). I came to the conclusion that i "want" them to be called thickness and main-size. With the dimensional quantities, we can represent this as thickness 4px and main-size 50%.
<highly-subjective>This also fixes the association with the output proportion to me. Because the config otherwise has some aspects of CSS (color naming), which {{this comment}} only perpetuates, the connotation of 50% is in my head just "50% of the container", which is obviously the window width/height depending on the tab axis.</highly-subjective>
Using main-size 50% is also compatible with later expanding it to be each tab's size, e.g. main-size tab=20px, main-size each=20px, and so on. I am open to change about the main-size name; and honestly the current length is perfectly acceptable too, so long as the cross size is renamed to thickness.

Another candidate for adding units is for the easing animation. it takes a duration-ms=200 property, which i think would be much more ergonomically spelled out duration=200ms. In this case, the units should be required, as duration=200 does not make sense. Likewise, we should rename Mod+ScrollWheelUp cooldown-ms=150 to Mod+ScrollWheelUp cooldown=150ms.

We should also allow (enforce?) a unit of deg for gradient angles. CSS enforces a unit here, and our documentation says basically "lol these props do the same thing as css", so we ought to have a similar way to write angles in my opinion.

[zkat]

One stupid trick to consider is to pick from one of the characters that are identifier-legal and use them as a prefix. Then your stuff works.

literally any of these: -<~!$@%^&*,:'|?> (and the backtick, which I can never remember how to escape)

[YaLTeR]

To entertain this idea some more, lists could use the - node name:

preset-column-widths {
    - 33%
    - 50%
    - 66%
    - 1000px
}

But yeah this also bumps into the first digit problem. All in all I'm not sure the workarounds make it worth it over just sticking to the current syntax. (And since I didn't want to rename things unnecessarily for this KDLv2 migration, even if it worked as is, I'm not sure I would go with it. It's not like the current syntax is unusable or anything.)

[JohnDowson]

FYI unquoted strings still follow classic “identifier” rules, so they can’t start with numbers

Uhhh I have a suggestion for KDL3

[zkat]

The reason most languages don’t do this is to make sure numerical literals can be clearly distinguished from identifiers.

changing this would mean a typo like 1_000_a000 would yield a string. Or, depending on the KIND of flexibility, it might result in just more added complexity to the unquoted string logic, which is already fairly heavy

that said: maybe limiting it strictly to suffixes might be ok.

I’ll file an issue for discussion

[zkat]

kdl-org/kdl#510 Feel free to come in with your thoughts. I think this might solve your use case