Language feature proposal: exclusive 'with'

[alexeymuranov]

A problem with with expression in Nix is that to know which values the identifiers in the scope of with dict; refer to, one needs to examine the entire dictionary/set dict [not quite true, see Edit 1], which could be defined in a different file or composed in a dynamic way. This is especially true if this with expression is itself in the scope of another with expression (i've got "bitten", or at least surprised, by this). I imagine this could be the reason why the analogous with statement was removed from ECMAScript strict mode.

How about introducing into Nix an exclusive version of with, let's call it only_with:

{ # ...
  packageOverrides = pkgs: ONLY_WITH pkgs; { # ...
    myTexLive = texlive.combine {
      inherit (texlive) scheme-basic
                        collection-fontsrecommended
                        collection-genericrecommended
                        collection-latex
                        collection-latexextra
                        collection-latexrecommended
                        collection-luatex
                        collection-mathextra
                        collection-xetex
                        latexmk;
    };

    myHaskellEnv = haskell.packages.ghc802.ghcWithPackages (
      haskellPkgs: ONLY_WITH haskellPkgs; [
        cabal-install
        cabal2nix
        yi
        vim  # *** Error! ***
      ]
    );
  };
}

What do you think?

---

Edit 1

From a comment below i discovered that

let
  x = 0;
  a = { x = 42; };
in
  (with a; x)

evaluates to 0. Is this documented anywhere?

While being a bit unexpected, this behaviour allows to determine which values variables in the scope of with foo; are bound to without inspecting foo ... unless there are nested with or optional arguments.

[thufschmitt]

Although I understand the rationale behind the behavior of with, I find it unpredictable and error-prone. So I'm fully in favor of adding an operator with slightly more sensible semantics.

[thufschmitt]

(However, this probably should go through an rfc).

[matthewbauer]

Maybe I don't understand the rationale but to me:

{ x ? "z" }:

let
  a = { x = "y"; };
in
  (with a; x)

should evaluate to "y" not "xz" which is not the case right now.

[copumpkin]

It should, but changing something that would silently change the meaning of tons of existing code seems difficult. We could force-enable my #1364 warning for a couple of releases, saying that the binding order will change and that you should update your code to not be ambiguous.

[alexeymuranov]

@matthewbauer, wow, i would say that this is a true bug, but the documentation is silent about what the correct behaviour really is.

Here is a slightly simpler example:

let
  x = 1;
  a = { x = 42; };
in
  (with a; x)

evaluates to 1.

P.S. In fact, this behaviour makes it easier to understand what the actual binding in the scope of with is, unless there are nested with.

[matthewbauer]

I think the low precedence of with can be useful but only when dealing with large attributes like using with pkgs;. You don't want the with to override yours variables. It seems like it needs to be more verbose though. Perhaps something like Haskell's hiding would be helpful? For example:

with pkgs hiding (darwin);

That would remove any legitimate use cases of this feature IMO.

[thufschmitt]

I think this is mostly a security feature, in order to avoid accidental (and invisible) name capture, so hiding wouldn't help much (except of course if nix throws some warning if you shadow a variable) as you can only use it if you're aware of the conflict.

[edolstra]

@regnat The current behaviour is more predictable. You can tell from the lexical scope what variable is going to be used. E.g. in

let foo = ...;
in with pkgs; ... foo ...

you don't have to worry that somebody adds an attribute named foo to pkgs, silently changing the semantics of your expression.

Adding a second with construct doesn't seem elegant language design to me. with is already a questionable feature, so having two would be really bad...

The docs could be improved though.

[alexeymuranov]

However:

(with { a = 1; };
  (with { a = 2; };
    a))
# => 2

(let a = 1; in
  (with { a = 2; };
    a))
# => 1

After learning all this, i think i will not be using with in Nix, and i will not mind at all if it will be deprecated :).

As to the proposed only_with, i think i would have been using it. I am not denying that there might exist even better alternatives.

[danbst]

@alexeymuranov
same with lets
let a = 1; in let a = 2; in a === 2

---

In general it seems like a bad idea to do haskellPkgs: with haskellPkgs; [.... The main usecase for with here is to reduce typing, and in this case there is another nice solution (not adopted by Haskell nixpkgs, unfortunately), using inherit.

You've even provided the example. inherit doesn't have overrides and gives an error when required attribute is absent.

[taktoa]

Personally, I like to use with { … }; … over let … in … because it is easier for my editor to correctly indent and looks more "consistent" with the rest of Nix's syntax. However, the low precedence of with can make this confusing. Perhaps it would be good to add a let { … }; … binding structure, where the curly braces are required. Then you can recover with-style semantics by inheriting inside the curly braces.

EDIT: just to be clear, this is 100% equivalent to the existing let … in … syntax, it just looks more consistent with assert …; … and with …; ….

[danbst]

I definitely like it. But I've stumbled upon this https://nixos.org/releases/nix/nix-0.5/manual/manual.html#id2526745, which says

5.1. Grammar

[8] | ExprSimple | ::= 
  ...
  | 'let' '{' Bind* '}' 

which means that was a case in Nix 0.5?

[edolstra]

The old let syntax requires a special attribute body, e.g.

nix-repl> let { x = 1; body = x; }
1

[taktoa]

hmm, that is very close to the syntax I want... the fact that variables are lexically bound in body is very confusing however (since it is at "the same level" as the variables).

[danbst]

It's possible even to preserve pkgs variables in that scenario ``` ... = builtins.attrValues { inherit (pkgs) foo bar lolcat; }; ``` In fact, it would be great if `*buildInputs` supported attribute set, and not only a list as input, so it would be clean usage for `inherit`-style sets. For example, that would allow /remove/ a build input in `overrideAttrs`: ``` foo.overrideAttrs (super: { buildInputsSet = lib.filterAttrs (n: v: n != "bar") super.buildInputsSet; }) ``` But speaking of `with`, it is often convenient to do `with lib;`. Should this be deprecated too? In this case it deserves a RFC. 2018-01-05 2:39 GMT+02:00 volth <notifications@github.com>:

…

I would totally deprecate with as a language feature. For syntax-sugar purposes, it could be implemented as a lib-function used like lib.With pkgs [ "foo" "bar" ] or lib.With pkgs "foo bar" Currently it is very easy to hit a problem writing the code like let lolcat = 0;# ....hundred of lines....in# ....hundred of lines....with pkgs; [ foo bar lolcat ] It is nowadays normal in nixpkgs to have huge lists after with pkgs; or with pythonPackages; expecting list of packages, not checking that there could be a same-name variable defined few scopes upwards which has priority over with-bindings. And pkgs has tons of generic names. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1361 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAtWkfbN4CPHlaP2zcn08JeY1uESAcZjks5tHW9agaJpZM4NJ9tq> .

[masaeedu]

I think the confusion is mitigated if you think of with as a "fallback" lexical scope. Any variables within a with block that are not otherwise bound are assumed to be bound to the nearest with. The mental "desugaring" of the variable reference looks different depending on whether a matching let binding is in scope:

• Let binding available: let x = 10; in (with { x = 42; }; x) -> let x = 10; in (let ctx = { x = 42; }; in x) -> 10
• Let binding not available: with { x = 42; }; x -> let ctx = { x = 42; }; in ctx.x -> 42
• With-shadowing: with { x = 10; }; with { x = 42; }; x -> let ctx = { x = 10; }; in (let ctx = { x = 42; }; in ctx.x) -> 42
• Both let-binding and with-scope missing: let y = 10; in x -> should not parse

A further measure to reduce confusion might be to syntactically distinguish with-lookups from let-lookups, by forcing users to type .e.g with { x = 42; }; .x or with { x = 42; }; ^x rather than with { x = 42; }; x. This way a with lookup can never be confused visually with a let-bound variable. For nested with blocks the intuitive let-based semantics in the desugaring apply, where the nearest context is the one the lookup will be performed on.

[Warbo]

I just stumbled into this while googling for something else, and thought I'd weigh in since it's concerning to see so many calls to deprecate with. If we're going to deprecate something, I'd much rather get rid of let. I avoid it in all of my code, and it only seems to complicate the language for no benefit:

• with works with attrsets, which are first-class values; let works with the current scope, which is a second-class "ambient" feature which we can't reference, pass around, inspect, etc.
• Altering how a let expression behaves requires altering the language; we can change how a with expression behaves by just feeding it a different attrset.
• with bindings can be recursive or non-recursive, precisely because they're just ordinary attrsets.
• with bindings can be open/extensible/dynamic by passing in a variable, or closed/static by passing in a set literal (either defining names explicitly, or using inherit). Again, this is precisely because we're working with ordinary attrsets.
• The usual justification for including let in a language doesn't apply to Nix. We can always lexically scope values using "one shot functions", e.g. ((x: y: ...) foo bar) (this was very common in Javascript, for example). This is bad since (a) the order of the definitions is significant (b) the matching of names to values is implicit (based on order) rather than explicit (c) the names may be very far away from their values (if the function body is large). To avoid this, many languages introduce let; but Nix can avoid them using named arguments instead: (({ x, y }: ...) { x = foo; y = bar; }). Hence there's no need for let. This has the added advantage that ordinary sets can be passed in, but is slightly problematic due to the names being repeated and definitions appearing after the expression; but that's precisely what with solves.
• with is terminated with a "standard" separator ;, just like other constructs (e.g. assert, name = value definitions, etc.); let introduces another keyword (in) solely for use as a terminator/separator.

Note that I do consider the "low precedence" behaviour of with to be a problem, since it violates the Principle of Least Surprise, but I appreciate that it may need to be kept for compatibility. However, I hardly ever run into it simply because I never use let (it also bites for function arguments, but it's a thankfully rare situation).