Discussion: complex nested with expressions

[orthoxerox]

As mentioned in one of the LDM notes, with syntax doesn't really pay for itself, since it's almost tasteless syntactic sugar over With methods. I want to restart the discussion from dotnet/roslyn#16459 in the new repo:

The withers should be nestable and transform compound assignment, like this example below, this makes them much more useful than bare With methods:

public static Order UpdateQuantity(Order order, Product product, Quantity newQuantity)
    => order with { Lines[product].Quantity = newQuantity };

public static Order SwitchProduct(Order order, Product oldProduct, Product newProduct)
    => order with {
        Lines[newProduct] ?= new OrderLine(newProduct, 0),
        Lines[newProduct].Quantity += order.Lines[oldProduct].Quantity,
        Lines = order.Lines.Remove(oldProduct)
    };

More details

1. with should accept nested immutable values:

e1 with { P1.P2 = e2 }
//is equivalent to
e1 with { P1 = e1.P1 with { P2 = e2} }
//or further
e1.With(P1: e1.P1.With(P2: e2))

2. with should accept compound assignment operators:

e1 with { P1 += e2 }
//is equivalent to
e1 with { P1 = e1.P1 + e2}
//or further
e1.With(P1: e1.P1 + e2)

3. with should work with immutable collections (they probably should support something like WithElement (AddOrUpdate?)):

e1 with { [e2] = e3 }
//is equivalent to
e1.WithElement(e2, e3)

4. Compound conditional assigment to indexes should use conditional access:

e1 with { [e2] ?= e3 }
//is equivalent to
e1.WithElement(e2, e1?[e2] ?? e3)

Now we can see how the examples will be transformed by the compiler:

Let's say we have the following immutable records:

public class Order(Customer customer, ImmutableMap<Product, OrderLine> Lines);
public class OrderLine(Product product, Quantity quantity);
public class Quantity(uint value);

e.g.:

var expr = new Order(
    customer,
    {
        [boots] = new OrderLine(boots, 100),
        [gloves] = new OrderLine(gloves, 200)
    });

public static Order UpdateQuantity(Order order, Product product, Quantity newQuantity)
    => order with { Lines[product].Quantity = newQuantity };

//is equivalent to

public static Order UpdateQuantity(Order order, Product product, Quantity newQuantity)
    => order.With(
		Lines: order.Lines.WithElement(
			product, 
			order.Lines[product].With(
				Quantity: = newQuantity)));

public static Order SwitchProduct(Order order, Product oldProduct, Product newProduct)
    => order with {
        Lines[newProduct] ?= new OrderLine(newProduct, 0),
        Lines[newProduct].Quantity += order.Lines[oldProduct].Quantity,
        Lines = order.Lines.Remove(oldProduct)
    };

//is equivalent to

public static Order SwitchProduct(Order order, Product oldProduct, Product newProduct)
    => order
		.With(
			Lines: order.Lines.WithElement(
				newProduct,
				order.Lines?[newProduct] ?? new OrderLine(newProduct, 0)))
		.With(
			Lines: order.Lines.WithElement(
				newProduct,
				order.Lines[newProduct].With(
					Quantity: order.Lines[newProduct].Quantity
						+ order.Lines[oldProduct].Quantity)))
		.With(
			Lines: order.Lines.Remove(oldProduct));			

Grammar

primary_expression
    : with_expression
    ;

with_expression
    : primary_expression 'with' '{' with_initializer_list '}'
    ;

with_initializer_list
    : with_initializer
    | with_initiaizer ',' with_initializer_list
    ;

with_initializer
    : with_path assignment_operator expression
    ;

with_path
    : with_member_access
    | with_element_access
    ;
    
with-member-access
    : with_path '.' identifier
    | identifier
    ;
    
with-element_access
    : with_path '[' argument_list ']'
    | '[' argument_list ']'
    ;

[alrz]

@orthoxerox Your examples are too complicated that lose the point. I'd suggest you to demonstrate a single case along with its equivalent code at a time. That might help to understand what is going on.

[orthoxerox]

@alrz Good idea, I've rewritten the OP.

[bradphelan]

@orthoxerox Thanks for proposing this. The current "withers" proposal really needs generalizing as you have proposed. In my own production code I use a variant of this post I made years ago.

https://gist.github.com/bradphelan/5395652

I used Expresion<Func<T,R>> objects with reflection to implement recursive wither type behavior. See the example of the Set method below.

I even have code for immutable updates of objects within immutable collections just not in the gist as posted.

A native solution for this would be great.

namespace Utils.Spec
{
    public class ImmutableObjectPatternSpec
    {
        [Fact]
        public void OhMyLord()
        {
            var a = new A
                ( p:10
                , b: new B
                    ( p: 20
                    , c : new C(30)));

            var a_ = a.Set(p => p.B.C.P, 10);

            a.Should().NotBe(a_);
            a.B.C.P.Should().Be(30);
            a_.B.C.P.Should().Be(10);

            
        }

    }
}

[gafter]

See #162 for the champion issue most closely related to this discussion. I've linked this disucssion from there.

[shaggygi]

I believe this thread is similar to what I was looking for in the issue [Question/Proposal] Immutable Parent Child API.

I was initially comparing an immutable parent/child scenario similar to what the team did in Roslyn with SyntaxTrees and SyntaxNodes.

Let's use the infamous Person class. A Person object can have zero, one or more children (which are also Person objects) and its children can have zero, one or more children (Descendants; also Person objects), etc., etc. The Person object can also have a Parent (Person object) and going up the tree, can have zero, one or more Ancestors (also Person objects).

So going back to the comparison with Roslyn, SyntaxNodes are updated and returned with the changes. In addition, the Parent and SyntaxTree are updated respectfully (or at least in a manner to lazily query for them when needed).

I get the basics of updating properties like WithFirstName(FirstName: "Bobby"). However, what if you wanted to replace a specific descendant person, similar to using SyntaxNode ReplaceNode method?

// This code is not exact, just trying to explain a little further.
Person granddad = new Person(// with a child; c1 and a couple grandchildren; gc1, gc2);
Person grandchild2 = granddad.Children[0].Children[1];  // Get gc2 from c1.
var gd = grandchild2.Parent.Parent; // granddad and gd should be equivalent.
Person gc3 = new Person(// with properties);  // Create a new grandchild person.
var gdUpdated  = granddad.ReplacePerson(grandchild2, gc3);  // Removed gc2 and replaced with gc3.

// At this point, gdUpdated would look like the following...
// gcUpdated
//    c1
//        gc1
//        gc3

// In addition, I didn't specify anything about granddad when instantiating grandchild gc3.
// However, a relative connection would be made when replacing the grandchildren.
// Meaning, finding the granddad below would be similar to finding above.
Person grandchild3 = granddad.Children[0].Children[1];  // Get gc3 from c1.
gd = grandchild3.Parent.Parent; // granddad and gd should be equivalent.

So my question is... how could/would you be able to do something similar using Records or Immutable Types and what is being discussed in related proposal(s)?

Any help is appreciated.

Updated: I noticed they mentioned Roslyn SyntaxTrees as a use-case for Records in the C# Design Meeting 2015-03-25

[mattwar]

This would sure that with syntax more compelling. However, it would require more expected convention on the part of immutable types. The current proposal only requires the type getting the with syntax to have a With method with all the corresponding parameters. This proposal would require the compiler to understand things like collections have WithElement and possibly others. Maybe that's not so bad.

[orthoxerox]

@mattwar indexed immutable collections in System.Collections.Immutable have SetItem that does what WithElement is supposed to do. @louthy's library uses this naming convention as well. It could probably be made a standard equivalent of []= in some framework design guidelines without much pain.

[mattwar]

@orthoxerox its not so much that existing libraries have conventions or even agree, its that the language with have to bless one of the patterns.

[orthoxerox]

Maybe it's not so bad that one pattern will have to be chosen.

[kofifus]

Note that With can be implemented quite elegantly and efficiently with C#7 and caching lambda expressions, see my https://github.com/kofifus/With or abaik https://github.com/ababik/Remute

With With you can do:

var org = new Organization("Organization", new Department("Development Department", new Employee("John", "Doe")));
var mutatedOrg = org.With(x => x.Sales.Manager.FirstName, "Foo");

Object.ReferenceEquals(org.Sales.Manager, mutatedOrg.Sales.Manager)); // false
Object.ReferenceEquals(org.Sales, mutatedOrg.Sales)); // false
Object.ReferenceEquals(org.Name, mutatedOrg.Name)); // true
Object.ReferenceEquals(org.Sales.Title, mutatedOrg.Sales.Title)); // true

Hope this helps

[alrz]

Instead of WithElements perhaps we could use SetItem which is already present in immutable collections.

[orthoxerox]

@alrz indeed

[jnm2]

I love this proposal! It would be a great quality-of-life improvement for me.

@orthoxerox What do you think about making the old value available so that non-with operations can be nested? For example:

state = state with { PerCompany.State.LineItems.Rows = Rows.Remove(lineItem) }

The names in scope would be Rows, LineItems, State, and PerCompany. If there is a name collision, closer names (inner names) would shadow names that are further away in the hierarchy or names defined outside the expression.

[ronnygunawan]

with_initializer
    : with_path assignment_operator expression
    : with_path 'with' '{' with_initializer_list '}'
    ;

state = state with { PerCompany.State.LineItems with { Rows = Rows.Remove(lineItem) } }

with_initializer_list will be scoped to LineItems

[mrange]

Seems you find a way to do nested withers that you like but I want to bring up a concept from FP world which is called lenses.

I have a somewhat rudimentary understanding of the math behind lenses but I think I got the concepts so I am taking a risk describing it here :).

Lenses allows focusing on properties in a nested structure and perform an "immutable update" of them.

A rudimentary lens for .NET could look like this

interface Lens<T, P>
{
   P Get(T t);
   T Set(T t, P p);
}

Often one define a few side of combinator functions for lenses but there's also usually some kind of code-generation to generate lenses for known types. As we are here inside a compiler it could be possible to create that on the fly.

// Vague proposal of lens construction syntax
Lens<MyClass, int> l = $MyClass.MyProperty.MyNestedProperty;
MyClass myUpdatedInstance = l.Set(myInstance, 2);

Lenses have applications outside just creation of objects such as if you want to have functions that operator on specific properties of a nested immutable structure. For example if I define a textbox that on updates should read and updates it's values from a nested property it could be like this

TextBox<T> CreateTextBox<T, P>(string label, Lens<T, P> lens);

Reading the previous issues a bit more closely I think this idea resembles the suggestion by @bradphelan somewhat.

[kofifus]

@mrange you can see some work toward lenses here https://stackoverflow.com/questions/68012124/record-lenses-expression-tree-for-with-expression