abstract types with fields

[JeffBezanson]

This would look something like

abstract Foo with
    x::Int
    y::String
end

which will cause every subtype of Foo to begin with those fields.

Some parts of the language internals already anticipate this; it's a matter of hooking up the syntax and filling in a few missing pieces.

[IainNZ]

👍 to this

[kmsquire]

+1 (!)

I'm wondering if the with keyword is useful, necessary, and/or deliberate?

[ivarne]

If he didn't have a with keyword, every declaration of abstract will need an end. Currently abstract is a oneliner, but type and immutable is mulitiline until a end marker.

[kmsquire]

Right, thanks @ivarne.

[aviks]

+1 this will be very useful!

[johnmyleswhite]

+1000

[StefanKarpinski]

I'd actually be ok with changing abstract to always require an end, although that would make this a breaking change, which it currently isn't. The with thing feels pretty clunky to me and our current abstract declarations have always felt a little jarringly open-ended to me.

[johnmyleswhite]

I kind of agree with Stefan. Making the visual appearance of abstract more like that of type and immutable seems like a gain to me.

[nalimilan]

Yeah, FWIW I was going to say the same.

[IainNZ]

Fourth-ed

[StefanKarpinski]

The big problem with making a syntactic change like that is it's going to become a watershed for all the code out there that declares abstract types, splitting that code into before and after versions. Since half of our community likes to live on the edge while the other half likes to use 0.2 (making up numbers here, but half-and-half seems reasonable), that's kind of a big problem. If there was some way we could deprecate the open-ended abstract type declaration, that would avoid the issue.

[johnmyleswhite]

Now that 0.2 is out, I actually think we should tell people not to use master for work that's not focused on the direct development of Julia itself. I intend to only work from 0.2 until the 0.3 release while developing packages.

[nalimilan]

Maybe you can hack a temporary thing to end an abstract block if the next line does not start with a field declaration? Backporting this to 0.2.x would allow moving progressively, then you would introduce a deprecation warning, and make it an error with 0.3.

[StefanKarpinski]

I think that's very reasonable, although it does cut down on the number of people testing out 0.3, which is unfortunate, but probably unavoidable.

[StefanKarpinski]

@nalimilan, yes, I was thinking something along those lines, but it does feel kind of awful.

[ivarne]

As a transitioning solution we might update 0.2.1 to allow a end on the same line after abstract. Then in 0.3 we might issue a warning if it is missing and in 0.4 we can require it. That makes this a rather lengthy process though.

Why don't we enable inheriting from type and immutable instead? It keeps the abstract keyword reserved for grouping types. It will also be cleaner if a immutable can't inherit from a type. Will it cause trouble somewhere if we have a abstract and a concrete type with the same name?

[StefanKarpinski]

I like the first approach, but it is very, very slow, unfortunately. We definitely cannot allow inheriting from type or immutable. The fact that concrete types are final is crucial. Otherwise when you write Array{Complex{Float64}} you can't store them inline because someone could subtype Complex and add more fields, which means that the things in the array might be bigger than 16 bytes. Game over for all numerical work.

[ivarne]

That is a good point. It will be too hard to know if Complex should be interpreted as an abstract or concrete type when it is used as a type parameter.

What about this?

abstract type Foo
    x::Int
    y::String
end

That does not introduce a new keyword, and it does not make old code break.

[JeffBezanson]

Very nice idea. So far that seems perfect.

[JeffBezanson]

That also potentially allows abstract immutable, which could require all subtypes to be immutable.

[IainNZ]

very cool

[StefanKarpinski]

Yes, I like that idea. We can also make abstract Foo end allowed – optionally for now – and eventually require the end and make allow leaving out the type like we do with immutable. Or maybe we just leave it the way it is.

[WestleyArgentum]

I'm unreasonably excited about this :)

[StefanKarpinski]

New language features are like Christmas.

[andrewcooke]

more support for this from https://groups.google.com/forum/#!topic/julia-users/6ohvsWpX6u0

(you're doing an amazing job here - i can't believe how far you've got and how good this is...)

[JeffBezanson]

There is a small question of how to handle constructors with this feature. The obvious thing is for it to behave as if you simply copy & pasted the parent type's fields into the new subtype declaration. However, this creates extra coupling, since changing the parent type can require changes to all subtype code:

abstract type Parent
    x
    y
end

type Child <: Parent
    z

    Child(q) = new(x, y, z)
end

The Child constructor has to know about the parent fields. Bug or feature?

[StefanKarpinski]

It's very non-local, which I don't care for. One thought is that the subtype would have to repeat the declaration and match it. I know that's not very DRY but it's an immediate, easy-to-diagnose error when it happens, and it means that the child declaration is completely self-contained. The point of having fields in the abstract type declaration is to allow the compiler to know that anything of that type will have those fields and know what offset they're at so that you can emit efficient generic code for accessing those fields for all things of that type without needing to know the precise subtype. I don't think the feature is really about avoiding typing fields.

[tknopp]

Isn't this a natural coupling which you will always have if you change a parent type?

Maybe it would be cleaner to have: Child(q) = new(Parent(x,y),z) i.e. the parent has to be the first value for new (and outer constructors as well).

[JeffBezanson]

It's an interesting point that all the value is in making sure the fields are there. Avoiding typing them is much less important.

[stevengj]

I guess with #20418 we could now consider abstract struct ... end

[wsshin]

I really hope this feature to be implemented soon. I am porting a C++ code, where a superclass with lots of member fields is inherited by hundreds of subclasses. I think the standard way of implementing this kind of situation in Julia is to copy all the fields of the superclass to subtypes as

abstract type AbstractMyType end

struct MyType1 <: AbstractMyType
    x1  # unique to MyType1
    a  # all fields from this line and below are common to AbstractMyType
    b
    c
    ...
end

struct MyType2 <: AbstractMyType
    x2  # unique to MyType2
    a  # all fields from this line and below are common to AbstractMyType
    b
    c
    ...
end

Copying many member fields to hundreds of subtype definitions is tedious. (I think #19383 was an attempt to overcome this problem.) Also, if I want to add additional member fields to the supertype, I need to add them to all the subtypes. The proposed feature will eliminate these problems.

In the mean time, my workaround was to define a type MyCore containing all the shared member fields and make it a common member field of subtypes:

struct MyCore
    a
    b
    c
    ...
end

abstract type AbstractMyType end

struct MyType1 <: AbstractMyType
    x1  # unique to MyType1
    core::MyCore  # common to AbstractMyType
end

struct MyType2 <: AbstractMyType
    x2  # unique to MyType2
    core::MyCore  # common to AbstractMyType
end

This approach solves the aforementioned problems, but it implicitly assumes that all the subtypes have core as a member. For example, consider the following function that handles all the subtypes of AbstractMyType:

myfun(m::AbstractMyType) = m.core.a + m.core.b + m.core.c

Here, I had to require that any subtype of AbstractMyType has core as a member field, but there is no mechanism to force this requirement to subtypes.

Recently I came up with a nice method to solve this issue using a parametric type. In this method, I replace AbstractMyType with a parametric type with a parameter that describes specs distinguishing individual subtypes, and define MyType1 and MyType2 as type aliases of AbstractMyType with type parameters corresponding to particular specs:

abstract type AbstractSpecs end

struct AbstractMyType{S<:AbstractSpecs}
    specs::S
    a  # all fields from this line and below are common to AbstractMyType
    b
    c
    ...
end

struct Specs1 <: AbstractSpecs
    x1  # unique to MyType1
end

struct Specs2 <: AbstractSpecs
    x2  # unique to MyType2
end

const MyType1 = AbstractMyType{Specs1}
const MyType2 = AbstractMyType{Specs2}

Now, you can define functions for AbstractMyType (which is not really abstract by the way) without referring to implicit member fields like core in the previous example:

myfun(m::AbstractMyType) = m.a + m.b + m.c

Also, unlike the method proposed in #19383, the <: relationship holds between the subtypes and supertype:

julia> MyType1 <: AbstractMyType
true

julia> MyType2 <: AbstractMyType
true

I think this is a practical method to use in my current situation of porting the C++ code, but I haven't seen it widely used in the Julia community. (Maybe I am ignorant.) Hope this helps people struggling with similar situations until the proposed feature is implemented.

[stevengj]

Complex parameterized types in Julia are far more common than abstract types with "hundreds" (or even dozens) of subtypes, so I would say in that sense that this is a standard technique in Julia. That being said, if your C++ code consists of hundreds of subclasses, my first reaction is that you should perhaps completely re-think the code organization when porting it to Julia.

[wsshin]

@stevengj, I also use parametric types extensively in all of my projects, but their particular usage, combined with type aliasing, to mimic the behavior of the inheritance in OOP languages as shown in the above example didn't occur to me until recently. When I needed inheritance, I always used an abstract type and defined its common member fields in all the subtypes (and manually defining the common member fields in all the subtypes will become unnecessary once abstract types with fields proposed in this thread are implemented).

The C++ code I am porting right now has hundreds of subclasses of one superclass, corresponding to hundreds of devices of an abstract device. Because the code supports hundreds of different devices, I think such a large number of subclasses are unavoidable. But I agree that the number of subclasses is overwhelming, and it will be nice to reduce it. I will look into the possibility of reducing the number of subclasses; maybe they can be grouped into a few subcategories. Thanks for the advice!

[StefanKarpinski]

One of the main issues I had with this feature was that when you add a field to an abstract super type you then cannot remove it. Which is especially unfortunate since #24960 because it's entirely possible that you might write generic code for an abstract type that accesses a field that isn't even there for some specific implementation. So now you find yourself in a situation where you want a way to delete a field in a subtype that was inherited from an abstract supertype. So now we need two language features. It seems far simpler to just put the .core field in all the implementations of a subtype — except, of course, for the ones that don't end up needing it.

[RaulDurand]

I find this explanation reasonable. However, I think that this feature would be very useful in some applications.

[zot]

One of the main issues I had with this feature was that when you add a field to an abstract super type you then cannot remove it. Which is especially unfortunate since #24960 because it's entirely possible that you might write generic code for an abstract type that accesses a field that isn't even there for some specific implementation. So now you find yourself in a situation where you want a way to delete a field in a subtype that was inherited from an abstract supertype. So now we need two language features. It seems far simpler to just put the .core field in all the implementations of a subtype — except, of course, for the ones that don't end up needing it.

Do structural interfaces help with this by potentially removing the need for inheritance?

[henriquebecker91]

I really hope this feature never gets implemented. Removing fields from abstract structures was one of the best design decisions of Julia.

[wsshin]

@henriquebecker91, could you elaborate? I'm curious about the advantages of removing fields from abstract structures.

[martinholters]

To throw a new thought into the discussion: Inheriting fields is subclassing, while Julia focuses on subtyping. In C++, subclassing and subtyping are mashed together, so the distinction may be non-obvious to many. But it might be interesting to consider what would happen if Julia gained subclassing as a distinct concept...
(To make the distinction obvious: every a square is a rectangle, so Square <: Rectangle would make sense, but subclassing would work better the other way round: Rectangle could inherit the single property width from Square and extend with a new property height.)

[opiateblush]

I find it very curious that such a feature is still up for debate after almost a decade. The more I read through the discussions about inheritance in Julia the more I feel back at school where people were complaining about their favorite music artist going "mainstream". As if it was important to preserve a unique feature just because of its uniqueness. I don't see any significant disadvantages by allowing inheritance of structure as it proposed in this issue. Here is why I think this is a good thing to have in Julia:

• people having a background in common OOP languages wouldn't have such a hard time adapting their "thinking patterns" when working with Julia (Well, that opinion is obviously based on subjective experiences, but my colleagues see it the same way)
• people tend to impose structural contracts (must have fields of certain names) on abstract types, which would make a lot more sense if that contract was enforced by design
• there are dozens of packages out there that address the missing inheritance problem (and its boilerplate symptom) in some way. Isn't that a sign that it might be a good idea to support that feature by default?

To be honest, I don't quite get the gist of abstract types with their current implementation. In the docs it says it's about inheritance of behavior. But the way I see it, there is nothing to inherit from. If you define an abstract type, the interfaces for it must be defined in the docs and cannot be enforced. Structure is irrelevant at this point, fine. So, now you can dispatch on that abstract type assuming that the actual implementation supports the interface defined in the docs. Great, but the actual implementation doesn't have to implement anything, it just throws an exception at runtime if it doesn't. Furthermore, if it was only about behavior, what about a different type that implements the interface but is not a concrete subtype of the abstract type? Well, if I wanted my function to work with any type that supports a specific behavior I couldn't dispatch on any type. What are abstract types there for then?

Don't get me wrong. I don't want to offend anyone and I might be a bit emotional right now, but I still find my points valid. I also find Julia a great language with huge potential to eliminate strange constructs of Python and C/C++ code (and similar) just to benefit from both their advantages. With Julia I can switch between the flexibility of Python, the speed C or anything in between just as I go. It's awesome!

[zot]

To be honest, I don't quite get the gist of abstract types with their current implementation. In the docs it says it's about inheritance of behavior. But the way I see it, there is nothing to inherit from. If you define an abstract type, the interfaces for it must be defined in the docs and cannot be enforced. Structure is irrelevant at this point, fine. So, now you can dispatch on that abstract type assuming that the actual implementation supports the interface defined in the docs. Great, but the actual implementation doesn't have to implement anything, it just throws an exception at runtime if it doesn't. Furthermore, if it was only about behavior, what about a different type that implements the interface but is not a concrete subtype of the abstract type? Well, if I wanted my function to work with any type that supports a specific behavior I couldn't dispatch on any type. What are abstract types there for then?

If you need to handle access in a general way, you can define getter methods (ala OOP) to access the parts you need and use them in the more general methods, like:

showquadrangle(q::Quadrangle) = println("$(nameof(typeof(q)))[$(width(q)) x $(length(q))]")
width(s::Square) = s.size
length(s::Square) = s.size
width(r::Rectangle) = r.width
length(r::Rectangle) = r.length

This is reminiscent of Smalltalk where you can't directly access instance variables in another object without using reflection, you have to use accessor methods for that.

[Deduction42]

I'm just getting bit by this problem too. I have very generic types of equipment that require a bare minimum set of fields for functions to work, and I'm specializing them so that they always have the same fields as the parent. If I was able to define a set of fields for an abstract type that must be available to all sub-types, it would make my code adhere better to DRY.

I'm aware from #24960 that getproperty can be used to overload property access, but if someone Author A defined a required field for an AbstractType but user B wants to subclass it in a way for that field to not be there, user B could simply

1. Build their own constructor that fills that property with a default value
2. Override that property access with the desired behaviour in "getproperty" higher up in the "if statement" block

Even if this hack were undesirable, I don't think it would be too hard to have a "@deletefield" macro to delete "inherited" fields from abstract types. I already wrote a macro that includes fields from another type and can even omit a set of fields if so desired. Field removal from an abstract parent should be quite rare, and the only need I'd see from it would be if there was a "lazy" way to fill that field with an inferred value on the fly via a getproperty branch.

From this, fields on Abstract Types should therefore be a sort of interface spec. If there is a field on an Abstract Type, then getproperty should be defined for it. Period. That way, any function that uses "getproperty" can be guaranteed that those properties exist. That's just how interfaces work. It's certainly less work and less error-prone than copy-pasting the same fields over and over again or re-implementing a complete set of constructors for every subtype.

If it turns out that abstract types with fields is a bad idea, would it be acceptable to have a macro that expands getproperty and propertynames that include the properties of a nested object?

[jkosata]

While this is under debate (maybe another decade :)) ), I hacked this together to bypass the issue, DefaultFields.jl
It lets you define supertype along with a macro, calling which then generates new types with the required fields added:

@with_fields abs_type a::Int b::Float64
@abs_type struct mystruct
            c::String
        end

julia> fieldnames(mystruct)
(:c, :b, :a)
julia> fieldtypes(mystruct)
(String, Float64, Int64)

Hope it helps!