Plumbing buffer allocation responsibility through the runtime and up into the lower-level stream dialect. #20240

benvanik · 2025-03-13T23:13:47Z

Adds support through low-level stream (cmd phase) through to runtime for indicating buffer ownership responsibility. These APIs allow deallocation responsibility tracking at runtime by both hosting applications and compiled modules and include provisions for ownership transfer and indeterminant lifetimes (analysis failures/external resources/etc).

Passes that emit these ops will be added in the future.

(see commits for more information, this is going to be a merge)

MaheshRavishankar · 2025-03-13T23:40:42Z

Thanks. I am assuming this is just a WIP branch... Did you want me to start reviewing any of the commits here (I am wrapping up somethings before I start reading the code here, but I can prioritize this if you want to start landing the pieces)

benvanik · 2025-03-13T23:43:44Z

No rush - I figured I'd split out the more durable parts of the trashfire that was my experimentation :)

compiler/src/iree/compiler/Dialect/HAL/IR/HALOps.td

compiler/src/iree/compiler/Dialect/HAL/IR/HALAttrs.td

compiler/src/iree/compiler/Dialect/HAL/IR/HALOps.cpp

compiler/src/iree/compiler/Dialect/HAL/IR/HALOps.td

runtime/src/iree/hal/device.h

krzysz00 · 2025-03-20T22:34:53Z

Meta comment: can I get a bit more of a PR description so I have a sense of what the goal of this change is?

compiler/src/iree/compiler/Dialect/HAL/IR/HALOps.td

benvanik · 2025-03-20T22:48:48Z

I've been discussing it with Mahesh - I'm not sure why you were added to the reviewers (possibly your comment) - I'll add more description, but feel free to skip this.

These APIs allow deallocation responsibility tracking at runtime by both hosting applications and compiled modules.

This allows allocations to be routed back to their original placement instead of whatever the compiler thinks is appropriate. This will mostly be useful for imported resources that the compiler cannot associate an affinity with.

This allows the compiler or host applications to indicate they don't trust any other code to deallocate the buffer prior to all handles to it being released. This is intended for when lifetime analysis in the compiler fails but could also be used to avoid letting the compiler reuse application buffers that may have weird lifetimes.

These are used to indicate how many conceptual "owners" claim lifetime responsibility for a resource.

It's the job of the verifier to ensure correct programs.

This allows sources and sinks to indicate transfers of ownership. It can roughly be thought of as an std::move on a std::shared_ptr.

benvanik requested a review from MaheshRavishankar March 13, 2025 23:13

benvanik force-pushed the users/benvanik/arc-foundation branch from aa33d3d to bf8dfed Compare March 13, 2025 23:26

benvanik force-pushed the users/benvanik/arc-foundation branch from bf8dfed to f546750 Compare March 13, 2025 23:42