Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

miri native-call support: all previously exposed provenance is accessible to the callee #137802

Merged
merged 2 commits into from
Mar 7, 2025
Merged

miri native-call support: all previously exposed provenance is accessible to the callee #137802

merged 2 commits into from
Mar 7, 2025

Conversation

RalfJung
Copy link
Member

@RalfJung RalfJung commented Feb 28, 2025 •
edited
Loading

When Miri invokes a native C function, the memory C can access needs to be "prepared": to avoid false positives, we need to consider all that memory initialized, and we need to consider it to have arbitrary provenance. So far we did this for all pointers passed to C, but not for pointers that were exposed already before the native call. This PR adjusts the logic so that we now "prepare" all memory that has ever been exposed.

This fixes cases such as:

  • cast a pointer to integer, send that integer to C, and access the memory there (test_pass_ptr_as_int)
  • send a pointer to some memory to C, which stores it somewhere; then in Rust store another pointer in that memory, and access that via C (test_pass_ptr_via_previously_shared_mem)

r? @oli-obk

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Feb 28, 2025
@rustbot
Copy link
Collaborator

rustbot commented Feb 28, 2025

Some changes occurred to the CTFE / Miri interpreter

cc @rust-lang/miri

Some changes occurred to the CTFE machinery

cc @rust-lang/wg-const-eval

The Miri subtree was changed

cc @rust-lang/miri

@RalfJung RalfJung force-pushed the miri-native-call-exposed branch from 11496de to 88f988c Compare February 28, 2025 15:07
@oli-obk
Copy link
Contributor

oli-obk commented Mar 5, 2025

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Mar 5, 2025

📌 Commit 88f988c has been approved by oli-obk

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 5, 2025
jieyouxu added a commit to jieyouxu/rust that referenced this pull request Mar 5, 2025
@jieyouxu
Rollup merge of rust-lang#137802 - RalfJung:miri-native-call-exposed,…
b8855d8 
… r=oli-obk

miri native-call support: all previously exposed provenance is accessible to the callee

When Miri invokes a native C function, the memory C can access needs to be "prepared": to avoid false positives, we need to consider all that memory initialized, and we need to consider it to have arbitrary provenance. So far we did this for all pointers passed to C, but not for pointers that were exposed already before the native call. This PR adjusts the logic so that we now "prepare" all memory that has ever been exposed.

This fixes cases such as:
- cast a pointer to integer, send that integer to C, and access the memory there (`test_pass_ptr_as_int`)
- send a pointer to some memory to C, which stores it somewhere; then in Rust store another pointer in that memory, and access that via C (`test_pass_ptr_via_previously_shared_mem`)

r? `@oli-obk`
@jieyouxu jieyouxu mentioned this pull request Mar 5, 2025
Closed
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 5, 2025
@bors
Auto merge of rust-lang#138059 - jieyouxu:rollup-wxy3xpx, r=jieyouxu
ee7cd30 
Rollup of 23 pull requests

Successful merges:

 - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion)
 - rust-lang#137327 (Undeprecate env::home_dir)
 - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book)
 - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden))
 - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var)
 - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck)
 - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums)
 - rust-lang#137744 (Re-add `Clone`-derive on `Thir`)
 - rust-lang#137758 (fix usage of ty decl macro fragments in attributes)
 - rust-lang#137764 (Ensure that negative auto impls are always applicable)
 - rust-lang#137772 (Fix char count in `Display` for `ByteStr`)
 - rust-lang#137798 (ci: use ubuntu 24 on arm large runner)
 - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee)
 - rust-lang#137805 (adjust Layout debug printing to match the internal field name)
 - rust-lang#137808 (Do not require that unsafe fields lack drop glue)
 - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists)
 - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN)
 - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT)
 - rust-lang#137920 (interpret/provenance_map: consistently use range_is_empty)
 - rust-lang#138038 (Update `compiler-builtins` to 0.1.151)
 - rust-lang#138041 (bootstrap and compiletest: Use `size_of_val` from the prelude instead of imported)
 - rust-lang#138046 (trim channel value in `get_closest_merge_commit`)
 - rust-lang#138053 (Increase the max. custom try jobs requested to `20`)

r? `@ghost`
`@rustbot` modify labels: rollup
jieyouxu added a commit to jieyouxu/rust that referenced this pull request Mar 5, 2025
@jieyouxu
Rollup merge of rust-lang#137802 - RalfJung:miri-native-call-exposed,…
890fa8c 
… r=oli-obk

miri native-call support: all previously exposed provenance is accessible to the callee

When Miri invokes a native C function, the memory C can access needs to be "prepared": to avoid false positives, we need to consider all that memory initialized, and we need to consider it to have arbitrary provenance. So far we did this for all pointers passed to C, but not for pointers that were exposed already before the native call. This PR adjusts the logic so that we now "prepare" all memory that has ever been exposed.

This fixes cases such as:
- cast a pointer to integer, send that integer to C, and access the memory there (`test_pass_ptr_as_int`)
- send a pointer to some memory to C, which stores it somewhere; then in Rust store another pointer in that memory, and access that via C (`test_pass_ptr_via_previously_shared_mem`)

r? ``@oli-obk``
@jieyouxu jieyouxu mentioned this pull request Mar 5, 2025
Closed
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 5, 2025
@bors
Auto merge of rust-lang#138071 - jieyouxu:rollup-a19djgi, r=jieyouxu
06306c0 
Rollup of 29 pull requests

Successful merges:

 - rust-lang#135733 (Implement `&pin const self` and `&pin mut self` sugars)
 - rust-lang#137107 (Override default `Write` methods for cursor-like types)
 - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion)
 - rust-lang#137327 (Undeprecate env::home_dir)
 - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book)
 - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden))
 - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var)
 - rust-lang#137612 (Update bootstrap to edition 2024)
 - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck)
 - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums)
 - rust-lang#137685 (self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux)
 - rust-lang#137744 (Re-add `Clone`-derive on `Thir`)
 - rust-lang#137758 (fix usage of ty decl macro fragments in attributes)
 - rust-lang#137764 (Ensure that negative auto impls are always applicable)
 - rust-lang#137772 (Fix char count in `Display` for `ByteStr`)
 - rust-lang#137798 (ci: use ubuntu 24 on arm large runner)
 - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee)
 - rust-lang#137805 (adjust Layout debug printing to match the internal field name)
 - rust-lang#137808 (Do not require that unsafe fields lack drop glue)
 - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists)
 - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN)
 - rust-lang#137827 (Add timestamp to unstable feature usage metrics)
 - rust-lang#137832 (Fix crash in BufReader::peek())
 - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT)
 - rust-lang#137920 (interpret/provenance_map: consistently use range_is_empty)
 - rust-lang#138038 (Update `compiler-builtins` to 0.1.151)
 - rust-lang#138046 (trim channel value in `get_closest_merge_commit`)
 - rust-lang#138052 (strip `-Wlinker-messages` wrappers from `rust-lld` rmake test)
 - rust-lang#138053 (Increase the max. custom try jobs requested to `20`)

r? `@ghost`
`@rustbot` modify labels: rollup
compiler-errors added a commit to compiler-errors/rust that referenced this pull request Mar 6, 2025
@compiler-errors
Rollup merge of rust-lang#137802 - RalfJung:miri-native-call-exposed,…
623c32b 
… r=oli-obk

miri native-call support: all previously exposed provenance is accessible to the callee

When Miri invokes a native C function, the memory C can access needs to be "prepared": to avoid false positives, we need to consider all that memory initialized, and we need to consider it to have arbitrary provenance. So far we did this for all pointers passed to C, but not for pointers that were exposed already before the native call. This PR adjusts the logic so that we now "prepare" all memory that has ever been exposed.

This fixes cases such as:
- cast a pointer to integer, send that integer to C, and access the memory there (`test_pass_ptr_as_int`)
- send a pointer to some memory to C, which stores it somewhere; then in Rust store another pointer in that memory, and access that via C (`test_pass_ptr_via_previously_shared_mem`)

r? ```@oli-obk```
@compiler-errors compiler-errors mentioned this pull request Mar 6, 2025
Closed
Noratrieb added a commit to Noratrieb/rust that referenced this pull request Mar 6, 2025
@Noratrieb
Rollup merge of rust-lang#137802 - RalfJung:miri-native-call-exposed,…
f1aad67 
… r=oli-obk

miri native-call support: all previously exposed provenance is accessible to the callee

When Miri invokes a native C function, the memory C can access needs to be "prepared": to avoid false positives, we need to consider all that memory initialized, and we need to consider it to have arbitrary provenance. So far we did this for all pointers passed to C, but not for pointers that were exposed already before the native call. This PR adjusts the logic so that we now "prepare" all memory that has ever been exposed.

This fixes cases such as:
- cast a pointer to integer, send that integer to C, and access the memory there (`test_pass_ptr_as_int`)
- send a pointer to some memory to C, which stores it somewhere; then in Rust store another pointer in that memory, and access that via C (`test_pass_ptr_via_previously_shared_mem`)

r? ````@oli-obk````
@Noratrieb Noratrieb mentioned this pull request Mar 6, 2025
Closed
Noratrieb added a commit to Noratrieb/rust that referenced this pull request Mar 6, 2025
@Noratrieb
Rollup merge of rust-lang#138079 - compiler-errors:rollup-8cax8qw, r=…
69143c0 
…compiler-errors

Rollup of 20 pull requests

Successful merges:

 - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion)
 - rust-lang#137327 (Undeprecate env::home_dir)
 - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book)
 - rust-lang#137502 (Don't include global asm in `mir_keys`, fix error body synthesis)
 - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden))
 - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var)
 - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck)
 - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums)
 - rust-lang#137744 (Re-add `Clone`-derive on `Thir`)
 - rust-lang#137758 (fix usage of ty decl macro fragments in attributes)
 - rust-lang#137764 (Ensure that negative auto impls are always applicable)
 - rust-lang#137772 (Fix char count in `Display` for `ByteStr`)
 - rust-lang#137798 (ci: use ubuntu 24 on arm large runner)
 - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee)
 - rust-lang#137805 (adjust Layout debug printing to match the internal field name)
 - rust-lang#137808 (Do not require that unsafe fields lack drop glue)
 - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists)
 - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN)
 - rust-lang#137868 (Add minimal platform support documentation for powerpc-unknown-linux-gnuspe)
 - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2025
@bors
Auto merge of rust-lang#138079 - compiler-errors:rollup-8cax8qw, r=co…
e1e166f 
…mpiler-errors

Rollup of 20 pull requests

Successful merges:

 - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion)
 - rust-lang#137327 (Undeprecate env::home_dir)
 - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book)
 - rust-lang#137502 (Don't include global asm in `mir_keys`, fix error body synthesis)
 - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden))
 - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var)
 - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck)
 - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums)
 - rust-lang#137744 (Re-add `Clone`-derive on `Thir`)
 - rust-lang#137758 (fix usage of ty decl macro fragments in attributes)
 - rust-lang#137764 (Ensure that negative auto impls are always applicable)
 - rust-lang#137772 (Fix char count in `Display` for `ByteStr`)
 - rust-lang#137798 (ci: use ubuntu 24 on arm large runner)
 - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee)
 - rust-lang#137805 (adjust Layout debug printing to match the internal field name)
 - rust-lang#137808 (Do not require that unsafe fields lack drop glue)
 - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists)
 - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN)
 - rust-lang#137868 (Add minimal platform support documentation for powerpc-unknown-linux-gnuspe)
 - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2025
@bors
Auto merge of rust-lang#138079 - compiler-errors:rollup-8cax8qw, r=co…
60c2a48 
…mpiler-errors

Rollup of 20 pull requests

Successful merges:

 - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion)
 - rust-lang#137327 (Undeprecate env::home_dir)
 - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book)
 - rust-lang#137502 (Don't include global asm in `mir_keys`, fix error body synthesis)
 - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden))
 - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var)
 - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck)
 - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums)
 - rust-lang#137744 (Re-add `Clone`-derive on `Thir`)
 - rust-lang#137758 (fix usage of ty decl macro fragments in attributes)
 - rust-lang#137764 (Ensure that negative auto impls are always applicable)
 - rust-lang#137772 (Fix char count in `Display` for `ByteStr`)
 - rust-lang#137798 (ci: use ubuntu 24 on arm large runner)
 - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee)
 - rust-lang#137805 (adjust Layout debug printing to match the internal field name)
 - rust-lang#137808 (Do not require that unsafe fields lack drop glue)
 - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists)
 - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN)
 - rust-lang#137868 (Add minimal platform support documentation for powerpc-unknown-linux-gnuspe)
 - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT)

r? `@ghost`
`@rustbot` modify labels: rollup
@compiler-errors compiler-errors mentioned this pull request Mar 6, 2025
Merged
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2025
@bors
Auto merge of rust-lang#138114 - compiler-errors:rollup-7xr4b69, r=co…
0e18167 
…mpiler-errors

Rollup of 25 pull requests

Successful merges:

 - rust-lang#135733 (Implement `&pin const self` and `&pin mut self` sugars)
 - rust-lang#135895 (Document workings of successors more clearly)
 - rust-lang#136922 (Pattern types: Avoid having to handle an Option for range ends in the type system or the HIR)
 - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion)
 - rust-lang#137327 (Undeprecate env::home_dir)
 - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book)
 - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden))
 - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var)
 - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck)
 - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums)
 - rust-lang#137744 (Re-add `Clone`-derive on `Thir`)
 - rust-lang#137758 (fix usage of ty decl macro fragments in attributes)
 - rust-lang#137764 (Ensure that negative auto impls are always applicable)
 - rust-lang#137772 (Fix char count in `Display` for `ByteStr`)
 - rust-lang#137798 (ci: use ubuntu 24 on arm large runner)
 - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee)
 - rust-lang#137805 (adjust Layout debug printing to match the internal field name)
 - rust-lang#137808 (Do not require that unsafe fields lack drop glue)
 - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists)
 - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN)
 - rust-lang#137834 (rustc_fluent_macro: use CARGO_CRATE_NAME instead of CARGO_PKG_NAME)
 - rust-lang#137868 (Add minimal platform support documentation for powerpc-unknown-linux-gnuspe)
 - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT)
 - rust-lang#137920 (interpret/provenance_map: consistently use range_is_empty)
 - rust-lang#138038 (Update `compiler-builtins` to 0.1.151)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2025
@bors
Auto merge of rust-lang#138114 - compiler-errors:rollup-7xr4b69, r=co…
98a4878 
…mpiler-errors

Rollup of 25 pull requests

Successful merges:

 - rust-lang#135733 (Implement `&pin const self` and `&pin mut self` sugars)
 - rust-lang#135895 (Document workings of successors more clearly)
 - rust-lang#136922 (Pattern types: Avoid having to handle an Option for range ends in the type system or the HIR)
 - rust-lang#137303 (Remove `MaybeForgetReturn` suggestion)
 - rust-lang#137327 (Undeprecate env::home_dir)
 - rust-lang#137358 (Match Ergonomics 2024: add context and examples to the unstable book)
 - rust-lang#137534 ([rustdoc] hide item that is not marked as doc(inline) and whose src is doc(hidden))
 - rust-lang#137565 (Try to point of macro expansion from resolver and method errors if it involves macro var)
 - rust-lang#137637 (Check dyn flavor before registering upcast goal on wide pointer cast in MIR typeck)
 - rust-lang#137643 (Add DWARF test case for non-C-like `repr128` enums)
 - rust-lang#137744 (Re-add `Clone`-derive on `Thir`)
 - rust-lang#137758 (fix usage of ty decl macro fragments in attributes)
 - rust-lang#137764 (Ensure that negative auto impls are always applicable)
 - rust-lang#137772 (Fix char count in `Display` for `ByteStr`)
 - rust-lang#137798 (ci: use ubuntu 24 on arm large runner)
 - rust-lang#137802 (miri native-call support: all previously exposed provenance is accessible to the callee)
 - rust-lang#137805 (adjust Layout debug printing to match the internal field name)
 - rust-lang#137808 (Do not require that unsafe fields lack drop glue)
 - rust-lang#137820 (Clarify why InhabitedPredicate::instantiate_opt exists)
 - rust-lang#137825 (Provide more context on resolve error caused from incorrect RTN)
 - rust-lang#137834 (rustc_fluent_macro: use CARGO_CRATE_NAME instead of CARGO_PKG_NAME)
 - rust-lang#137868 (Add minimal platform support documentation for powerpc-unknown-linux-gnuspe)
 - rust-lang#137910 (Improve error message for `AsyncFn` trait failure for RPIT)
 - rust-lang#137920 (interpret/provenance_map: consistently use range_is_empty)
 - rust-lang#138038 (Update `compiler-builtins` to 0.1.151)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit b7b2179 into rust-lang:master Mar 7, 2025
6 checks passed
@rustbot rustbot added this to the 1.87.0 milestone Mar 7, 2025
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 7, 2025
@rust-timer
Unrolled build for rust-lang#137802
b1a1194 
Rollup merge of rust-lang#137802 - RalfJung:miri-native-call-exposed, r=oli-obk

miri native-call support: all previously exposed provenance is accessible to the callee

When Miri invokes a native C function, the memory C can access needs to be "prepared": to avoid false positives, we need to consider all that memory initialized, and we need to consider it to have arbitrary provenance. So far we did this for all pointers passed to C, but not for pointers that were exposed already before the native call. This PR adjusts the logic so that we now "prepare" all memory that has ever been exposed.

This fixes cases such as:
- cast a pointer to integer, send that integer to C, and access the memory there (`test_pass_ptr_as_int`)
- send a pointer to some memory to C, which stores it somewhere; then in Rust store another pointer in that memory, and access that via C (`test_pass_ptr_via_previously_shared_mem`)

r? `````@oli-obk`````
@RalfJung RalfJung deleted the miri-native-call-exposed branch March 10, 2025 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@oli-obk oli-obk

Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
1.87.0
Development

Successfully merging this pull request may close these issues.
4 participants
@RalfJung @rustbot @oli-obk @bors