From dd88dbfee54b8a34c6e3dcb28003e020478439b1 Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Fri, 24 Dec 2021 07:00:21 +1100 Subject: [PATCH 1/4] Move `x.py` intro section before first use, and shorten it. --- src/getting-started.md | 33 ++++++++++----------------------- 1 file changed, 10 insertions(+), 23 deletions(-) diff --git a/src/getting-started.md b/src/getting-started.md index a2f507e76..880e0ea74 100644 --- a/src/getting-started.md +++ b/src/getting-started.md @@ -62,7 +62,7 @@ advice.**][suggested] ### System Requirements [**See this chapter for detailed software requirements.**](./building/prerequisites.md) -Most notably, you will need Python 2 or 3 to run `x.py`. +Most notably, you will need Python 2 or 3. There are no hard hardware requirements, but building the compiler is computationally expensive, so a beefier machine will help, and I wouldn't @@ -117,6 +117,15 @@ git submodule update --init --recursive git submodule update ``` +### `x.py` Intro + +`rustc` is a [bootstrapping] compiler, which makes it more complex than a +typical Rust program. As a result, you cannot use Cargo to build it. Instead +you must use the special tool `x.py`. It is used for the things Cargo is +normally used for: building, testing, creating releases, formatting, etc. + +[bootstrapping]: ./building/bootstrapping.md + ### Configuring the Compiler The compiler has a configuration file which contains a ton of settings. We will @@ -166,28 +175,6 @@ download-ci-llvm = true ["Updating LLVM" section]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html?highlight=download-ci-llvm#feature-updates -### `x.py` Intro - -`rustc` is a _bootstrapping_ compiler, which means that it is written in Rust -and thus needs to be compiled by itself. So where do you -get the original compiler from? We use the current beta compiler -to build a new compiler. Then, we use that compiler to build itself. Thus, -`rustc` has a 2-stage build. You can read more about bootstrapping -[here][boot], but you don't need to know much more to contribute. - -[boot]: ./building/bootstrapping.md - -We have a special tool `x.py` that drives this process. It is used for -building the compiler, the standard libraries, and `rustdoc`. It is also used -for driving CI and building the final release artifacts. - -Unfortunately, a proper 2-stage build takes a long time depending on your -hardware, but it is the only correct way to build everything (e.g. it's what -the CI and release processes use). **However, in most cases, you can get by -without a full 2-stage build**. In the following section, we give instructions -for how to do "the correct thing", but then we also give various tips to speed -things up. - ### Building and Testing `rustc` Here is a summary of the different commands for reference, but you probably From bdbbc3844daa5a3d688993164fea6ae1ae38ab37 Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Fri, 24 Dec 2021 07:45:49 +1100 Subject: [PATCH 2/4] Improve `x.py setup` docs. In "Getting Started", strip it back to the bare minimum. Some of this is moved into the later section. In the later section, add notable details like config.toml.example how and `profile` works. Also make the config.toml example more concise. --- src/building/how-to-build-and-run.md | 38 +++++++++++++------------ src/getting-started.md | 42 +++------------------------- 2 files changed, 24 insertions(+), 56 deletions(-) diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index ae3aabf12..09e2a8bb7 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -20,34 +20,36 @@ cd rust ## Create a `config.toml` -To start, run `./x.py setup`. This will create a `config.toml` with reasonable defaults. +To start, run `./x.py setup`. This will do some initialization and create a +`config.toml` for you with reasonable defaults. These defaults are specified +indirectly via the `profile` setting, which points to one of the TOML files in +`src/bootstrap/defaults.` -You may also want to change some of the following settings (and possibly others, such as +Alternatively, you can write `config.toml` by hand. See `config.toml.example` +for all the available settings and explanations of them. The following settings +are of particular interest, and `config.toml.example` has full explanations. + +You may want to change some of the following settings (and possibly others, such as `llvm.ccache`): ```toml [llvm] # Whether to use Rust CI built LLVM instead of locally building it. -download-ci-llvm = true - -# Indicates whether the LLVM assertions are enabled or not -assertions = true +download-ci-llvm = true # Download a pre-built LLVM? +assertions = true # LLVM assertions on? +ccache = "/path/to/ccache" # Use ccache when building LLVM? [rust] -# Whether or not to leave debug! and trace! calls in the rust binary. -# Overrides the `debug-assertions` option, if defined. -# -# Defaults to rust.debug-assertions value -# -# If you see a message from `tracing` saying -# `max_level_info` is enabled and means logging won't be shown, -# set this value to `true`. -debug-logging = true - -# Whether to always use incremental compilation when building rustc -incremental = true +debug-logging = true # Leave debug! and trace! calls in rustc? +incremental = true # Build rustc with incremental compilation? ``` +If you set `download-ci-llvm = true`, in some circumstances, such as when +updating the version of LLVM used by `rustc`, you may want to temporarily +disable this feature. See the ["Updating LLVM" section] for more. + +["Updating LLVM" section]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html?highlight=download-ci-llvm#feature-updates + If you have already built `rustc` and you change settings related to LLVM, then you may have to execute `rm -rf build` for subsequent configuration changes to take effect. Note that `./x.py clean` will not cause a rebuild of LLVM. diff --git a/src/getting-started.md b/src/getting-started.md index 880e0ea74..84b48b760 100644 --- a/src/getting-started.md +++ b/src/getting-started.md @@ -128,52 +128,18 @@ normally used for: building, testing, creating releases, formatting, etc. ### Configuring the Compiler -The compiler has a configuration file which contains a ton of settings. We will -provide some recommendations here that should work for most, but [check out -this chapter for more info][config]. - -[config]: ./building/how-to-build-and-run.md#create-a-configtoml - In the top level of the repo: ```sh $ ./x.py setup ``` -This will walk you through an interactive setup for `x.py` that looks like this: +This will do some initialization and walk you through an interactive setup to +create `config.toml`, the primary configuration file. -``` -$ ./x.py setup -Welcome to the Rust project! What do you want to do with x.py? -a) Contribute to the standard library -b) Contribute to the compiler -c) Contribute to the compiler, and also modify LLVM or codegen -d) Install Rust from source -Please choose one (a/b/c/d): a -`x.py` will now use the configuration at /home/joshua/rustc2/src/bootstrap/defaults/config.toml.library -To get started, try one of the following commands: -- `x.py check` -- `x.py build` -- `x.py test library/std` -- `x.py doc` -For more suggestions, see https://rustc-dev-guide.rust-lang.org/building/suggested.html -``` +See [this chapter][config] for more info about configuration. -Note that by default, `./x.py setup` will use CI-built LLVM if available for your -platform so that you don't need to build LLVM in addition to building the -compiler. In some circumstances, such as when updating the version of LLVM used -by `rustc`, you may want to temporarily disable this feature. See the ["Updating -LLVM" section] for more. - -If you want to download LLVM from CI without running `./x.py setup`, you can set -the `download-ci-llvm` option to `true` in your `config.toml`: - -```toml -[llvm] -download-ci-llvm = true -``` - -["Updating LLVM" section]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html?highlight=download-ci-llvm#feature-updates +[config]: ./building/how-to-build-and-run.md#create-a-configtoml ### Building and Testing `rustc` From 513d4031ceb4f39362cafbad8466528b1b076ccd Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Fri, 24 Dec 2021 08:00:22 +1100 Subject: [PATCH 3/4] Move details about the repository. Less detail in "Getting Started", more in the later sections. --- src/building/how-to-build-and-run.md | 18 ++++++++++++++++++ src/getting-started.md | 27 +-------------------------- 2 files changed, 19 insertions(+), 26 deletions(-) diff --git a/src/building/how-to-build-and-run.md b/src/building/how-to-build-and-run.md index 09e2a8bb7..4fe1da6d0 100644 --- a/src/building/how-to-build-and-run.md +++ b/src/building/how-to-build-and-run.md @@ -11,6 +11,12 @@ see [the next page](./prerequisites.md). ## Get the source code +The main repository is [`rust-lang/rust`][repo]. This contains the compiler, +the standard library (including `core`, `alloc`, `test`, `proc_macro`, etc), +and a bunch of tools (e.g. `rustdoc`, the bootstrapping infrastructure, etc). + +[repo]: https://github.com/rust-lang/rust + The very first step to work on `rustc` is to clone the repository: ```bash @@ -18,6 +24,18 @@ git clone https://github.com/rust-lang/rust.git cd rust ``` +There are also submodules for things like LLVM, `clippy`, `miri`, etc. The +build tool will automatically clone and sync these for you. But if you want to, +you can do the following: + +```sh +# first time +git submodule update --init --recursive + +# subsequent times (to pull new commits) +git submodule update +``` + ## Create a `config.toml` To start, run `./x.py setup`. This will do some initialization and create a diff --git a/src/getting-started.md b/src/getting-started.md index 84b48b760..4b3fa0cba 100644 --- a/src/getting-started.md +++ b/src/getting-started.md @@ -44,21 +44,6 @@ just create noise, so we ask that you be mindful of the fact that the ## Cloning and Building -The main repository is [`rust-lang/rust`][repo]. This contains the compiler, -the standard library (including `core`, `alloc`, `test`, `proc_macro`, etc), -and a bunch of tools (e.g. `rustdoc`, the bootstrapping infrastructure, etc). - -[repo]: https://github.com/rust-lang/rust - -There are also a bunch of submodules for things like LLVM, `clippy`, `miri`, -etc. You don't need to clone these immediately, but the build tool will -automatically clone and sync them (more on this later). - -[**Take a look at the "Suggested Workflows" chapter for some helpful -advice.**][suggested] - -[suggested]: ./building/suggested.md - ### System Requirements [**See this chapter for detailed software requirements.**](./building/prerequisites.md) @@ -104,17 +89,7 @@ You can just do a normal git clone: ```sh git clone https://github.com/rust-lang/rust.git -``` - -You don't need to clone the submodules at this time. But if you want to, you -can do the following: - -```sh -# first time -git submodule update --init --recursive - -# subsequent times (to pull new commits) -git submodule update +cd rust ``` ### `x.py` Intro From 08c0d2ea448af614a3c821bcc63b34bfd37e7080 Mon Sep 17 00:00:00 2001 From: Nicholas Nethercote Date: Fri, 24 Dec 2021 08:20:13 +1100 Subject: [PATCH 4/4] Move details about the prereqs. Less detail in "Getting Started", more in the later sections. --- src/building/prerequisites.md | 35 +++++++++++++++++++---- src/getting-started.md | 52 +++++++++++------------------------ 2 files changed, 45 insertions(+), 42 deletions(-) diff --git a/src/building/prerequisites.md b/src/building/prerequisites.md index be5d96ad9..ec2e8a860 100644 --- a/src/building/prerequisites.md +++ b/src/building/prerequisites.md @@ -43,17 +43,40 @@ see [the `rust-lang/rust` README](https://github.com/rust-lang/rust#building-on- ## Hardware -These are not so much requirements as _recommendations_: - -* ~15GB of free disk space (~25GB or more if doing incremental builds). -* \>= 8GB RAM -* \>= 2 cores -* Internet access +You will need an internet connection to build. The bootstrapping process +involves updating git submodules and downloading a beta compiler. It doesn't +need to be super fast, but that can help. + +There are no strict hardware requirements, but building the compiler is +computationally expensive, so a beefier machine will help, and I wouldn't +recommend trying to build on a Raspberry Pi! We recommend the following. +* 30GB+ of free disk space. Otherwise, you will have to keep + clearing incremental caches. More space is better, the compiler is a bit of a + hog; it's a problem we are aware of. +* 8GB+ RAM +* 2+ cores. Having more cores really helps. 10 or 20 or more is not too many! Beefier machines will lead to much faster builds. If your machine is not very powerful, a common strategy is to only use `./x.py check` on your local machine and let the CI build test your changes when you push to a PR branch. +Building the compiler takes more than half an hour on my moderately powerful +laptop. The first time you build the compiler, LLVM will also be built unless +you use CI-built LLVM ([see here][config]). + +Like `cargo`, the build system will use as many cores as possible. Sometimes +this can cause you to run low on memory. You can use `-j` to adjust the number +concurrent jobs. If a full build takes more than ~45 minutes to an hour, you +are probably spending most of the time swapping memory in and out; try using +`-j1`. + +If you don't have too much free disk space, you may want to turn off +incremental compilation ([see here][config]). This will make compilation take +longer (especially after a rebase), but will save a ton of space from the +incremental caches. + +[config]: ./how-to-build-and-run.md#create-a-configtoml + ## `rustc` and toolchain installation Follow the installation given in the [Rust book][install] to install a working diff --git a/src/getting-started.md b/src/getting-started.md index 4b3fa0cba..40e68cf0e 100644 --- a/src/getting-started.md +++ b/src/getting-started.md @@ -46,42 +46,22 @@ just create noise, so we ask that you be mindful of the fact that the ### System Requirements -[**See this chapter for detailed software requirements.**](./building/prerequisites.md) -Most notably, you will need Python 2 or 3. - -There are no hard hardware requirements, but building the compiler is -computationally expensive, so a beefier machine will help, and I wouldn't -recommend trying to build on a Raspberry Pi :P - -- Recommended >=30GB of free disk space; otherwise, you will have to keep - clearing incremental caches. More space is better, the compiler is a bit of a - hog; it's a problem we are aware of. -- Recommended >=8GB RAM. -- Recommended >=2 cores; having more cores really helps. -- You will need an internet connection to build; the bootstrapping process - involves updating git submodules and downloading a beta compiler. It doesn't - need to be super fast, but that can help. - -Building the compiler takes more than half an hour on my moderately powerful -laptop. The first time you build the compiler, LLVM will also be built unless -you use CI-built LLVM ([see below][configsec]). - -[configsec]: #configuring-the-compiler - -Like `cargo`, the build system will use as many cores as possible. Sometimes -this can cause you to run low on memory. You can use `-j` to adjust the number -concurrent jobs. If a full build takes more than ~45 minutes to an hour, -you are probably spending most of the time swapping memory in and out; -try using `-j1`. - -On a slow machine, the build times for rustc are very painful. Consider using -`./x.py check` instead of a full build and letting the automated tests run -when you push to GitHub. - -If you don't have too much free disk space, you may want to turn off -incremental compilation ([see below][configsec]). This will make -compilation take longer (especially after a rebase), -but will save a ton of space from the incremental caches. +Internet access is required. + +The most notable software requirement is that you will need Python 2 or 3, but +there are various others. + +The following hardware is recommended. +* 30GB+ of free disk space. +* 8GB+ RAM +* 2+ cores + +More powerful machines will lead to much faster builds. There are various +strategies to work around lesser hardware in the following chapters. + +See [this chapter][prereqs] for more details about software and hardware prerequisites. + +[prereqs]: ./building/prerequisites.md ### Cloning