Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dumps unbounded amounts of data in diagnostics (27M in three lines for one error!) for large input lines (and thinks about them for minutes first) #137680

Open
nabijaczleweli opened this issue Feb 26, 2025 · 2 comments
Labels
A-diagnostics Area: Messages for errors, warnings, and lints D-verbose Diagnostics: Too much output caused by a single piece of incorrect code. I-compiletime Issue: Problems and improvements with respect to compile times. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@nabijaczleweli
Copy link
Contributor

nabijaczleweli commented Feb 26, 2025

Code

(The actual data in my use-case is indices in a 100³ gilbert curve but isn't relevant here.)

$ { echo '['; seq $(( 100 * 100 * 100 )) | sed 's/.*/(0,0,0),/'; echo ']'; } > 100.rs
$ tr '\n' ' ' < 100.rs > 100s.rs
const GRID_SIZE: usize = 100;
static GRID_GILBERT: [(u8,u8,u8); GRID_SIZE*GRID_SIZE*GRID_SIZE] = include!("100s.rs");
fn main() {}

this is the good state

$ time rustc bugowcy.rs
warning: constant `GRID_SIZE` is never used
 --> bugowcy.rs:1:7
  |
1 | const GRID_SIZE: usize = 100;
  |       ^^^^^^^^^
  |
  = note: `#[warn(dead_code)]` on by default

warning: static `GRID_GILBERT` is never used
 --> bugowcy.rs:2:8
  |
2 | static GRID_GILBERT: [(u8,u8,u8); GRID_SIZE*GRID_SIZE*GRID_SIZE] = include!("100s.rs");
  |        ^^^^^^^^^^^^

warning: 2 warnings emitted


real    0m31.891s
user    0m0.000s
sys     0m0.015s

Change line 2 to

static GRID_GILBERT: &[(u8,u8,u8)] = include!("100s.rs");

to trigger

Current output

$ time rustc bugowcy.rs > err 2>&1

real    2m12.617s
user    0m0.000s
sys     0m0.015s
$ wc -cl err
      16 27000433 err
$ less -S err
error[E0308]: mismatched types
 --> 100s.rs:1:1
  |
1 | [ (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (>
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^>
  |
  = note: expected reference `&'static [(u8, u8, u8)]`
                 found array `[({integer}, {integer}, {integer}); 1000000]`
help: consider borrowing here
  |
1 | &[ (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), (0,0,0), >
  | +

error: aborting due to 1 previous error

For more information about this error, try `rustc --explain E0308`.

Yes that is 27 megabytes of diagnostic output, basically repeating the input thrice (twice verbatim, once with ^^^^s).

Desired output

Either a bounded line output or just a bailout. Honestly just a bailout. It'd be nice if it took like 30s instead of 2:20 💀

Rationale and extra context

I think in this case (and in general) having rustc churn for two full minutes is. Not good. And value to the user is bigger when you get the error without source-level diagnostics.

Rust Version

$ rustc --version --verbose
rustc 1.84.1 (e71f9a9a9 2025-01-27)
binary: rustc
commit-hash: e71f9a9a98b0faf423844bf0ba7438f29dc27d58
commit-date: 2025-01-27
host: x86_64-pc-windows-gnu
release: 1.84.1
LLVM version: 19.1.5
@nabijaczleweli nabijaczleweli added A-diagnostics Area: Messages for errors, warnings, and lints T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Feb 26, 2025
@nabijaczleweli
Copy link
Contributor Author

To replicate the experience of doing this, you can download err.gz and then do

$ sleep 138; zcat err.gz

@nabijaczleweli nabijaczleweli changed the title Dumps unbounded amounts of data in diagnostics (27M for one error!) for large input lines (and thinks about them for minutes first) Dumps unbounded amounts of data in diagnostics (27M in three lines for one error!) for large input lines (and thinks about them for minutes first) Feb 26, 2025
@fmease fmease added D-verbose Diagnostics: Too much output caused by a single piece of incorrect code. I-compiletime Issue: Problems and improvements with respect to compile times. labels Feb 26, 2025
@estebank
Copy link
Contributor

estebank commented Feb 26, 2025

I believe there is a similar open ticket for this. Edit: #125581.

Note that the rendering output already trims the output to fit the terminal width when possible. Namely, when there are spans for code that is wider than the terminal, we remove "margins" to the left and right. But that logic operates under the assumption that if there are spans, people want to see them and their associated code, so there's a bounding box on the left-most span and the right-most span, from which the trimming cannot go past. And as an extension of that, there's no logic to trim code from a single, long, span (like the one being presented here). I believe we can do that, likely only if there's a single span, or if the output is so extreme (like in this case) that there's no point on trying to do pretty output. One option would be for cases where there are spans that are >3 times the terminal width to trim the middle of them, as long as they don't overlap with another span, and only for that line. This doesn't keep the shape of the user's code as they wrote it, but it is better than nothing.

For suggestions, the situation is a bit harder as well, because we currently don't trim those, because we want people to be able to copy-paste from that ouptut, but it might be that we want to apply a similar heuristic of "if there's only one code change in a long line of code, and it is wider than N times the terminal, trim to the right while still including the code change".

I think that the compile time slowdown is likely unrelated to the verbosity, likely some accidentally quadratic code in the suggestion machinery for this error.

estebank added a commit to estebank/rust that referenced this issue Feb 27, 2025
…al width

When encountering a single line span that is wider than the terminal, we keep context at the start and end of the span but otherwise remove the code from the middle. This is somewhat independent from whether the left and right margins of the output have been trimmed as well.

```
error[E0308]: mismatched types
  --> $DIR/long-span.rs:6:15
   |
LL | ... = [0, 0, 0, 0, ..., 0, 0];
   |       ^^^^^^^^^^^^^...^^^^^^^ expected `u8`, found `[{integer}; 1681]`
```

Address part of rust-lang#137680 (missing handling of the long suggestion). Fix rust-lang#125581.
estebank added a commit to estebank/rust that referenced this issue Feb 27, 2025
…al width

When encountering a single line span that is wider than the terminal, we keep context at the start and end of the span but otherwise remove the code from the middle. This is somewhat independent from whether the left and right margins of the output have been trimmed as well.

```
error[E0308]: mismatched types
  --> $DIR/long-span.rs:6:15
   |
LL | ... = [0, 0, 0, 0, ..., 0, 0];
   |       ^^^^^^^^^^^^^...^^^^^^^ expected `u8`, found `[{integer}; 1681]`
```

Address part of rust-lang#137680 (missing handling of the long suggestion). Fix rust-lang#125581.
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Mar 7, 2025
On long spans, trim the middle of them to make them fit in the terminal width

When encountering a single line span that is wider than the terminal, we keep context at the start and end of the span but otherwise remove the code from the middle. This is somewhat independent from whether the left and right margins of the output have been trimmed as well.

```
error[E0308]: mismatched types
  --> $DIR/long-span.rs:6:15
   |
LL | ... = [0, 0, 0, 0, ..., 0, 0];
   |       ^^^^^^^^^^^^^...^^^^^^^ expected `u8`, found `[{integer}; 1681]`
```

Address part of rust-lang#137680 (missing handling of the long suggestion). Fix rust-lang#125581.

---

Change the way that underline positions are calculated by delaying using the "visual" column position until the last possible moment, instead using the "file"/byte position in the file, and then calculating visual positioning as late as possible. This should make the underlines more resilient to non-1-width unicode chars.

Unfortunately, as part of this change (which fixes some visual bugs) comes with the loss of some eager tab codepoint handling, but the output remains legible despite some minor regression on the "margin trimming" logic.

---

`-Zteach` is perma-unstable, barely used, the highlighting logic buggy and the flag being passed around is tech-debt. We should likely remove `-Zteach` in its entirely.
estebank added a commit to estebank/rust that referenced this issue Mar 7, 2025
…al width

When encountering a single line span that is wider than the terminal, we keep context at the start and end of the span but otherwise remove the code from the middle. This is somewhat independent from whether the left and right margins of the output have been trimmed as well.

```
error[E0308]: mismatched types
  --> $DIR/long-span.rs:6:15
   |
LL | ... = [0, 0, 0, 0, ..., 0, 0];
   |       ^^^^^^^^^^^^^...^^^^^^^ expected `u8`, found `[{integer}; 1681]`
```

Address part of rust-lang#137680 (missing handling of the long suggestion). Fix rust-lang#125581.
jhpratt added a commit to jhpratt/rust that referenced this issue Mar 8, 2025
On long spans, trim the middle of them to make them fit in the terminal width

When encountering a single line span that is wider than the terminal, we keep context at the start and end of the span but otherwise remove the code from the middle. This is somewhat independent from whether the left and right margins of the output have been trimmed as well.

```
error[E0308]: mismatched types
  --> $DIR/long-span.rs:6:15
   |
LL | ... = [0, 0, 0, 0, ..., 0, 0];
   |       ^^^^^^^^^^^^^...^^^^^^^ expected `u8`, found `[{integer}; 1681]`
```

Address part of rust-lang#137680 (missing handling of the long suggestion). Fix rust-lang#125581.

---

Change the way that underline positions are calculated by delaying using the "visual" column position until the last possible moment, instead using the "file"/byte position in the file, and then calculating visual positioning as late as possible. This should make the underlines more resilient to non-1-width unicode chars.

Unfortunately, as part of this change (which fixes some visual bugs) comes with the loss of some eager tab codepoint handling, but the output remains legible despite some minor regression on the "margin trimming" logic.

---

`-Zteach` is perma-unstable, barely used, the highlighting logic buggy and the flag being passed around is tech-debt. We should likely remove `-Zteach` in its entirely.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Mar 8, 2025
Rollup merge of rust-lang#137757 - estebank:trim-spans, r=davidtwco

On long spans, trim the middle of them to make them fit in the terminal width

When encountering a single line span that is wider than the terminal, we keep context at the start and end of the span but otherwise remove the code from the middle. This is somewhat independent from whether the left and right margins of the output have been trimmed as well.

```
error[E0308]: mismatched types
  --> $DIR/long-span.rs:6:15
   |
LL | ... = [0, 0, 0, 0, ..., 0, 0];
   |       ^^^^^^^^^^^^^...^^^^^^^ expected `u8`, found `[{integer}; 1681]`
```

Address part of rust-lang#137680 (missing handling of the long suggestion). Fix rust-lang#125581.

---

Change the way that underline positions are calculated by delaying using the "visual" column position until the last possible moment, instead using the "file"/byte position in the file, and then calculating visual positioning as late as possible. This should make the underlines more resilient to non-1-width unicode chars.

Unfortunately, as part of this change (which fixes some visual bugs) comes with the loss of some eager tab codepoint handling, but the output remains legible despite some minor regression on the "margin trimming" logic.

---

`-Zteach` is perma-unstable, barely used, the highlighting logic buggy and the flag being passed around is tech-debt. We should likely remove `-Zteach` in its entirely.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-diagnostics Area: Messages for errors, warnings, and lints D-verbose Diagnostics: Too much output caused by a single piece of incorrect code. I-compiletime Issue: Problems and improvements with respect to compile times. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

3 participants