mirror of
https://github.com/nushell/nushell.git
synced 2025-05-07 08:22:58 +00:00
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Wind
|
2f18b9c856
|
Enable nushell error with backtrace (#14945)
# Description After this pr, nushell is able to raise errors with a backtrace, which should make users easier to debug. To enable the feature, users need to set env variable via `$env.NU_BACKTRACE = 1`. But yeah it might not work perfectly, there are some corner cases which might not be handled. I think it should close #13379 in another way. ### About the change The implementation mostly contained with 2 parts: 1. introduce a new `ChainedError` struct as well as a new `ShellError::ChainedError` variant. If `eval_instruction` returned an error, it converts the error to `ShellError::ChainedError`. `ChainedError` struct is responsable to display errors properly. It needs to handle the following 2 cases: - if we run a function which runs `error make` internally, it needs to display the error itself along with caller span. - if we run a `error make` directly, or some commands directly returns an error, we just want nushell raise an error about `error make`. 2. Attach caller spans to `ListStream` and `ByteStream`, because they are lazy streams, and *only* contains the span that runs it directly(like `^false`, for example), so nushell needs to add all caller spans to the stream. For example: in `def a [] { ^false }; def b [] { a; 33 }; b`, when we run `b`, which runs `a`, which runs `^false`, the `ByteStream` only contains the span of `^false`, we need to make it contains the span of `a`, so nushell is able to get all spans if something bad happened. This behavior is happened after running `Instruction::Call`, if it returns a `ByteStream` and `ListStream`, it will call `push_caller_span` method to attach call spans. # User-Facing Changes It's better to demostrate how it works by examples, given the following definition: ```nushell > $env.NU_BACKTRACE = 1 > def a [x] { if $x == 3 { error make {msg: 'a custom error'}}} > def a_2 [x] { if $x == 3 { ^false } else { $x } } > def a_3 [x] { if $x == 3 { [1 2 3] | each {error make {msg: 'a custom error inside list stream'} } } } > def b [--list-stream --external] { if $external == true { # error with non-zero exit code, which is generated from external command. a_2 1; a_2 3; a_2 2 } else if $list_stream == true { # error generated by list-stream a_3 1; a_3 3; a_3 2 } else { # error generated by command directly a 1; a 2; a 3 } } ``` Run `b` directly shows the following error: <details> ```nushell Error: chained_error × oops ╭─[entry #27:1:1] 1 │ b · ┬ · ╰── error happened when running this ╰──── Error: chained_error × oops ╭─[entry #26:10:19] 9 │ # error generated by command directly 10 │ a 1; a 2; a 3 · ┬ · ╰── error happened when running this 11 │ } ╰──── Error: × a custom error ╭─[entry #6:1:26] 1 │ def a [x] { if $x == 3 { error make {msg: 'a custom error'}}} · ─────┬──── · ╰── originates from here ╰──── ``` </details> Run `b --list-stream` shows the following error <details> ```nushell Error: chained_error × oops ╭─[entry #28:1:1] 1 │ b --list-stream · ┬ · ╰── error happened when running this ╰──── Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #26:7:16] 6 │ # error generated by list-stream 7 │ a_3 1; a_3 3; a_3 2 · ─┬─ · ╰── source value 8 │ } else { ╰──── Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #23:1:29] 1 │ def a_3 [x] { if $x == 3 { [1 2 3] | each {error make {msg: 'a custom error inside list stream'} } } } · ┬ · ╰── source value ╰──── Error: × a custom error inside list stream ╭─[entry #23:1:44] 1 │ def a_3 [x] { if $x == 3 { [1 2 3] | each {error make {msg: 'a custom error inside list stream'} } } } · ─────┬──── · ╰── originates from here ╰──── ``` </details> Run `b --external` shows the following error: <details> ```nushell Error: chained_error × oops ╭─[entry #29:1:1] 1 │ b --external · ┬ · ╰── error happened when running this ╰──── Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #26:4:16] 3 │ # error with non-zero exit code, which is generated from external command. 4 │ a_2 1; a_2 3; a_2 2 · ─┬─ · ╰── source value 5 │ } else if $list_stream == true { ╰──── Error: nu:🐚:non_zero_exit_code × External command had a non-zero exit code ╭─[entry #7:1:29] 1 │ def a_2 [x] { if $x == 3 { ^false } else { $x } } · ──┬── · ╰── exited with code 1 ╰──── ``` </details> It also added a message to guide the usage of NU_BACKTRACE, see the last line in the following example: ```shell ls asdfasd Error: nu:🐚:io::not_found × I/O error ╰─▶ × Entity not found ╭─[entry #17:1:4] 1 │ ls asdfasd · ───┬─── · ╰── Entity not found ╰──── help: The error occurred at '/home/windsoilder/projects/nushell/asdfasd' set the `NU_BACKTRACE=1` environment variable to display a backtrace. ``` # Tests + Formatting Added some tests for the behavior. # After Submitting |
||
Piepmatz
|
66bc0542e0
|
Refactor I/O Errors (#14927)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
As mentioned in #10698, we have too many `ShellError` variants, with
some even overlapping in meaning. This PR simplifies and improves I/O
error handling by restructuring `ShellError` related to I/O issues.
Previously, `ShellError::IOError` only contained a message string,
making it convenient but overly generic. It was widely used without
providing spans (#4323).
This PR introduces a new `ShellError::Io` variant that consolidates
multiple I/O-related errors (except for `ShellError::NetworkFailure`,
which remains distinct for now). The new `ShellError::Io` variant
replaces the following:
- `FileNotFound`
- `FileNotFoundCustom`
- `IOInterrupted`
- `IOError`
- `IOErrorSpanned`
- `NotADirectory`
- `DirectoryNotFound`
- `MoveNotPossible`
- `CreateNotPossible`
- `ChangeAccessTimeNotPossible`
- `ChangeModifiedTimeNotPossible`
- `RemoveNotPossible`
- `ReadingFile`
## The `IoError`
`IoError` includes the following fields:
1. **`kind`**: Extends `std::io::ErrorKind` to specify the type of I/O
error without needing new `ShellError` variants. This aligns with the
approach used in `std::io::Error`. This adds a second dimension to error
reporting by combining the `kind` field with `ShellError` variants,
making it easier to describe errors in more detail. As proposed by
@kubouch in [#design-discussion on
Discord](https://discord.com/channels/601130461678272522/615329862395101194/1323699197165178930),
this helps reduce the number of `ShellError` variants. In the error
report, the `kind` field is displayed as the "source" of the error,
e.g., "I/O error," followed by the specific kind of I/O error.
2. **`span`**: A non-optional field to encourage providing spans for
better error reporting (#4323).
3. **`path`**: Optional `PathBuf` to give context about the file or
directory involved in the error (#7695). If provided, it’s shown as a
help entry in error reports.
4. **`additional_context`**: Allows adding custom messages when the
span, kind, and path are insufficient. This is rendered in the error
report at the labeled span.
5. **`location`**: Sometimes, I/O errors occur in the engine itself and
are not caused directly by user input. In such cases, if we don’t have a
span and must set it to `Span::unknown()`, we need another way to
reference the error. For this, the `location` field uses the new
`Location` struct, which records the Rust file and line number where the
error occurred. This ensures that we at least know the Rust code
location that failed, helping with debugging. To make this work, a new
`location!` macro was added, which retrieves `file!`, `line!`, and
`column!` values accurately. If `Location::new` is used directly, it
issues a warning to remind developers to use the macro instead, ensuring
consistent and correct usage.
### Constructor Behavior
`IoError` provides five constructor methods:
- `new` and `new_with_additional_context`: Used for errors caused by
user input and require a valid (non-unknown) span to ensure precise
error reporting.
- `new_internal` and `new_internal_with_path`: Used for internal errors
where a span is not available. These methods require additional context
and the `Location` struct to pinpoint the source of the error in the
engine code.
- `factory`: Returns a closure that maps an `std::io::Error` to an
`IoError`. This is useful for handling multiple I/O errors that share
the same span and path, streamlining error handling in such cases.
## New Report Look
This is simulation how the I/O errors look like (the `open crates` is
simulated to show how internal errors are referenced now):
## `Span::test_data()`
To enable better testing, `Span::test_data()` now returns a value
distinct from `Span::unknown()`. Both `Span::test_data()` and
`Span::unknown()` refer to invalid source code, but having a separate
value for test data helps identify issues during testing while keeping
spans unique.
## Cursed Sneaky Error Transfers
I removed the conversions between `std::io::Error` and `ShellError` as
they often removed important information and were used too broadly to
handle I/O errors. This also removed the problematic implementation
found here:
|
||
Ian Manske
|
fce6146576
|
Refactor config updates (#13802)
# Description This PR standardizes updates to the config through a new `UpdateFromValue` trait. For now, this trait is private in case we need to make changes to it. Note that this PR adds some additional `ShellError` cases to create standard error messages for config errors. A follow-up PR will move usages of the old error cases to these new ones. This PR also uses `Type::custom` in lots of places (e.g., for string enums). Not sure if this is something we want to encourage. # User-Facing Changes Should be none. |
||
|
Ian Manske
|
3d008e2c4e
|
Error on non-zero exit statuses (#13515)
# Description This PR makes it so that non-zero exit codes and termination by signal are treated as a normal `ShellError`. Currently, these are silent errors. That is, if an external command fails, then it's code block is aborted, but the parent block can sometimes continue execution. E.g., see #8569 and this example: ```nushell [1 2] | each { ^false } ``` Before this would give: ``` ╭───┬──╮ │ 0 │ │ │ 1 │ │ ╰───┴──╯ ``` Now, this shows an error: ``` Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #1:1:2] 1 │ [1 2] | each { ^false } · ┬ · ╰── source value ╰──── Error: nu:🐚:non_zero_exit_code × External command had a non-zero exit code ╭─[entry #1:1:17] 1 │ [1 2] | each { ^false } · ──┬── · ╰── exited with code 1 ╰──── ``` This PR fixes #12874, fixes #5960, fixes #10856, and fixes #5347. This PR also partially addresses #10633 and #10624 (only the last command of a pipeline is currently checked). It looks like #8569 is already fixed, but this PR will make sure it is definitely fixed (fixes #8569). # User-Facing Changes - Non-zero exit codes and termination by signal now cause an error to be thrown. - The error record value passed to a `catch` block may now have an `exit_code` column containing the integer exit code if the error was due to an external command. - Adds new config values, `display_errors.exit_code` and `display_errors.termination_signal`, which determine whether an error message should be printed in the respective error cases. For non-interactive sessions, these are set to `true`, and for interactive sessions `display_errors.exit_code` is false (via the default config). # Tests Added a few tests. # After Submitting - Update docs and book. - Future work: - Error if other external commands besides the last in a pipeline exit with a non-zero exit code. Then, deprecate `do -c` since this will be the default behavior everywhere. - Add a better mechanism for exit codes and deprecate `$env.LAST_EXIT_CODE` (it's buggy). |
||
|
Ian Manske
|
0918050ac8
|
Deprecate group in favor of chunks (#13377)
# Description The name of the `group` command is a little unclear/ambiguous. Everything I look at it, I think of `group-by`. I think `chunks` more clearly conveys what the `group` command does. Namely, it divides the input list into chunks of a certain size. For example, [`slice::chunks`](https://doc.rust-lang.org/std/primitive.slice.html#method.chunks) has the same name. So, this PR adds a new `chunks` command to replace the now deprecated `group` command. The `chunks` command is a refactored version of `group`. As such, there is a small performance improvement: ```nushell # $data is a very large list > bench { $data | chunks 2 } --rounds 30 | get mean 474ms 921µs 190ns # deprecation warning was disabled here for fairness > bench { $data | group 2 } --rounds 30 | get mean 592ms 702µs 440ns > bench { $data | chunks 200 } --rounds 30 | get mean 374ms 188µs 318ns > bench { $data | group 200 } --rounds 30 | get mean 481ms 264µs 869ns > bench { $data | chunks 1 } --rounds 30 | get mean 642ms 574µs 42ns > bench { $data | group 1 } --rounds 30 | get mean 981ms 602µs 513ns ``` # User-Facing Changes - `group` command has been deprecated in favor of new `chunks` command. - `chunks` errors when given a chunk size of `0` whereas `group` returns chunks with one element. # Tests + Formatting Added tests for `chunks`, since `group` did not have any tests. # After Submitting Update book if necessary. |
||
Devyn Cairns
|
d7392f1f3b
|
Internal representation (IR) compiler and evaluator (#13330)
# Description This PR adds an internal representation language to Nushell, offering an alternative evaluator based on simple instructions, stream-containing registers, and indexed control flow. The number of registers required is determined statically at compile-time, and the fixed size required is allocated upon entering the block. Each instruction is associated with a span, which makes going backwards from IR instructions to source code very easy. Motivations for IR: 1. **Performance.** By simplifying the evaluation path and making it more cache-friendly and branch predictor-friendly, code that does a lot of computation in Nushell itself can be sped up a decent bit. Because the IR is fairly easy to reason about, we can also implement optimization passes in the future to eliminate and simplify code. 2. **Correctness.** The instructions mostly have very simple and easily-specified behavior, so hopefully engine changes are a little bit easier to reason about, and they can be specified in a more formal way at some point. I have made an effort to document each of the instructions in the docs for the enum itself in a reasonably specific way. Some of the errors that would have happened during evaluation before are now moved to the compilation step instead, because they don't make sense to check during evaluation. 3. **As an intermediate target.** This is a good step for us to bring the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at some point, as code generated from new AST can be directly compared to code generated from old AST. If the IR code is functionally equivalent, it will behave the exact same way. 4. **Debugging.** With a little bit more work, we can probably give control over advancing the virtual machine that `IrBlock`s run on to some sort of external driver, making things like breakpoints and single stepping possible. Tools like `view ir` and [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than before to see what exactly is going on with your Nushell code. The goal is to eventually replace the AST evaluator entirely, once we're sure it's working just as well. You can help dogfood this by running Nushell with `$env.NU_USE_IR` set to some value. The environment variable is checked when Nushell starts, so config runs with IR, or it can also be set on a line at the REPL to change it dynamically. It is also checked when running `do` in case within a script you want to just run a specific piece of code with or without IR. # Example ```nushell view ir { |data| mut sum = 0 for n in $data { $sum += $n } $sum } ``` ```gas # 3 registers, 19 instructions, 0 bytes of data 0: load-literal %0, int(0) 1: store-variable var 904, %0 # let 2: drain %0 3: drop %0 4: load-variable %1, var 903 5: iterate %0, %1, end 15 # for, label(1), from(14:) 6: store-variable var 905, %0 7: load-variable %0, var 904 8: load-variable %2, var 905 9: binary-op %0, Math(Plus), %2 10: span %0 11: store-variable var 904, %0 12: load-literal %0, nothing 13: drain %0 14: jump 5 15: drop %0 # label(0), from(5:) 16: drain %0 17: load-variable %0, var 904 18: return %0 ``` # Benchmarks All benchmarks run on a base model Mac Mini M1. ## Iterative Fibonacci sequence This is about as best case as possible, making use of the much faster control flow. Most code will not experience a speed improvement nearly this large. ```nushell def fib [n: int] { mut a = 0 mut b = 1 for _ in 2..=$n { let c = $a + $b $a = $b $b = $c } $b } use std bench bench { 0..50 | each { |n| fib $n } } ``` IR disabled: ``` ╭───────┬─────────────────╮ │ mean │ 1ms 924µs 665ns │ │ min │ 1ms 700µs 83ns │ │ max │ 3ms 450µs 125ns │ │ std │ 395µs 759ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ``` IR enabled: ``` ╭───────┬─────────────────╮ │ mean │ 452µs 820ns │ │ min │ 427µs 417ns │ │ max │ 540µs 167ns │ │ std │ 17µs 158ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ```  ## [gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu) IR disabled: ``` ╭───┬──────────────────╮ │ 0 │ 27ms 929µs 958ns │ │ 1 │ 21ms 153µs 459ns │ │ 2 │ 18ms 639µs 666ns │ │ 3 │ 19ms 554µs 583ns │ │ 4 │ 13ms 383µs 375ns │ │ 5 │ 11ms 328µs 208ns │ │ 6 │ 5ms 659µs 542ns │ ╰───┴──────────────────╯ ``` IR enabled: ``` ╭───┬──────────────────╮ │ 0 │ 22ms 662µs │ │ 1 │ 17ms 221µs 792ns │ │ 2 │ 14ms 786µs 708ns │ │ 3 │ 13ms 876µs 834ns │ │ 4 │ 13ms 52µs 875ns │ │ 5 │ 11ms 269µs 666ns │ │ 6 │ 6ms 942µs 500ns │ ╰───┴──────────────────╯ ``` ## [random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu) I got pretty random results out of this benchmark so I decided not to include it. Not clear why. # User-Facing Changes - IR compilation errors may appear even if the user isn't evaluating with IR. - IR evaluation can be enabled by setting the `NU_USE_IR` environment variable to any value. - New command `view ir` pretty-prints the IR for a block, and `view ir --json` can be piped into an external tool like [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir). # Tests + Formatting All tests are passing with `NU_USE_IR=1`, and I've added some more eval tests to compare the results for some very core operations. I will probably want to add some more so we don't have to always check `NU_USE_IR=1 toolkit test --workspace` on a regular basis. # After Submitting - [ ] release notes - [ ] further documentation of instructions? - [ ] post-release: publish `nu_plugin_explore_ir` |
||
|
Devyn Cairns
|
efe25e3f58
|
Better generic errors for plugins (and perhaps scripts) (#12236)
# Description This makes `LabeledError` much more capable of representing close to everything a `miette::Diagnostic` can, including `ShellError`, and allows plugins to generate multiple error spans, codes, help, etc. `LabeledError` is now embeddable within `ShellError` as a transparent variant. This could also be used to improve `error make` and `try/catch` to reflect `LabeledError` exactly in the future. Also cleaned up some errors in existing plugins. # User-Facing Changes Breaking change for plugins. Nicer errors for users. |
||
Stefan Holderbach
|
f695ba408a
|
Restructure nu-protocol in more meaningful units (#11917)
This is partially "feng-shui programming" of moving things to new separate places. The later commits include "`git blame` tollbooths" by moving out chunks of code into new files, which requires an extra step to track things with `git blame`. We can negiotiate if you want to keep particular things in their original place. If egregious I tried to add a bit of documentation. If I see something that is unused/unnecessarily `pub` I will try to remove that. - Move `nu_protocol::Exportable` to `nu-parser` - Guess doccomment for `Exportable` - Move `Unit` enum from `value` to `AST` - Move engine state `Variable` def into its folder - Move error-related files in `nu-protocol` subdir - Move `pipeline_data` module into its own folder - Move `stream.rs` over into the `pipeline_data` mod - Move `PipelineMetadata` into its own file - Doccomment `PipelineMetadata` - Remove unused `is_leap_year` in `value/mod` - Note about criminal `type_compatible` helper - Move duration fmting into new `value/duration.rs` - Move filesize fmting logic to new `value/filesize` - Split reexports from standard imports in `value/mod` - Doccomment trait `CustomValue` - Polish doccomments and intradoc links |