Bahex 301d1370c4

Add input support to generate (#14804) ...

- closes #8523 

# Description

This PR adds pipeline input support to `generate`.
- Without input, `generate` keeps its current behavior.
- With input, each invocation of the closure is provided an item from
the input stream as pipeline input (`$in`). If/when the input stream
runs out, `generate` also stops.

Before this PR, there is no filter command that is both stateful _and_
streaming.

This PR also refactors `std/iter scan` to use `generate`, making it
streaming and more performant over larger inputs.

# User-Facing Changes
- `generate` now supports pipeline input, passing each element to the
closure as `$in` until it runs out
- `std/iter scan` is now streaming

# Tests + Formatting
Added tests to validate the new feature.

- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib

# After Submitting
N/A

2025-01-14 11:44:31 -06:00

..

assignment

Change append operator to concatenation operator (#14344)

2024-11-24 10:59:54 -08:00

base

Replace the old encode base64 and decode base64 with new-base64 commands (#14018)

2024-10-08 11:01:43 +08:00

bytes

Implementing ByteStream interuption on infinite stream (#13552)

2025-01-11 13:28:08 -08:00

conversions

Change behavior of into record on lists to be more useful (#13637)

2024-08-22 11:38:43 +02:00

database

more closure serialization (#14698)

2025-01-07 11:51:22 -06:00

date

fix format date based on users locale (#11908)

2024-02-20 11:08:49 -06:00

debug

Make timeit take only closures as an argument (#14483)

2024-12-10 23:08:53 +08:00

hash_

Replace the old encode base64 and decode base64 with new-base64 commands (#14018)

2024-10-08 11:01:43 +08:00

math

better error message for "sum", "product", and "sum_of_squares" (#14711)

2024-12-31 16:04:23 -06:00

move_

update uutils crates (#14371)

2024-11-17 19:31:36 -06:00

network

Fix commands::network::http::*::*_timeout tests on non-english system (#14640)

2024-12-19 17:15:27 -06:00

path

test(path self): Add tests (#14607)

2024-12-17 17:01:23 +01:00

platform

ansi name for clear-scrollback code (#14184)

2024-10-29 07:01:32 -05:00

query

Feature cleanup (#7182)

2022-11-22 16:58:11 -08:00

random

support filesize arguments in random binary/chars (#14068)

2024-10-12 14:49:05 +08:00

skip

Add string/binary type color to ByteStream (#12897)

2024-05-20 00:35:32 +00:00

str_

Improves commands that support range input (#13113)

2024-06-18 07:19:13 -05:00

take

Add string/binary type color to ByteStream (#12897)

2024-05-20 00:35:32 +00:00

url

Change append operator to concatenation operator (#14344)

2024-11-24 10:59:54 -08:00

alias.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

all.rs

Rewrite run_external.rs (#12921)

2024-05-23 02:05:27 +00:00

any.rs

Rewrite run_external.rs (#12921)

2024-05-23 02:05:27 +00:00

append.rs

fix(nu-command/tests): further remove unnecessary pipeline() and cwd() (#8793)

2023-04-07 14:09:55 -07:00

break_.rs

don't allow break/continue in each and items command (#13398)

2024-07-19 00:21:02 -07:00

cal.rs

Suppress column index for default cal output (#13188)

2024-06-22 07:41:29 -05:00

cd.rs

Path migration part 4: various tests (#13373)

2024-08-03 10:09:13 +02:00

chunk_by.rs

Implement chunk_by operation (#14410)

2024-11-29 13:37:27 -08:00

chunks.rs

Deprecate group in favor of chunks (#13377)

2024-07-16 03:49:00 +00:00

compact.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

complete.rs

Make assignment and const consistent with let/mut (#13385)

2024-07-30 18:55:22 -05:00

config_env_default.rs

Command: Add config env/nu --default to print defaults (#10480)

2023-09-25 08:00:59 -05:00

config_nu_default.rs

Command: Add config env/nu --default to print defaults (#10480)

2023-09-25 08:00:59 -05:00

continue_.rs

fix(nu-command/tests): further remove unnecessary pipeline() and cwd() (#8793)

2023-04-07 14:09:55 -07:00

debug_info.rs

Make debug info lazy (#10728)

2023-10-24 12:48:05 -05:00

def.rs

make command signature parsing more strict (#14309)

2024-11-18 08:01:52 +08:00

default.rs

Remove default list-diving behaviour (#13386)

2024-07-16 03:54:24 +00:00

detect_columns.rs

Improves commands that support range input (#13113)

2024-06-18 07:19:13 -05:00

do_.rs

deprecate --ignore-shell-errors and --ignore-program-errors in do (#14385)

2024-11-27 09:36:30 +08:00

drop.rs

Refactor drop columns to fix issues (#10903)

2023-11-09 13:51:46 +01:00

du.rs

du: add -l/--long flag, remove -a/--all flag (#14407)

2024-12-10 11:22:56 -06:00

each.rs

Remove group command (#14056)

2024-10-11 06:43:12 -05:00

echo.rs

Change echo to print when not redirected (#10338)

2023-09-13 06:35:01 +12:00

empty.rs

fix(nu-command/tests): further remove unnecessary pipeline() and cwd() (#8793)

2023-04-07 14:09:55 -07:00

error_make.rs

Change the error style during tests to plain (#13061)

2024-06-18 21:37:24 -07:00

every.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

exec.rs

Isolate tests from user config (#12437)

2024-04-10 06:27:46 +08:00

export_def.rs

Removes unnecessary cwd and pipeline from various tests (#9202)

2023-05-17 18:55:26 -05:00

fill.rs

Removes unnecessary cwd and pipeline from various tests (#9202)

2023-05-17 18:55:26 -05:00

filter.rs

Fix return in filter closure eval (#12292)

2024-03-26 17:50:36 +01:00

find.rs

fixed issue with find not working with symbols that should be escaped (#13792)

2024-09-06 07:22:03 +08:00

first.rs

Add string/binary type color to ByteStream (#12897)

2024-05-20 00:35:32 +00:00

flatten.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

for_.rs

Change echo to print when not redirected (#10338)

2023-09-13 06:35:01 +12:00

format.rs

Span ID Refactor (Step 2): Use SpanId of expressions in some places (#13102)

2024-06-09 12:15:53 +03:00

generate.rs

Add input support to generate (#14804)

2025-01-14 11:44:31 -06:00

get.rs

Make get const (#14751)

2025-01-04 16:41:03 -05:00

glob.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

griddle.rs

make grid throw an error when not enough columns (#12672)

2024-04-26 06:33:00 -05:00

group_by.rs

Add run-time type checking for command pipeline input (#14741)

2025-01-08 23:09:47 +01:00

headers.rs

Fix: remove unnecessary r#"..."# (#8670) (#9764)

2023-07-21 17:32:37 +02:00

help.rs

Change the usage misnomer to "description" (#13598)

2024-08-22 12:02:08 +02:00

histogram.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

ignore.rs

Change the ignore command to use drain() instead of collecting a value (#12120)

2024-03-08 02:18:26 -05:00

insert.rs

Remove lazy records (#12682)

2024-05-03 08:36:10 +08:00

inspect.rs

fix inspect and explore panics on empty records (#13893)

2024-09-25 07:48:16 -05:00

interleave.rs

Isolate tests from user config (#12437)

2024-04-10 06:27:46 +08:00

into_datetime.rs

add table -> table to into datetime (#9775)

2023-07-23 20:14:51 +02:00

into_filesize.rs

fix wrong casting with into filesize (#13110)

2024-06-10 10:43:17 +08:00

into_int.rs

add --signed flag for binary into int conversions (#11902)

2024-02-27 15:05:26 +00:00

join.rs

support table literal syntax in join right-table argument (#14190)

2024-10-29 06:37:44 -05:00

last.rs

Add string/binary type color to ByteStream (#12897)

2024-05-20 00:35:32 +00:00

length.rs

support binary input in length (#14224)

2024-11-04 03:39:24 +00:00

let_.rs

fix error when exporting consts with type signatures in modules (#14118)

2024-10-22 11:54:31 +02:00

lines.rs

fix panic with lines on an error (#9967)

2023-08-09 14:12:58 +02:00

loop_.rs

Change echo to print when not redirected (#10338)

2023-09-13 06:35:01 +12:00

ls.rs

make ls return "Permission denied" for CWD instead of empty results (#14310)

2024-11-15 12:09:02 +08:00

match_.rs

support raw strings in match patterns (#14573)

2024-12-13 06:55:57 -06:00

merge_deep.rs

Add merge deep command (#14525)

2024-12-18 06:36:04 -06:00

merge.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

mktemp.rs

Path migration part 4: various tests (#13373)

2024-08-03 10:09:13 +02:00

mod.rs

Remove deprecated commands (#14726)

2025-01-07 07:37:51 +08:00

mut_.rs

raise ParseError if assign to a non-variable or non-mutable-variable (#14405)

2024-11-29 23:02:21 +01:00

nu_check.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

open.rs

open: Assign content_type metadata for filetypes not handled with a from converter (#14670)

2025-01-01 03:05:43 +01:00

par_each.rs

Fix: remove unnecessary r#"..."# (#8670) (#9764)

2023-07-21 17:32:37 +02:00

parse.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

prepend.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

print.rs

Add --raw switch to print for binary data (#13597)

2024-08-12 17:29:25 +08:00

range.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

redirection.rs

Overhaul $in expressions (#13357)

2024-07-17 16:02:42 -05:00

reduce.rs

Add run-time type checking for command pipeline input (#14741)

2025-01-08 23:09:47 +01:00

reject.rs

Improve CellPath display output (#14197)

2024-11-02 10:28:10 -05:00

rename.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

return_.rs

Fix return setting last exit code (#14120)

2024-10-18 03:05:58 +00:00

reverse.rs

Clean up tests containing unnecessary cwd: tokens (#9692)

2023-07-17 18:43:51 +02:00

rm.rs

Path migration part 4: various tests (#13373)

2024-08-03 10:09:13 +02:00

roll.rs

Fix: remove unnecessary r#"..."# (#8670) (#9764)

2023-07-21 17:32:37 +02:00

rotate.rs

Fix panic in rotate; Add safe record creation function (#11718)

2024-02-03 13:23:16 +02:00

run_external.rs

Don't expand ndots if prefixed with ./ (#14755)

2025-01-05 17:07:34 -05:00

save.rs

fix wrong error msg of save command on windows (#14699)

2025-01-07 07:36:42 +08:00

select.rs

allow select to stream more (#14492)

2024-12-03 20:45:31 -06:00

semicolon.rs

Slim down tests (#9021)

2023-04-28 13:25:44 +02:00

seq_char.rs

Seq char update will work on all char (#14261)

2024-11-15 21:05:29 +01:00

seq_date.rs

Fix panic in seq date (#11871)

2024-02-17 10:51:20 +02:00

seq.rs

Slim down tests (#9021)

2023-04-28 13:25:44 +02:00

sort_by.rs

to json -r not removing whitespaces fix (#11948)

2024-03-20 22:14:31 +01:00

sort.rs

Rework sorting and add cell path and closure comparators to sort-by (#13154)

2024-10-09 19:18:16 -07:00

source_env.rs

Path migration part 2: nu-test-support (#13329)

2024-07-12 02:43:10 +00:00

split_column.rs

Add --number flag to split column (#13831)

2024-09-12 07:16:33 -05:00

split_row.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00

table.rs

Let table only check for use_ansi_coloring config value (#14798)

2025-01-10 19:24:16 -06:00

tee.rs

Make tee work more nicely with non-collections (#13652)

2024-09-01 19:03:46 +02:00

terminal.rs

Add is-terminal to determine if stdin/out/err are a terminal (#10970)

2023-11-21 20:48:39 -06:00

to_text.rs

Make to text line endings consistent for list (streams) (#14166)

2024-11-05 09:33:54 +01:00

transpose.rs

Defensive handling of errors when transposing (#14096)

2024-10-22 11:30:48 -05:00

try_.rs

add rendered and json error messages in try/catch (#14082)

2024-10-20 23:14:11 +02:00

ucp.rs

cp: disable unsupported reflink mode in freebsd builds (#14677)

2024-12-26 07:56:42 -06:00

ulimit.rs

FreeBSD compatibility patches (#11869)

2024-02-17 20:04:59 +01:00

umkdir.rs

[umkdir][tests] get umask instead of assuming it (#14046)

2024-10-11 14:13:42 +02:00

uname.rs

Initial implementation for uutils uname (#11684)

2024-03-25 16:51:50 -05:00

uniq_by.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

uniq.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

update.rs

Remove lazy records (#12682)

2024-05-03 08:36:10 +08:00

upsert.rs

Remove lazy records (#12682)

2024-05-03 08:36:10 +08:00

use_.rs

Change tests which may invoke externals to use non-conflicting names (#14516)

2024-12-04 19:26:48 -06:00

utouch.rs

Make utouch the new touch (#14721)

2025-01-02 06:26:46 -06:00

where_.rs

Fix ignored clippy lints (#12160)

2024-03-11 19:46:04 +01:00

which.rs

change the output of which to be more explicit (#9646)

2023-07-20 19:10:53 -05:00

while_.rs

Change echo to print when not redirected (#10338)

2023-09-13 06:35:01 +12:00

window.rs

Refactor window (#13401)

2024-07-19 04:16:09 +00:00

with_env.rs

Improve with-env robustness (#12523)

2024-04-16 19:08:58 +08:00

wrap.rs

Remove file I/O from tests that don't need it (#11182)

2023-11-29 23:21:34 +01:00

zip.rs

Avoid taking unnecessary ownership of intermediates (#12740)

2024-05-04 00:53:15 +00:00