An AI agent skill for writing tests in the `nushell` repo.

A developer created an AI agent skill for writing tests in the Nushell repository, emphasizing the use of modern `nu_test_support` infrastructure over older subprocess-style testing. The skill provides guidelines for in-process tests, direct value assertions, and explicit dependencies.

| name | nushell-testing | |||||| |---|---|---|---|---|---|---|---| | description | Use when writing or refactoring Nushell Rust tests that use nu test support, NuTester, Playground, rstest, deps, plugins, CompleteResult, or when replacing old nu /nu with plugins tests. | |||||| | license | MIT | |||||| | compatibility | opencode | |||||| | metadata | | Use this skill when writing new tests or refactoring old tests in the Nushell repository. The goal is to use the modern nu test support infrastructure directly, keep tests small and explicit, and avoid subprocess-style testing unless the behavior under test truly crosses the nu binary boundary. The most important rule: do not use nu in new tests. Do not introduce nu , nu with plugins , nu with std , or nu repl code . Some of these APIs may still exist for compatibility in older branches, but new and refactored tests should use test and NuTester instead. Prefer in-process tests. test returns a NuTester that runs Nushell code in-process against the current crate code. This is faster, more deterministic, and avoids stale nu binaries. Assert values, not text rendering. If a command returns a list, record, bool, int, table, duration, or structured error, assert that structured value directly. Do not add to json , to nuon , to text , str join , or manual output collapsing just to make assertion easier. Make required external artifacts explicit. If a test needs a compiled nu binary, use deps NU . If it needs a plugin, use deps NU PLUGIN EXAMPLE or the appropriate plugin dependency. Do not call add nu to path in new code. Keep tests local and readable. Avoid tiny helper functions that hide the testing infra. Prefer a few explicit lines in the test body, rstest cases for repeated shapes, and constants only for large shared data or setup snippets. Return Result . Most Nushell tests should be fn test name - Result , where Result comes from nu test support::prelude:: . Let ? propagate TestError from NuTester and assertion helpers. Use the newest guidance over older examples. Older refactors may use add nu to path ; newer test infra replaces that with deps NU . Most test files should start with the prelude: use nu test support::prelude:: ; Add specific imports when needed: use nu protocol::Signals; use pretty assertions::assert matches; use rstest::rstest; Do not import test value , test record , or test table again; those test macros are already provided by nu test support::prelude:: . Do not import the old macros for new tests: // Bad use nu test support::nu; use nu test support::{nu, nu repl code}; use nu test support::nu with plugins; Why: even when nu is still re-exported by compatibility modules or the prelude on some branches, that is not permission to use it in new tests. The direction is to remove these macros from tests. For a single snippet and a single expected value, chain test .run ... .expect value eq ... : php use nu test support::prelude:: ; test fn can average range - Result { test .run "0..5 | math avg" .expect value eq 2.5 } This is better than a nu test because it compares a real Value converted through IntoValue , not collapsed stdout text. Do not write this: js use nu test support::nu; test fn can average range { let actual = nu "0..5 | math avg" ; assert eq actual.out, "2.5" ; } Use a local variable named code only for long or multiline snippets: use nu test support::prelude:: ; test fn format filesize respects float precision for fractional values - Result { let code = " $env.config = $env.config | upsert float precision 5 1024B | format filesize kB "; test .run code .expect value eq "1.02400 kB" } Guidelines for multiline snippets: Use let code = "..." or let code = r "..." when the snippet is long or multiline. For short snippets, pass a one-line string directly to run ... or run with data ... . Keep Nushell code readable; do not compress everything into one line unless it is genuinely clearer. If the expected output is a multiline Rust string, prefer indoc::indoc . Use cwd on the tester when the snippet reads files relative to a fixture directory: php use nu test support::prelude:: ; test fn reads cargo sample - Result { let code = r " open cargo sample.toml | get package | format pattern "{name} has license {license}" " ; test .cwd "tests/fixtures/formats" .run code .expect value eq "nu has license ISC" } cwd accepts repo-relative paths and sandbox paths. Prefer it over cd in the snippet when the test setup itself should start in a specific directory. run extracts the result into any type implementing FromValue . expect value eq accepts any type implementing IntoValue . Never write run::<T ... or run with data::<T ... . If you extract a value, let the left-hand side define the type with let value: T = test .run ... ? . If you call expect value eq , expect shell error , expect parse error , or similar, the helper already fixes the needed type. Simple expected values should be raw Rust values when they are homogeneous or scalar: php test fn filters ints - Result { test .run " 3, 1, 2 | sort" .expect value eq 1, 2, 3 } Use test value , test record , and test table for mixed, nested, record, or table-like data. Prefer test value for nested structures instead of nesting test record inside test record : php use nu test support::prelude:: ; test fn insert at end of list - Result { test .run " 1, 2, 3 | insert 3 abc" .expect value eq test value 1, 2, 3, "abc" } php use nu test support::prelude:: ; test fn inserts into nested record - Result { test .run "{a: {}} | insert a.b.c 0" .expect value eq test value { a: { b: { c: 0 } } } } php use nu test support::prelude:: ; test fn insert uses enumerate index - Result { let code = " a ; 7 6 | enumerate | insert b {|el| $el.index + 1 + $el.item.a } | flatten "; test .run code .expect value eq test table "index", "a", "b" ; 0, 7, 8 , 1, 6, 8 , } Prefer these forms over serialized output: // Bad unless serialization itself is under test test fn insert at end of list - Result { test .run " 1, 2, 3 | insert 3 abc | to json --raw" .expect value eq r " 1,2,3,"abc" " } php // Good test fn insert at end of list - Result { test .run " 1, 2, 3 | insert 3 abc" .expect value eq test value 1, 2, 3, "abc" } Exceptions are real rendering and serialization tests, such as to json , to nuon , table , or to text , where the string representation is the behavior under test. When assertions need Rust logic, extract a typed value: php test fn splits empty path - Result { let outcome: bool = test .cwd "tests" .run "echo '' | path split | is-empty" ?; assert outcome ; Ok } If the test only verifies that a command succeeds and returns a value of a certain type, bind to : php test fn is terminal accepts stdin flag - Result { let : bool = test .run "is-terminal --stdin" ?; Ok } If you only care that a command succeeds and returns any value: php test fn help does not error - Result { let : Value = test .run "overlay hide --help" ?; Ok } If the command should return nothing , use let or expect value eq : php test fn definition compiles - Result { let = test .run "def helper { 'ok' }" ?; Ok } Use run with data when the test needs external data. This keeps the Nushell snippet stable and avoids unsafe or noisy format use: php use nu test support::prelude:: ; test fn reads input record - Result { let input = test record { "name" = "nu", "license" = "MIT", }; test .run with data "$in | format pattern '{name} has license {license}'", input .expect value eq "nu has license MIT" } Prefer this over injecting values into source with format : php // Bad when only the value changes test fn uppercases name - Result { let name = "nu"; test .run format "'{name}' | str upcase" .expect value eq "NU" } php // Good test fn uppercases name - Result { let name = "nu"; test .run with data "$in | str upcase", name .expect value eq "NU" } format is fine when you are intentionally generating Nushell syntax, such as an option flag or a large code fixture, and the generated code remains obvious. Use one mutable NuTester when later snippets depend on definitions, environment changes, overlays, or variables from earlier snippets: php test fn use main def env - Result { let mut tester = test ; let = tester.run r "module spam { export def --env main { $env.SPAM = "spam" } }" ?; let = tester.run "use spam" ?; let = tester.run "spam" ?; tester.run "$env.SPAM" .expect value eq "spam" } Use run multiple when the test intentionally needs to run several pipelines after each other on the same tester and only the final value matters: php test fn new overlay from const name - Result { let commands = "const mod = 'spam'", "overlay new $mod", "overlay list | last | get name", ; test .run multiple commands .expect value eq "spam" } Prefer explicit repeated run calls when intermediate results matter, when some steps are setup with nothing output, or when the source-unit boundaries are important for understanding the test: php test fn use main def env - Result { let mut tester = test ; let = tester.run r "module spam { export def --env main { $env.SPAM = "spam" } }" ?; let = tester.run "use spam" ?; let = tester.run "spam" ?; tester.run "$env.SPAM" .expect value eq "spam" } Do not use nu repl code . If the test wants REPL-like separate source units, use run multiple for a simple sequence where only the final value matters, or repeated tester.run ... calls when intermediate steps matter. Be deliberate about source-unit boundaries: If behavior must occur in one parse/source unit, put it in one code block and run once. If behavior must persist across source units, use repeated run calls on one mutable tester. If scenarios are independent, use separate test calls or rstest cases. Example of independent scenarios using fresh testers: php test fn module main not found - Result { let mut tester = test ; let = tester.run "module spam {}" ?; tester .run "use spam main" .expect error code eq "nu::parser::export not found" ?; let mut tester = test ; let = tester.run "module spam {}" ?; tester .run "use spam main " .expect error code eq "nu::parser::export not found" } Use Playground::setup when the test needs a sandboxed filesystem: use nu test support::{fs::Stub::EmptyFile, prelude:: }; test fn rm deletes file - Result { Playground::setup "rm deletes file", |dirs, sandbox| { sandbox.with files & EmptyFile "delete-me.txt" ; test .cwd dirs.test .run "rm delete-me.txt; 'delete-me.txt' | path exists" .expect value eq false } } Rules for playground tests: Return the Playground::setup result directly from the test. Use test .cwd dirs.test when the snippet should run in the sandbox. Use FileWithContentToBeTrimmed for indented file contents. When the Rust side needs to read a file, do not use file contents path . Use std::fs::read to string path ? or fs::read to string path ? if std::fs is imported. io::Error converts into TestError , so this works naturally in tests that return Result : use std::fs; use nu test support::{fs::Stub::FileWithContent, prelude:: }; test fn writes expected file - Result { Playground::setup "writes expected file", |dirs, sandbox| { sandbox.with files & FileWithContent "input.txt", "hello" ; let = test .cwd dirs.test .run "open input.txt | str upcase | save output.txt" ?; let contents = fs::read to string dirs.test .join "output.txt" ?; assert eq contents, "HELLO" ; Ok } } Do not create a playground when no filesystem behavior is being tested. Do not write this shape: php // Bad test fn bad playground shape - Result { Playground::setup "case", |dirs, sandbox| { sandbox.with files & EmptyFile "x" ; test .cwd dirs.test .run "ls" ?; Ok } ; Ok } Write this instead: php test fn good playground shape - Result { Playground::setup "case", |dirs, sandbox| { sandbox.with files & EmptyFile "x" ; let : Value = test .cwd dirs.test .run "ls" ?; Ok } } Use rstest when several tests have the same shape and only data changes: use nu test support::prelude:: ; use rstest::rstest; rstest case::null "null", case::true "true", true case::false "false", false fn top level values from json case json: &str, case expected: impl IntoValue - Result { test .run with data "from json", json .expect value eq expected } This is better than a loop inside one test because every case is reported independently. When combining rstest with deps , env , exp , serial , or other harness attributes, explicitly use the harness test macro: use nu test support::prelude:: ; use rstest::rstest; rstest case::child lib dirs "$env.NU LIB DIRS | describe", "list<string " case::child plugin dirs "$NU PLUGIN DIRS | describe", "list<string " nu test support::test deps NU fn child process defaults case child code: &str, case expected: impl IntoValue - Result { test .run with data "let child code; nu -n -c $child code", child code .expect value eq expected } For per-case dependencies, follow nearby repository style and place the dependency attribute next to the case it belongs to. Prefer structured error assertions. If the exact error code is enough, use expect error code eq : php test fn insert past end of list - Result { test .run " 1, 2, 3 | insert 5 abc" .expect error code eq "nu::shell::access beyond end" } If the variant or fields matter, extract the error and match it: use nu test support::prelude:: ; use pretty assertions::assert matches; test fn insert nested path into empty list errors without underflow - Result { let err = test .run " | insert 0.0 1" .expect shell error ?; assert matches err, ShellError::AccessEmptyContent { .. } ; Ok } Parse and compile errors have dedicated helpers: use nu protocol::Type; use nu test support::prelude:: ; use pretty assertions::assert matches; test fn parse function signature switch is bool - Result { let err = test .run "def foo --bar { let baz: int = $bar }" .expect parse error ?; assert matches err, ParseError::TypeMismatch Type::Int, Type::Bool, ; Ok } Use ShellErrorExt for generic or labeled errors when needed: php test fn glob without match reports missing file - Result { let err = test .run "ls root3 " .expect shell error ?; let msg = err.generic msg ?; assert contains "file or folder not found", msg ; Ok } Avoid this old style: js // Bad test fn insert past end of list { let actual = nu " 1, 2, 3 | insert 5 abc" ; assert actual.err.contains "too large" ; } Use expect value eq for Nushell values. Use assert contains needle, haystack for positive contains checks. Use pretty assertions::assert eq or pretty assertions::assert str eq for large Rust-side values or rendered strings when it improves diffs. Use assert matches for error variants and enum shapes. Avoid assert haystack.contains needle ; it gives weaker diagnostics than assert contains . Avoid assert eq actual.out, ... because new tests should not produce actual.out in the first place. Use assert contains not needle, haystack for negative containment checks. Prefer it over hand-written assert haystack.contains needle because it matches the positive assert contains style and gives a consistent failure message: php test fn overlay list does not contain hidden overlay - Result { let names: String = test .run "overlay list | get name | str join ' '" ?; assert contains not "spam", names ; Ok } Most tests should not spawn the nu binary. Spawn it only when testing behavior that exists at the process boundary, such as CLI flags, script-file execution, process exit behavior, or child process environment defaults. When spawning nu , declare deps NU : use nu test support::prelude:: ; test deps NU fn source file relative to file - Result { let result: CompleteResult = test .cwd "tests/parsing/samples" .run "nu -n source file relative.nu | complete" ?; assert eq result.exit code, 0 ; assert eq result.stdout.trim , "5" ; assert eq result.stderr, "" ; Ok } Use CompleteResult for complete output instead of indexing a record: php test deps NU fn child nu reports missing script - Result { let result: CompleteResult = test .run "nu -n missing-script.nu | complete" ?; assert ne result.exit code, 0 ; assert contains "file not found", result.stderr ; Ok } Do not use add nu to path in new tests: php // Bad test fn child version - Result { test .add nu to path .run "nu -n -c 'version | get version'" .expect value eq env "CARGO PKG VERSION" } php // Good test deps NU fn child version - Result { test .run "nu -n -c 'version | get version'" .expect value eq env "CARGO PKG VERSION" } Do not call test binaries through nu --testbin ... . That route is deprecated for tests. Declare the testbin dependency and call the binary directly: php // Bad test deps NU fn bad testbin call - Result { test .run "nu --testbin cococo a b c" .expect value eq "a b c" } php // Good test deps TESTBIN COCOCO fn calls testbin directly - Result { test .run "cococo a b c" .expect value eq "a b c" } Use the appropriate TESTBIN constant from the prelude, such as TESTBIN COCOCO , TESTBIN MEOW , or TESTBIN REPEATER . Add NU only if the behavior under test specifically requires a child Nushell process in addition to the testbin. Plugin tests should not use nu with plugins . Declare plugin dependencies with deps and call plugin commands directly in-process: use nu test support::prelude:: ; test deps NU PLUGIN EXAMPLE fn call to json - Result { test .run " 42 | example call-decl 'to json' {indent: 4}" .expect value eq " \n 42\n " } use nu test support::prelude:: ; test deps NU PLUGIN EXAMPLE fn plugin receives config - Result { let code = r " $env.config = { plugins: { example: { path: "some/path" nested: { bool: true, string: "Hello Example " } } } } example config " ; test .run code .expect value eq test value { path: "some/path", nested: { bool: true, string: "Hello Example " } } } If a plugin test specifically needs an actual child nu process plus a plugin path, declare both dependencies and pass the plugin path as data: test deps NU, NU PLUGIN INC fn plugin process exits when nushell exits - Result { let pid: u32 = test .run with data r "nu -n --plugins $in -c "'2.0.0' | inc -m; plugin list .0.pid" | into int" , NU PLUGIN INC.path , ?; let mut tester = test ; let = tester.run "sleep 500ms" ?; let = tester.run with data "let pid = $in", pid ?; tester .run "ps | where pid == $pid | is-empty" .expect value eq true } Use serial for plugin tests that check process identity, start/stop timing, or shared process state. Do not mark all plugin tests serial by default. By default, NuTester keeps $env.PATH unset or minimal for deterministic tests. Use deps NU for the Nushell binary. Use plugin dependencies for plugins. Use deps TESTBIN X for test binaries and invoke them directly by name. Do not route testbins through nu --testbin . Use inherit path only when the test intentionally relies on arbitrary system commands. Use inherit rust toolchain env for tests that intentionally spawn cargo , rustc , or rustup . Example: php test fn cargo is available when inheriting rust toolchain env - Result { test .inherit rust toolchain env .run "cargo --version | split row ' ' | get 0" .expect value eq "cargo" } Do not use inherit path as a workaround for missing deps . The custom harness supports attributes on tests: serial runs tests sequentially. Use it for process-wide state, heavy IO contention, plugin PID lifecycle tests, or other unavoidable shared-state cases. Do not use global locks instead. env KEY = "value" sets process environment variables for a test group. Use this when engine setup or harness grouping depends on an environment variable. exp OPTION enables an experimental option. Use it instead of setting experimental options manually in a nu call. deps ... declares binaries or plugins that must be built and made available to test . Example: use nu experimental::REORDER CELL PATHS; use nu test support::prelude:: ; test exp REORDER CELL PATHS fn update table cell respects reorder option - Result { let code = " let a = foo ; bar let b = $a | update foo.0 'baz' $b.0.foo "; test .run code .expect value eq "baz" } For locale-sensitive tests, use env NU TEST LOCALE OVERRIDE = "en US.UTF-8" when the locale must affect harness/engine setup, or test .locale en when configuring only the tester is enough. For commands whose output is intentionally a string rendering, assert the string directly. Use indoc to keep expected output readable. use indoc::indoc; use nu test support::prelude:: ; test fn table padding zero - Result { test .run with data " let data = $in $env.config.table.padding = {left: 0, right: 0} $data | table --width=80 ", test table "a", "b", "c" ; 1, 2, 3 , 4, 5, 1, 2, 3 , , .expect value eq indoc {" ╭─┬─┬─┬──────────────╮ │ │a│b│ c │ ├─┼─┼─┼──────────────┤ │0│1│2│ 3│ │1│4│5│ list 3 items │ ╰─┴─┴─┴──────────────╯ "} } Notes for rendering tests: Use run with data for complex input so the test focuses on rendering. Use ansi strip when colors are not the behavior under test. When colors are the behavior under test, assert the ANSI string explicitly. Use pretty assertions::assert str eq when comparing large rendered strings extracted as String . Use this checklist when converting old tests. - Remove use nu test support::nu , nu with plugins , and nu repl code imports. - Add use nu test support::prelude:: ; and only specific extra imports that are needed. - Change fn test name to fn test name - Result . - Replace let actual = nu ... with test .run ... , test .cwd ... .run ... , or let mut tester = test ; tester.run ... . - Replace assert eq actual.out, "..." with .expect value eq ... or typed extraction. - Replace assert actual.err.contains ... with .expect error code eq ... , .expect shell error ? , .expect parse error ? , or .expect compile error ? . - Replace plugin macro usage with deps NU PLUGIN ... and direct plugin commands. - Replace add nu to path with deps NU . - Replace nu --testbin name ... with deps TESTBIN NAME and direct name ... calls. - Replace nu repl code with run multiple or a mutable NuTester and repeated run calls, depending on which is clearer. - Replace serialization-only assertions with structured values unless serialization is the feature under test. - Use rstest when the refactor exposes several same-shaped tests. - Keep the test body explicit rather than moving a small sequence into a helper. Example refactor: js // Old use nu test support::nu; test fn insert into list { let actual = nu " 1, 2, 3 | insert 1 abc | to json -r" ; assert eq actual.out, r " 1,"abc",2,3 " ; } php // New use nu test support::prelude:: ; test fn insert into list - Result { test .run " 1, 2, 3 | insert 1 abc" .expect value eq test value 1, "abc", 2, 3 } Error refactor: js // Old test fn list unknown long flag { let actual = nu "ls --full-path" ; assert actual.err.contains "Did you mean: --full-paths ?" ; } // New use nu protocol::ParseError; use nu test support::prelude:: ; use pretty assertions::assert matches; test fn list unknown long flag - Result { let err = test .run "ls --full-path" .expect parse error ?; assert matches err, ParseError::UnknownFlag , , , help if help == "Did you mean: --full-paths ?" ; Ok } Plugin refactor: js // Old test fn call reduce { let result = nu with plugins cwd: ".", plugin: "nu plugin example" , " 1 2 3 | example call-decl 'reduce' {fold: 10} { |it, acc| $it + $acc }" ; assert eq "16", result.out ; } php // New test deps NU PLUGIN EXAMPLE fn call reduce - Result { test .run " 1 2 3 | example call-decl 'reduce' {fold: 10} {|it, acc| $it + $acc}" .expect value eq 16 } Multiple source-unit refactor: // Acceptable when only the final value matters test fn new overlay from const name - Result { let commands = "const mod = 'spam'", "overlay new $mod", "overlay list | last | get name", ; test .run multiple commands .expect value eq "spam" } // Prefer this when intermediate steps or assertions matter test fn new overlay from const name - Result { let mut tester = test ; let = tester.run "const mod = 'spam'" ?; let = tester.run "overlay new $mod" ?; tester.run "overlay list | last | get name" .expect value eq "spam" } Do not use nu for new tests. Do not use nu with plugins . Do not use nu repl code . Do not use add nu to path . Do not call testbins via nu --testbin ; use deps TESTBIN ... and call them directly. Do not assert actual.out , actual.err , or actual.status from nu output. Do not write run::<T ... or run with data::<T ... ; annotate the left-hand binding or use the expectation helpers. Do not depend on NU just because a test used to spawn nu ; first ask whether the behavior can be tested in-process. Do not depend on a plugin or binary without deps ... . Do not serialize values to JSON, NUON, markdown, text, or joined strings only to assert them. Do not use Playground::setup when no filesystem behavior is needed. Do not use file contents path ; use std::fs::read to string path ? instead. Do not emulate serialization or isolation with global mutexes; use serial when serial execution is truly required. Do not create helper functions just to wrap test .run or hide a small command sequence. Do not manually set process environment with std::env::set var ; use env or test .env ... . Do not manually enable experimental options in the snippet or old macro options; use exp . Do not import test value , test record , or test table from nu protocol when nu test support::prelude:: is already imported; the prelude provides the test macros. Use test .run code .expect value eq expected for one snippet and one expected value. Use let mut tester = test ; tester.run ... for shared state across source units. Use test .run multiple commands for a simple sequence of pipelines where only the last output is asserted. Use one multiline code block for behavior that must parse or execute in one source unit. Use run with data when external Rust data is the input. Use Playground::setup plus .cwd dirs.test for sandboxed files. Use rstest for same-shaped cases. Use expect error code eq when the code is the contract. Use expect shell error , expect parse error , or expect compile error when the error shape matters. Use deps NU plus CompleteResult when testing a child nu process. Use deps TESTBIN ... when testing commands that need a testbin, then call the testbin directly. Use deps NU PLUGIN ... when testing plugin commands in-process. Use serial only for unavoidable shared state, timing-sensitive process lifecycle, or heavy IO conflicts. Above all, keep the test explicit, type-aware, and free of nu .