marsh

Split and join

split runs parallel commands on private copies of your project, called forks. join shows their output and patches. It never applies a patch.

From a marsh shell, ask two agents to work independently:

split {
  a: claude -p 'append gamma to notes.txt'
  b: pi -p 'append gamma to notes.txt'
} | join --keep

--keep leaves the results under .marsh/split/ID/out/. Review a branch's diff.patch, apply it with git apply, then remove the split with marsh splits rm ID. Keep results whenever you want to read or apply them after the pipeline ends.

Registered agents mount their own fork of the project. A branch's shell commands, including tests, run with your shell VM privileges and can reach your real project (branch types).

The shell examples below assume notes.txt contains alpha and beta on separate lines. Use a scratch project to try them. Agent examples also need provider credentials.

What split touches

  • marsh creates .marsh/split/ID/ for the snapshot, forks, and results. In a Git project, it also creates .marsh/.gitignore if absent. It leaves the original repository's index, objects, and refs untouched.
  • Branches run in the shell VM or in a Kit's VM, not on the Mac.
  • In Git repositories, ignored files are not copied into forks. A split's root cannot be a linked Git worktree or a submodule checkout with a .git file; use the main checkout.
  • marsh split works from the marsh shell and from plain bash on the Mac. split { } works only in the marsh shell. Inside a job, only ::: branches work.
  • join limits previews to 128 KiB of stdout per branch, 128 KiB of diff across branches, and the last 4 KiB of a failed branch's stderr. The whole rendering is capped at 192 KiB. Files in out/ hold the full captured bytes.

The snapshot comes from your current files, including uncommitted edits and nonignored untracked files. It is not a checkout of HEAD. marsh uses copy-on-write filesystem clones, falling back to ordinary copies on filesystems without cloning support. Other copy errors fail the split. Forks live and mount at their own host paths, such as PROJECT/.marsh/split/ID/a. In a shell branch, an absolute path to the original project still reaches your live files. A registered Kit command mounts the fork, not the original project root.

Applying a result

git apply needs no index. It works from the marsh shell and from a Mac terminal:

split_id=$(marsh split -n -b fix='printf "alpha\nbeta\ngamma\n" > notes.txt')
marsh join -- sh -c 'cat >/dev/null; git apply "$SPLIT_DIR/fix/diff.patch"' <<<"$split_id"
grep -q gamma notes.txt

Merging when your tree has moved

If your tree has changed since the split, git apply --3way can attempt a three-way merge. It stages the result and can leave conflicts. Commit or stage your own edits first.

Two details matter:

  • SPLIT_OBJECTS holds file versions Git may not have. Pass it as GIT_ALTERNATE_OBJECT_DIRECTORIES.
  • Your index must match your working files. If Git reports a stale index after changes from the other side of the mount, run git update-index -q --refresh.

The example keeps an uncommitted line from the starting snapshot and merges two later edits to different lines. The objects from the split are needed because that starting version was never committed to the original repository.

set -e
printf 'alpha\nbeta\nmiddle\nuncommitted\n' > notes.txt
handle=$(marsh split -n -b fix='sed s/alpha/ALPHA/ notes.txt > next; mv next notes.txt')
printf 'alpha\nbeta\nmiddle\nlocal\n' > notes.txt
git add notes.txt
marsh join -- sh -c 'cat >/dev/null
  export GIT_ALTERNATE_OBJECT_DIRECTORIES="$SPLIT_OBJECTS"
  git apply --3way "$SPLIT_DIR/fix/diff.patch"' <<<"$handle"
grep -qx ALPHA notes.txt
grep -qx local notes.txt

Inside marsh (Brush form)

split {
  upper: tr a-z A-Z < notes.txt > n && mv n notes.txt
  count: wc -l < notes.txt
} | join | less -FX

Each label: command line is one branch. Without a label, a branch is named after its command.

  • Labels start with a letter or _, followed by letters, digits, _, or -, up to 64 characters. They must be unique, ignoring case; out, base, and owner are reserved.
  • A split has at most 16 branches.
  • Commands after join read its output on stdin and run in your real project directory (in the marsh shell, the shell VM's view of it). They see SPLIT_ID, SPLIT_DIR, and SPLIT_MANIFEST, plus SPLIT_OBJECTS for Git projects. These variables are scoped to the downstream stages.
  • Every branch in split { } is a shell branch (see below). For ::: branches, use the CLI form.

Where one branch ends and the next begins

A new branch starts after {, after a newline, or after ,.

After ;, a new branch starts only if the next word is a label. A label matches [A-Za-z_][A-Za-z0-9_-]*:.

Everything else is ordinary shell syntax inside the current branch. That includes other ;, &&, ||, { ...; }, ( ... ), and case ... ;;.

InputResult
a: cd x; make; b: cd y; maketwo branches, two commands each
x; yone branch
a: echo b: cone branch (only the first word after ; counts)
x; echo: hitwo branches, the second named echo

A command whose name ends in : is read as a label. Quote it to run it as a command: 'echo:' hi.

split { upper: tr a-z A-Z < notes.txt > n; mv n notes.txt; count: { wc -l; echo done; } < notes.txt } | join

What join prints

For each branch, join prints a header with the status and where it ran (shell-vm, or kit:NAME for a ::: branch), then its stdout, its stderr if it failed, and its diff. -- LABEL: no changes -- means capture found no net changes relative to the snapshot. It does not describe new files excluded by ignore rules, shared-home writes, or edits the branch later reverted. A diff cut at the limit ends with (diff truncated; full diff: PATH).

A binary change appears as binary file changed: PATH (N bytes; full patch in $SPLIT_DIR/LABEL/diff.patch). diff.patch has the full bytes.

A fanout inside a branch

A branch can run a fanout on its own fork. join shows the branch's collect output, and marsh jobs --tree draws a fanout (...) node under the branch:

split {
  fix: claude -p "fix TestDialTimeout"
  review: fanout { races: codex exec "review net/ for races", leaks: claude -p "look for goroutine leaks" } | collect
} | join

From plain bash (CLI form)

marsh split -b upper='tr a-z A-Z < notes.txt > n && mv n notes.txt' \
            -b count='wc -l < notes.txt' </dev/null | marsh join

marsh split prints one line of JSON, the handle (it contains "id"). It exits with the first nonzero branch status, in the order the branches were written. marsh join reads the handle.

Input to the branches:

  • marsh split reads stdin once and gives every branch the same bytes.
  • With -n, or when stdin is a terminal, branches get empty input.

With -- CMD, marsh join runs CMD with its output on stdin and the SPLIT_* variables set. It removes the split when CMD exits 0, unless --keep was set or cleanup is uncertain. A nonzero CMD status keeps it.

CMD runs where you invoke marsh join: on the Mac from a Mac terminal, in the shell VM from a marsh shell, or in the container from a job.

OptionEffect
marsh split -nGive branches empty stdin
marsh join --jsonEmit the manifest instead of the text rendering
marsh join --keepRetain forks and results after joining
marsh join --timingPrint phase timings on stderr
marsh join -- CMD ARG...Run a command with the rendering on stdin and SPLIT_* set

Shell branches and ::: branches

A shell branch, written as -b LABEL=STRING or inside split { }, runs your code in a fresh Brush shell in the shell VM. It starts in its fork, even when your interactive shell is Bash or zsh.

Its ordinary commands (make, sed, a script) are not confined: they can write anywhere your shell user can, including your live project. Capture records only changes in the fork. Use shell branches for tests and scripts you trust. Registered commands they start still run in Kit containers that mount the fork.

An argv branch, written as ::: LABEL CMD ARG..., runs one registered command in a fresh container. Its project mount is the fork, along with the Git metadata the fork needs:

  • The index, objects, and refs are writable, with HEAD naming a private marsh-split branch. The command can stage and commit without changing your original repository.
  • The original object store and the fork's fixed configuration files are read-only mounts.

Every Kit job also mounts the shared guest home read/write. Login files and session state saved there, such as ~/.claude and ~/.codex, remain visible to other jobs in that home.

marsh split -n -b tests='grep -c . notes.txt' \
            ::: review codex exec 'fix the typo in notes.txt' | marsh join

-b takes one word, LABEL=STRING. ::: starts an argv branch, and the next ::: ends it. Words pass through unchanged, so an argument cannot be :::.

Results

$SPLIT_DIR = <project>/.marsh/split/<id>/out
  manifest.json            ($SPLIT_MANIFEST) each branch's state, status, and where it ran
  <label>/stdout, stderr   the branch's output bytes
  <label>/status           "exited 0", "failed: ...", "rejected: ...", "cancelled"
  <label>/files            one "A|M|D|T<TAB>path" line per changed path
  <label>/diff.patch       a git-apply patch (absent when nothing changed)

In a Git repository, marsh creates .marsh/.gitignore containing * when that file is absent, so Git ignores new split state. An existing ignore file is left unchanged.

What is captured

In a Git repository, new files that the project's ignore rules exclude are not captured, as with git status. The rules are .gitignore, .git/info/exclude, and core.excludesFile. __pycache__/ from running Python is an example.

A file included in the snapshot is always compared, even if a branch adds an ignore rule that matches it. Ignore filtering applies only to Git projects; a plain directory's .gitignore does not exclude files.

Capture allows 32 MiB per changed file and 256 MiB of changed bytes per branch, with a five-second deadline per text diff. Exceeding a capture limit rejects that branch's result. .marsh, nested .git entries, and special files are excluded from the initial snapshot; creating a nested .git in a fork causes capture to reject it.

Directories that are not Git repositories

A plain directory splits the same way. The patches still apply with git apply, which needs no repository. marsh writes no Git files there.

marsh split -n -b fix='printf "alpha\nbeta\ngamma\n" > notes.txt' |
  marsh join -- sh -c 'cat >/dev/null; git apply "$SPLIT_DIR/fix/diff.patch"'
grep -q gamma notes.txt && test ! -e .git && test ! -e .marsh/.gitignore

Failures, pipefail, Ctrl-C

marsh split returns the first nonzero branch status in declaration order. Bare marsh join returns that status too, keeping failed results. With marsh join -- CMD, CMD runs even if branches failed, and its status becomes join's status. A successful CMD removes the results unless --keep was set or cleanup is uncertain.

In Brush's split { } | join | CMD form, PIPESTATUS records the split stage, join stage, and downstream stages. The last stage determines the pipeline's status unless pipefail is set. Retention still follows the last stage's status, even with pipefail: a successful CMD can remove a failed branch's results. Use join --keep if you need them afterward.

If CMD fails, marsh keeps the split and says why:

marsh split -n -b bad='echo oops >&2; exit 4' | marsh join -- sh -c 'cat; exit 1'

Ctrl-C

Ctrl-C during split (or a split { } pipeline) cancels every branch and anything they started. Each gets SIGINT, then SIGKILL after 10 seconds. Nothing after join runs.

When every branch is confirmed stopped, marsh removes the split directory.

If a branch cannot be confirmed, for example because its VM was lost, marsh keeps the split and the message names the branches. Run marsh workers reset all before marsh splits rm.

Inspecting and removing kept splits

h=$(marsh split -n -b bad='exit 4')
marsh join --keep -- true <<<"$h"   # prints why it was kept and how to remove it
marsh splits                        # id, state, dir; one line per branch
id=$(sed 's/.*"id":"\([^"]*\)".*/\1/' <<<"$h")
marsh splits rm "$id"

Other commands:

CommandEffect
marsh splits IDdescribes a split, including one join already removed (states and times, no files)
marsh splits cancel IDcancels a running split
marsh statuscounts active, awaiting, and kept splits
marsh join --timingprints a timing breakdown on stderr

A split waiting for join that no process has held open for 60 seconds is kept. Its creator can still join it.

--timing reports these phases:

  • run: the slowest branch, including Kit VM boots.
  • consumer: the commands after join.
  • snapshot, forks, capture: marsh's own overhead.

Splits allow 16 branches, 64 MiB of input, and 16 MiB of combined branch output. They have a one-hour deadline and can nest three splits deep. Nested splits snapshot the calling branch's fork and accept only argv branches. Branches share an eight-slot session pool, while registered commands also count against the daemon's job limits. Only the creator may join a split, including one retained after lease expiry.

Seeing a split in marsh jobs --tree

In your session, marsh jobs --tree draws each split with its branches and the jobs they started, even after join removed the files. A job started in a downstream stage says (consumes split ID).

A shell branch's row (SPLIT/label) times the whole branch. The job rows under it time each Kit job from its start. The gap between them is shell startup plus any Kit VM boot.

Next: Fanout and collect, Agents running agents, design: workspaces.

Edit this page on GitHub