Benchmarks

This page publishes the macro CLI benchmark results produced by make bench-macro. It is intended as a technical reference for engineers evaluating runtime characteristics, not as a cross-machine league table.

Two datasets are shown:

• A checked-in Apple M5 Max snapshot generated from local benchmark exports.
• A Linux CI snapshot regenerated during the GitHub Pages deploy for the latest published release.

Compare tools within the same snapshot. Absolute numbers across different machines are useful for rough orientation only.

Snapshot Overview

Methodology

• The runner is hyperfine with 3 warmups and 10 measured runs per case.
• shuck is measured with check --no-cache so the results reflect parsing and linting work rather than cache reuse.
• The comparison command is shellcheck --severity=style <fixture> on the same input.
• --ignore-failure is intentional. These fixtures contain lint findings, so non-zero exit codes are expected and the benchmark is measuring runtime rather than success state.
• Each fixture is benchmarked independently, and the all case benchmarks one invocation over the entire vendored corpus.

Reproducing Results

Refresh the checked-in local snapshot

make bench-macro-site-local

Generate a CI-style dataset manually

./scripts/benchmarks/setup.sh hyperfine shellcheck
./scripts/benchmarks/run.sh
python3 ./scripts/benchmarks/export_website_data.py \
  --repo-root . \
  --bench-dir .cache \
  --output website/generated/benchmarks/ci-latest.json \
  --dataset-id ci-latest \
  --dataset-name "GitHub Actions latest release snapshot" \
  --dataset-description "Generated during the website deploy workflow for the latest published release." \
  --environment-kind ci \
  --environment-label "GitHub Actions ubuntu-latest"

Benchmark Corpus

The benchmark corpus is vendored into the repository so every run uses the same inputs. The current corpus contains 5 files, 10,671 lines, and 318.0 KiB of shell source.