head/tail for JSON, YAML — but structure‑aware. Get a compact preview that shows both the shape and representative values of your data, all within a strict byte budget. (Just like head/tail, hson can also work with unstructured text files.)

Available as:

• CLI (see Usage)
• Python library (see Python Bindings)

Install

Using Cargo:

cargo install headson

Note: the CLI installs as hson. All examples below use hson ....

From source:

cargo build --release
target/release/hson --help

Features

• Budgeted output: specify exactly how much you want to see
• Output formats: auto | json | yaml | text
  • Styles: strict | default | detailed
    • JSON family: strict → strict JSON, default → human‑friendly Pseudo, detailed → JS with inline comments
    • YAML: always YAML; strict has no comments, default uses “# …”, detailed uses “# N more …”
    • Text: prints raw lines. In default style, omissions are shown as a single line …; in detailed, as … N more lines …. strict omits array‑level summaries.
• Multiple inputs: preview many files at once with a shared or per‑file budget
• Fast: processes gigabyte‑scale files in seconds (mostly disk‑bound)
• Available as a CLI app and as a Python library

Fits into command line workflows

If you’re comfortable with tools like head and tail, use hson when you want a quick, structured peek into a JSON file without dumping the entire thing.

• head/tail operate on bytes/lines - their output is not optimized for tree structures
• jq: you need to craft filters to preview large JSON files
• hson: head/tail for trees—zero‑config by default; force text with -i text when you want raw lines

Usage

hson [FLAGS] [INPUT...]

• INPUT (optional, repeatable): file path(s). If omitted, reads from stdin. Multiple input files are supported.
• Prints the preview to stdout. On parse errors, exits non‑zero and prints an error to stderr.

Common flags:

• -c, --bytes <BYTES>: per‑file output budget (bytes). For multiple inputs, default total budget is <BYTES> * number_of_inputs.
• -u, --chars <CHARS>: per‑file output budget (Unicode code points). Behaves like --bytes but counts characters instead of bytes.
• -C, --global-bytes <BYTES>: total output budget across all inputs. With --bytes, the effective total is the smaller of the two.
• -f, --format <auto|json|yaml|text>: output format (default: auto).
  • Auto: stdin → JSON family; filesets → per‑file based on extension (.json → JSON family, .yaml/.yml → YAML, unknown → Text).
• -t, --template <strict|default|detailed>: output style (default: default).
  • JSON family: strict → strict JSON; default → Pseudo; detailed → JS with inline comments.
  • YAML: always YAML; style only affects comments (strict none, default “# …”, detailed “# N more …”).
• -i, --input-format <json|yaml|text>: ingestion format (default: json). For filesets in auto format, ingestion is chosen by extensions.
• -m, --compact: no indentation, no spaces, no newlines
• --no-newline: single line output
• --no-header: suppress fileset section headers (useful when embedding output in scripts)
• --tree: render filesets as a directory tree with inline previews (keeps code line numbers); uses per-file auto formatting.
• --no-space: no space after : in objects
• --indent <STR>: indentation unit (default: two spaces)
• --string-cap <N>: max graphemes to consider per string (default: 500)
• --grep <REGEX>: guarantee inclusion of values/keys/lines matching the regex (ripgrep‑style). Matches + ancestors are “free”; budgets apply to everything else. If matches consume all headroom, only the must‑keep path is shown. Colors follow the normal on/auto/off rules; when grep is active, syntax colors are suppressed and only the match highlights are colored. JSON/YAML structural punctuation is not highlighted—only the matching key/value text.
• --head: prefer the beginning of arrays when truncating (keep first N). Strings are unaffected. Display styles place omission markers accordingly; strict JSON remains unannotated. Mutually exclusive with --tail.
• --tail: prefer the end of arrays when truncating (keep last N). Strings are unaffected. Display styles place omission markers accordingly; strict JSON remains unannotated. Mutually exclusive with --head.

Notes:

• Multiple inputs:
  • With newlines enabled, file sections are rendered with human‑readable headers (pass --no-header to suppress them). In compact/single‑line modes, headers are omitted.
  • Order: inputs are sorted by git frecency (via frecenfile) when available, then by mtime; pass --no-sort to keep the original input order without repo scanning.
  • Fairness: fileset nodes are interleaved round‑robin during selection so tight budgets don’t starve later files.
• In --format auto, each file uses its own best format: JSON family for .json, YAML for .yaml/.yml.
  • Unknown extensions are treated as Text (raw lines) — safe for logs and .txt files.
  • --global-bytes may truncate or omit entire files to respect the total budget.
  • The tool finds the largest preview that fits the budget; even if extremely tight, you still get a minimal, valid preview.
  • Directories and binary files are ignored; a notice is printed to stderr for each. Stdin reads the stream as‑is.
  • Head vs Tail sampling: these options bias which part of arrays are kept before rendering. Display styles may still insert internal gap markers to honor very small budgets; strict JSON stays unannotated.

Working with multiple files

• Budgets: per-file caps (--bytes/--chars/--lines) apply to each input; global caps (--global-*) constrain the combined output. Default byte budget scales by input count when no globals are set.
• Sorting: inputs are pre-sorted by git frecency (frecenfile) with last-modified-time fallback so recently touched files appear first. Pass --no-sort to preserve the order you provided and skip repo scanning.
• Headers: fileset sections get ==> headers when newlines are enabled; hide them with --no-header. Compact and single-line modes omit headers automatically.
• Formats: in --format auto, each file picks JSON/YAML/Text based on extension; unknowns fall back to Text so mixed filesets “just work.”

Grep mode

Use --grep <REGEX> to guarantee inclusion of values/keys/lines matching the regex (ripgrep-style). Matches plus their ancestors are “free” against budgets; everything else must fit the remaining headroom.

• Matching: values/lines are checked; object keys match too, except for filenames in filesets (no filename-only hits).
• Colors: only the matching text is highlighted; syntax colors are suppressed in grep mode. Disable color entirely with --no-color.
• Weak grep: --weak-grep <REGEX> biases priority toward matches but does not guarantee inclusion, expand budgets, or filter filesets. Budgets stay exact and matches can still be pruned if they do not fit.
• Filesets (strong --grep only):
  • Default (--grep-show=matching): files without matches are dropped from the render and summary. If no files match at all, the output is empty and the CLI prints a notice to stderr.
  • --grep-show=all: keep non-matching files in the render; only matching files are highlighted.
  • Headers respect --no-header as usual.
• Mutual exclusion: --grep-show requires --grep and cannot be used with --weak-grep; --weak-grep cannot be combined with --grep.
• Context: there are no explicit -C/-B/-A style flags; per-file budgets decide how much surrounding structure/lines can stay alongside the must-keep matches.
• Budgets: matches and ancestors always render; remaining budget determines what else can appear. Extremely tight budgets may show only the must-keep path.
• Text/code: works with -i text and code-like files; when using --format auto, file extensions still decide ingest/rendering.

Tree mode

Use --tree to render filesets as a directory tree (like tree) with inline structured previews instead of per-file headers. Works with grep/weak-grep; matches are shown inside the tree.

• Layout: classic tree branches (├─, │, └─) with continuous guides; code gutters stay visible under the tree prefix.
• Headers: --tree is mutually exclusive with --no-header; tree mode never prints ==> headers and relies on the tree structure instead. Files are still auto-formatted per extension (--format must be auto for filesets).
• Budgets: tree scaffolding is treated like headers (free unless you set --count-headers); per-file/global budgets still apply to file content and omission markers. Tight budgets can truncate file previews within the tree, and entire files may be omitted under tiny global line budgets—omitted entries are reported as … N more items on the relevant folder/root.
• Sorting: respects --no-sort; otherwise uses the usual frecency/mtime ordering before tree grouping.
• Fairness: file contents are interleaved round‑robin in the priority order so later files still surface under tight budgets.

Budget Modes

• Bytes (-c/--bytes, -C/--global-bytes)

  • Measures UTF‑8 bytes in the output.
  • Default per‑file budget is 500 bytes when neither --lines nor --chars is provided.
  • Multiple inputs: total default budget is <BYTES> * number_of_inputs; --global-bytes caps the total.

• Characters (-u/--chars)

  • Measures Unicode code points (not grapheme clusters).

• Lines (-n/--lines, -N/--global-lines)

  • Caps the number of lines in the output.
  • Incompatible with --no-newline.
  • Multiple inputs: defaults to <LINES> * number_of_inputs; --global-lines caps the total.
  • Fileset headers, blank separators, and summary lines do not count toward the line cap by default; only actual content lines are considered. Pass -H/--count-headers to include headers/summaries in the line budget.

• Interactions and precedence

  • All active budgets are enforced simultaneously. The render must satisfy all of: bytes (if set), chars (if set), and lines (if set). The strictest cap wins.
  • When only lines are specified, no implicit byte cap applies. When neither lines nor chars are specified, a 500‑byte default applies.

Quick one‑liners:

• Peek a big JSON stream (keeps structure):

  zstdcat huge.json.zst | hson -c 800 -f json -t default
  

• Many files with a fixed overall size:

  hson -C 1200 -f json -t strict logs/*.json
  

• Glance at a file, JavaScript‑style comments for omissions:

  hson -c 400 -f json -t detailed data.json
  

• YAML with detailed comments:

  hson -c 400 -f yaml -t detailed config.yaml
  

Text mode

• Single file (auto):

  hson -c 200 notes.txt
  

• Force Text ingest/output (useful when mixing with other extensions, or when the extension suggests JSON/YAML):

  hson -c 200 -i text -f text notes.txt
  # Force text ingest even if the file looks like JSON
  hson -i text notes.json
  

• Styles on Text:

  • default: omission as a standalone … line.
  • detailed: omission as … N more lines ….
  • strict: no array‑level omission line (individual long lines may still truncate with …).

Note: Filesets always render with per-file auto templates. When you need to preview a directory of mixed formats, skip -f text and let -f auto pick the right renderer for each entry.

Show help:

hson --help

Note: flags align with head/tail conventions (-c/--bytes, -C/--global-bytes).

Examples: head vs hson

Input:

{"users":[{"id":1,"name":"Ana","roles":["admin","dev"]},{"id":2,"name":"Bo"}],"meta":{"count":2,"source":"db"}}

Naive cut (can break mid‑token):

jq -c . users.json | head -c 80
# {"users":[{"id":1,"name":"Ana","roles":["admin","dev"]},{"id":2,"name":"Bo"}],"me

Structured preview with hson (JSON family, default style → Pseudo):

hson -c 120 -f json -t default users.json
# {
#   users: [
#     { id: 1, name: "Ana", roles: [ "admin", … ] },
#     …
#   ]
#   meta: { count: 2, … }
# }

Machine‑readable preview (JSON family, strict style → strict JSON):

hson -c 120 -f json -t strict users.json
# {"users":[{"id":1,"name":"Ana","roles":["admin"]}],"meta":{"count":2}}

Terminal Demos

Regenerate locally:

• Place tapes under docs/tapes (e.g., docs/tapes/demo.tape)
• Run: cargo make tapes
• Outputs are written to docs/assets/tapes

Python Bindings

A thin Python extension module is available on PyPI as headson.

• Install: pip install headson (ABI3 wheels for Python 3.10+ on Linux/macOS/Windows).
• API:
  • headson.summarize(text: str, *, format: str = "auto", style: str = "default", input_format: str = "json", byte_budget: int | None = None, skew: str = "balanced") -> str
    • format: "auto" | "json" | "yaml" (auto maps to JSON family for single inputs)
    • style: "strict" | "default" | "detailed"
    • input_format: "json" | "yaml" (ingestion)
    • byte_budget: maximum output size in bytes (default: 500)
    • skew: "balanced" | "head" | "tail" (affects display styles; strict JSON remains unannotated)

Examples:

import json
import headson

data = {"foo": [1, 2, 3], "bar": {"x": "y"}}
preview = headson.summarize(json.dumps(data), format="json", style="strict", byte_budget=200)
print(preview)

# Prefer the tail of arrays (annotations show with style="default"/"detailed")
print(
    headson.summarize(
        json.dumps(list(range(100))),
        format="json",
        style="detailed",
        byte_budget=80,
        skew="tail",
    )
)

# YAML support
doc = "root:\n  items: [1,2,3,4,5,6,7,8,9,10]\n"
print(headson.summarize(doc, format="yaml", style="default", input_format="yaml", byte_budget=60))

Source Code Support

Source code support is a challenging area. While headson's algorithm and code structure would allow for the use of completely accurate parsing using language-specific parsers using tree-sitter, this would increase the complexity of the application and its number of dependencies.

Instead of attempting a deep parse of source code files, we convert them into nested arrays based on a heuristic that understands indentation patterns in the file.

When headson detects a code-like file, it uses a set of additional heuristics:

• Atomic line ingest: each line is treated as an atomic string so omission markers never split a code line.
• Depth-aware sampling:
  • We attempt to include more of the top level of the source code in order to give a good overview of classes, function and constants at the top level.
  • Nested blocks (function bodies, loops) prefer to omit lines in the middle to attempt to preserve natural "block" boundaries
• Header priority: lines that introduce a nested block (e.g., def foo():) get a small priority boost to ensure they survive tight budgets.

Algorithm

Footnotes

• ^[1] Optimized tree representation: An arena‑style tree stored in flat, contiguous buffers. Each node records its kind and value plus index ranges into shared child and key arrays. Arrays are ingested in a single pass and may be deterministically pre‑sampled: the first element is always kept; additional elements are selected via a fixed per‑index inclusion test; for kept elements, original indices are stored and full lengths are counted. This enables accurate omission info and internal gap markers later, while minimizing pointer chasing.
• ^[2] Priority order: Nodes are scored so previews surface representative structure and values first. Arrays can favor head/mid/tail coverage (default) or strictly the head; tail preference flips head/tail when configured. Object properties are ordered by key, and strings expand by grapheme with early characters prioritized over very deep expansions.
• ^[3] Choose top N nodes (binary search): Iteratively picks N so that the rendered preview fits within the byte budget, looping between “choose N” and a render attempt to converge quickly.
• ^[4] Render attempt: Serializes the currently included nodes using the selected template. Omission summaries and per-file section headers appear in display templates (pseudo/js); json remains strict. For arrays, display templates may insert internal gap markers between non‑contiguous kept items using original indices.
• ^[5] Diagram source: The Algorithm diagram is generated from docs/diagrams/algorithm.mmd. Regenerate the SVG with cargo make diagrams before releasing.

License

MIT