docs: clarify spec v1.5 changes

This commit is contained in:
Johann Schopplich
2025-11-10 13:21:25 +01:00
parent ac17a8d260
commit 661dcbe5f8
2 changed files with 14 additions and 14 deletions

View File

@@ -80,7 +80,7 @@ See [benchmarks](#benchmarks) for concrete comparisons across different data str
- 🍱 **Minimal syntax:** removes redundant punctuation (braces, brackets, most quotes)
- 📐 **Indentation-based structure:** like YAML, uses whitespace instead of braces
- 🧺 **Tabular arrays:** declare keys once, stream data as rows
- 🔗 **Optional key folding (v1.5):** collapses single-key wrapper chains into dotted paths (e.g., `data.metadata.items`) to reduce indentation and tokens
- 🔗 **Optional key folding (spec v1.5):** collapses single-key wrapper chains into dotted paths (e.g., `data.metadata.items`) to reduce indentation and tokens
[^1]: For flat tabular data, CSV is more compact. TOON adds minimal overhead to provide explicit structure and validation that improves LLM reliability.
@@ -737,9 +737,9 @@ cat data.toon | npx @toon-format/cli --decode
| `--length-marker` | Add `#` prefix to array lengths (e.g., `items[#3]`) |
| `--stats` | Show token count estimates and savings (encode only) |
| `--no-strict` | Disable strict validation when decoding |
| `--key-folding <mode>` | Key folding mode: `off`, `safe` (default: `off`) - collapses nested chains (v1.5) |
| `--flatten-depth <number>` | Maximum segments to fold (default: `Infinity`) - requires `--key-folding safe` (v1.5) |
| `--expand-paths <mode>` | Path expansion mode: `off`, `safe` (default: `off`) - reconstructs dotted keys (v1.5) |
| `--key-folding <mode>` | Key folding mode: `off`, `safe` (default: `off`) - collapses nested chains |
| `--flatten-depth <number>` | Maximum segments to fold (default: `Infinity`) - requires `--key-folding safe` |
| `--expand-paths <mode>` | Path expansion mode: `off`, `safe` (default: `off`) - reconstructs dotted keys |
### Examples
@@ -756,7 +756,7 @@ npx @toon-format/cli data.json --delimiter "|" --length-marker -o output.toon
# Lenient decoding (skip validation)
npx @toon-format/cli data.toon --no-strict -o output.json
# Key folding for nested data (v1.5)
# Key folding for nested data (spec v1.5)
npx @toon-format/cli data.json --key-folding safe -o output.toon
# Stdin workflows
@@ -806,7 +806,7 @@ user:
### Key Folding (Optional)
New in v1.5: Optionally collapse single-key wrapper chains into dotted paths to reduce tokens. Enable with `keyFolding: 'safe'`.
New in spec v1.5: Optionally collapse single-key wrapper chains into dotted paths to reduce tokens. Enable with `keyFolding: 'safe'`.
Standard nesting:
@@ -1016,8 +1016,8 @@ Converts any JSON-serializable value to TOON format.
- `indent?: number` Number of spaces per indentation level (default: `2`)
- `delimiter?: ',' | '\t' | '|'` Delimiter for array values and tabular rows (default: `','`)
- `lengthMarker?: '#' | false` Optional marker to prefix array lengths (default: `false`)
- `keyFolding?: 'off' | 'safe'` Enable key folding to collapse single-key wrapper chains into dotted paths (default: `'off'`). When `'safe'`, only valid identifier segments are folded (v1.5)
- `flattenDepth?: number` Maximum number of segments to fold when `keyFolding` is enabled (default: `Infinity`). Values 0-1 have no practical effect (v1.5)
- `keyFolding?: 'off' | 'safe'` Enable key folding to collapse single-key wrapper chains into dotted paths (default: `'off'`). When `'safe'`, only valid identifier segments are folded
- `flattenDepth?: number` Maximum number of segments to fold when `keyFolding` is enabled (default: `Infinity`). Values 0-1 have no practical effect
**Returns:**
@@ -1139,7 +1139,7 @@ Converts a TOON-formatted string back to JavaScript values.
- `options` Optional decoding options:
- `indent?: number` Expected number of spaces per indentation level (default: `2`)
- `strict?: boolean` Enable strict validation (default: `true`)
- `expandPaths?: 'off' | 'safe'` Enable path expansion to reconstruct dotted keys into nested objects (default: `'off'`). Pairs with `keyFolding: 'safe'` for lossless round-trips (v1.5)
- `expandPaths?: 'off' | 'safe'` Enable path expansion to reconstruct dotted keys into nested objects (default: `'off'`). Pairs with `keyFolding: 'safe'` for lossless round-trips
**Returns:**

View File

@@ -65,9 +65,9 @@ cat data.toon | toon --decode
| `--length-marker` | Add `#` prefix to array lengths (e.g., `items[#3]`) |
| `--stats` | Show token count estimates and savings (encode only) |
| `--no-strict` | Disable strict validation when decoding |
| `--key-folding <mode>` | Enable key folding: `off`, `safe` (default: `off`) - v1.5 |
| `--flatten-depth <number>` | Maximum folded segment count when key folding is enabled (default: `Infinity`) - v1.5 |
| `--expand-paths <mode>` | Enable path expansion: `off`, `safe` (default: `off`) - v1.5 |
| `--key-folding <mode>` | Enable key folding: `off`, `safe` (default: `off`) |
| `--flatten-depth <number>` | Maximum folded segment count when key folding is enabled (default: `Infinity`) |
| `--expand-paths <mode>` | Enable path expansion: `off`, `safe` (default: `off`) |
## Advanced Examples
@@ -122,7 +122,7 @@ cat large-dataset.json | toon --delimiter "\t" > output.toon
jq '.results' data.json | toon > filtered.toon
```
### Key Folding (v1.5)
### Key Folding (spec v1.5)
Collapse nested wrapper chains to reduce tokens:
@@ -196,7 +196,7 @@ toon data.json --key-folding safe --delimiter "\t" --stats -o output.toon
- **Token analysis** to see potential savings before sending to LLMs
- **Pipeline integration** with existing JSON-based workflows
- **Flexible formatting** with delimiter and indentation options
- **Key folding (v1.5)** to collapse nested wrappers for additional token savings
- **Key folding** to collapse nested wrappers for additional token savings
## Related