mirror of
https://github.com/voson-wang/toon.git
synced 2026-01-29 15:24:10 +08:00
docs: clarify spec v1.5 changes
This commit is contained in:
18
README.md
18
README.md
@@ -80,7 +80,7 @@ See [benchmarks](#benchmarks) for concrete comparisons across different data str
|
|||||||
- 🍱 **Minimal syntax:** removes redundant punctuation (braces, brackets, most quotes)
|
- 🍱 **Minimal syntax:** removes redundant punctuation (braces, brackets, most quotes)
|
||||||
- 📐 **Indentation-based structure:** like YAML, uses whitespace instead of braces
|
- 📐 **Indentation-based structure:** like YAML, uses whitespace instead of braces
|
||||||
- 🧺 **Tabular arrays:** declare keys once, stream data as rows
|
- 🧺 **Tabular arrays:** declare keys once, stream data as rows
|
||||||
- 🔗 **Optional key folding (v1.5):** collapses single-key wrapper chains into dotted paths (e.g., `data.metadata.items`) to reduce indentation and tokens
|
- 🔗 **Optional key folding (spec v1.5):** collapses single-key wrapper chains into dotted paths (e.g., `data.metadata.items`) to reduce indentation and tokens
|
||||||
|
|
||||||
[^1]: For flat tabular data, CSV is more compact. TOON adds minimal overhead to provide explicit structure and validation that improves LLM reliability.
|
[^1]: For flat tabular data, CSV is more compact. TOON adds minimal overhead to provide explicit structure and validation that improves LLM reliability.
|
||||||
|
|
||||||
@@ -737,9 +737,9 @@ cat data.toon | npx @toon-format/cli --decode
|
|||||||
| `--length-marker` | Add `#` prefix to array lengths (e.g., `items[#3]`) |
|
| `--length-marker` | Add `#` prefix to array lengths (e.g., `items[#3]`) |
|
||||||
| `--stats` | Show token count estimates and savings (encode only) |
|
| `--stats` | Show token count estimates and savings (encode only) |
|
||||||
| `--no-strict` | Disable strict validation when decoding |
|
| `--no-strict` | Disable strict validation when decoding |
|
||||||
| `--key-folding <mode>` | Key folding mode: `off`, `safe` (default: `off`) - collapses nested chains (v1.5) |
|
| `--key-folding <mode>` | Key folding mode: `off`, `safe` (default: `off`) - collapses nested chains |
|
||||||
| `--flatten-depth <number>` | Maximum segments to fold (default: `Infinity`) - requires `--key-folding safe` (v1.5) |
|
| `--flatten-depth <number>` | Maximum segments to fold (default: `Infinity`) - requires `--key-folding safe` |
|
||||||
| `--expand-paths <mode>` | Path expansion mode: `off`, `safe` (default: `off`) - reconstructs dotted keys (v1.5) |
|
| `--expand-paths <mode>` | Path expansion mode: `off`, `safe` (default: `off`) - reconstructs dotted keys |
|
||||||
|
|
||||||
### Examples
|
### Examples
|
||||||
|
|
||||||
@@ -756,7 +756,7 @@ npx @toon-format/cli data.json --delimiter "|" --length-marker -o output.toon
|
|||||||
# Lenient decoding (skip validation)
|
# Lenient decoding (skip validation)
|
||||||
npx @toon-format/cli data.toon --no-strict -o output.json
|
npx @toon-format/cli data.toon --no-strict -o output.json
|
||||||
|
|
||||||
# Key folding for nested data (v1.5)
|
# Key folding for nested data (spec v1.5)
|
||||||
npx @toon-format/cli data.json --key-folding safe -o output.toon
|
npx @toon-format/cli data.json --key-folding safe -o output.toon
|
||||||
|
|
||||||
# Stdin workflows
|
# Stdin workflows
|
||||||
@@ -806,7 +806,7 @@ user:
|
|||||||
|
|
||||||
### Key Folding (Optional)
|
### Key Folding (Optional)
|
||||||
|
|
||||||
New in v1.5: Optionally collapse single-key wrapper chains into dotted paths to reduce tokens. Enable with `keyFolding: 'safe'`.
|
New in spec v1.5: Optionally collapse single-key wrapper chains into dotted paths to reduce tokens. Enable with `keyFolding: 'safe'`.
|
||||||
|
|
||||||
Standard nesting:
|
Standard nesting:
|
||||||
|
|
||||||
@@ -1016,8 +1016,8 @@ Converts any JSON-serializable value to TOON format.
|
|||||||
- `indent?: number` – Number of spaces per indentation level (default: `2`)
|
- `indent?: number` – Number of spaces per indentation level (default: `2`)
|
||||||
- `delimiter?: ',' | '\t' | '|'` – Delimiter for array values and tabular rows (default: `','`)
|
- `delimiter?: ',' | '\t' | '|'` – Delimiter for array values and tabular rows (default: `','`)
|
||||||
- `lengthMarker?: '#' | false` – Optional marker to prefix array lengths (default: `false`)
|
- `lengthMarker?: '#' | false` – Optional marker to prefix array lengths (default: `false`)
|
||||||
- `keyFolding?: 'off' | 'safe'` – Enable key folding to collapse single-key wrapper chains into dotted paths (default: `'off'`). When `'safe'`, only valid identifier segments are folded (v1.5)
|
- `keyFolding?: 'off' | 'safe'` – Enable key folding to collapse single-key wrapper chains into dotted paths (default: `'off'`). When `'safe'`, only valid identifier segments are folded
|
||||||
- `flattenDepth?: number` – Maximum number of segments to fold when `keyFolding` is enabled (default: `Infinity`). Values 0-1 have no practical effect (v1.5)
|
- `flattenDepth?: number` – Maximum number of segments to fold when `keyFolding` is enabled (default: `Infinity`). Values 0-1 have no practical effect
|
||||||
|
|
||||||
**Returns:**
|
**Returns:**
|
||||||
|
|
||||||
@@ -1139,7 +1139,7 @@ Converts a TOON-formatted string back to JavaScript values.
|
|||||||
- `options` – Optional decoding options:
|
- `options` – Optional decoding options:
|
||||||
- `indent?: number` – Expected number of spaces per indentation level (default: `2`)
|
- `indent?: number` – Expected number of spaces per indentation level (default: `2`)
|
||||||
- `strict?: boolean` – Enable strict validation (default: `true`)
|
- `strict?: boolean` – Enable strict validation (default: `true`)
|
||||||
- `expandPaths?: 'off' | 'safe'` – Enable path expansion to reconstruct dotted keys into nested objects (default: `'off'`). Pairs with `keyFolding: 'safe'` for lossless round-trips (v1.5)
|
- `expandPaths?: 'off' | 'safe'` – Enable path expansion to reconstruct dotted keys into nested objects (default: `'off'`). Pairs with `keyFolding: 'safe'` for lossless round-trips
|
||||||
|
|
||||||
**Returns:**
|
**Returns:**
|
||||||
|
|
||||||
|
|||||||
@@ -65,9 +65,9 @@ cat data.toon | toon --decode
|
|||||||
| `--length-marker` | Add `#` prefix to array lengths (e.g., `items[#3]`) |
|
| `--length-marker` | Add `#` prefix to array lengths (e.g., `items[#3]`) |
|
||||||
| `--stats` | Show token count estimates and savings (encode only) |
|
| `--stats` | Show token count estimates and savings (encode only) |
|
||||||
| `--no-strict` | Disable strict validation when decoding |
|
| `--no-strict` | Disable strict validation when decoding |
|
||||||
| `--key-folding <mode>` | Enable key folding: `off`, `safe` (default: `off`) - v1.5 |
|
| `--key-folding <mode>` | Enable key folding: `off`, `safe` (default: `off`) |
|
||||||
| `--flatten-depth <number>` | Maximum folded segment count when key folding is enabled (default: `Infinity`) - v1.5 |
|
| `--flatten-depth <number>` | Maximum folded segment count when key folding is enabled (default: `Infinity`) |
|
||||||
| `--expand-paths <mode>` | Enable path expansion: `off`, `safe` (default: `off`) - v1.5 |
|
| `--expand-paths <mode>` | Enable path expansion: `off`, `safe` (default: `off`) |
|
||||||
|
|
||||||
## Advanced Examples
|
## Advanced Examples
|
||||||
|
|
||||||
@@ -122,7 +122,7 @@ cat large-dataset.json | toon --delimiter "\t" > output.toon
|
|||||||
jq '.results' data.json | toon > filtered.toon
|
jq '.results' data.json | toon > filtered.toon
|
||||||
```
|
```
|
||||||
|
|
||||||
### Key Folding (v1.5)
|
### Key Folding (spec v1.5)
|
||||||
|
|
||||||
Collapse nested wrapper chains to reduce tokens:
|
Collapse nested wrapper chains to reduce tokens:
|
||||||
|
|
||||||
@@ -196,7 +196,7 @@ toon data.json --key-folding safe --delimiter "\t" --stats -o output.toon
|
|||||||
- **Token analysis** to see potential savings before sending to LLMs
|
- **Token analysis** to see potential savings before sending to LLMs
|
||||||
- **Pipeline integration** with existing JSON-based workflows
|
- **Pipeline integration** with existing JSON-based workflows
|
||||||
- **Flexible formatting** with delimiter and indentation options
|
- **Flexible formatting** with delimiter and indentation options
|
||||||
- **Key folding (v1.5)** to collapse nested wrappers for additional token savings
|
- **Key folding** to collapse nested wrappers for additional token savings
|
||||||
|
|
||||||
## Related
|
## Related
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user