github/toon
1
0
Fork 0
mirror of https://github.com/voson-wang/toon.git synced 2026-01-29 23:34:10 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: streaming decode functionality with event-based parsing (closes #131)

Browse Source
This commit is contained in:
Johann Schopplich
2025-11-21 22:29:57 +01:00
parent 9ebad53ea3
commit 6c57a14009
19 changed files with 2220 additions and 431 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
docs/cli/index.md
View File

@@ -108,19 +108,25 @@ cat data.toon | toon --decode
Both encoding and decoding operations use streaming output, writing incrementally without building the full output string in memory. This makes the CLI efficient for large datasets without requiring additional configuration.
**JSON → TOON (Encode)**
- Streams TOON lines to output
- No full TOON string in memory
**JSON → TOON (Encode)**:
**TOON → JSON (Decode)**
- Streams JSON tokens to output
- No full JSON string in memory
- Streams TOON lines to output.
- No full TOON string in memory.
**TOON → JSON (Decode)**:
- Uses the same event-based streaming decoder as the `decodeStream` API in `@toon-format/toon`.
- Streams JSON tokens to output.
- No full JSON string in memory.
- When `--expand-paths safe` is enabled, falls back to non-streaming decode internally to apply deep-merge expansion before writing JSON.
Process large files with minimal memory usage:
```bash
# Encode large JSON file with minimal memory usage
# Encode large JSON file
toon huge-dataset.json -o output.toon
# Decode large TOON file with minimal memory usage
# Decode large TOON file
toon huge-dataset.toon -o output.json
# Process millions of records efficiently via stdin