mirror of
https://github.com/voson-wang/toon.git
synced 2026-01-29 23:34:10 +08:00
chore: upgrade key features
This commit is contained in:
21
README.md
21
README.md
@@ -12,7 +12,7 @@ AI is becoming cheaper and more accessible, but larger context windows allow for
|
|||||||
- Writing inline primitive arrays without spaces
|
- Writing inline primitive arrays without spaces
|
||||||
|
|
||||||
> [!TIP]
|
> [!TIP]
|
||||||
> Wrap your JSON in `encode()` before sending it to LLMs and save ~1/2 of the token cost!
|
> Wrap your JSON in `encode()` before sending it to LLMs and save ~1/2 of the token cost for structured data!
|
||||||
|
|
||||||
## Token Benchmarks
|
## Token Benchmarks
|
||||||
|
|
||||||
@@ -205,12 +205,11 @@ users[2]{id,name,role}:
|
|||||||
|
|
||||||
## Key Features
|
## Key Features
|
||||||
|
|
||||||
- 📉 **Token-efficient:** typically 30–60% fewer tokens vs JSON on GPT-style tokenizers
|
- 💸 **Token-efficient:** typically 30–60% fewer tokens vs JSON on GPT-style tokenizers, based on real benchmarks
|
||||||
- 📊 **Tabular arrays:** write object keys once, list rows beneath
|
- 🎛️ **Deterministic, tokenizer-aware output:** minimal quoting and stable ordering keep payloads compact and reproducible
|
||||||
- ✂️ **Minimal quoting:** only when required (e.g., commas, colons, ambiguous primitives)
|
- 🧺 **Tabular arrays without repetition:** declare uniform keys once, then stream rows for dense datasets
|
||||||
- 📐 **Indentation-based structure:** no braces/brackets for objects
|
- 📐 **Readable yet concise structure:** indentation replaces braces so nested data stays scannable without extra tokens
|
||||||
- 🎯 **Inline primitive arrays:** written without spaces after commas
|
- 🔢 **LLM-friendly guardrails:** explicit lengths and field lists help models validate and reproduce structured responses
|
||||||
- 🎲 **Deterministic:** stable key order, no trailing spaces/newline
|
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
@@ -576,13 +575,13 @@ items[2]{sku,name,qty,price}:
|
|||||||
|
|
||||||
**Benefits:**
|
**Benefits:**
|
||||||
|
|
||||||
- Tabs are single characters and often tokenize more efficiently than commas
|
- Tabs are single characters and often tokenize more efficiently than commas.
|
||||||
- Tabs rarely appear in natural text, reducing the need for quote-escaping
|
- Tabs rarely appear in natural text, reducing the need for quote-escaping.
|
||||||
|
|
||||||
**Considerations:**
|
**Considerations:**
|
||||||
|
|
||||||
- Some terminals and editors may collapse or expand tabs visually
|
- Some terminals and editors may collapse or expand tabs visually.
|
||||||
- String values containing tabs will still require quoting
|
- String values containing tabs will still require quoting.
|
||||||
|
|
||||||
##### Pipe Delimiter (`|`)
|
##### Pipe Delimiter (`|`)
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user