From c2b0e3f40497d5803f7d8c04c25862197deed05d Mon Sep 17 00:00:00 2001 From: Johann Schopplich Date: Mon, 27 Oct 2025 20:02:05 +0100 Subject: [PATCH] docs: add format comparison --- README.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 1136970..73163c7 100644 --- a/README.md +++ b/README.md @@ -4,11 +4,7 @@ **Token-Oriented Object Notation** is a compact, human-readable format designed for passing structured data to Large Language Models with significantly reduced token usage. -In other words, if YAML and CSV had a baby, optimized for LLM contexts. -TOON borrows YAML's indentation-based structure for nested objects and CSV's tabular format for uniform data rows, then optimizes both for token efficiency in LLM contexts. - -> [!TIP] -> Wrap your JSON in `encode()` before sending it to LLMs and save ~1/2 of the token cost for structured data! +TOON excels at **uniform complex objects** – multiple fields per row, same structure across items. It borrows YAML's indentation-based structure for nested objects and CSV's tabular format for uniform data rows, then optimizes both for token efficiency in LLM contexts. ## Why TOON? @@ -31,8 +27,6 @@ users[2]{id,name,role}: 2,Bob,user ``` -I built TOON to save tokens when sending large datasets to LLMs at work, where I tend to have uniform arrays of objects that benefit from the tabular format. -
Another reason @@ -40,6 +34,16 @@ I built TOON to save tokens when sending large datasets to LLMs at work, where I
+## Format Comparison + +Format familiarity matters as much as token count. + +- **CSV:** best for uniform tables. +- **JSON:** best for non-uniform data. +- **TOON:** best for uniform complex (but not deeply nested) objects. + +TOON switches to list format for non-uniform arrays. In those cases, JSON can be cheaper at scale. + ## Key Features - 💸 **Token-efficient:** typically 30–60% fewer tokens than JSON