diff --git a/README.md b/README.md index a1358ac..954abc6 100644 --- a/README.md +++ b/README.md @@ -136,6 +136,26 @@ hikes[3]{id,name,distanceKm,elevationGain,companion,wasSunny}: - 🧺 **Tabular Arrays:** Uniform arrays of objects collapse into tables that declare fields once and stream row values line by line. - 🌐 **Multi-Language Ecosystem:** Spec-driven implementations in TypeScript, Python, Go, Rust, .NET, and other languages. +## Media Type & File Extension + +For HTTP and content-type–aware contexts, TOON defines: + +- **Media type (provisional):** `text/toon` +- **Charset:** Always UTF-8 (`charset=utf-8` MAY be specified; if omitted, UTF-8 MUST be assumed) +- **File extension:** `.toon` + +Example HTTP usage: + +```http +GET /resource HTTP/1.1 +Accept: text/toon + +HTTP/1.1 200 OK +Content-Type: text/toon +``` + +See [SPEC.md §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type) for details. + ## When Not to Use TOON TOON excels with uniform arrays of objects, but there are cases where other formats are better: @@ -889,17 +909,20 @@ Follow the detailed [LLM integration guide](https://toonformat.dev/guide/llm-pro Comprehensive guides, references, and resources to help you get the most out of the TOON format and tools. -**Getting Started** +**Getting Started:** + - [Introduction & Installation](https://toonformat.dev/guide/getting-started) – What TOON is, when to use it, first steps - [Format Overview](https://toonformat.dev/guide/format-overview) – Complete syntax with examples - [Benchmarks](https://toonformat.dev/guide/benchmarks) – Accuracy & token efficiency results -**Tools & Integration** +**Tools & Integration:** + - [CLI](https://toonformat.dev/cli/) – Command-line tool for JSON↔TOON conversions - [Using TOON with LLMs](https://toonformat.dev/guide/llm-prompts) – Prompting strategies & validation - [Playgrounds](https://toonformat.dev/ecosystem/tools-and-playgrounds) – Interactive tools -**Reference** +**Reference:** + - [API Reference](https://toonformat.dev/reference/api) – TypeScript/JavaScript encode/decode API - [Syntax Cheatsheet](https://toonformat.dev/reference/syntax-cheatsheet) – Quick format lookup - [Specification v2.0](https://github.com/toon-format/spec/blob/main/SPEC.md) – Normative rules for implementers @@ -907,7 +930,7 @@ Comprehensive guides, references, and resources to help you get the most out of ## Other Implementations > [!NOTE] -> When implementing TOON in other languages, please follow the [specification](https://github.com/toon-format/spec/blob/main/SPEC.md) (currently v2.0) to ensure compatibility across implementations. The [conformance tests](https://github.com/toon-format/spec/tree/main/tests) provide language-agnostic test fixtures that validate your implementations. +> When implementing TOON in other languages, please follow the [Specification](https://github.com/toon-format/spec/blob/main/SPEC.md) (currently v2.0) to ensure compatibility across implementations. The [conformance tests](https://github.com/toon-format/spec/tree/main/tests) provide language-agnostic test fixtures that validate your implementations. ### Official Implementations diff --git a/docs/cli/index.md b/docs/cli/index.md index 7001de7..9d1dcee 100644 --- a/docs/cli/index.md +++ b/docs/cli/index.md @@ -87,6 +87,8 @@ cat data.toon | toon --decode ::: +By convention, TOON files use the `.toon` extension and the provisional media type `text/toon` (see [spec §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type)). + ### Standard Input Omit the input argument or use `-` to read from stdin. This enables piping data directly from other commands: diff --git a/docs/ecosystem/implementations.md b/docs/ecosystem/implementations.md index 85ae664..acf6f1e 100644 --- a/docs/ecosystem/implementations.md +++ b/docs/ecosystem/implementations.md @@ -1,6 +1,6 @@ # Implementations -TOON has official and community implementations across multiple programming languages. All implementations are intended to conform to the same [specification](https://github.com/toon-format/spec) to ensure compatibility and interoperability. +TOON has official and community implementations across multiple programming languages. All implementations are intended to conform to the same [Specification](https://github.com/toon-format/spec) to ensure compatibility and interoperability. The code examples throughout this documentation site use the TypeScript implementation by default, but the format and concepts apply equally to all languages. diff --git a/docs/ecosystem/tools-and-playgrounds.md b/docs/ecosystem/tools-and-playgrounds.md index 694de22..058be20 100644 --- a/docs/ecosystem/tools-and-playgrounds.md +++ b/docs/ecosystem/tools-and-playgrounds.md @@ -53,4 +53,4 @@ const toon = encode(data) const data = decode(toon) ``` -See the [API reference](/reference/api) for details. +See the [API Reference](/reference/api) for details. diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index cedd77b..d145da6 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -234,8 +234,20 @@ console.log(JSON.stringify(data, null, 2)) Round-tripping is lossless: `decode(encode(x))` always equals `x` (after normalization of non-JSON types like `Date`, `NaN`, etc.). +## Media Type & File Extension + +When using TOON over HTTP or in content-type–aware systems: + +- Use `text/toon` as the media type (provisional; see [spec §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type)) +- Use `.toon` as the standard file extension +- TOON is always UTF-8 encoded; `charset=utf-8` is optional + +Example: + +```http +Content-Type: text/toon; charset=utf-8 +``` + ## Where to Go Next -Now that you've seen your first TOON document, read the [Format Overview](/guide/format-overview) for complete syntax details (objects, arrays, quoting rules, key folding), then explore [Using TOON with LLMs](/guide/llm-prompts) to see how to use it effectively in prompts. For implementation details, check the [API reference](/reference/api) (TypeScript) or the [specification](/reference/spec) (language-agnostic normative rules). - -For large datasets or streaming use-cases, see `encodeLines`, `decodeFromLines`, and `decodeStream` in the [API reference](/reference/api). +Now that you've seen your first TOON document, read the [Format Overview](/guide/format-overview) for complete syntax details (objects, arrays, quoting rules, key folding), then explore [Using TOON with LLMs](/guide/llm-prompts) to see how to use it effectively in prompts. For implementation details, check the [API Reference](/reference/api) (TypeScript) or the [Specification](/reference/spec) (language-agnostic normative rules). diff --git a/docs/guide/llm-prompts.md b/docs/guide/llm-prompts.md index 7ab2d30..3696a50 100644 --- a/docs/guide/llm-prompts.md +++ b/docs/guide/llm-prompts.md @@ -83,7 +83,7 @@ catch (error) { } ``` -Strict mode checks counts, indentation, and escaping so you can detect truncation or malformed TOON. For complete details, see the [API reference](/reference/api#decode). +Strict mode checks counts, indentation, and escaping so you can detect truncation or malformed TOON. For complete details, see the [API Reference](/reference/api#decode). ## Delimiter Choices for Token Efficiency @@ -116,7 +116,7 @@ The CLI also supports streaming for memory-efficient JSON-to-TOON conversion: toon large-dataset.json --output output.toon ``` -This streaming approach prevents out-of-memory errors when preparing large context windows for LLMs. For complete details on `encodeLines()`, see the [API reference](/reference/api#encodelines). +This streaming approach prevents out-of-memory errors when preparing large context windows for LLMs. For complete details on `encodeLines()`, see the [API Reference](/reference/api#encodelines). **Consuming streaming LLM outputs:** If your LLM client exposes streaming text and you buffer by lines, you can decode TOON incrementally: diff --git a/docs/reference/spec.md b/docs/reference/spec.md index 55721cd..825e86e 100644 --- a/docs/reference/spec.md +++ b/docs/reference/spec.md @@ -11,6 +11,11 @@ You don't need this page to *use* TOON. It's mainly for implementers and contrib **Spec v{{ $spec.version }}** (2025-11-10) is the current stable version. +The spec defines a provisional media type and file extension in §18.2: + +- **Media type:** `text/toon` (provisional, UTF-8 only) +- **File extension:** `.toon` + ## Guided Tour of the Spec ### Core Concepts diff --git a/docs/reference/syntax-cheatsheet.md b/docs/reference/syntax-cheatsheet.md index 647ccd9..c3eb878 100644 --- a/docs/reference/syntax-cheatsheet.md +++ b/docs/reference/syntax-cheatsheet.md @@ -1,6 +1,6 @@ # Syntax Cheatsheet -Quick reference for mapping JSON to TOON format. For rigorous, normative syntax rules and edge cases, see the [specification](/reference/spec). +Quick reference for mapping JSON to TOON format. For rigorous, normative syntax rules and edge cases, see the [Specification](/reference/spec). ## Objects