From 27c6fc55193199fa62e6b117ae2cb4300415ad27 Mon Sep 17 00:00:00 2001 From: Johann Schopplich Date: Sun, 23 Nov 2025 19:05:04 +0100 Subject: [PATCH] docs: simplify stream decode intructions --- README.md | 65 ++++------------------------------- docs/guide/getting-started.md | 18 +++------- 2 files changed, 10 insertions(+), 73 deletions(-) diff --git a/README.md b/README.md index 954abc6..9c5eaa1 100644 --- a/README.md +++ b/README.md @@ -138,23 +138,7 @@ hikes[3]{id,name,distanceKm,elevationGain,companion,wasSunny}: ## Media Type & File Extension -For HTTP and content-type–aware contexts, TOON defines: - -- **Media type (provisional):** `text/toon` -- **Charset:** Always UTF-8 (`charset=utf-8` MAY be specified; if omitted, UTF-8 MUST be assumed) -- **File extension:** `.toon` - -Example HTTP usage: - -```http -GET /resource HTTP/1.1 -Accept: text/toon - -HTTP/1.1 200 OK -Content-Type: text/toon -``` - -See [SPEC.md §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type) for details. +By convention, TOON files use the `.toon` extension and the provisional media type `text/toon` for HTTP and content-type–aware contexts. TOON documents are always UTF-8 encoded; the `charset=utf-8` parameter may be specified but defaults to UTF-8 when omitted. See [SPEC.md §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type) for normative details. ## When Not to Use TOON @@ -797,45 +781,8 @@ for (const line of encodeLines(largeData)) { } ``` -**Streaming decode:** - -```ts -import { decodeFromLines, decodeStreamSync } from '@toon-format/toon' - -// 1. Lines → value (build full JSON value) -const value = decodeFromLines([ - 'users[2]{id,name}:', - ' 1,Alice', - ' 2,Bob', -]) -// { users: [{ id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }] } - -// 2. Lines → events (for custom streaming consumers) -const lines = [ - 'users[2]{id,name}:', - ' 1,Alice', - ' 2,Bob', -] -for (const event of decodeStreamSync(lines)) { - // { type: 'startObject' }, { type: 'key', key: 'users' }, ... -} -``` - -**Async streaming decode:** - -```ts -// 3. Async streaming from files or network -import { createReadStream } from 'node:fs' -import { createInterface } from 'node:readline' -import { decodeStream } from '@toon-format/toon' - -const fileStream = createReadStream('data.toon', 'utf-8') -const rl = createInterface({ input: fileStream }) - -for await (const event of decodeStream(rl)) { - // Process events as they arrive -} -``` +> [!TIP] +> For streaming decode APIs, see [`decodeFromLines()`](/reference/api#decodeFromLines-lines-options) and [`decodeStream()`](/reference/api#decodeStream-source-options). ## Playgrounds @@ -909,19 +856,19 @@ Follow the detailed [LLM integration guide](https://toonformat.dev/guide/llm-pro Comprehensive guides, references, and resources to help you get the most out of the TOON format and tools. -**Getting Started:** +### Getting Started - [Introduction & Installation](https://toonformat.dev/guide/getting-started) – What TOON is, when to use it, first steps - [Format Overview](https://toonformat.dev/guide/format-overview) – Complete syntax with examples - [Benchmarks](https://toonformat.dev/guide/benchmarks) – Accuracy & token efficiency results -**Tools & Integration:** +### Tools & Integration - [CLI](https://toonformat.dev/cli/) – Command-line tool for JSON↔TOON conversions - [Using TOON with LLMs](https://toonformat.dev/guide/llm-prompts) – Prompting strategies & validation - [Playgrounds](https://toonformat.dev/ecosystem/tools-and-playgrounds) – Interactive tools -**Reference:** +### References - [API Reference](https://toonformat.dev/reference/api) – TypeScript/JavaScript encode/decode API - [Syntax Cheatsheet](https://toonformat.dev/reference/syntax-cheatsheet) – Quick format lookup diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md index d145da6..b9f0040 100644 --- a/docs/guide/getting-started.md +++ b/docs/guide/getting-started.md @@ -177,6 +177,10 @@ yarn global add @toon-format/cli For full CLI documentation, see the [CLI reference](/cli/). +## Media Type & File Extension + +TOON files conventionally use the `.toon` extension. For HTTP transmission, the provisional media type is `text/toon`, always with UTF-8 encoding. While you may specify `charset=utf-8` explicitly, it's optional – UTF-8 is the default assumption. This follows the registration process outlined in [spec §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type). + ## Your First Example The examples below use the TypeScript library for demonstration, but the same operations work in any language with a TOON implementation. @@ -234,20 +238,6 @@ console.log(JSON.stringify(data, null, 2)) Round-tripping is lossless: `decode(encode(x))` always equals `x` (after normalization of non-JSON types like `Date`, `NaN`, etc.). -## Media Type & File Extension - -When using TOON over HTTP or in content-type–aware systems: - -- Use `text/toon` as the media type (provisional; see [spec §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type)) -- Use `.toon` as the standard file extension -- TOON is always UTF-8 encoded; `charset=utf-8` is optional - -Example: - -```http -Content-Type: text/toon; charset=utf-8 -``` - ## Where to Go Next Now that you've seen your first TOON document, read the [Format Overview](/guide/format-overview) for complete syntax details (objects, arrays, quoting rules, key folding), then explore [Using TOON with LLMs](/guide/llm-prompts) to see how to use it effectively in prompts. For implementation details, check the [API Reference](/reference/api) (TypeScript) or the [Specification](/reference/spec) (language-agnostic normative rules).