docs: simplify stream decode intructions

This commit is contained in:
Johann Schopplich
2025-11-23 19:05:04 +01:00
parent a200a9fa54
commit 27c6fc5519
2 changed files with 10 additions and 73 deletions

View File

@@ -138,23 +138,7 @@ hikes[3]{id,name,distanceKm,elevationGain,companion,wasSunny}:
## Media Type & File Extension ## Media Type & File Extension
For HTTP and content-typeaware contexts, TOON defines: By convention, TOON files use the `.toon` extension and the provisional media type `text/toon` for HTTP and content-typeaware contexts. TOON documents are always UTF-8 encoded; the `charset=utf-8` parameter may be specified but defaults to UTF-8 when omitted. See [SPEC.md §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type) for normative details.
- **Media type (provisional):** `text/toon`
- **Charset:** Always UTF-8 (`charset=utf-8` MAY be specified; if omitted, UTF-8 MUST be assumed)
- **File extension:** `.toon`
Example HTTP usage:
```http
GET /resource HTTP/1.1
Accept: text/toon
HTTP/1.1 200 OK
Content-Type: text/toon
```
See [SPEC.md §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type) for details.
## When Not to Use TOON ## When Not to Use TOON
@@ -797,45 +781,8 @@ for (const line of encodeLines(largeData)) {
} }
``` ```
**Streaming decode:** > [!TIP]
> For streaming decode APIs, see [`decodeFromLines()`](/reference/api#decodeFromLines-lines-options) and [`decodeStream()`](/reference/api#decodeStream-source-options).
```ts
import { decodeFromLines, decodeStreamSync } from '@toon-format/toon'
// 1. Lines → value (build full JSON value)
const value = decodeFromLines([
'users[2]{id,name}:',
' 1,Alice',
' 2,Bob',
])
// { users: [{ id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }] }
// 2. Lines → events (for custom streaming consumers)
const lines = [
'users[2]{id,name}:',
' 1,Alice',
' 2,Bob',
]
for (const event of decodeStreamSync(lines)) {
// { type: 'startObject' }, { type: 'key', key: 'users' }, ...
}
```
**Async streaming decode:**
```ts
// 3. Async streaming from files or network
import { createReadStream } from 'node:fs'
import { createInterface } from 'node:readline'
import { decodeStream } from '@toon-format/toon'
const fileStream = createReadStream('data.toon', 'utf-8')
const rl = createInterface({ input: fileStream })
for await (const event of decodeStream(rl)) {
// Process events as they arrive
}
```
## Playgrounds ## Playgrounds
@@ -909,19 +856,19 @@ Follow the detailed [LLM integration guide](https://toonformat.dev/guide/llm-pro
Comprehensive guides, references, and resources to help you get the most out of the TOON format and tools. Comprehensive guides, references, and resources to help you get the most out of the TOON format and tools.
**Getting Started:** ### Getting Started
- [Introduction & Installation](https://toonformat.dev/guide/getting-started) What TOON is, when to use it, first steps - [Introduction & Installation](https://toonformat.dev/guide/getting-started) What TOON is, when to use it, first steps
- [Format Overview](https://toonformat.dev/guide/format-overview) Complete syntax with examples - [Format Overview](https://toonformat.dev/guide/format-overview) Complete syntax with examples
- [Benchmarks](https://toonformat.dev/guide/benchmarks) Accuracy & token efficiency results - [Benchmarks](https://toonformat.dev/guide/benchmarks) Accuracy & token efficiency results
**Tools & Integration:** ### Tools & Integration
- [CLI](https://toonformat.dev/cli/) Command-line tool for JSON↔TOON conversions - [CLI](https://toonformat.dev/cli/) Command-line tool for JSON↔TOON conversions
- [Using TOON with LLMs](https://toonformat.dev/guide/llm-prompts) Prompting strategies & validation - [Using TOON with LLMs](https://toonformat.dev/guide/llm-prompts) Prompting strategies & validation
- [Playgrounds](https://toonformat.dev/ecosystem/tools-and-playgrounds) Interactive tools - [Playgrounds](https://toonformat.dev/ecosystem/tools-and-playgrounds) Interactive tools
**Reference:** ### References
- [API Reference](https://toonformat.dev/reference/api) TypeScript/JavaScript encode/decode API - [API Reference](https://toonformat.dev/reference/api) TypeScript/JavaScript encode/decode API
- [Syntax Cheatsheet](https://toonformat.dev/reference/syntax-cheatsheet) Quick format lookup - [Syntax Cheatsheet](https://toonformat.dev/reference/syntax-cheatsheet) Quick format lookup

View File

@@ -177,6 +177,10 @@ yarn global add @toon-format/cli
For full CLI documentation, see the [CLI reference](/cli/). For full CLI documentation, see the [CLI reference](/cli/).
## Media Type & File Extension
TOON files conventionally use the `.toon` extension. For HTTP transmission, the provisional media type is `text/toon`, always with UTF-8 encoding. While you may specify `charset=utf-8` explicitly, it's optional UTF-8 is the default assumption. This follows the registration process outlined in [spec §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type).
## Your First Example ## Your First Example
The examples below use the TypeScript library for demonstration, but the same operations work in any language with a TOON implementation. The examples below use the TypeScript library for demonstration, but the same operations work in any language with a TOON implementation.
@@ -234,20 +238,6 @@ console.log(JSON.stringify(data, null, 2))
Round-tripping is lossless: `decode(encode(x))` always equals `x` (after normalization of non-JSON types like `Date`, `NaN`, etc.). Round-tripping is lossless: `decode(encode(x))` always equals `x` (after normalization of non-JSON types like `Date`, `NaN`, etc.).
## Media Type & File Extension
When using TOON over HTTP or in content-typeaware systems:
- Use `text/toon` as the media type (provisional; see [spec §18.2](https://github.com/toon-format/spec/blob/main/SPEC.md#182-provisional-media-type))
- Use `.toon` as the standard file extension
- TOON is always UTF-8 encoded; `charset=utf-8` is optional
Example:
```http
Content-Type: text/toon; charset=utf-8
```
## Where to Go Next ## Where to Go Next
Now that you've seen your first TOON document, read the [Format Overview](/guide/format-overview) for complete syntax details (objects, arrays, quoting rules, key folding), then explore [Using TOON with LLMs](/guide/llm-prompts) to see how to use it effectively in prompts. For implementation details, check the [API Reference](/reference/api) (TypeScript) or the [Specification](/reference/spec) (language-agnostic normative rules). Now that you've seen your first TOON document, read the [Format Overview](/guide/format-overview) for complete syntax details (objects, arrays, quoting rules, key folding), then explore [Using TOON with LLMs](/guide/llm-prompts) to see how to use it effectively in prompts. For implementation details, check the [API Reference](/reference/api) (TypeScript) or the [Specification](/reference/spec) (language-agnostic normative rules).