From e174c46ba1b4917c47dd96e611dce37626cdb9e0 Mon Sep 17 00:00:00 2001 From: Johann Schopplich Date: Mon, 3 Nov 2025 17:36:47 +0100 Subject: [PATCH] fix: add README files to published packages --- .github/workflows/release.yml | 6 ++ packages/cli/README.md | 137 ++++++++++++++++++++++++++++++++++ 2 files changed, 143 insertions(+) create mode 100644 packages/cli/README.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index abc0bc9..4164cf0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -31,6 +31,12 @@ jobs: - run: pnpm install - run: pnpm run build + - name: Copy READMEs for npm packages + run: | + cp README.md packages/toon/README.md + echo "✓ Copied root README to @toon-format/toon" + echo "✓ CLI package uses its own README" + - name: Publish to npm run: npm install -g npm@latest && pnpm -r publish --access public --no-git-checks env: diff --git a/packages/cli/README.md b/packages/cli/README.md new file mode 100644 index 0000000..6d5faf6 --- /dev/null +++ b/packages/cli/README.md @@ -0,0 +1,137 @@ +# @toon-format/cli + +Command-line tool for converting between JSON and TOON formats. + +[TOON (Token-Oriented Object Notation)](https://toonformat.dev) is a compact, human-readable serialization format designed for passing structured data to Large Language Models with significantly reduced token usage. + +## Installation + +```bash +# npm +npm install -g @toon-format/cli + +# pnpm +pnpm add -g @toon-format/cli + +# yarn +yarn global add @toon-format/cli +``` + +Or use directly with `npx`: + +```bash +npx @toon-format/cli [options] [input] +``` + +## Usage + +```bash +toon [options] [input] +``` + +**Standard input:** Omit the input argument or use `-` to read from stdin. This enables piping data directly from other commands. + +**Auto-detection:** The CLI automatically detects the operation based on file extension (`.json` → encode, `.toon` → decode). When reading from stdin, use `--encode` or `--decode` flags to specify the operation (defaults to encode). + +### Basic Examples + +```bash +# Encode JSON to TOON (auto-detected) +toon input.json -o output.toon + +# Decode TOON to JSON (auto-detected) +toon data.toon -o output.json + +# Output to stdout +toon input.json + +# Pipe from stdin +cat data.json | toon +echo '{"name": "Ada"}' | toon + +# Decode from stdin +cat data.toon | toon --decode +``` + +## Options + +| Option | Description | +| ------ | ----------- | +| `-o, --output ` | Output file path (prints to stdout if omitted) | +| `-e, --encode` | Force encode mode (overrides auto-detection) | +| `-d, --decode` | Force decode mode (overrides auto-detection) | +| `--delimiter ` | Array delimiter: `,` (comma), `\t` (tab), `\|` (pipe) | +| `--indent ` | Indentation size (default: `2`) | +| `--length-marker` | Add `#` prefix to array lengths (e.g., `items[#3]`) | +| `--stats` | Show token count estimates and savings (encode only) | +| `--no-strict` | Disable strict validation when decoding | + +## Advanced Examples + +### Token Statistics + +Show token savings when encoding: + +```bash +toon data.json --stats -o output.toon +``` + +Example output: +``` +✓ Encoded to TOON + Input: 15,145 tokens (JSON) + Output: 8,745 tokens (TOON) + Saved: 6,400 tokens (42.3% reduction) +``` + +### Alternative Delimiters + +#### Tab-separated (often more token-efficient) + +```bash +toon data.json --delimiter "\t" -o output.toon +``` + +#### Pipe-separated with length markers + +```bash +toon data.json --delimiter "|" --length-marker -o output.toon +``` + +### Lenient Decoding + +Skip validation for faster processing: + +```bash +toon data.toon --no-strict -o output.json +``` + +### Stdin Workflows + +```bash +# Convert API response to TOON +curl https://api.example.com/data | toon --stats + +# Process large dataset +cat large-dataset.json | toon --delimiter "\t" > output.toon + +# Chain with other tools +jq '.results' data.json | toon > filtered.toon +``` + +## Why Use the CLI? + +- **Quick conversions** between formats without writing code +- **Token analysis** to see potential savings before sending to LLMs +- **Pipeline integration** with existing JSON-based workflows +- **Flexible formatting** with delimiter and indentation options + +## Related + +- [@toon-format/toon](https://www.npmjs.com/package/@toon-format/toon) - JavaScript/TypeScript library +- [Full specification](https://github.com/toon-format/spec) - Complete format documentation +- [Website](https://toonformat.dev) - Interactive examples and guides + +## License + +[MIT](https://github.com/toon-format/toon/blob/main/LICENSE) License © 2025-PRESENT [Johann Schopplich](https://github.com/johannschopplich)