test: add LLM retrieval accuracy tests

2026-01-29 23:34:10 +08:00 · 2025-10-27 11:48:33 +01:00
parent eb8f7e28e1
commit 3c840259fe
25 changed files with 21404 additions and 723 deletions
--- a/benchmarks/results/token-efficiency.md
+++ b/benchmarks/results/token-efficiency.md
@@ -0,0 +1,141 @@
+### Token Efficiency
+
+```
+⭐ GitHub Repositories       ██████████████░░░░░░░░░░░   8,745 tokens  (JSON: 15,145)  💰 42.3% saved
+📈 Analytics Time Series     ██████████░░░░░░░░░░░░░░░   3,631 tokens  (JSON:  9,024)  💰 59.8% saved
+👥 API Response              ██████████████░░░░░░░░░░░   2,593 tokens  (JSON:  4,589)  💰 43.5% saved
+🛒 E-commerce Order          ███████████████░░░░░░░░░░     203 tokens  (JSON:    338)  💰 39.9% saved
+```
+
+**Total:** 15,172 tokens (TOON) vs 29,096 tokens (JSON) → 47.9% savings
+
+<details>
+<summary><strong>View detailed examples</strong></summary>
+
+#### ⭐ GitHub Repositories
+
+**Configuration:** Top 100 GitHub repositories with stars, forks, and metadata
+
+**Savings:** 6,400 tokens (42.3% reduction)
+
+**JSON** (15,145 tokens):
+
+```json
+{
+  "repositories": [
+    {
+      "id": 28457823,
+      "name": "freeCodeCamp",
+      "repo": "freeCodeCamp/freeCodeCamp",
+      "description": "freeCodeCamp.org's open-source codebase and curriculum. Learn math, programming,...",
+      "createdAt": "2014-12-24T17:49:19Z",
+      "updatedAt": "2025-10-27T07:40:58Z",
+      "pushedAt": "2025-10-26T11:31:08Z",
+      "stars": 430828,
+      "watchers": 8582,
+      "forks": 42136,
+      "defaultBranch": "main"
+    },
+    {
+      "id": 132750724,
+      "name": "build-your-own-x",
+      "repo": "codecrafters-io/build-your-own-x",
+      "description": "Master programming by recreating your favorite technologies from scratch.",
+      "createdAt": "2018-05-09T12:03:18Z",
+      "updatedAt": "2025-10-27T07:43:25Z",
+      "pushedAt": "2025-10-10T18:45:01Z",
+      "stars": 430102,
+      "watchers": 6322,
+      "forks": 40388,
+      "defaultBranch": "master"
+    },
+    {
+      "id": 21737465,
+      "name": "awesome",
+      "repo": "sindresorhus/awesome",
+      "description": "😎 Awesome lists about all kinds of interesting topics",
+      "createdAt": "2014-07-11T13:42:37Z",
+      "updatedAt": "2025-10-27T07:44:27Z",
+      "pushedAt": "2025-10-23T17:26:53Z",
+      "stars": 409760,
+      "watchers": 8016,
+      "forks": 32015,
+      "defaultBranch": "main"
+    }
+  ]
+}
+```
+
+**TOON** (8,745 tokens):
+
+```
+repositories[3]{id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch}:
+  28457823,freeCodeCamp,freeCodeCamp/freeCodeCamp,"freeCodeCamp.org's open-source codebase and curriculum. Learn math, programming,...","2014-12-24T17:49:19Z","2025-10-27T07:40:58Z","2025-10-26T11:31:08Z",430828,8582,42136,main
+  132750724,build-your-own-x,codecrafters-io/build-your-own-x,Master programming by recreating your favorite technologies from scratch.,"2018-05-09T12:03:18Z","2025-10-27T07:43:25Z","2025-10-10T18:45:01Z",430102,6322,40388,master
+  21737465,awesome,sindresorhus/awesome,😎 Awesome lists about all kinds of interesting topics,"2014-07-11T13:42:37Z","2025-10-27T07:44:27Z","2025-10-23T17:26:53Z",409760,8016,32015,main
+```
+
+---
+
+#### 📈 Analytics Time Series
+
+**Configuration:** 180 days of web metrics (views, clicks, conversions, revenue)
+
+**Savings:** 5,393 tokens (59.8% reduction)
+
+**JSON** (9,024 tokens):
+
+```json
+{
+  "metrics": [
+    {
+      "date": "2024-12-31",
+      "views": 3769,
+      "clicks": 400,
+      "conversions": 59,
+      "revenue": 198.98
+    },
+    {
+      "date": "2025-01-01",
+      "views": 5742,
+      "clicks": 463,
+      "conversions": 28,
+      "revenue": 295.77
+    },
+    {
+      "date": "2025-01-02",
+      "views": 3669,
+      "clicks": 336,
+      "conversions": 102,
+      "revenue": 624.23
+    },
+    {
+      "date": "2025-01-03",
+      "views": 1332,
+      "clicks": 304,
+      "conversions": 99,
+      "revenue": 113.06
+    },
+    {
+      "date": "2025-01-04",
+      "views": 1444,
+      "clicks": 222,
+      "conversions": 88,
+      "revenue": 986.69
+    }
+  ]
+}
+```
+
+**TOON** (3,631 tokens):
+
+```
+metrics[5]{date,views,clicks,conversions,revenue}:
+  2024-12-31,3769,400,59,198.98
+  2025-01-01,5742,463,28,295.77
+  2025-01-02,3669,336,102,624.23
+  2025-01-03,1332,304,99,113.06
+  2025-01-04,1444,222,88,986.69
+```
+
+</details>