diff --git a/README.md b/README.md index 2acf0a1..3c0ce90 100644 --- a/README.md +++ b/README.md @@ -75,7 +75,7 @@ See [benchmarks](#benchmarks) for concrete comparisons across different data str ## Key Features -- ๐Ÿ’ธ **Token-efficient:** typically 30โ€“60% fewer tokens than JSON[^1] +- ๐Ÿ’ธ **Token-efficient:** typically 30-60% fewer tokens on large uniform arrays vs formatted JSON[^1] - ๐Ÿคฟ **LLM-friendly guardrails:** explicit lengths and fields enable validation - ๐Ÿฑ **Minimal syntax:** removes redundant punctuation (braces, brackets, most quotes) - ๐Ÿ“ **Indentation-based structure:** like YAML, uses whitespace instead of braces @@ -108,19 +108,19 @@ Datasets with nested or semi-uniform structures. CSV excluded as it cannot prope ``` ๐Ÿ›’ E-commerce orders with nested structures โ”Š Tabular: 33% โ”‚ - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 72,743 tokens - โ”œโ”€ vs JSON (โˆ’33.1%) 108,731 tokens - โ”œโ”€ vs JSON compact (+5.5%) 68,936 tokens - โ”œโ”€ vs YAML (โˆ’14.1%) 84,724 tokens - โ””โ”€ vs XML (โˆ’40.5%) 122,313 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 72,771 tokens + โ”œโ”€ vs JSON (โˆ’33.1%) 108,806 tokens + โ”œโ”€ vs JSON compact (+5.5%) 68,975 tokens + โ”œโ”€ vs YAML (โˆ’14.2%) 84,780 tokens + โ””โ”€ vs XML (โˆ’40.5%) 122,406 tokens ๐Ÿงพ Semi-uniform event logs โ”Š Tabular: 50% โ”‚ - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 153,223 tokens - โ”œโ”€ vs JSON (โˆ’15.0%) 180,196 tokens - โ”œโ”€ vs JSON compact (+19.9%) 127,740 tokens - โ”œโ”€ vs YAML (โˆ’0.8%) 154,514 tokens - โ””โ”€ vs XML (โˆ’25.2%) 204,800 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 153,211 tokens + โ”œโ”€ vs JSON (โˆ’15.0%) 180,176 tokens + โ”œโ”€ vs JSON compact (+19.9%) 127,731 tokens + โ”œโ”€ vs YAML (โˆ’0.8%) 154,505 tokens + โ””โ”€ vs XML (โˆ’25.2%) 204,777 tokens ๐Ÿงฉ Deeply nested configuration โ”Š Tabular: 0% โ”‚ @@ -131,11 +131,11 @@ Datasets with nested or semi-uniform structures. CSV excluded as it cannot prope โ””โ”€ vs XML (โˆ’37.4%) 1,008 tokens โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Total โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 226,597 tokens - โ”œโ”€ vs JSON (โˆ’21.8%) 289,846 tokens - โ”œโ”€ vs JSON compact (+14.9%) 197,240 tokens - โ”œโ”€ vs YAML (โˆ’5.5%) 239,911 tokens - โ””โ”€ vs XML (โˆ’30.9%) 328,121 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 226,613 tokens + โ”œโ”€ vs JSON (โˆ’21.8%) 289,901 tokens + โ”œโ”€ vs JSON compact (+14.9%) 197,270 tokens + โ”œโ”€ vs YAML (โˆ’5.6%) 239,958 tokens + โ””โ”€ vs XML (โˆ’31.0%) 328,191 tokens ``` #### Flat-Only Track @@ -145,21 +145,21 @@ Datasets with flat tabular structures where CSV is applicable. ``` ๐Ÿ‘ฅ Uniform employee records โ”Š Tabular: 100% โ”‚ - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 46,956 tokens - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 49,827 tokens (+6.1% vs CSV) - โ”œโ”€ vs JSON (โˆ’60.7%) 126,854 tokens - โ”œโ”€ vs JSON compact (โˆ’36.8%) 78,850 tokens - โ”œโ”€ vs YAML (โˆ’50.0%) 99,701 tokens - โ””โ”€ vs XML (โˆ’66.0%) 146,440 tokens + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 46,954 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 49,831 tokens (+6.1% vs CSV) + โ”œโ”€ vs JSON (โˆ’60.7%) 126,860 tokens + โ”œโ”€ vs JSON compact (โˆ’36.8%) 78,856 tokens + โ”œโ”€ vs YAML (โˆ’50.0%) 99,706 tokens + โ””โ”€ vs XML (โˆ’66.0%) 146,444 tokens ๐Ÿ“ˆ Time-series analytics data โ”Š Tabular: 100% โ”‚ - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 8,396 tokens - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 9,128 tokens (+8.7% vs CSV) - โ”œโ”€ vs JSON (โˆ’59.0%) 22,258 tokens - โ”œโ”€ vs JSON compact (โˆ’35.8%) 14,224 tokens - โ”œโ”€ vs YAML (โˆ’48.9%) 17,871 tokens - โ””โ”€ vs XML (โˆ’65.7%) 26,629 tokens + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 8,388 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 9,120 tokens (+8.7% vs CSV) + โ”œโ”€ vs JSON (โˆ’59.0%) 22,250 tokens + โ”œโ”€ vs JSON compact (โˆ’35.8%) 14,216 tokens + โ”œโ”€ vs YAML (โˆ’48.9%) 17,863 tokens + โ””โ”€ vs XML (โˆ’65.7%) 26,621 tokens โญ Top 100 GitHub repositories โ”Š Tabular: 100% โ”‚ @@ -171,12 +171,12 @@ Datasets with flat tabular structures where CSV is applicable. โ””โ”€ vs XML (โˆ’48.8%) 17,095 tokens โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Total โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 63,865 tokens - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 67,700 tokens (+6.0% vs CSV) - โ”œโ”€ vs JSON (โˆ’58.8%) 164,257 tokens - โ”œโ”€ vs JSON compact (โˆ’35.2%) 104,529 tokens - โ”œโ”€ vs YAML (โˆ’48.2%) 130,701 tokens - โ””โ”€ vs XML (โˆ’64.4%) 190,164 tokens + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 63,855 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 67,696 tokens (+6.0% vs CSV) + โ”œโ”€ vs JSON (โˆ’58.8%) 164,255 tokens + โ”œโ”€ vs JSON compact (โˆ’35.2%) 104,527 tokens + โ”œโ”€ vs YAML (โˆ’48.2%) 130,698 tokens + โ””โ”€ vs XML (โˆ’64.4%) 190,160 tokens ```
@@ -186,64 +186,64 @@ Datasets with flat tabular structures where CSV is applicable. **Savings:** 13,130 tokens (59.0% reduction vs JSON) -**JSON** (22,258 tokens): +**JSON** (22,250 tokens): ```json { "metrics": [ { "date": "2025-01-01", - "views": 7708, - "clicks": 595, - "conversions": 69, - "revenue": 15369.93, - "bounceRate": 0.35 + "views": 5715, + "clicks": 211, + "conversions": 28, + "revenue": 7976.46, + "bounceRate": 0.47 }, { "date": "2025-01-02", - "views": 5894, - "clicks": 381, - "conversions": 21, - "revenue": 2112.12, - "bounceRate": 0.3 + "views": 7103, + "clicks": 393, + "conversions": 28, + "revenue": 8360.53, + "bounceRate": 0.32 }, { "date": "2025-01-03", - "views": 6835, - "clicks": 422, - "conversions": 35, - "revenue": 4525.73, + "views": 7248, + "clicks": 378, + "conversions": 24, + "revenue": 3212.57, "bounceRate": 0.5 }, { "date": "2025-01-04", - "views": 5325, - "clicks": 305, - "conversions": 22, - "revenue": 2445.3, - "bounceRate": 0.44 + "views": 2927, + "clicks": 77, + "conversions": 11, + "revenue": 1211.69, + "bounceRate": 0.62 }, { "date": "2025-01-05", - "views": 2974, - "clicks": 61, - "conversions": 6, - "revenue": 956.57, - "bounceRate": 0.47 + "views": 3530, + "clicks": 82, + "conversions": 8, + "revenue": 462.77, + "bounceRate": 0.56 } ] } ``` -**TOON** (9,128 tokens): +**TOON** (9,120 tokens): ``` metrics[5]{date,views,clicks,conversions,revenue,bounceRate}: - 2025-01-01,7708,595,69,15369.93,0.35 - 2025-01-02,5894,381,21,2112.12,0.3 - 2025-01-03,6835,422,35,4525.73,0.5 - 2025-01-04,5325,305,22,2445.3,0.44 - 2025-01-05,2974,61,6,956.57,0.47 + 2025-01-01,5715,211,28,7976.46,0.47 + 2025-01-02,7103,393,28,8360.53,0.32 + 2025-01-03,7248,378,24,3212.57,0.5 + 2025-01-04,2927,77,11,1211.69,0.62 + 2025-01-05,3530,82,8,462.77,0.56 ``` --- @@ -317,7 +317,7 @@ repositories[3]{id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watc -Benchmarks test LLM comprehension across different input formats using 204 data retrieval questions on 4 models. +Benchmarks test LLM comprehension across different input formats using 209 data retrieval questions on 4 models.
Show Dataset Catalog @@ -332,6 +332,11 @@ Benchmarks test LLM comprehension across different input formats using 204 data | Top 100 GitHub repositories | 100 | uniform | โœ“ | 100% | | Semi-uniform event logs | 75 | semi-uniform | โœ— | 50% | | Deeply nested configuration | 11 | deep | โœ— | 0% | +| Valid complete dataset (control) | 20 | uniform | โœ“ | 100% | +| Array truncated: 3 rows removed from end | 17 | uniform | โœ“ | 100% | +| Extra rows added beyond declared length | 23 | uniform | โœ“ | 100% | +| Inconsistent field count (missing salary in row 10) | 20 | uniform | โœ“ | 100% | +| Missing required fields (no email in multiple rows) | 20 | uniform | โœ“ | 100% | **Structure classes:** - **uniform**: All objects have identical fields with primitive values @@ -350,67 +355,69 @@ Benchmarks test LLM comprehension across different input formats using 204 data Each format's overall performance, balancing accuracy against token cost: ``` -TOON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“ 17.2 โ”‚ 75.5% acc โ”‚ 4,389 tokens -CSV โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘ 16.6 โ”‚ 67.8% acc โ”‚ 4,080 tokens -JSON compact โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘ 14.7 โ”‚ 73.3% acc โ”‚ 4,982 tokens -YAML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 12.1 โ”‚ 72.4% acc โ”‚ 5,976 tokens -JSON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 10.0 โ”‚ 72.4% acc โ”‚ 7,260 tokens -XML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 8.4 โ”‚ 69.0% acc โ”‚ 8,251 tokens +TOON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“ 26.9 โ”‚ 73.9% acc โ”‚ 2,744 tokens +JSON compact โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘ 22.9 โ”‚ 70.7% acc โ”‚ 3,081 tokens +YAML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 18.6 โ”‚ 69.0% acc โ”‚ 3,719 tokens +JSON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 15.3 โ”‚ 69.7% acc โ”‚ 4,545 tokens +XML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 13.0 โ”‚ 67.1% acc โ”‚ 5,167 tokens ``` -TOON achieves **75.5%** accuracy (vs JSON's 72.4%) while using **39.5% fewer tokens**. +TOON achieves **73.9%** accuracy (vs JSON's 69.7%) while using **39.6% fewer tokens**. + +**Note on CSV:** Excluded from ranking as it only supports 436/209 questions (flat tabular data only). While CSV is highly token-efficient for simple tabular data, it cannot represent nested structures that other formats handle. #### Per-Model Accuracy -Accuracy across 4 LLMs on 204 data retrieval questions: +Accuracy across 4 LLMs on 209 data retrieval questions: ``` claude-haiku-4-5-20251001 -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 62.3% (127/204) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 56.9% (116/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.9% (114/204) - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.9% (112/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.9% (112/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 47.1% (49/104) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 59.8% (125/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 57.4% (120/209) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 56.0% (117/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.5% (116/209) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.0% (115/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 50.5% (55/109) gemini-2.5-flash -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 91.2% (186/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.7% (183/204) - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 87.7% (179/204) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 87.7% (179/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 87.3% (178/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 85.6% (89/104) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 87.6% (183/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 86.2% (94/109) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 82.3% (172/209) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 79.4% (166/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 79.4% (166/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘ 77.0% (161/209) gpt-5-nano - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 93.6% (191/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 90.4% (94/104) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.7% (183/204) -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.2% (182/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.2% (182/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 81.4% (166/204) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 90.9% (190/209) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 90.9% (190/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.0% (186/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.0% (97/109) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 87.1% (182/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 80.9% (169/209) grok-4-fast-non-reasoning -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 59.3% (121/204) - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 56.9% (116/204) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.4% (113/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.9% (112/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 52.5% (107/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 48.1% (50/104) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 57.4% (120/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.5% (116/209) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.5% (114/209) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 53.6% (112/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 52.6% (110/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 52.3% (57/109) ``` -**Key tradeoff:** TOON achieves **75.5% accuracy** (vs JSON's 72.4%) while using **39.5% fewer tokens** on these datasets. +**Key tradeoff:** TOON achieves **73.9% accuracy** (vs JSON's 69.7%) while using **39.6% fewer tokens** on these datasets.
Performance by dataset, model, and question type #### Performance by Question Type -| Question Type | TOON | JSON compact | JSON | YAML | XML | CSV | +| Question Type | TOON | JSON compact | JSON | CSV | YAML | XML | | ------------- | ---- | ---- | ---- | ---- | ---- | ---- | -| Field Retrieval | 100.0% | 98.9% | 99.6% | 99.3% | 98.5% | 100.0% | -| Aggregation | 56.3% | 52.4% | 53.2% | 53.2% | 47.2% | 40.5% | -| Filtering | 58.9% | 58.3% | 54.2% | 53.1% | 50.5% | 49.1% | -| Structure Awareness | 89.0% | 85.0% | 82.0% | 85.0% | 79.0% | 84.4% | +| Field Retrieval | 99.6% | 99.3% | 99.3% | 100.0% | 98.2% | 98.9% | +| Aggregation | 54.4% | 47.2% | 48.8% | 44.0% | 47.6% | 41.3% | +| Filtering | 56.3% | 57.3% | 50.5% | 49.1% | 51.0% | 47.9% | +| Structure Awareness | 88.0% | 83.0% | 83.0% | 85.9% | 80.0% | 80.0% | +| Structural Validation | 70.0% | 45.0% | 50.0% | 80.0% | 60.0% | 80.0% | #### Performance by Dataset @@ -418,64 +425,119 @@ grok-4-fast-non-reasoning | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `csv` | 70.7% | 2,337 | 116/164 | -| `toon` | 72.0% | 2,483 | 118/164 | -| `json-compact` | 71.3% | 3,943 | 117/164 | -| `yaml` | 70.1% | 4,969 | 115/164 | -| `json-pretty` | 72.6% | 6,347 | 119/164 | -| `xml` | 70.7% | 7,314 | 116/164 | +| `csv` | 72.0% | 2,352 | 118/164 | +| `toon` | 73.8% | 2,518 | 121/164 | +| `json-compact` | 69.5% | 3,953 | 114/164 | +| `yaml` | 68.3% | 4,982 | 112/164 | +| `json-pretty` | 68.3% | 6,360 | 112/164 | +| `xml` | 69.5% | 7,324 | 114/164 | ##### E-commerce orders with nested structures | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `toon` | 83.5% | 7,197 | 137/164 | -| `json-compact` | 79.3% | 6,784 | 130/164 | -| `yaml` | 78.7% | 8,334 | 129/164 | -| `json-pretty` | 78.7% | 10,700 | 129/164 | -| `xml` | 73.8% | 12,013 | 121/164 | +| `toon` | 81.1% | 7,232 | 133/164 | +| `json-compact` | 76.8% | 6,794 | 126/164 | +| `yaml` | 75.6% | 8,347 | 124/164 | +| `json-pretty` | 76.2% | 10,713 | 125/164 | +| `xml` | 74.4% | 12,023 | 122/164 | ##### Time-series analytics data | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `toon` | 75.8% | 1,513 | 91/120 | -| `csv` | 72.5% | 1,391 | 87/120 | -| `json-compact` | 70.0% | 2,339 | 84/120 | -| `yaml` | 70.0% | 2,936 | 84/120 | -| `json-pretty` | 71.7% | 3,663 | 86/120 | -| `xml` | 71.7% | 4,374 | 86/120 | +| `csv` | 73.3% | 1,406 | 88/120 | +| `toon` | 72.5% | 1,548 | 87/120 | +| `json-compact` | 71.7% | 2,349 | 86/120 | +| `yaml` | 71.7% | 2,949 | 86/120 | +| `json-pretty` | 68.3% | 3,676 | 82/120 | +| `xml` | 68.3% | 4,384 | 82/120 | ##### Top 100 GitHub repositories | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `toon` | 64.4% | 8,745 | 85/132 | -| `csv` | 59.8% | 8,513 | 79/132 | -| `json-compact` | 60.6% | 11,455 | 80/132 | -| `yaml` | 61.4% | 13,129 | 81/132 | -| `json-pretty` | 59.1% | 15,145 | 78/132 | -| `xml` | 51.5% | 17,095 | 68/132 | +| `toon` | 62.9% | 8,780 | 83/132 | +| `csv` | 61.4% | 8,528 | 81/132 | +| `yaml` | 59.8% | 13,142 | 79/132 | +| `json-compact` | 55.3% | 11,465 | 73/132 | +| `json-pretty` | 56.1% | 15,158 | 74/132 | +| `xml` | 48.5% | 17,105 | 64/132 | ##### Semi-uniform event logs | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `json-compact` | 67.5% | 4,809 | 81/120 | -| `yaml` | 63.3% | 5,814 | 76/120 | -| `toon` | 62.5% | 5,764 | 75/120 | -| `json-pretty` | 59.2% | 6,784 | 71/120 | -| `xml` | 55.0% | 7,699 | 66/120 | +| `json-compact` | 63.3% | 4,819 | 76/120 | +| `toon` | 57.5% | 5,799 | 69/120 | +| `json-pretty` | 59.2% | 6,797 | 71/120 | +| `yaml` | 48.3% | 5,827 | 58/120 | +| `xml` | 46.7% | 7,709 | 56/120 | ##### Deeply nested configuration | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `json-compact` | 91.4% | 564 | 106/116 | -| `toon` | 94.8% | 631 | 110/116 | -| `yaml` | 91.4% | 673 | 106/116 | -| `json-pretty` | 93.1% | 919 | 108/116 | -| `xml` | 91.4% | 1,008 | 106/116 | +| `json-compact` | 92.2% | 574 | 107/116 | +| `toon` | 95.7% | 666 | 111/116 | +| `yaml` | 91.4% | 686 | 106/116 | +| `json-pretty` | 94.0% | 932 | 109/116 | +| `xml` | 92.2% | 1,018 | 107/116 | + +##### Valid complete dataset (control) + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `toon` | 100.0% | 544 | 4/4 | +| `json-compact` | 100.0% | 795 | 4/4 | +| `yaml` | 100.0% | 1,003 | 4/4 | +| `json-pretty` | 100.0% | 1,282 | 4/4 | +| `csv` | 25.0% | 492 | 1/4 | +| `xml` | 0.0% | 1,467 | 0/4 | + +##### Array truncated: 3 rows removed from end + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 100.0% | 425 | 4/4 | +| `xml` | 100.0% | 1,251 | 4/4 | +| `toon` | 0.0% | 474 | 0/4 | +| `json-compact` | 0.0% | 681 | 0/4 | +| `json-pretty` | 0.0% | 1,096 | 0/4 | +| `yaml` | 0.0% | 859 | 0/4 | + +##### Extra rows added beyond declared length + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 100.0% | 566 | 4/4 | +| `toon` | 75.0% | 621 | 3/4 | +| `xml` | 100.0% | 1,692 | 4/4 | +| `yaml` | 75.0% | 1,157 | 3/4 | +| `json-compact` | 50.0% | 917 | 2/4 | +| `json-pretty` | 50.0% | 1,476 | 2/4 | + +##### Inconsistent field count (missing salary in row 10) + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 75.0% | 489 | 3/4 | +| `yaml` | 100.0% | 996 | 4/4 | +| `toon` | 100.0% | 1,019 | 4/4 | +| `json-compact` | 75.0% | 790 | 3/4 | +| `xml` | 100.0% | 1,458 | 4/4 | +| `json-pretty` | 75.0% | 1,274 | 3/4 | + +##### Missing required fields (no email in multiple rows) + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 100.0% | 329 | 4/4 | +| `xml` | 100.0% | 1,411 | 4/4 | +| `toon` | 75.0% | 983 | 3/4 | +| `yaml` | 25.0% | 960 | 1/4 | +| `json-pretty` | 25.0% | 1,230 | 1/4 | +| `json-compact` | 0.0% | 755 | 0/4 | #### Performance by Model @@ -483,45 +545,45 @@ grok-4-fast-non-reasoning | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `toon` | 62.3% | 127/204 | -| `json-pretty` | 56.9% | 116/204 | -| `yaml` | 55.9% | 114/204 | -| `json-compact` | 54.9% | 112/204 | -| `xml` | 54.9% | 112/204 | -| `csv` | 47.1% | 49/104 | +| `toon` | 59.8% | 125/209 | +| `json-pretty` | 57.4% | 120/209 | +| `yaml` | 56.0% | 117/209 | +| `xml` | 55.5% | 116/209 | +| `json-compact` | 55.0% | 115/209 | +| `csv` | 50.5% | 55/109 | ##### gemini-2.5-flash | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `toon` | 91.2% | 186/204 | -| `yaml` | 89.7% | 183/204 | -| `json-compact` | 87.7% | 179/204 | -| `json-pretty` | 87.7% | 179/204 | -| `xml` | 87.3% | 178/204 | -| `csv` | 85.6% | 89/104 | +| `toon` | 87.6% | 183/209 | +| `csv` | 86.2% | 94/109 | +| `json-compact` | 82.3% | 172/209 | +| `yaml` | 79.4% | 166/209 | +| `xml` | 79.4% | 166/209 | +| `json-pretty` | 77.0% | 161/209 | ##### gpt-5-nano | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `json-compact` | 93.6% | 191/204 | -| `csv` | 90.4% | 94/104 | -| `json-pretty` | 89.7% | 183/204 | -| `toon` | 89.2% | 182/204 | -| `yaml` | 89.2% | 182/204 | -| `xml` | 81.4% | 166/204 | +| `toon` | 90.9% | 190/209 | +| `json-compact` | 90.9% | 190/209 | +| `json-pretty` | 89.0% | 186/209 | +| `csv` | 89.0% | 97/109 | +| `yaml` | 87.1% | 182/209 | +| `xml` | 80.9% | 169/209 | ##### grok-4-fast-non-reasoning | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `toon` | 59.3% | 121/204 | -| `json-compact` | 56.9% | 116/204 | -| `json-pretty` | 55.4% | 113/204 | -| `yaml` | 54.9% | 112/204 | -| `xml` | 52.5% | 107/204 | -| `csv` | 48.1% | 50/104 | +| `toon` | 57.4% | 120/209 | +| `json-pretty` | 55.5% | 116/209 | +| `json-compact` | 54.5% | 114/209 | +| `yaml` | 53.6% | 112/209 | +| `xml` | 52.6% | 110/209 | +| `csv` | 52.3% | 57/109 |
@@ -534,8 +596,9 @@ This benchmark tests **LLM comprehension and data retrieval accuracy** across di #### Datasets Tested -Six datasets designed to test different structural patterns: +Eleven datasets designed to test different structural patterns and validation capabilities: +**Primary datasets:** 1. **Tabular** (100 employee records): Uniform objects with identical fields โ€“ optimal for TOON's tabular format. 2. **Nested** (50 e-commerce orders): Complex structures with nested customer objects and item arrays. 3. **Analytics** (60 days of metrics): Time-series data with dates and numeric values. @@ -543,21 +606,28 @@ Six datasets designed to test different structural patterns: 5. **Event Logs** (75 logs): Semi-uniform data with ~50% flat logs and ~50% with nested error objects. 6. **Nested Config** (1 configuration): Deeply nested configuration with minimal tabular eligibility. +**Structural validation datasets:** +7. **Control**: Valid complete dataset (baseline for validation) +8. **Truncated**: Array with 3 rows removed from end (tests [N] length detection) +9. **Extra rows**: Array with 3 additional rows beyond declared length +10. **Width mismatch**: Inconsistent field count (missing salary in row 10) +11. **Missing fields**: Systematic field omissions (no email in multiple rows) + #### Question Types -204 questions are generated dynamically across four categories: +209 questions are generated dynamically across five categories: - **Field retrieval (33%)**: Direct value lookups or values that can be read straight off a record (including booleans and simple counts such as array lengths) - Example: "What is Alice's salary?" โ†’ `75000` - Example: "How many items are in order ORD-0042?" โ†’ `3` - Example: "What is the customer name for order ORD-0042?" โ†’ `John Doe` -- **Aggregation (31%)**: Dataset-level totals and averages plus single-condition filters (counts, sums, min/max comparisons) +- **Aggregation (30%)**: Dataset-level totals and averages plus single-condition filters (counts, sums, min/max comparisons) - Example: "How many employees work in Engineering?" โ†’ `17` - Example: "What is the total revenue across all orders?" โ†’ `45123.50` - Example: "How many employees have salary > 80000?" โ†’ `23` -- **Filtering (24%)**: Multi-condition queries requiring compound logic (AND constraints across fields) +- **Filtering (23%)**: Multi-condition queries requiring compound logic (AND constraints across fields) - Example: "How many employees in Sales have salary > 80000?" โ†’ `5` - Example: "How many active employees have more than 10 years of experience?" โ†’ `8` @@ -566,18 +636,23 @@ Six datasets designed to test different structural patterns: - Example: "List the field names for employees" โ†’ `id, name, email, department, salary, yearsExperience, active` - Example: "What is the department of the last employee?" โ†’ `Sales` +- **Structural validation (2%)**: Tests ability to detect incomplete, truncated, or corrupted data using structural metadata + - Example: "Is this data complete and valid?" โ†’ `YES` (control dataset) or `NO` (corrupted datasets) + - Tests TOON's [N] length validation and {fields} consistency checking + - Demonstrates CSV's lack of structural validation capabilities + #### Evaluation Process -1. **Format conversion**: Each dataset is converted to all 6 formats (TOON, JSON compact, JSON, YAML, XML, CSV). +1. **Format conversion**: Each dataset is converted to all 6 formats (TOON, JSON compact, JSON, CSV, YAML, XML). 2. **Query LLM**: Each model receives formatted data + question in a prompt and extracts the answer. -3. **Validate with LLM-as-judge**: `gpt-5-nano` validates if the answer is semantically correct (e.g., `50000` = `$50,000`, `Engineering` = `engineering`, `2025-01-01` = `January 1, 2025`). +3. **Validate deterministically**: Answers are validated using type-aware comparison (e.g., `50000` = `$50,000`, `Engineering` = `engineering`, `2025-01-01` = `January 1, 2025`) without requiring an LLM judge. #### Models & Configuration - **Models tested**: `claude-haiku-4-5-20251001`, `gemini-2.5-flash`, `gpt-5-nano`, `grok-4-fast-non-reasoning` - **Token counting**: Using `gpt-tokenizer` with `o200k_base` encoding (GPT-5 tokenizer) - **Temperature**: Not set (models use their defaults) -- **Total evaluations**: 204 questions ร— 6 formats ร— 4 models = 4,896 LLM calls +- **Total evaluations**: 209 questions ร— 6 formats ร— 4 models = 5,016 LLM calls
@@ -782,6 +857,9 @@ items[1]: status: active ``` +> [!NOTE] +> Tabular format requires identical field sets across all objects (same keys, order doesn't matter) and primitive values only (strings, numbers, booleans, null). + #### Mixed and Non-Uniform Arrays Arrays that don't meet the tabular requirements use list format: diff --git a/benchmarks/README.md b/benchmarks/README.md index e995c58..1f3e463 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -34,10 +34,10 @@ Results are saved to `results/token-efficiency.md`. Tests how well LLMs can answer questions about data in different formats (TOON, JSON, JSON compact, XML, YAML, CSV): -1. Generate ~200 questions across 6 datasets (CSV only included for datasets with flat/tabular structure) +1. Generate 209 questions across 11 datasets (6 primary + 5 structural validation; CSV only included for datasets with flat/tabular structure) 2. Convert each dataset to all supported formats 3. Query each LLM with formatted data + question -4. Validate answers using `gpt-5-nano` as judge +4. Validate answers deterministically using type-aware comparison (no LLM judge needed) 5. Aggregate metrics and generate report ### Setup @@ -95,10 +95,22 @@ src/ โ”œโ”€โ”€ datasets.ts # Test data generators โ”œโ”€โ”€ evaluate.ts # LLM evaluation โ”œโ”€โ”€ formatters.ts # Format converters -โ”œโ”€โ”€ questions.ts # Question generation +โ”œโ”€โ”€ normalize.ts # Answer normalization โ”œโ”€โ”€ report.ts # Markdown reports โ”œโ”€โ”€ storage.ts # Result caching -โ””โ”€โ”€ utils.ts # Helpers +โ”œโ”€โ”€ types.ts # Type definitions +โ”œโ”€โ”€ utils.ts # Helpers +โ””โ”€โ”€ questions/ # Question generators + โ”œโ”€โ”€ analytics.ts + โ”œโ”€โ”€ event-logs.ts + โ”œโ”€โ”€ github.ts + โ”œโ”€โ”€ index.ts + โ”œโ”€โ”€ nested-config.ts + โ”œโ”€โ”€ nested.ts + โ”œโ”€โ”€ structural-validation.ts + โ”œโ”€โ”€ structure.ts + โ”œโ”€โ”€ tabular.ts + โ””โ”€โ”€ utils.ts data/ โ””โ”€โ”€ github-repos.json # Top 100 GitHub repos results/ diff --git a/benchmarks/results/accuracy/models/claude-haiku-4-5-20251001 b/benchmarks/results/accuracy/models/claude-haiku-4-5-20251001 index 3bacc16..c406372 100644 --- a/benchmarks/results/accuracy/models/claude-haiku-4-5-20251001 +++ b/benchmarks/results/accuracy/models/claude-haiku-4-5-20251001 @@ -1 +1 @@ -[{"questionId":"q1","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7888,"outputTokens":6,"latencyMs":1210.7189159999998},{"questionId":"q1","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":4779,"outputTokens":6,"latencyMs":1558.90225},{"questionId":"q1","format":"toon","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":3028,"outputTokens":6,"latencyMs":993.7559580000002},{"questionId":"q1","format":"csv","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2874,"outputTokens":6,"latencyMs":950.8272919999999},{"questionId":"q1","format":"xml","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":9373,"outputTokens":6,"latencyMs":1375.1998330000001},{"questionId":"q1","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5779,"outputTokens":6,"latencyMs":1128.227125},{"questionId":"q2","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7887,"outputTokens":4,"latencyMs":1651.9582919999998},{"questionId":"q2","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":4778,"outputTokens":4,"latencyMs":1260.3342499999999},{"questionId":"q2","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":3027,"outputTokens":4,"latencyMs":1160.4352919999997},{"questionId":"q2","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2873,"outputTokens":4,"latencyMs":1646.9346250000003},{"questionId":"q2","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":9372,"outputTokens":4,"latencyMs":1170.066167},{"questionId":"q2","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5778,"outputTokens":4,"latencyMs":979.0120420000003},{"questionId":"q3","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7892,"outputTokens":12,"latencyMs":1439.5705829999997},{"questionId":"q3","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4783,"outputTokens":12,"latencyMs":1139.6998750000002},{"questionId":"q3","format":"toon","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":3032,"outputTokens":12,"latencyMs":1220.2398750000002},{"questionId":"q3","format":"csv","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2878,"outputTokens":12,"latencyMs":1192.820667},{"questionId":"q3","format":"xml","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":9377,"outputTokens":12,"latencyMs":1761.4941250000002},{"questionId":"q3","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5783,"outputTokens":12,"latencyMs":1134.9589159999996},{"questionId":"q4","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":7890,"outputTokens":5,"latencyMs":1524.2077499999996},{"questionId":"q4","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":4781,"outputTokens":5,"latencyMs":1096.6124169999985},{"questionId":"q4","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":3030,"outputTokens":5,"latencyMs":1190.3290840000009},{"questionId":"q4","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":2876,"outputTokens":5,"latencyMs":1345.7378750000007},{"questionId":"q4","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":9375,"outputTokens":5,"latencyMs":1039.2939170000009},{"questionId":"q4","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":5781,"outputTokens":5,"latencyMs":1205.7180420000004},{"questionId":"q5","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"no","actual":"No","isCorrect":true,"inputTokens":7885,"outputTokens":4,"latencyMs":1211.265958},{"questionId":"q5","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"no","actual":"No","isCorrect":true,"inputTokens":4776,"outputTokens":4,"latencyMs":954.2045409999992},{"questionId":"q5","format":"toon","model":"claude-haiku-4-5-20251001","expected":"no","actual":"No","isCorrect":true,"inputTokens":3025,"outputTokens":4,"latencyMs":1534.906167000001},{"questionId":"q5","format":"csv","model":"claude-haiku-4-5-20251001","expected":"no","actual":"No","isCorrect":true,"inputTokens":2871,"outputTokens":4,"latencyMs":909.1663750000007},{"questionId":"q5","format":"xml","model":"claude-haiku-4-5-20251001","expected":"no","actual":"No","isCorrect":true,"inputTokens":9370,"outputTokens":4,"latencyMs":1309.5269580000004},{"questionId":"q5","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"no","actual":"No","isCorrect":true,"inputTokens":5776,"outputTokens":4,"latencyMs":1092.505333000001},{"questionId":"q6","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7888,"outputTokens":6,"latencyMs":1320.5142080000005},{"questionId":"q6","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":4779,"outputTokens":6,"latencyMs":1646.9417919999996},{"questionId":"q6","format":"toon","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":3028,"outputTokens":6,"latencyMs":1673.666792},{"questionId":"q6","format":"csv","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2874,"outputTokens":6,"latencyMs":823.2306659999995},{"questionId":"q6","format":"xml","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":9373,"outputTokens":6,"latencyMs":1228.8472079999992},{"questionId":"q6","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5779,"outputTokens":6,"latencyMs":1084.815458000001},{"questionId":"q7","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7888,"outputTokens":4,"latencyMs":1318.6587080000008},{"questionId":"q7","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":4779,"outputTokens":4,"latencyMs":1401.8515419999985},{"questionId":"q7","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":3028,"outputTokens":4,"latencyMs":2094.0003340000003},{"questionId":"q7","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2874,"outputTokens":4,"latencyMs":1238.8982910000013},{"questionId":"q7","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":9373,"outputTokens":4,"latencyMs":1496.2972500000033},{"questionId":"q7","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5779,"outputTokens":4,"latencyMs":1630.1005420000001},{"questionId":"q8","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7890,"outputTokens":12,"latencyMs":1197.1183339999989},{"questionId":"q8","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4781,"outputTokens":12,"latencyMs":1133.1408330000013},{"questionId":"q8","format":"toon","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":3030,"outputTokens":12,"latencyMs":1181.2194170000002},{"questionId":"q8","format":"csv","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2876,"outputTokens":12,"latencyMs":938.3579160000008},{"questionId":"q8","format":"xml","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":9375,"outputTokens":12,"latencyMs":1326.2782499999994},{"questionId":"q8","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5781,"outputTokens":12,"latencyMs":1691.4011250000003},{"questionId":"q9","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":7890,"outputTokens":5,"latencyMs":1291.4621250000018},{"questionId":"q9","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":4781,"outputTokens":5,"latencyMs":1420.1972079999978},{"questionId":"q9","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":3030,"outputTokens":5,"latencyMs":1249.945875000005},{"questionId":"q9","format":"csv","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":2876,"outputTokens":5,"latencyMs":1295.317084000002},{"questionId":"q9","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":9375,"outputTokens":5,"latencyMs":1406.5817499999976},{"questionId":"q9","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":5781,"outputTokens":5,"latencyMs":1243.1099170000016},{"questionId":"q10","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":7887,"outputTokens":4,"latencyMs":1155.9180409999972},{"questionId":"q10","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"true","isCorrect":true,"inputTokens":4778,"outputTokens":4,"latencyMs":1116.8734999999942},{"questionId":"q10","format":"toon","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":3027,"outputTokens":4,"latencyMs":1121.8300000000017},{"questionId":"q10","format":"csv","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":2873,"outputTokens":4,"latencyMs":1121.8768750000017},{"questionId":"q10","format":"xml","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":9372,"outputTokens":4,"latencyMs":1378.8897920000018},{"questionId":"q10","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":5778,"outputTokens":4,"latencyMs":1108.507749999997},{"questionId":"q11","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7887,"outputTokens":6,"latencyMs":1046.3552499999932},{"questionId":"q11","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":4778,"outputTokens":6,"latencyMs":1322.560124999989},{"questionId":"q11","format":"toon","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":3027,"outputTokens":6,"latencyMs":1340.5672090000007},{"questionId":"q11","format":"csv","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2873,"outputTokens":6,"latencyMs":4085.911666},{"questionId":"q11","format":"xml","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":9372,"outputTokens":6,"latencyMs":1510.3084590000071},{"questionId":"q11","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5778,"outputTokens":6,"latencyMs":1964.2606249999953},{"questionId":"q12","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7884,"outputTokens":4,"latencyMs":1527.3185419999936},{"questionId":"q12","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":4775,"outputTokens":4,"latencyMs":1221.1387910000049},{"questionId":"q12","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":3024,"outputTokens":4,"latencyMs":1185.782791000005},{"questionId":"q12","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2870,"outputTokens":4,"latencyMs":907.760374999998},{"questionId":"q12","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":9369,"outputTokens":4,"latencyMs":1579.382958000002},{"questionId":"q12","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5775,"outputTokens":4,"latencyMs":1491.0770420000044},{"questionId":"q13","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":7883,"outputTokens":5,"latencyMs":1112.994709000006},{"questionId":"q13","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":4774,"outputTokens":5,"latencyMs":1232.392000000007},{"questionId":"q13","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"14","isCorrect":false,"inputTokens":3023,"outputTokens":5,"latencyMs":1173.924165999997},{"questionId":"q13","format":"csv","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":2869,"outputTokens":5,"latencyMs":1432.9519999999902},{"questionId":"q13","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"14","isCorrect":false,"inputTokens":9368,"outputTokens":5,"latencyMs":1202.4103749999922},{"questionId":"q13","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":5774,"outputTokens":5,"latencyMs":2270.728667000003},{"questionId":"q14","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"13","isCorrect":false,"inputTokens":7883,"outputTokens":5,"latencyMs":1083.098874999996},{"questionId":"q14","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":4774,"outputTokens":5,"latencyMs":1151.881165999992},{"questionId":"q14","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"14","isCorrect":false,"inputTokens":3023,"outputTokens":5,"latencyMs":1085.5253749999974},{"questionId":"q14","format":"csv","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":2869,"outputTokens":5,"latencyMs":924.369458000001},{"questionId":"q14","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":9368,"outputTokens":5,"latencyMs":1374.0424169999897},{"questionId":"q14","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":5774,"outputTokens":5,"latencyMs":980.1530000000057},{"questionId":"q15","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":7883,"outputTokens":5,"latencyMs":1287.5655830000032},{"questionId":"q15","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":4774,"outputTokens":5,"latencyMs":1573.3662909999985},{"questionId":"q15","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"12","isCorrect":false,"inputTokens":3023,"outputTokens":5,"latencyMs":1020.0604170000006},{"questionId":"q15","format":"csv","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":2869,"outputTokens":5,"latencyMs":1598.8126670000056},{"questionId":"q15","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":9368,"outputTokens":5,"latencyMs":1058.768916000001},{"questionId":"q15","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":5774,"outputTokens":5,"latencyMs":1353.5325410000078},{"questionId":"q16","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"91","actual":"93","isCorrect":false,"inputTokens":7888,"outputTokens":5,"latencyMs":1129.9145829999907},{"questionId":"q16","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"91","actual":"82","isCorrect":false,"inputTokens":4779,"outputTokens":5,"latencyMs":1462.791207999995},{"questionId":"q16","format":"toon","model":"claude-haiku-4-5-20251001","expected":"91","actual":"87","isCorrect":false,"inputTokens":3028,"outputTokens":5,"latencyMs":1151.7741249999963},{"questionId":"q16","format":"csv","model":"claude-haiku-4-5-20251001","expected":"91","actual":"73","isCorrect":false,"inputTokens":2874,"outputTokens":5,"latencyMs":1031.250874999998},{"questionId":"q16","format":"xml","model":"claude-haiku-4-5-20251001","expected":"91","actual":"88","isCorrect":false,"inputTokens":9373,"outputTokens":5,"latencyMs":1306.8380409999954},{"questionId":"q16","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"91","actual":"89","isCorrect":false,"inputTokens":5779,"outputTokens":5,"latencyMs":1339.9416249999922},{"questionId":"q17","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"67","actual":"54","isCorrect":false,"inputTokens":7888,"outputTokens":5,"latencyMs":1175.3184580000088},{"questionId":"q17","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"67","actual":"57","isCorrect":false,"inputTokens":4779,"outputTokens":5,"latencyMs":1040.4461660000088},{"questionId":"q17","format":"toon","model":"claude-haiku-4-5-20251001","expected":"67","actual":"42","isCorrect":false,"inputTokens":3028,"outputTokens":5,"latencyMs":1735.4916669999948},{"questionId":"q17","format":"csv","model":"claude-haiku-4-5-20251001","expected":"67","actual":"42","isCorrect":false,"inputTokens":2874,"outputTokens":5,"latencyMs":1980.5700420000067},{"questionId":"q17","format":"xml","model":"claude-haiku-4-5-20251001","expected":"67","actual":"52","isCorrect":false,"inputTokens":9373,"outputTokens":5,"latencyMs":1852.883999999991},{"questionId":"q17","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"67","actual":"58","isCorrect":false,"inputTokens":5779,"outputTokens":5,"latencyMs":1273.8227920000063},{"questionId":"q18","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"41","actual":"35","isCorrect":false,"inputTokens":7888,"outputTokens":5,"latencyMs":1153.0166250000038},{"questionId":"q18","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"41","actual":"23","isCorrect":false,"inputTokens":4779,"outputTokens":5,"latencyMs":1144.4543750000012},{"questionId":"q18","format":"toon","model":"claude-haiku-4-5-20251001","expected":"41","actual":"28","isCorrect":false,"inputTokens":3028,"outputTokens":5,"latencyMs":1371.0743329999968},{"questionId":"q18","format":"csv","model":"claude-haiku-4-5-20251001","expected":"41","actual":"27","isCorrect":false,"inputTokens":2874,"outputTokens":5,"latencyMs":1047.8320419999945},{"questionId":"q18","format":"xml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"32","isCorrect":false,"inputTokens":9373,"outputTokens":5,"latencyMs":1350.8212920000078},{"questionId":"q18","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"31","isCorrect":false,"inputTokens":5779,"outputTokens":5,"latencyMs":1249.1499590000021},{"questionId":"q19","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":7884,"outputTokens":5,"latencyMs":1147.088957999993},{"questionId":"q19","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":4775,"outputTokens":5,"latencyMs":1231.8347499999945},{"questionId":"q19","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":3024,"outputTokens":5,"latencyMs":1181.9470829999918},{"questionId":"q19","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":2870,"outputTokens":5,"latencyMs":1210.713625000004},{"questionId":"q19","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9369,"outputTokens":5,"latencyMs":1015.4177080000081},{"questionId":"q19","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":5775,"outputTokens":5,"latencyMs":1197.7371249999997},{"questionId":"q20","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"99,884.99","isCorrect":false,"inputTokens":7885,"outputTokens":9,"latencyMs":1740.2427079999907},{"questionId":"q20","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"98947.92","isCorrect":false,"inputTokens":4776,"outputTokens":8,"latencyMs":1919.1532910000096},{"questionId":"q20","format":"toon","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"99,758.17","isCorrect":false,"inputTokens":3025,"outputTokens":9,"latencyMs":991.9548329999961},{"questionId":"q20","format":"csv","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"98,450.51","isCorrect":false,"inputTokens":2871,"outputTokens":9,"latencyMs":1093.8427919999958},{"questionId":"q20","format":"xml","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"100,112.89","isCorrect":false,"inputTokens":9370,"outputTokens":9,"latencyMs":1150.9055829999998},{"questionId":"q20","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"100594.47","isCorrect":false,"inputTokens":5776,"outputTokens":8,"latencyMs":1337.222041999994},{"questionId":"q21","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"78","actual":"76","isCorrect":false,"inputTokens":7882,"outputTokens":5,"latencyMs":1153.1522499999992},{"questionId":"q21","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"78","actual":"74","isCorrect":false,"inputTokens":4773,"outputTokens":5,"latencyMs":1197.8059159999975},{"questionId":"q21","format":"toon","model":"claude-haiku-4-5-20251001","expected":"78","actual":"79","isCorrect":false,"inputTokens":3022,"outputTokens":5,"latencyMs":990.6914170000091},{"questionId":"q21","format":"csv","model":"claude-haiku-4-5-20251001","expected":"78","actual":"73","isCorrect":false,"inputTokens":2868,"outputTokens":5,"latencyMs":1091.2322080000013},{"questionId":"q21","format":"xml","model":"claude-haiku-4-5-20251001","expected":"78","actual":"75","isCorrect":false,"inputTokens":9367,"outputTokens":5,"latencyMs":1264.0100410000014},{"questionId":"q21","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"78","actual":"75","isCorrect":false,"inputTokens":5773,"outputTokens":5,"latencyMs":1240.9275419999904},{"questionId":"q22","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":7882,"outputTokens":5,"latencyMs":1038.4012499999953},{"questionId":"q22","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":4773,"outputTokens":5,"latencyMs":1624.5698339999944},{"questionId":"q22","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"17","isCorrect":false,"inputTokens":3022,"outputTokens":5,"latencyMs":1036.7119579999999},{"questionId":"q22","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"21","isCorrect":false,"inputTokens":2868,"outputTokens":5,"latencyMs":1471.7847499999916},{"questionId":"q22","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":9367,"outputTokens":5,"latencyMs":1161.8851250000007},{"questionId":"q22","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":5773,"outputTokens":5,"latencyMs":975.5237920000218},{"questionId":"q23","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"12","actual":"11","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":1124.1700419999834},{"questionId":"q23","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"12","actual":"11","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":1086.0098339999968},{"questionId":"q23","format":"toon","model":"claude-haiku-4-5-20251001","expected":"12","actual":"9","isCorrect":false,"inputTokens":3030,"outputTokens":5,"latencyMs":978.0476670000062},{"questionId":"q23","format":"csv","model":"claude-haiku-4-5-20251001","expected":"12","actual":"10","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":1166.4321670000209},{"questionId":"q23","format":"xml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"9","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1433.4018750000105},{"questionId":"q23","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"10","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":1403.7286659999809},{"questionId":"q24","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":1363.1690829999861},{"questionId":"q24","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":955.9364579999819},{"questionId":"q24","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":3030,"outputTokens":5,"latencyMs":1449.7054160000116},{"questionId":"q24","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":859.7894169999927},{"questionId":"q24","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1033.2879580000008},{"questionId":"q24","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":1073.560959000024},{"questionId":"q25","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":1314.960166999983},{"questionId":"q25","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":1119.2924160000111},{"questionId":"q25","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":3030,"outputTokens":5,"latencyMs":1050.9931250000081},{"questionId":"q25","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":929.4169170000241},{"questionId":"q25","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1351.7019580000197},{"questionId":"q25","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":1356.250874999998},{"questionId":"q26","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"12","actual":"8","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":1629.827250000002},{"questionId":"q26","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"12","actual":"8","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":1319.8203750000102},{"questionId":"q26","format":"toon","model":"claude-haiku-4-5-20251001","expected":"12","actual":"6","isCorrect":false,"inputTokens":3030,"outputTokens":5,"latencyMs":1146.393624999997},{"questionId":"q26","format":"csv","model":"claude-haiku-4-5-20251001","expected":"12","actual":"6","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":1217.7608330000076},{"questionId":"q26","format":"xml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"7","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1496.661374999996},{"questionId":"q26","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"7","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":1132.3672500000102},{"questionId":"q27","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":1762.1454580000136},{"questionId":"q27","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":1277.2397089999868},{"questionId":"q27","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":3030,"outputTokens":5,"latencyMs":1539.9044580000045},{"questionId":"q27","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":1269.3111249999783},{"questionId":"q27","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1308.213584000012},{"questionId":"q27","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":169351.407875},{"questionId":"q28","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"63","actual":"73","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":1061.3787089999823},{"questionId":"q28","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"63","actual":"69","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":1147.2813749999914},{"questionId":"q28","format":"toon","model":"claude-haiku-4-5-20251001","expected":"63","actual":"67","isCorrect":false,"inputTokens":3030,"outputTokens":5,"latencyMs":1170.139165999979},{"questionId":"q28","format":"csv","model":"claude-haiku-4-5-20251001","expected":"63","actual":"62","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":905.4461660000088},{"questionId":"q28","format":"xml","model":"claude-haiku-4-5-20251001","expected":"63","actual":"62","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":992.9407079999801},{"questionId":"q28","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"63","actual":"73","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":970.3287909999781},{"questionId":"q29","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"53","actual":"54","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":1444.4941249999974},{"questionId":"q29","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"53","actual":"52","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":1377.6631669999915},{"questionId":"q29","format":"toon","model":"claude-haiku-4-5-20251001","expected":"53","actual":"53","isCorrect":true,"inputTokens":3030,"outputTokens":5,"latencyMs":977.0651660000149},{"questionId":"q29","format":"csv","model":"claude-haiku-4-5-20251001","expected":"53","actual":"42","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":987.6547079999873},{"questionId":"q29","format":"xml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"52","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":999.5382089999912},{"questionId":"q29","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"54","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":1191.6241659999941},{"questionId":"q30","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"39","actual":"34","isCorrect":false,"inputTokens":7890,"outputTokens":5,"latencyMs":2102.0624580000003},{"questionId":"q30","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"39","actual":"32","isCorrect":false,"inputTokens":4781,"outputTokens":5,"latencyMs":1129.8053329999966},{"questionId":"q30","format":"toon","model":"claude-haiku-4-5-20251001","expected":"39","actual":"32","isCorrect":false,"inputTokens":3030,"outputTokens":5,"latencyMs":1590.237792},{"questionId":"q30","format":"csv","model":"claude-haiku-4-5-20251001","expected":"39","actual":"31","isCorrect":false,"inputTokens":2876,"outputTokens":5,"latencyMs":1150.0739580000227},{"questionId":"q30","format":"xml","model":"claude-haiku-4-5-20251001","expected":"39","actual":"28","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1479.726290999999},{"questionId":"q30","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"39","actual":"31","isCorrect":false,"inputTokens":5781,"outputTokens":5,"latencyMs":1500.8358339999977},{"questionId":"q31","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":7891,"outputTokens":5,"latencyMs":1061.221292000002},{"questionId":"q31","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":4782,"outputTokens":5,"latencyMs":1173.3693330000096},{"questionId":"q31","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":3031,"outputTokens":5,"latencyMs":1393.310832999996},{"questionId":"q31","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":2877,"outputTokens":5,"latencyMs":1029.9407919999794},{"questionId":"q31","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":9376,"outputTokens":5,"latencyMs":1778.361917000002},{"questionId":"q31","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":5782,"outputTokens":5,"latencyMs":1278.9144169999927},{"questionId":"q32","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"8","actual":"7","isCorrect":false,"inputTokens":7891,"outputTokens":5,"latencyMs":1410.4881250000035},{"questionId":"q32","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":4782,"outputTokens":5,"latencyMs":1530.299583999993},{"questionId":"q32","format":"toon","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":3031,"outputTokens":5,"latencyMs":884.0975000000035},{"questionId":"q32","format":"csv","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":2877,"outputTokens":5,"latencyMs":1189.612792},{"questionId":"q32","format":"xml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":9376,"outputTokens":5,"latencyMs":1241.3236669999897},{"questionId":"q32","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":5782,"outputTokens":5,"latencyMs":1148.9409159999923},{"questionId":"q33","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"15","actual":"8","isCorrect":false,"inputTokens":7891,"outputTokens":5,"latencyMs":1047.500417000003},{"questionId":"q33","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"15","actual":"9","isCorrect":false,"inputTokens":4782,"outputTokens":5,"latencyMs":1049.9271670000162},{"questionId":"q33","format":"toon","model":"claude-haiku-4-5-20251001","expected":"15","actual":"7","isCorrect":false,"inputTokens":3031,"outputTokens":5,"latencyMs":1201.0838330000115},{"questionId":"q33","format":"csv","model":"claude-haiku-4-5-20251001","expected":"15","actual":"8","isCorrect":false,"inputTokens":2877,"outputTokens":5,"latencyMs":891.4600420000206},{"questionId":"q33","format":"xml","model":"claude-haiku-4-5-20251001","expected":"15","actual":"9","isCorrect":false,"inputTokens":9376,"outputTokens":5,"latencyMs":1333.9015420000069},{"questionId":"q33","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"15","actual":"9","isCorrect":false,"inputTokens":5782,"outputTokens":5,"latencyMs":1727.5371670000022},{"questionId":"q34","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"12","actual":"13","isCorrect":false,"inputTokens":7884,"outputTokens":5,"latencyMs":989.0173340000038},{"questionId":"q34","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"12","actual":"11","isCorrect":false,"inputTokens":4775,"outputTokens":5,"latencyMs":1275.1794999999984},{"questionId":"q34","format":"toon","model":"claude-haiku-4-5-20251001","expected":"12","actual":"13","isCorrect":false,"inputTokens":3024,"outputTokens":5,"latencyMs":1190.0951249999925},{"questionId":"q34","format":"csv","model":"claude-haiku-4-5-20251001","expected":"12","actual":"12","isCorrect":true,"inputTokens":2870,"outputTokens":5,"latencyMs":1034.840750000003},{"questionId":"q34","format":"xml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"13","isCorrect":false,"inputTokens":9369,"outputTokens":5,"latencyMs":1197.2422079999815},{"questionId":"q34","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"13","isCorrect":false,"inputTokens":5775,"outputTokens":5,"latencyMs":1683.550041999988},{"questionId":"q35","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":7884,"outputTokens":5,"latencyMs":1073.9468750000233},{"questionId":"q35","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"13","isCorrect":false,"inputTokens":4775,"outputTokens":5,"latencyMs":1278.042792000022},{"questionId":"q35","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":3024,"outputTokens":5,"latencyMs":1796.531290999992},{"questionId":"q35","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":2870,"outputTokens":5,"latencyMs":2176.8507499999832},{"questionId":"q35","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"12","isCorrect":false,"inputTokens":9369,"outputTokens":5,"latencyMs":1126.361957999994},{"questionId":"q35","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"12","isCorrect":false,"inputTokens":5775,"outputTokens":5,"latencyMs":1000.8374170000025},{"questionId":"q36","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":13020,"outputTokens":7,"latencyMs":1877.2704999999842},{"questionId":"q36","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":7981,"outputTokens":7,"latencyMs":1061.2825410000223},{"questionId":"q36","format":"toon","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8298,"outputTokens":7,"latencyMs":1279.4919169999775},{"questionId":"q36","format":"xml","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":14525,"outputTokens":7,"latencyMs":2190.0840000000026},{"questionId":"q36","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":9423,"outputTokens":7,"latencyMs":1279.1115829999908},{"questionId":"q37","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":13020,"outputTokens":4,"latencyMs":1326.8059999999823},{"questionId":"q37","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":7981,"outputTokens":4,"latencyMs":1126.6603340000147},{"questionId":"q37","format":"toon","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8298,"outputTokens":4,"latencyMs":1360.8319999999949},{"questionId":"q37","format":"xml","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":14525,"outputTokens":4,"latencyMs":1159.0872500000114},{"questionId":"q37","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":9423,"outputTokens":4,"latencyMs":999.0764579999959},{"questionId":"q38","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":13020,"outputTokens":7,"latencyMs":1195.0188330000092},{"questionId":"q38","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":7981,"outputTokens":7,"latencyMs":2066.192208000022},{"questionId":"q38","format":"toon","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8298,"outputTokens":7,"latencyMs":1108.658791999973},{"questionId":"q38","format":"xml","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":14525,"outputTokens":7,"latencyMs":1606.5897910000058},{"questionId":"q38","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":9423,"outputTokens":7,"latencyMs":1122.106625000015},{"questionId":"q39","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":13020,"outputTokens":4,"latencyMs":1778.1102079999982},{"questionId":"q39","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":7981,"outputTokens":4,"latencyMs":1312.048959000007},{"questionId":"q39","format":"toon","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8298,"outputTokens":4,"latencyMs":1387.956750000012},{"questionId":"q39","format":"xml","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":14525,"outputTokens":4,"latencyMs":1334.5222080000094},{"questionId":"q39","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":9423,"outputTokens":4,"latencyMs":1689.652666000009},{"questionId":"q40","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":13020,"outputTokens":8,"latencyMs":1423.433834000025},{"questionId":"q40","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":7981,"outputTokens":8,"latencyMs":997.4557500000228},{"questionId":"q40","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8298,"outputTokens":8,"latencyMs":1305.189000000013},{"questionId":"q40","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":14525,"outputTokens":8,"latencyMs":1829.152167000022},{"questionId":"q40","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":9423,"outputTokens":8,"latencyMs":1535.2441249999974},{"questionId":"q41","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":13020,"outputTokens":4,"latencyMs":1362.3707500000019},{"questionId":"q41","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":7981,"outputTokens":4,"latencyMs":1149.0101670000004},{"questionId":"q41","format":"toon","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8298,"outputTokens":4,"latencyMs":1340.8606670000008},{"questionId":"q41","format":"xml","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":14525,"outputTokens":4,"latencyMs":1366.5337500000023},{"questionId":"q41","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":9423,"outputTokens":4,"latencyMs":1261.2614590000012},{"questionId":"q42","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":13020,"outputTokens":8,"latencyMs":1366.4395829999994},{"questionId":"q42","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":7981,"outputTokens":8,"latencyMs":1169.115334000002},{"questionId":"q42","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8298,"outputTokens":8,"latencyMs":1726.6568330000155},{"questionId":"q42","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":14525,"outputTokens":8,"latencyMs":2412.5611250000075},{"questionId":"q42","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":9423,"outputTokens":8,"latencyMs":1188.3911250000237},{"questionId":"q43","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13020,"outputTokens":4,"latencyMs":1254.5480409999727},{"questionId":"q43","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7981,"outputTokens":4,"latencyMs":1356.305250000005},{"questionId":"q43","format":"toon","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8298,"outputTokens":4,"latencyMs":1367.7742500000168},{"questionId":"q43","format":"xml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":14525,"outputTokens":4,"latencyMs":1786.5491669999901},{"questionId":"q43","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9423,"outputTokens":4,"latencyMs":1337.3919160000223},{"questionId":"q44","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":13021,"outputTokens":11,"latencyMs":1836.185792000033},{"questionId":"q44","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":7982,"outputTokens":11,"latencyMs":1526.692333999963},{"questionId":"q44","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8299,"outputTokens":11,"latencyMs":134599.87291600002},{"questionId":"q44","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":14526,"outputTokens":11,"latencyMs":1764.925709000032},{"questionId":"q44","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":9424,"outputTokens":11,"latencyMs":2104.7243339999695},{"questionId":"q45","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":13021,"outputTokens":16,"latencyMs":2079.898333000019},{"questionId":"q45","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":7982,"outputTokens":16,"latencyMs":2299.298167000001},{"questionId":"q45","format":"toon","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8299,"outputTokens":16,"latencyMs":2531.651708999998},{"questionId":"q45","format":"xml","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":14526,"outputTokens":16,"latencyMs":1717.160166000016},{"questionId":"q45","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":9424,"outputTokens":16,"latencyMs":1261.9375},{"questionId":"q46","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":13021,"outputTokens":10,"latencyMs":1568.283999999985},{"questionId":"q46","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":7982,"outputTokens":10,"latencyMs":977.8154170000344},{"questionId":"q46","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8299,"outputTokens":10,"latencyMs":1274.0614589999896},{"questionId":"q46","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":14526,"outputTokens":10,"latencyMs":1344.8440829999745},{"questionId":"q46","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":9424,"outputTokens":10,"latencyMs":1758.2577499999898},{"questionId":"q47","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":13020,"outputTokens":5,"latencyMs":1221.24679200002},{"questionId":"q47","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"10","isCorrect":false,"inputTokens":7981,"outputTokens":5,"latencyMs":1632.9894580000546},{"questionId":"q47","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":8298,"outputTokens":5,"latencyMs":1078.8186250000144},{"questionId":"q47","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":14525,"outputTokens":5,"latencyMs":1238.5565829999978},{"questionId":"q47","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":9423,"outputTokens":5,"latencyMs":1557.3261250000214},{"questionId":"q48","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":13021,"outputTokens":9,"latencyMs":1143.566083999991},{"questionId":"q48","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":7982,"outputTokens":9,"latencyMs":1315.0375840000343},{"questionId":"q48","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8299,"outputTokens":9,"latencyMs":1936.8320829999866},{"questionId":"q48","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":14526,"outputTokens":9,"latencyMs":1378.4906670000055},{"questionId":"q48","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":9424,"outputTokens":9,"latencyMs":1440.9520839999896},{"questionId":"q49","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":13021,"outputTokens":13,"latencyMs":1431.4234169999836},{"questionId":"q49","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":7982,"outputTokens":13,"latencyMs":1096.4912079999922},{"questionId":"q49","format":"toon","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8299,"outputTokens":13,"latencyMs":1349.268665999989},{"questionId":"q49","format":"xml","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":14526,"outputTokens":13,"latencyMs":1583.6609579999931},{"questionId":"q49","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":9424,"outputTokens":13,"latencyMs":1671.1608750000014},{"questionId":"q50","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":13021,"outputTokens":10,"latencyMs":1557.6222919999855},{"questionId":"q50","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":7982,"outputTokens":10,"latencyMs":1279.2233750000014},{"questionId":"q50","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":8299,"outputTokens":10,"latencyMs":1045.5043750000186},{"questionId":"q50","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":14526,"outputTokens":10,"latencyMs":1203.7452500000363},{"questionId":"q50","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":9424,"outputTokens":10,"latencyMs":1044.3679589999956},{"questionId":"q51","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":13020,"outputTokens":5,"latencyMs":1203.4841659999802},{"questionId":"q51","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":7981,"outputTokens":5,"latencyMs":990.5502080000006},{"questionId":"q51","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":8298,"outputTokens":5,"latencyMs":2314.0848750000005},{"questionId":"q51","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":14525,"outputTokens":5,"latencyMs":1664.362124999985},{"questionId":"q51","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":9423,"outputTokens":5,"latencyMs":1207.373041999992},{"questionId":"q52","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":13016,"outputTokens":5,"latencyMs":1132.1158750000177},{"questionId":"q52","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":7977,"outputTokens":5,"latencyMs":964.0262909999583},{"questionId":"q52","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":8294,"outputTokens":5,"latencyMs":1290.6799159999937},{"questionId":"q52","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":14521,"outputTokens":5,"latencyMs":1771.8612089999951},{"questionId":"q52","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":9419,"outputTokens":5,"latencyMs":989.1434170000139},{"questionId":"q53","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":13016,"outputTokens":5,"latencyMs":1462.7887080000364},{"questionId":"q53","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":7977,"outputTokens":5,"latencyMs":984.8299159999588},{"questionId":"q53","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":8294,"outputTokens":5,"latencyMs":1356.3089170000167},{"questionId":"q53","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":14521,"outputTokens":5,"latencyMs":1269.7774590000045},{"questionId":"q53","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":9419,"outputTokens":5,"latencyMs":1580.1444999999949},{"questionId":"q54","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":13016,"outputTokens":5,"latencyMs":1715.1656250000233},{"questionId":"q54","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":7977,"outputTokens":5,"latencyMs":1272.5985830000136},{"questionId":"q54","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":8294,"outputTokens":5,"latencyMs":1287.9930000000168},{"questionId":"q54","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":14521,"outputTokens":5,"latencyMs":1951.4127499999595},{"questionId":"q54","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":9419,"outputTokens":5,"latencyMs":1229.9152080000495},{"questionId":"q55","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"40,895.96","isCorrect":false,"inputTokens":13016,"outputTokens":9,"latencyMs":2050.2400830000406},{"questionId":"q55","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"39847.36","isCorrect":false,"inputTokens":7977,"outputTokens":8,"latencyMs":4447.450750000018},{"questionId":"q55","format":"toon","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"39447.04","isCorrect":false,"inputTokens":8294,"outputTokens":8,"latencyMs":1377.8202080000192},{"questionId":"q55","format":"xml","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"37,847.74","isCorrect":false,"inputTokens":14521,"outputTokens":9,"latencyMs":1616.2727920000325},{"questionId":"q55","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"41,822.34","isCorrect":false,"inputTokens":9419,"outputTokens":9,"latencyMs":1325.8112090000068},{"questionId":"q56","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"732.89","isCorrect":false,"inputTokens":13014,"outputTokens":7,"latencyMs":136942.92754200002},{"questionId":"q56","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"768.79","isCorrect":false,"inputTokens":7975,"outputTokens":7,"latencyMs":1218.7471659999574},{"questionId":"q56","format":"toon","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"772.89","isCorrect":false,"inputTokens":8292,"outputTokens":7,"latencyMs":1441.0296669999952},{"questionId":"q56","format":"xml","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"805.42","isCorrect":false,"inputTokens":14519,"outputTokens":7,"latencyMs":1610.0354170000064},{"questionId":"q56","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"762.89","isCorrect":false,"inputTokens":9417,"outputTokens":7,"latencyMs":1444.1710409999941},{"questionId":"q57","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":13015,"outputTokens":5,"latencyMs":1477.2014999999665},{"questionId":"q57","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":7976,"outputTokens":5,"latencyMs":1185.721458999964},{"questionId":"q57","format":"toon","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":8293,"outputTokens":5,"latencyMs":2056.8777919999557},{"questionId":"q57","format":"xml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":14520,"outputTokens":5,"latencyMs":1333.0480000000098},{"questionId":"q57","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":9418,"outputTokens":5,"latencyMs":1429.2252909999806},{"questionId":"q58","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":13014,"outputTokens":8,"latencyMs":1367.4437499999767},{"questionId":"q58","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":7975,"outputTokens":8,"latencyMs":1527.104250000033},{"questionId":"q58","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8292,"outputTokens":8,"latencyMs":1224.509458000015},{"questionId":"q58","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":14519,"outputTokens":8,"latencyMs":2327.4375829999917},{"questionId":"q58","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":9417,"outputTokens":8,"latencyMs":1940.742957999988},{"questionId":"q59","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"43","actual":"48","isCorrect":false,"inputTokens":13018,"outputTokens":5,"latencyMs":1190.3244169999962},{"questionId":"q59","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"43","actual":"48","isCorrect":false,"inputTokens":7979,"outputTokens":5,"latencyMs":1532.6525419999962},{"questionId":"q59","format":"toon","model":"claude-haiku-4-5-20251001","expected":"43","actual":"44","isCorrect":false,"inputTokens":8296,"outputTokens":5,"latencyMs":1179.2960829999647},{"questionId":"q59","format":"xml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"45","isCorrect":false,"inputTokens":14523,"outputTokens":5,"latencyMs":137004.635167},{"questionId":"q59","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"48","isCorrect":false,"inputTokens":9421,"outputTokens":5,"latencyMs":1339.1403750000172},{"questionId":"q60","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"37","actual":"41","isCorrect":false,"inputTokens":13018,"outputTokens":5,"latencyMs":1236.763833999983},{"questionId":"q60","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"37","actual":"38","isCorrect":false,"inputTokens":7979,"outputTokens":5,"latencyMs":2161.9954580000485},{"questionId":"q60","format":"toon","model":"claude-haiku-4-5-20251001","expected":"37","actual":"34","isCorrect":false,"inputTokens":8296,"outputTokens":5,"latencyMs":1194.7575419999775},{"questionId":"q60","format":"xml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"38","isCorrect":false,"inputTokens":14523,"outputTokens":5,"latencyMs":1315.2825420000008},{"questionId":"q60","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"34","isCorrect":false,"inputTokens":9421,"outputTokens":5,"latencyMs":1363.0747919999994},{"questionId":"q61","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"28","actual":"32","isCorrect":false,"inputTokens":13018,"outputTokens":5,"latencyMs":1074.8357499999693},{"questionId":"q61","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"28","actual":"31","isCorrect":false,"inputTokens":7979,"outputTokens":5,"latencyMs":1676.432833000028},{"questionId":"q61","format":"toon","model":"claude-haiku-4-5-20251001","expected":"28","actual":"28","isCorrect":true,"inputTokens":8296,"outputTokens":5,"latencyMs":1122.5760829999927},{"questionId":"q61","format":"xml","model":"claude-haiku-4-5-20251001","expected":"28","actual":"27","isCorrect":false,"inputTokens":14523,"outputTokens":5,"latencyMs":1429.8751249999623},{"questionId":"q61","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"28","actual":"28","isCorrect":true,"inputTokens":9421,"outputTokens":5,"latencyMs":1676.8320420000236},{"questionId":"q62","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":13022,"outputTokens":5,"latencyMs":1407.8011249999981},{"questionId":"q62","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"8","actual":"7","isCorrect":false,"inputTokens":7983,"outputTokens":5,"latencyMs":1221.6820000000298},{"questionId":"q62","format":"toon","model":"claude-haiku-4-5-20251001","expected":"8","actual":"6","isCorrect":false,"inputTokens":8300,"outputTokens":5,"latencyMs":1366.9517920000362},{"questionId":"q62","format":"xml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":14527,"outputTokens":5,"latencyMs":1228.7712080000201},{"questionId":"q62","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":9425,"outputTokens":5,"latencyMs":1266.3425830000197},{"questionId":"q63","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6","actual":"8","isCorrect":false,"inputTokens":13022,"outputTokens":5,"latencyMs":1335.2824580000015},{"questionId":"q63","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6","actual":"7","isCorrect":false,"inputTokens":7983,"outputTokens":5,"latencyMs":2025.3399169999757},{"questionId":"q63","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":8300,"outputTokens":5,"latencyMs":1620.6212919999962},{"questionId":"q63","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":14527,"outputTokens":5,"latencyMs":1218.0760840000003},{"questionId":"q63","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"7","isCorrect":false,"inputTokens":9425,"outputTokens":5,"latencyMs":1507.2060410000267},{"questionId":"q64","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":13022,"outputTokens":5,"latencyMs":1744.654582999996},{"questionId":"q64","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"6","isCorrect":false,"inputTokens":7983,"outputTokens":5,"latencyMs":1201.6237919999985},{"questionId":"q64","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":8300,"outputTokens":5,"latencyMs":1788.556333000015},{"questionId":"q64","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":14527,"outputTokens":5,"latencyMs":1798.2694170000032},{"questionId":"q64","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":9425,"outputTokens":5,"latencyMs":1371.8953749999637},{"questionId":"q65","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":13022,"outputTokens":5,"latencyMs":1428.5923750000075},{"questionId":"q65","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":7983,"outputTokens":5,"latencyMs":1213.5106249999953},{"questionId":"q65","format":"toon","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":8300,"outputTokens":5,"latencyMs":1153.3391249999986},{"questionId":"q65","format":"xml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":14527,"outputTokens":5,"latencyMs":1358.6227500000386},{"questionId":"q65","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":9425,"outputTokens":5,"latencyMs":1041.3409159999574},{"questionId":"q66","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"4","isCorrect":false,"inputTokens":13023,"outputTokens":5,"latencyMs":1429.8350419999915},{"questionId":"q66","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":7984,"outputTokens":5,"latencyMs":1130.6302500000456},{"questionId":"q66","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":8301,"outputTokens":5,"latencyMs":1183.4113329999964},{"questionId":"q66","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"6","isCorrect":false,"inputTokens":14528,"outputTokens":5,"latencyMs":1403.4962079999968},{"questionId":"q66","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"6","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1552.7885419999948},{"questionId":"q67","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"6","isCorrect":false,"inputTokens":13023,"outputTokens":5,"latencyMs":1547.673749999958},{"questionId":"q67","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":7984,"outputTokens":5,"latencyMs":1488.8348750000005},{"questionId":"q67","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"6","isCorrect":false,"inputTokens":8301,"outputTokens":5,"latencyMs":1294.1888749999925},{"questionId":"q67","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":14528,"outputTokens":5,"latencyMs":1580.0848339999793},{"questionId":"q67","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"6","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1412.2284579999978},{"questionId":"q68","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":13023,"outputTokens":5,"latencyMs":1436.3784999999916},{"questionId":"q68","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":7984,"outputTokens":5,"latencyMs":135647.30895799998},{"questionId":"q68","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":8301,"outputTokens":5,"latencyMs":1149.2568330000504},{"questionId":"q68","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":14528,"outputTokens":5,"latencyMs":1191.73770899995},{"questionId":"q68","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1037.708541000029},{"questionId":"q69","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"20","actual":"23","isCorrect":false,"inputTokens":13026,"outputTokens":5,"latencyMs":2555.6043340000324},{"questionId":"q69","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"20","actual":"23","isCorrect":false,"inputTokens":7987,"outputTokens":5,"latencyMs":1268.627708999964},{"questionId":"q69","format":"toon","model":"claude-haiku-4-5-20251001","expected":"20","actual":"20","isCorrect":true,"inputTokens":8304,"outputTokens":5,"latencyMs":1064.138750000042},{"questionId":"q69","format":"xml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"26","isCorrect":false,"inputTokens":14531,"outputTokens":5,"latencyMs":1462.3702090000152},{"questionId":"q69","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"23","isCorrect":false,"inputTokens":9429,"outputTokens":5,"latencyMs":1708.8065829999978},{"questionId":"q70","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"19","actual":"18","isCorrect":false,"inputTokens":13026,"outputTokens":5,"latencyMs":1303.9015840000357},{"questionId":"q70","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"19","actual":"23","isCorrect":false,"inputTokens":7987,"outputTokens":5,"latencyMs":1000.4688749999623},{"questionId":"q70","format":"toon","model":"claude-haiku-4-5-20251001","expected":"19","actual":"19","isCorrect":true,"inputTokens":8304,"outputTokens":5,"latencyMs":1066.8717499999912},{"questionId":"q70","format":"xml","model":"claude-haiku-4-5-20251001","expected":"19","actual":"18","isCorrect":false,"inputTokens":14531,"outputTokens":5,"latencyMs":1540.689624999999},{"questionId":"q70","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"19","actual":"23","isCorrect":false,"inputTokens":9429,"outputTokens":5,"latencyMs":1036.3092079999624},{"questionId":"q71","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4095,"outputTokens":6,"latencyMs":1445.6645410000347},{"questionId":"q71","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2406,"outputTokens":6,"latencyMs":890.475541999971},{"questionId":"q71","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1552,"outputTokens":6,"latencyMs":1115.657125000027},{"questionId":"q71","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1460,"outputTokens":6,"latencyMs":1021.6982919999864},{"questionId":"q71","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4797,"outputTokens":6,"latencyMs":2825.67654200003},{"questionId":"q71","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3126,"outputTokens":6,"latencyMs":1105.7810000000172},{"questionId":"q72","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4095,"outputTokens":8,"latencyMs":1006.6233340000035},{"questionId":"q72","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2406,"outputTokens":8,"latencyMs":1475.1360420000274},{"questionId":"q72","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1552,"outputTokens":8,"latencyMs":1306.760500000033},{"questionId":"q72","format":"csv","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1460,"outputTokens":8,"latencyMs":1152.217249999987},{"questionId":"q72","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4797,"outputTokens":8,"latencyMs":1159.180541000038},{"questionId":"q72","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3126,"outputTokens":8,"latencyMs":1042.360791999963},{"questionId":"q73","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4096,"outputTokens":7,"latencyMs":964.712332999974},{"questionId":"q73","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2407,"outputTokens":7,"latencyMs":1077.9677080000401},{"questionId":"q73","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1553,"outputTokens":7,"latencyMs":888.1847500000149},{"questionId":"q73","format":"csv","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1461,"outputTokens":7,"latencyMs":2202.4551659999997},{"questionId":"q73","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4798,"outputTokens":7,"latencyMs":2407.378541999962},{"questionId":"q73","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3127,"outputTokens":7,"latencyMs":1260.8838750000577},{"questionId":"q74","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":4097,"outputTokens":5,"latencyMs":1068.901750000019},{"questionId":"q74","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":2408,"outputTokens":5,"latencyMs":2280.0547910000314},{"questionId":"q74","format":"toon","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":1554,"outputTokens":5,"latencyMs":1518.1229170000297},{"questionId":"q74","format":"csv","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":1462,"outputTokens":5,"latencyMs":1005.2864169999957},{"questionId":"q74","format":"xml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":4799,"outputTokens":5,"latencyMs":1377.9812500000116},{"questionId":"q74","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":3128,"outputTokens":5,"latencyMs":896.8197919999948},{"questionId":"q75","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4095,"outputTokens":6,"latencyMs":1041.8436249999795},{"questionId":"q75","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2406,"outputTokens":6,"latencyMs":1591.857291000022},{"questionId":"q75","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1552,"outputTokens":6,"latencyMs":134153.22737500007},{"questionId":"q75","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1460,"outputTokens":6,"latencyMs":2621.437166000018},{"questionId":"q75","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4797,"outputTokens":6,"latencyMs":1206.6165000000037},{"questionId":"q75","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3126,"outputTokens":6,"latencyMs":978.7292090000119},{"questionId":"q76","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4095,"outputTokens":8,"latencyMs":1551.3816670000087},{"questionId":"q76","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2406,"outputTokens":8,"latencyMs":1336.6815410000272},{"questionId":"q76","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1552,"outputTokens":8,"latencyMs":1139.947125000006},{"questionId":"q76","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1460,"outputTokens":8,"latencyMs":932.0748749999912},{"questionId":"q76","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4797,"outputTokens":8,"latencyMs":1333.9932910000207},{"questionId":"q76","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3126,"outputTokens":8,"latencyMs":1271.1030000000028},{"questionId":"q77","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4096,"outputTokens":7,"latencyMs":981.6602919999859},{"questionId":"q77","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2407,"outputTokens":7,"latencyMs":811.8191670000087},{"questionId":"q77","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1553,"outputTokens":7,"latencyMs":1184.6716250000172},{"questionId":"q77","format":"csv","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1461,"outputTokens":7,"latencyMs":1062.654166000022},{"questionId":"q77","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4798,"outputTokens":7,"latencyMs":1256.8684169999906},{"questionId":"q77","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3127,"outputTokens":7,"latencyMs":1064.5108340000152},{"questionId":"q78","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":4097,"outputTokens":5,"latencyMs":1105.1710420000018},{"questionId":"q78","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":2408,"outputTokens":5,"latencyMs":1352.293042000034},{"questionId":"q78","format":"toon","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":1554,"outputTokens":5,"latencyMs":1141.2828749999753},{"questionId":"q78","format":"csv","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":1462,"outputTokens":5,"latencyMs":1094.8242090000422},{"questionId":"q78","format":"xml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":4799,"outputTokens":5,"latencyMs":1199.8344159999979},{"questionId":"q78","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":3128,"outputTokens":5,"latencyMs":1093.0980829999899},{"questionId":"q79","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4095,"outputTokens":6,"latencyMs":1205.9528330000467},{"questionId":"q79","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2406,"outputTokens":6,"latencyMs":1070.842541000049},{"questionId":"q79","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1552,"outputTokens":6,"latencyMs":1010.1033329999773},{"questionId":"q79","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1460,"outputTokens":6,"latencyMs":1085.5609579999582},{"questionId":"q79","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4797,"outputTokens":6,"latencyMs":1161.1155410000356},{"questionId":"q79","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3126,"outputTokens":6,"latencyMs":1122.71541600005},{"questionId":"q80","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":4092,"outputTokens":5,"latencyMs":1073.2076670000097},{"questionId":"q80","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":2403,"outputTokens":5,"latencyMs":995.7284590000054},{"questionId":"q80","format":"toon","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":1549,"outputTokens":5,"latencyMs":845.3497089999728},{"questionId":"q80","format":"csv","model":"claude-haiku-4-5-20251001","expected":"60","actual":"59","isCorrect":false,"inputTokens":1457,"outputTokens":5,"latencyMs":886.6102499999688},{"questionId":"q80","format":"xml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":4794,"outputTokens":5,"latencyMs":1157.310165999981},{"questionId":"q80","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":3123,"outputTokens":5,"latencyMs":1132.964666999993},{"questionId":"q81","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"352,378","isCorrect":false,"inputTokens":4093,"outputTokens":7,"latencyMs":1095.3754999999655},{"questionId":"q81","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"319,555","isCorrect":false,"inputTokens":2404,"outputTokens":7,"latencyMs":1041.2718749999767},{"questionId":"q81","format":"toon","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"330,557","isCorrect":false,"inputTokens":1550,"outputTokens":7,"latencyMs":1020.2267090000096},{"questionId":"q81","format":"csv","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"349,968","isCorrect":false,"inputTokens":1458,"outputTokens":7,"latencyMs":1130.9903330000234},{"questionId":"q81","format":"xml","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"325,831","isCorrect":false,"inputTokens":4795,"outputTokens":7,"latencyMs":1567.1822919999831},{"questionId":"q81","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"182,588","isCorrect":false,"inputTokens":3124,"outputTokens":7,"latencyMs":1206.90533400001},{"questionId":"q82","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1370","isCorrect":false,"inputTokens":4094,"outputTokens":6,"latencyMs":1258.9610000000102},{"questionId":"q82","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1617","isCorrect":false,"inputTokens":2405,"outputTokens":6,"latencyMs":1399.7614169999724},{"questionId":"q82","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1625","isCorrect":false,"inputTokens":1551,"outputTokens":6,"latencyMs":1181.7479590000585},{"questionId":"q82","format":"csv","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1442","isCorrect":false,"inputTokens":1459,"outputTokens":6,"latencyMs":1267.9241250000196},{"questionId":"q82","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1396","isCorrect":false,"inputTokens":4796,"outputTokens":6,"latencyMs":1313.2502080000704},{"questionId":"q82","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1356","isCorrect":false,"inputTokens":3125,"outputTokens":6,"latencyMs":1171.9902500000317},{"questionId":"q83","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"409,821.89","isCorrect":false,"inputTokens":4091,"outputTokens":9,"latencyMs":1235.642332999967},{"questionId":"q83","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"378,891.97","isCorrect":false,"inputTokens":2402,"outputTokens":9,"latencyMs":1311.002666000044},{"questionId":"q83","format":"toon","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"361,548.62","isCorrect":false,"inputTokens":1548,"outputTokens":9,"latencyMs":1150.6039999999339},{"questionId":"q83","format":"csv","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"428,639.15","isCorrect":false,"inputTokens":1456,"outputTokens":9,"latencyMs":1094.9247500000056},{"questionId":"q83","format":"xml","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"381,832.14","isCorrect":false,"inputTokens":4793,"outputTokens":9,"latencyMs":1011.7133749999339},{"questionId":"q83","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"389,498.54","isCorrect":false,"inputTokens":3122,"outputTokens":9,"latencyMs":1043.9226250000065},{"questionId":"q84","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.5355","isCorrect":false,"inputTokens":4089,"outputTokens":8,"latencyMs":1295.4472079999978},{"questionId":"q84","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.5355","isCorrect":false,"inputTokens":2400,"outputTokens":8,"latencyMs":1916.0808340000222},{"questionId":"q84","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.5317","isCorrect":true,"inputTokens":1546,"outputTokens":8,"latencyMs":1264.7692080000415},{"questionId":"q84","format":"csv","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.5229","isCorrect":false,"inputTokens":1454,"outputTokens":8,"latencyMs":1405.785125000053},{"questionId":"q84","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4791,"outputTokens":7,"latencyMs":1364.1582499999786},{"questionId":"q84","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.5348","isCorrect":true,"inputTokens":3120,"outputTokens":8,"latencyMs":1299.8592499999795},{"questionId":"q85","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":4094,"outputTokens":5,"latencyMs":950.6797500000102},{"questionId":"q85","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"23","isCorrect":false,"inputTokens":2405,"outputTokens":5,"latencyMs":1537.8788749999367},{"questionId":"q85","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"18","isCorrect":false,"inputTokens":1551,"outputTokens":5,"latencyMs":741.9191250000149},{"questionId":"q85","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1459,"outputTokens":5,"latencyMs":1347.7071670000441},{"questionId":"q85","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"20","isCorrect":false,"inputTokens":4796,"outputTokens":5,"latencyMs":978.4684160000179},{"questionId":"q85","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":3125,"outputTokens":5,"latencyMs":1221.0832079999382},{"questionId":"q86","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"42","actual":"27","isCorrect":false,"inputTokens":4094,"outputTokens":5,"latencyMs":1657.6141670000507},{"questionId":"q86","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"42","actual":"28","isCorrect":false,"inputTokens":2405,"outputTokens":5,"latencyMs":1112.2490830000024},{"questionId":"q86","format":"toon","model":"claude-haiku-4-5-20251001","expected":"42","actual":"38","isCorrect":false,"inputTokens":1551,"outputTokens":5,"latencyMs":1090.7140420000069},{"questionId":"q86","format":"csv","model":"claude-haiku-4-5-20251001","expected":"42","actual":"38","isCorrect":false,"inputTokens":1459,"outputTokens":5,"latencyMs":908.3815840000752},{"questionId":"q86","format":"xml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"27","isCorrect":false,"inputTokens":4796,"outputTokens":5,"latencyMs":1333.344291999936},{"questionId":"q86","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"27","isCorrect":false,"inputTokens":3125,"outputTokens":5,"latencyMs":1417.703749999986},{"questionId":"q87","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"20","actual":"18","isCorrect":false,"inputTokens":4102,"outputTokens":5,"latencyMs":811.7225829999661},{"questionId":"q87","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"20","actual":"23","isCorrect":false,"inputTokens":2413,"outputTokens":5,"latencyMs":1470.5692910000216},{"questionId":"q87","format":"toon","model":"claude-haiku-4-5-20251001","expected":"20","actual":"21","isCorrect":false,"inputTokens":1559,"outputTokens":5,"latencyMs":1062.294792000088},{"questionId":"q87","format":"csv","model":"claude-haiku-4-5-20251001","expected":"20","actual":"18","isCorrect":false,"inputTokens":1467,"outputTokens":5,"latencyMs":1246.1075420000125},{"questionId":"q87","format":"xml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"15","isCorrect":false,"inputTokens":4804,"outputTokens":5,"latencyMs":1080.7989589999197},{"questionId":"q87","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"16","isCorrect":false,"inputTokens":3133,"outputTokens":5,"latencyMs":1536.6151249998948},{"questionId":"q88","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"14","actual":"8","isCorrect":false,"inputTokens":4102,"outputTokens":5,"latencyMs":1650.748459000024},{"questionId":"q88","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"14","actual":"10","isCorrect":false,"inputTokens":2413,"outputTokens":5,"latencyMs":1152.1895840000361},{"questionId":"q88","format":"toon","model":"claude-haiku-4-5-20251001","expected":"14","actual":"10","isCorrect":false,"inputTokens":1559,"outputTokens":5,"latencyMs":1257.109167000046},{"questionId":"q88","format":"csv","model":"claude-haiku-4-5-20251001","expected":"14","actual":"8","isCorrect":false,"inputTokens":1467,"outputTokens":5,"latencyMs":1134.593125000014},{"questionId":"q88","format":"xml","model":"claude-haiku-4-5-20251001","expected":"14","actual":"9","isCorrect":false,"inputTokens":4804,"outputTokens":5,"latencyMs":1552.6940409999806},{"questionId":"q88","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"14","actual":"8","isCorrect":false,"inputTokens":3133,"outputTokens":5,"latencyMs":1000.2084160000086},{"questionId":"q89","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":4100,"outputTokens":5,"latencyMs":1432.9054579999065},{"questionId":"q89","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":2411,"outputTokens":5,"latencyMs":1118.0298750000075},{"questionId":"q89","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"18","isCorrect":false,"inputTokens":1557,"outputTokens":5,"latencyMs":1100.108125000028},{"questionId":"q89","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1465,"outputTokens":5,"latencyMs":879.8744579999475},{"questionId":"q89","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":4802,"outputTokens":5,"latencyMs":932.8511250000447},{"questionId":"q89","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":3131,"outputTokens":5,"latencyMs":1239.4788750000298},{"questionId":"q90","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":4100,"outputTokens":5,"latencyMs":1117.268167000031},{"questionId":"q90","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"17","isCorrect":false,"inputTokens":2411,"outputTokens":5,"latencyMs":1466.201749999891},{"questionId":"q90","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1557,"outputTokens":5,"latencyMs":1142.5247499999823},{"questionId":"q90","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1465,"outputTokens":5,"latencyMs":1317.812707999954},{"questionId":"q90","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":4802,"outputTokens":5,"latencyMs":1471.6635420001112},{"questionId":"q90","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":3131,"outputTokens":5,"latencyMs":1131.126290999935},{"questionId":"q91","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"20","actual":"16","isCorrect":false,"inputTokens":4100,"outputTokens":5,"latencyMs":1457.6941660000011},{"questionId":"q91","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"20","actual":"15","isCorrect":false,"inputTokens":2411,"outputTokens":5,"latencyMs":1692.0967919999966},{"questionId":"q91","format":"toon","model":"claude-haiku-4-5-20251001","expected":"20","actual":"16","isCorrect":false,"inputTokens":1557,"outputTokens":5,"latencyMs":1209.3823329999577},{"questionId":"q91","format":"csv","model":"claude-haiku-4-5-20251001","expected":"20","actual":"15","isCorrect":false,"inputTokens":1465,"outputTokens":5,"latencyMs":1165.7072499999776},{"questionId":"q91","format":"xml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"13","isCorrect":false,"inputTokens":4802,"outputTokens":5,"latencyMs":1366.3539579999633},{"questionId":"q91","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"15","isCorrect":false,"inputTokens":3131,"outputTokens":5,"latencyMs":1627.3735419999575},{"questionId":"q92","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"32","actual":"23","isCorrect":false,"inputTokens":4101,"outputTokens":5,"latencyMs":941.233499999973},{"questionId":"q92","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"32","actual":"23","isCorrect":false,"inputTokens":2412,"outputTokens":5,"latencyMs":1174.533167000045},{"questionId":"q92","format":"toon","model":"claude-haiku-4-5-20251001","expected":"32","actual":"31","isCorrect":false,"inputTokens":1558,"outputTokens":5,"latencyMs":1270.3798330000136},{"questionId":"q92","format":"csv","model":"claude-haiku-4-5-20251001","expected":"32","actual":"25","isCorrect":false,"inputTokens":1466,"outputTokens":5,"latencyMs":135450.89775},{"questionId":"q92","format":"xml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"31","isCorrect":false,"inputTokens":4803,"outputTokens":5,"latencyMs":1089.6465000000317},{"questionId":"q92","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"18","isCorrect":false,"inputTokens":3132,"outputTokens":5,"latencyMs":1323.09620800009},{"questionId":"q93","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":4101,"outputTokens":5,"latencyMs":1097.0248750000028},{"questionId":"q93","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"9","actual":"12","isCorrect":false,"inputTokens":2412,"outputTokens":5,"latencyMs":2636.873957999982},{"questionId":"q93","format":"toon","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":1558,"outputTokens":5,"latencyMs":1240.7609159999993},{"questionId":"q93","format":"csv","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":1466,"outputTokens":5,"latencyMs":860.1172090000473},{"questionId":"q93","format":"xml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"7","isCorrect":false,"inputTokens":4803,"outputTokens":5,"latencyMs":1052.20229200006},{"questionId":"q93","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":3132,"outputTokens":5,"latencyMs":1558.8638329999521},{"questionId":"q94","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"18","isCorrect":false,"inputTokens":4102,"outputTokens":5,"latencyMs":955.4143750000512},{"questionId":"q94","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":2413,"outputTokens":5,"latencyMs":981.6348749999888},{"questionId":"q94","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":1559,"outputTokens":5,"latencyMs":1137.1802499999758},{"questionId":"q94","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"12","isCorrect":false,"inputTokens":1467,"outputTokens":5,"latencyMs":1008.8686250000028},{"questionId":"q94","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"27","isCorrect":false,"inputTokens":4804,"outputTokens":5,"latencyMs":1335.1314589999383},{"questionId":"q94","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"17","isCorrect":false,"inputTokens":3133,"outputTokens":5,"latencyMs":1422.817041000002},{"questionId":"q95","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"17","isCorrect":false,"inputTokens":4102,"outputTokens":5,"latencyMs":1395.5275830000173},{"questionId":"q95","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"18","isCorrect":false,"inputTokens":2413,"outputTokens":5,"latencyMs":1240.6792499999283},{"questionId":"q95","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"17","isCorrect":false,"inputTokens":1559,"outputTokens":5,"latencyMs":1357.294208999956},{"questionId":"q95","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"12","isCorrect":false,"inputTokens":1467,"outputTokens":5,"latencyMs":931.3478330000071},{"questionId":"q95","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":4804,"outputTokens":5,"latencyMs":1304.6751670000376},{"questionId":"q95","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":3133,"outputTokens":5,"latencyMs":1023.4539159999695},{"questionId":"q96","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17431,"outputTokens":6,"latencyMs":1549.0179999999236},{"questionId":"q96","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":12566,"outputTokens":6,"latencyMs":1073.2592919999734},{"questionId":"q96","format":"toon","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":9329,"outputTokens":6,"latencyMs":1650.7731250000652},{"questionId":"q96","format":"csv","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":9147,"outputTokens":6,"latencyMs":1706.1569590000436},{"questionId":"q96","format":"xml","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":19821,"outputTokens":6,"latencyMs":2284.9616660000756},{"questionId":"q96","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":14506,"outputTokens":6,"latencyMs":1872.9165000000503},{"questionId":"q97","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17430,"outputTokens":6,"latencyMs":1756.9818749999395},{"questionId":"q97","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":12565,"outputTokens":6,"latencyMs":1143.423208000022},{"questionId":"q97","format":"toon","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":9328,"outputTokens":6,"latencyMs":2193.2150839999085},{"questionId":"q97","format":"csv","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":9146,"outputTokens":6,"latencyMs":1128.1155410000356},{"questionId":"q97","format":"xml","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":19820,"outputTokens":6,"latencyMs":1524.0118340000045},{"questionId":"q97","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":14505,"outputTokens":6,"latencyMs":1436.5846250000177},{"questionId":"q98","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17425,"outputTokens":6,"latencyMs":1296.6286669999827},{"questionId":"q98","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":12560,"outputTokens":6,"latencyMs":1323.7782080000034},{"questionId":"q98","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":9323,"outputTokens":6,"latencyMs":59378.73687500006},{"questionId":"q98","format":"csv","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":9141,"outputTokens":6,"latencyMs":1815.1144589999458},{"questionId":"q98","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":19815,"outputTokens":6,"latencyMs":1427.9329579999903},{"questionId":"q98","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":14500,"outputTokens":6,"latencyMs":1279.6807920000283},{"questionId":"q99","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":17430,"outputTokens":4,"latencyMs":1292.262624999974},{"questionId":"q99","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":12565,"outputTokens":4,"latencyMs":1375.4475000000093},{"questionId":"q99","format":"toon","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":9328,"outputTokens":4,"latencyMs":1169.7131659999723},{"questionId":"q99","format":"csv","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":9146,"outputTokens":4,"latencyMs":2760.7117500000168},{"questionId":"q99","format":"xml","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":19820,"outputTokens":4,"latencyMs":2885.7982499999925},{"questionId":"q99","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":14505,"outputTokens":4,"latencyMs":1770.374792000046},{"questionId":"q100","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17425,"outputTokens":6,"latencyMs":1842.8854999999749},{"questionId":"q100","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":12560,"outputTokens":6,"latencyMs":1059.2116670000833},{"questionId":"q100","format":"toon","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":9323,"outputTokens":6,"latencyMs":1134.6034999999683},{"questionId":"q100","format":"csv","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":9141,"outputTokens":6,"latencyMs":1111.7190420000115},{"questionId":"q100","format":"xml","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":19815,"outputTokens":6,"latencyMs":1788.3842920000898},{"questionId":"q100","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":14500,"outputTokens":6,"latencyMs":1219.1583330000285},{"questionId":"q101","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17430,"outputTokens":6,"latencyMs":1336.5955419999082},{"questionId":"q101","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":12565,"outputTokens":6,"latencyMs":1979.233790999977},{"questionId":"q101","format":"toon","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":9328,"outputTokens":6,"latencyMs":1398.4806249999674},{"questionId":"q101","format":"csv","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":9146,"outputTokens":6,"latencyMs":1486.5824589999393},{"questionId":"q101","format":"xml","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":19820,"outputTokens":6,"latencyMs":2036.6534999998985},{"questionId":"q101","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":14505,"outputTokens":6,"latencyMs":1340.5284159999574},{"questionId":"q102","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":17432,"outputTokens":5,"latencyMs":1678.1820420000004},{"questionId":"q102","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":12567,"outputTokens":5,"latencyMs":1717.26087499992},{"questionId":"q102","format":"toon","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":9330,"outputTokens":5,"latencyMs":1414.3522499999963},{"questionId":"q102","format":"csv","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":9148,"outputTokens":5,"latencyMs":1452.5142910000868},{"questionId":"q102","format":"xml","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":19822,"outputTokens":5,"latencyMs":1461.0910419999855},{"questionId":"q102","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":14507,"outputTokens":5,"latencyMs":1208.626042000018},{"questionId":"q103","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":17426,"outputTokens":4,"latencyMs":1446.889542000019},{"questionId":"q103","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":12561,"outputTokens":4,"latencyMs":1275.819041000097},{"questionId":"q103","format":"toon","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":9324,"outputTokens":4,"latencyMs":1618.3254160000943},{"questionId":"q103","format":"csv","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":9142,"outputTokens":4,"latencyMs":1324.5384159999667},{"questionId":"q103","format":"xml","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":19816,"outputTokens":4,"latencyMs":1540.8082500000019},{"questionId":"q103","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":14501,"outputTokens":4,"latencyMs":1639.3106249999255},{"questionId":"q104","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17433,"outputTokens":6,"latencyMs":1416.6109160000924},{"questionId":"q104","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":12568,"outputTokens":6,"latencyMs":1462.0648339999607},{"questionId":"q104","format":"toon","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":9331,"outputTokens":6,"latencyMs":2445.9155830000527},{"questionId":"q104","format":"csv","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":9149,"outputTokens":6,"latencyMs":1240.1038749999134},{"questionId":"q104","format":"xml","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":19823,"outputTokens":6,"latencyMs":1565.9135839999653},{"questionId":"q104","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":14508,"outputTokens":6,"latencyMs":1406.3525420000078},{"questionId":"q105","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17429,"outputTokens":6,"latencyMs":1525.3579579999205},{"questionId":"q105","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":12564,"outputTokens":6,"latencyMs":1152.4085000000196},{"questionId":"q105","format":"toon","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":9327,"outputTokens":6,"latencyMs":1770.4646250000224},{"questionId":"q105","format":"csv","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":9145,"outputTokens":6,"latencyMs":1353.4057909999974},{"questionId":"q105","format":"xml","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":19819,"outputTokens":6,"latencyMs":1444.9497499999125},{"questionId":"q105","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":14504,"outputTokens":6,"latencyMs":135790.69074999995},{"questionId":"q106","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17434,"outputTokens":6,"latencyMs":1451.4468329999363},{"questionId":"q106","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":12569,"outputTokens":6,"latencyMs":1201.844000000041},{"questionId":"q106","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":9332,"outputTokens":6,"latencyMs":1058.9308750000782},{"questionId":"q106","format":"csv","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":9150,"outputTokens":6,"latencyMs":1200.5313340000575},{"questionId":"q106","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":19824,"outputTokens":6,"latencyMs":1919.609999999986},{"questionId":"q106","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":14509,"outputTokens":6,"latencyMs":1765.9685419999296},{"questionId":"q107","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":17422,"outputTokens":5,"latencyMs":1923.7036670000525},{"questionId":"q107","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":12557,"outputTokens":5,"latencyMs":2376.041458999971},{"questionId":"q107","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9320,"outputTokens":5,"latencyMs":1554.609208000009},{"questionId":"q107","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9138,"outputTokens":5,"latencyMs":1288.4187920000404},{"questionId":"q107","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":19812,"outputTokens":5,"latencyMs":1580.9837919999845},{"questionId":"q107","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":14497,"outputTokens":5,"latencyMs":1666.154332999955},{"questionId":"q108","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"17525947","isCorrect":false,"inputTokens":17425,"outputTokens":7,"latencyMs":1619.4902499999152},{"questionId":"q108","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"14842571","isCorrect":false,"inputTokens":12560,"outputTokens":7,"latencyMs":1170.5864580000052},{"questionId":"q108","format":"toon","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"13746686","isCorrect":false,"inputTokens":9323,"outputTokens":7,"latencyMs":1525.2280420000898},{"questionId":"q108","format":"csv","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"15746895","isCorrect":false,"inputTokens":9141,"outputTokens":7,"latencyMs":1232.1970420000143},{"questionId":"q108","format":"xml","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"12968527","isCorrect":false,"inputTokens":19815,"outputTokens":7,"latencyMs":3372.6276249999646},{"questionId":"q108","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"13824638","isCorrect":false,"inputTokens":14500,"outputTokens":7,"latencyMs":1895.8765839999542},{"questionId":"q109","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1,447,821","isCorrect":false,"inputTokens":17426,"outputTokens":9,"latencyMs":2032.5368330000201},{"questionId":"q109","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1346787","isCorrect":false,"inputTokens":12561,"outputTokens":7,"latencyMs":1355.346416000044},{"questionId":"q109","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1368618","isCorrect":false,"inputTokens":9324,"outputTokens":7,"latencyMs":1350.4067920000525},{"questionId":"q109","format":"csv","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1285547","isCorrect":false,"inputTokens":9142,"outputTokens":7,"latencyMs":1598.6938329999102},{"questionId":"q109","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1,534,956","isCorrect":false,"inputTokens":19816,"outputTokens":9,"latencyMs":1369.1255839999067},{"questionId":"q109","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1,273,892","isCorrect":false,"inputTokens":14501,"outputTokens":9,"latencyMs":1250.2912920000963},{"questionId":"q110","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"127,784.68","isCorrect":false,"inputTokens":17424,"outputTokens":9,"latencyMs":1589.4543339998927},{"questionId":"q110","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"```json\n{\"average_stars\":135847.16}\n```","isCorrect":false,"inputTokens":12559,"outputTokens":18,"latencyMs":1812.6839160000673},{"questionId":"q110","format":"toon","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"132,276.45","isCorrect":false,"inputTokens":9322,"outputTokens":9,"latencyMs":1077.4147919999668},{"questionId":"q110","format":"csv","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"133,892.28","isCorrect":false,"inputTokens":9140,"outputTokens":9,"latencyMs":1132.0060000000522},{"questionId":"q110","format":"xml","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"127,574.81","isCorrect":false,"inputTokens":19814,"outputTokens":9,"latencyMs":1978.7705419999547},{"questionId":"q110","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"137,841.45","isCorrect":false,"inputTokens":14499,"outputTokens":9,"latencyMs":1737.108540999936},{"questionId":"q111","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":17426,"outputTokens":5,"latencyMs":2407.2787089999765},{"questionId":"q111","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"41","actual":"25","isCorrect":false,"inputTokens":12561,"outputTokens":5,"latencyMs":1335.2449169999454},{"questionId":"q111","format":"toon","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":9324,"outputTokens":5,"latencyMs":1641.0892500000773},{"questionId":"q111","format":"csv","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":9142,"outputTokens":5,"latencyMs":1659.8185000000522},{"questionId":"q111","format":"xml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"31","isCorrect":false,"inputTokens":19816,"outputTokens":5,"latencyMs":1373.9072500000475},{"questionId":"q111","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":14501,"outputTokens":5,"latencyMs":1089.438124999986},{"questionId":"q112","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"53","actual":"65","isCorrect":false,"inputTokens":17426,"outputTokens":5,"latencyMs":1256.0232499999693},{"questionId":"q112","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"53","actual":"59","isCorrect":false,"inputTokens":12561,"outputTokens":5,"latencyMs":1270.2216250000056},{"questionId":"q112","format":"toon","model":"claude-haiku-4-5-20251001","expected":"53","actual":"56","isCorrect":false,"inputTokens":9324,"outputTokens":5,"latencyMs":1080.4613749999553},{"questionId":"q112","format":"csv","model":"claude-haiku-4-5-20251001","expected":"53","actual":"48","isCorrect":false,"inputTokens":9142,"outputTokens":5,"latencyMs":1195.3386249999749},{"questionId":"q112","format":"xml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"58","isCorrect":false,"inputTokens":19816,"outputTokens":5,"latencyMs":1716.5474580000155},{"questionId":"q112","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"61","isCorrect":false,"inputTokens":14501,"outputTokens":5,"latencyMs":1436.8687090000603},{"questionId":"q113","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":17426,"outputTokens":5,"latencyMs":1288.2891250000102},{"questionId":"q113","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"77","actual":"65","isCorrect":false,"inputTokens":12561,"outputTokens":5,"latencyMs":1878.7123329999158},{"questionId":"q113","format":"toon","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":9324,"outputTokens":5,"latencyMs":1174.7566660000011},{"questionId":"q113","format":"csv","model":"claude-haiku-4-5-20251001","expected":"77","actual":"97","isCorrect":false,"inputTokens":9142,"outputTokens":5,"latencyMs":1585.0327920001},{"questionId":"q113","format":"xml","model":"claude-haiku-4-5-20251001","expected":"77","actual":"79","isCorrect":false,"inputTokens":19816,"outputTokens":5,"latencyMs":1991.5329589999747},{"questionId":"q113","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":14501,"outputTokens":5,"latencyMs":1472.81004199991},{"questionId":"q114","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"37","actual":"60","isCorrect":false,"inputTokens":17426,"outputTokens":5,"latencyMs":1837.4823340000585},{"questionId":"q114","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"37","actual":"50","isCorrect":false,"inputTokens":12561,"outputTokens":5,"latencyMs":1154.4151249999413},{"questionId":"q114","format":"toon","model":"claude-haiku-4-5-20251001","expected":"37","actual":"35","isCorrect":false,"inputTokens":9324,"outputTokens":5,"latencyMs":1429.2106670000358},{"questionId":"q114","format":"csv","model":"claude-haiku-4-5-20251001","expected":"37","actual":"47","isCorrect":false,"inputTokens":9142,"outputTokens":5,"latencyMs":135280.49025000003},{"questionId":"q114","format":"xml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"50","isCorrect":false,"inputTokens":19816,"outputTokens":5,"latencyMs":2030.1534580000443},{"questionId":"q114","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"73","isCorrect":false,"inputTokens":14501,"outputTokens":5,"latencyMs":1371.425333000021},{"questionId":"q115","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"16","actual":"10","isCorrect":false,"inputTokens":17426,"outputTokens":5,"latencyMs":1489.0051250000251},{"questionId":"q115","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"16","actual":"13","isCorrect":false,"inputTokens":12561,"outputTokens":5,"latencyMs":1466.1329169999808},{"questionId":"q115","format":"toon","model":"claude-haiku-4-5-20251001","expected":"16","actual":"20","isCorrect":false,"inputTokens":9324,"outputTokens":5,"latencyMs":1293.8040409999667},{"questionId":"q115","format":"csv","model":"claude-haiku-4-5-20251001","expected":"16","actual":"20","isCorrect":false,"inputTokens":9142,"outputTokens":5,"latencyMs":1199.2742919999873},{"questionId":"q115","format":"xml","model":"claude-haiku-4-5-20251001","expected":"16","actual":"5","isCorrect":false,"inputTokens":19816,"outputTokens":5,"latencyMs":2370.445834000013},{"questionId":"q115","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"16","actual":"3","isCorrect":false,"inputTokens":14501,"outputTokens":5,"latencyMs":2260.0115830000723},{"questionId":"q116","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"49","actual":"31","isCorrect":false,"inputTokens":17427,"outputTokens":5,"latencyMs":1656.3866249999264},{"questionId":"q116","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"49","actual":"47","isCorrect":false,"inputTokens":12562,"outputTokens":5,"latencyMs":1364.529125000001},{"questionId":"q116","format":"toon","model":"claude-haiku-4-5-20251001","expected":"49","actual":"52","isCorrect":false,"inputTokens":9325,"outputTokens":5,"latencyMs":1118.893415999948},{"questionId":"q116","format":"csv","model":"claude-haiku-4-5-20251001","expected":"49","actual":"32","isCorrect":false,"inputTokens":9143,"outputTokens":5,"latencyMs":1354.8706660000607},{"questionId":"q116","format":"xml","model":"claude-haiku-4-5-20251001","expected":"49","actual":"23","isCorrect":false,"inputTokens":19817,"outputTokens":5,"latencyMs":1742.3912080000155},{"questionId":"q116","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"49","actual":"47","isCorrect":false,"inputTokens":14502,"outputTokens":5,"latencyMs":1213.965957999928},{"questionId":"q117","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"23","actual":"23","isCorrect":true,"inputTokens":17427,"outputTokens":5,"latencyMs":1659.6879999999655},{"questionId":"q117","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"23","actual":"23","isCorrect":true,"inputTokens":12562,"outputTokens":5,"latencyMs":1768.546834000037},{"questionId":"q117","format":"toon","model":"claude-haiku-4-5-20251001","expected":"23","actual":"25","isCorrect":false,"inputTokens":9325,"outputTokens":5,"latencyMs":1113.714917000034},{"questionId":"q117","format":"csv","model":"claude-haiku-4-5-20251001","expected":"23","actual":"31","isCorrect":false,"inputTokens":9143,"outputTokens":5,"latencyMs":1769.5054169999203},{"questionId":"q117","format":"xml","model":"claude-haiku-4-5-20251001","expected":"23","actual":"16","isCorrect":false,"inputTokens":19817,"outputTokens":5,"latencyMs":1303.7927500000224},{"questionId":"q117","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"23","actual":"23","isCorrect":true,"inputTokens":14502,"outputTokens":5,"latencyMs":1178.9949999999953},{"questionId":"q118","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":17427,"outputTokens":5,"latencyMs":1245.5005830000155},{"questionId":"q118","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"5","isCorrect":false,"inputTokens":12562,"outputTokens":5,"latencyMs":1179.517332999967},{"questionId":"q118","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"15","isCorrect":false,"inputTokens":9325,"outputTokens":5,"latencyMs":1234.552708000061},{"questionId":"q118","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4","actual":"27","isCorrect":false,"inputTokens":9143,"outputTokens":5,"latencyMs":980.5235000000102},{"questionId":"q118","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"15","isCorrect":false,"inputTokens":19817,"outputTokens":5,"latencyMs":1948.8275419999845},{"questionId":"q118","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"14","isCorrect":false,"inputTokens":14502,"outputTokens":5,"latencyMs":1154.1357909999788},{"questionId":"q119","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"57","actual":"14","isCorrect":false,"inputTokens":17435,"outputTokens":5,"latencyMs":1782.1672910000198},{"questionId":"q119","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"57","actual":"28","isCorrect":false,"inputTokens":12570,"outputTokens":5,"latencyMs":1546.312917000032},{"questionId":"q119","format":"toon","model":"claude-haiku-4-5-20251001","expected":"57","actual":"8","isCorrect":false,"inputTokens":9333,"outputTokens":5,"latencyMs":1557.8855420000618},{"questionId":"q119","format":"csv","model":"claude-haiku-4-5-20251001","expected":"57","actual":"16","isCorrect":false,"inputTokens":9151,"outputTokens":5,"latencyMs":1257.063957999926},{"questionId":"q119","format":"xml","model":"claude-haiku-4-5-20251001","expected":"57","actual":"15","isCorrect":false,"inputTokens":19825,"outputTokens":5,"latencyMs":2910.703749999986},{"questionId":"q119","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"57","actual":"15","isCorrect":false,"inputTokens":14510,"outputTokens":5,"latencyMs":1233.1566660000244},{"questionId":"q120","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"43","actual":"47","isCorrect":false,"inputTokens":17435,"outputTokens":5,"latencyMs":1282.1466670000227},{"questionId":"q120","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"43","actual":"36","isCorrect":false,"inputTokens":12570,"outputTokens":5,"latencyMs":1384.6129580000415},{"questionId":"q120","format":"toon","model":"claude-haiku-4-5-20251001","expected":"43","actual":"32","isCorrect":false,"inputTokens":9333,"outputTokens":5,"latencyMs":1190.936417000019},{"questionId":"q120","format":"csv","model":"claude-haiku-4-5-20251001","expected":"43","actual":"23","isCorrect":false,"inputTokens":9151,"outputTokens":5,"latencyMs":1255.6295420000097},{"questionId":"q120","format":"xml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"21","isCorrect":false,"inputTokens":19825,"outputTokens":5,"latencyMs":1539.4536249999655},{"questionId":"q120","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"42","isCorrect":false,"inputTokens":14510,"outputTokens":5,"latencyMs":1191.5602909999434},{"questionId":"q121","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"25","actual":"15","isCorrect":false,"inputTokens":17435,"outputTokens":5,"latencyMs":1456.7998339999467},{"questionId":"q121","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"25","actual":"16","isCorrect":false,"inputTokens":12570,"outputTokens":5,"latencyMs":1273.3514590000268},{"questionId":"q121","format":"toon","model":"claude-haiku-4-5-20251001","expected":"25","actual":"15","isCorrect":false,"inputTokens":9333,"outputTokens":5,"latencyMs":1030.0079169999808},{"questionId":"q121","format":"csv","model":"claude-haiku-4-5-20251001","expected":"25","actual":"12","isCorrect":false,"inputTokens":9151,"outputTokens":5,"latencyMs":1296.0598750000354},{"questionId":"q121","format":"xml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"12","isCorrect":false,"inputTokens":19825,"outputTokens":5,"latencyMs":1479.1339589999989},{"questionId":"q121","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"13","isCorrect":false,"inputTokens":14510,"outputTokens":5,"latencyMs":1348.386708999984},{"questionId":"q122","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6","actual":"12","isCorrect":false,"inputTokens":17435,"outputTokens":5,"latencyMs":1851.1320410000626},{"questionId":"q122","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6","actual":"17","isCorrect":false,"inputTokens":12570,"outputTokens":5,"latencyMs":1513.4139590000268},{"questionId":"q122","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6","actual":"12","isCorrect":false,"inputTokens":9333,"outputTokens":5,"latencyMs":1019.4977920000674},{"questionId":"q122","format":"csv","model":"claude-haiku-4-5-20251001","expected":"6","actual":"21","isCorrect":false,"inputTokens":9151,"outputTokens":5,"latencyMs":1252.4444580000127},{"questionId":"q122","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"12","isCorrect":false,"inputTokens":19825,"outputTokens":5,"latencyMs":2239.8731249999255},{"questionId":"q122","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"8","isCorrect":false,"inputTokens":14510,"outputTokens":5,"latencyMs":1138.81104099995},{"questionId":"q123","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"12","isCorrect":false,"inputTokens":17435,"outputTokens":5,"latencyMs":2972.753999999957},{"questionId":"q123","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"3","isCorrect":false,"inputTokens":12570,"outputTokens":5,"latencyMs":1550.2462499999674},{"questionId":"q123","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"8","isCorrect":false,"inputTokens":9333,"outputTokens":5,"latencyMs":1167.2528329999186},{"questionId":"q123","format":"csv","model":"claude-haiku-4-5-20251001","expected":"1","actual":"4","isCorrect":false,"inputTokens":9151,"outputTokens":5,"latencyMs":1822.666458999971},{"questionId":"q123","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"5","isCorrect":false,"inputTokens":19825,"outputTokens":5,"latencyMs":1701.1414589999476},{"questionId":"q123","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"4","isCorrect":false,"inputTokens":14510,"outputTokens":5,"latencyMs":2044.1263749999925},{"questionId":"q124","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":7732,"outputTokens":4,"latencyMs":1322.2519999999786},{"questionId":"q124","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":5170,"outputTokens":4,"latencyMs":1445.716625000001},{"questionId":"q124","format":"toon","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6190,"outputTokens":4,"latencyMs":1237.5652079998981},{"questionId":"q124","format":"xml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":8737,"outputTokens":4,"latencyMs":1288.4407499999506},{"questionId":"q124","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6136,"outputTokens":4,"latencyMs":1240.3781670000171},{"questionId":"q125","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7732,"outputTokens":7,"latencyMs":1251.3257500000764},{"questionId":"q125","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5170,"outputTokens":7,"latencyMs":1564.0911250000354},{"questionId":"q125","format":"toon","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6190,"outputTokens":7,"latencyMs":1095.515916000004},{"questionId":"q125","format":"xml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8737,"outputTokens":7,"latencyMs":1325.0977089999942},{"questionId":"q125","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6136,"outputTokens":7,"latencyMs":1380.611125000054},{"questionId":"q126","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":7733,"outputTokens":5,"latencyMs":1183.3869580000173},{"questionId":"q126","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":5171,"outputTokens":5,"latencyMs":1186.1116250000196},{"questionId":"q126","format":"toon","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":6191,"outputTokens":5,"latencyMs":1086.6458339999663},{"questionId":"q126","format":"xml","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":8738,"outputTokens":5,"latencyMs":1429.5812500000466},{"questionId":"q126","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":6137,"outputTokens":5,"latencyMs":1231.497667000047},{"questionId":"q127","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7733,"outputTokens":6,"latencyMs":1359.0183330000145},{"questionId":"q127","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5171,"outputTokens":6,"latencyMs":952.6933339999523},{"questionId":"q127","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6191,"outputTokens":6,"latencyMs":934.7892499999143},{"questionId":"q127","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":8738,"outputTokens":6,"latencyMs":1337.0739590000594},{"questionId":"q127","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6137,"outputTokens":6,"latencyMs":1141.6887080000015},{"questionId":"q128","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":7732,"outputTokens":4,"latencyMs":984.6171669999603},{"questionId":"q128","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":5170,"outputTokens":4,"latencyMs":1598.2621250000084},{"questionId":"q128","format":"toon","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6190,"outputTokens":4,"latencyMs":1665.3170829999726},{"questionId":"q128","format":"xml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":8737,"outputTokens":4,"latencyMs":1075.0322500000475},{"questionId":"q128","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6136,"outputTokens":4,"latencyMs":1032.5187920000171},{"questionId":"q129","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7732,"outputTokens":7,"latencyMs":1314.7635410000803},{"questionId":"q129","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5170,"outputTokens":7,"latencyMs":1124.9454169999808},{"questionId":"q129","format":"toon","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6190,"outputTokens":7,"latencyMs":1178.4954579999903},{"questionId":"q129","format":"xml","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":8737,"outputTokens":7,"latencyMs":1263.9604169999948},{"questionId":"q129","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6136,"outputTokens":7,"latencyMs":990.7301250000019},{"questionId":"q130","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":7733,"outputTokens":5,"latencyMs":134578.79599999997},{"questionId":"q130","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":5171,"outputTokens":5,"latencyMs":968.1252910000039},{"questionId":"q130","format":"toon","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":6191,"outputTokens":5,"latencyMs":1090.8137080000015},{"questionId":"q130","format":"xml","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":8738,"outputTokens":5,"latencyMs":1143.6139579999726},{"questionId":"q130","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":6137,"outputTokens":5,"latencyMs":1058.187957999995},{"questionId":"q131","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":7733,"outputTokens":5,"latencyMs":1351.676000000094},{"questionId":"q131","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":5171,"outputTokens":5,"latencyMs":1969.7786250000354},{"questionId":"q131","format":"toon","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":6191,"outputTokens":5,"latencyMs":998.9934170000488},{"questionId":"q131","format":"xml","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":8738,"outputTokens":5,"latencyMs":1755.1462909999536},{"questionId":"q131","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":6137,"outputTokens":5,"latencyMs":1366.4382500000065},{"questionId":"q132","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":7732,"outputTokens":4,"latencyMs":1946.048041000031},{"questionId":"q132","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":5170,"outputTokens":4,"latencyMs":1648.8376670000143},{"questionId":"q132","format":"toon","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6190,"outputTokens":4,"latencyMs":1219.62491699995},{"questionId":"q132","format":"xml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":8737,"outputTokens":4,"latencyMs":1589.6692500000354},{"questionId":"q132","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6136,"outputTokens":4,"latencyMs":1863.625250000041},{"questionId":"q133","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7732,"outputTokens":7,"latencyMs":7655.55554099998},{"questionId":"q133","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5170,"outputTokens":7,"latencyMs":1638.7547499999637},{"questionId":"q133","format":"toon","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6190,"outputTokens":7,"latencyMs":1624.8859589999774},{"questionId":"q133","format":"xml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8737,"outputTokens":7,"latencyMs":136946.34433400002},{"questionId":"q133","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6136,"outputTokens":7,"latencyMs":1423.7594589999644},{"questionId":"q134","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"75","actual":"80","isCorrect":false,"inputTokens":7716,"outputTokens":5,"latencyMs":1250.6163330000127},{"questionId":"q134","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"75","actual":"61","isCorrect":false,"inputTokens":5154,"outputTokens":5,"latencyMs":1449.7023339999141},{"questionId":"q134","format":"toon","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":6174,"outputTokens":5,"latencyMs":1335.3158340000082},{"questionId":"q134","format":"xml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"100","isCorrect":false,"inputTokens":8721,"outputTokens":5,"latencyMs":1690.8688339999644},{"questionId":"q134","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":6120,"outputTokens":5,"latencyMs":1194.3599170000525},{"questionId":"q135","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2447.44","isCorrect":false,"inputTokens":7717,"outputTokens":8,"latencyMs":1532.6048749999609},{"questionId":"q135","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2461.32","isCorrect":false,"inputTokens":5155,"outputTokens":8,"latencyMs":1174.643458000035},{"questionId":"q135","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2445.85","isCorrect":false,"inputTokens":6175,"outputTokens":8,"latencyMs":1453.3343330000062},{"questionId":"q135","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2465.45","isCorrect":false,"inputTokens":8722,"outputTokens":8,"latencyMs":1407.3757079999195},{"questionId":"q135","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2425.52","isCorrect":false,"inputTokens":6121,"outputTokens":8,"latencyMs":1727.172457999899},{"questionId":"q136","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"29","actual":"30","isCorrect":false,"inputTokens":7717,"outputTokens":5,"latencyMs":4074.6192919999594},{"questionId":"q136","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"29","actual":"26","isCorrect":false,"inputTokens":5155,"outputTokens":5,"latencyMs":1351.1561249999795},{"questionId":"q136","format":"toon","model":"claude-haiku-4-5-20251001","expected":"29","actual":"30","isCorrect":false,"inputTokens":6175,"outputTokens":5,"latencyMs":1079.7601659999928},{"questionId":"q136","format":"xml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"31","isCorrect":false,"inputTokens":8722,"outputTokens":5,"latencyMs":1080.8913750000065},{"questionId":"q136","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"30","isCorrect":false,"inputTokens":6121,"outputTokens":5,"latencyMs":1051.8147090000566},{"questionId":"q137","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"17","isCorrect":true,"inputTokens":7717,"outputTokens":5,"latencyMs":1117.4450410000281},{"questionId":"q137","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":5155,"outputTokens":5,"latencyMs":960.0351670000236},{"questionId":"q137","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":6175,"outputTokens":5,"latencyMs":1307.409958000062},{"questionId":"q137","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"17","isCorrect":true,"inputTokens":8722,"outputTokens":5,"latencyMs":1157.3227499999339},{"questionId":"q137","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":6121,"outputTokens":5,"latencyMs":1473.7385840000352},{"questionId":"q138","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"29","actual":"23","isCorrect":false,"inputTokens":7717,"outputTokens":5,"latencyMs":1617.0194589999737},{"questionId":"q138","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"29","actual":"23","isCorrect":false,"inputTokens":5155,"outputTokens":5,"latencyMs":4893.344209000003},{"questionId":"q138","format":"toon","model":"claude-haiku-4-5-20251001","expected":"29","actual":"28","isCorrect":false,"inputTokens":6175,"outputTokens":5,"latencyMs":994.5907920000609},{"questionId":"q138","format":"xml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"26","isCorrect":false,"inputTokens":8722,"outputTokens":5,"latencyMs":1068.0511249999981},{"questionId":"q138","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"25","isCorrect":false,"inputTokens":6121,"outputTokens":5,"latencyMs":1093.1777499999152},{"questionId":"q139","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":7720,"outputTokens":5,"latencyMs":1259.5825000000186},{"questionId":"q139","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":5158,"outputTokens":5,"latencyMs":1579.743708999944},{"questionId":"q139","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":6178,"outputTokens":5,"latencyMs":952.0351250000531},{"questionId":"q139","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"12","isCorrect":false,"inputTokens":8725,"outputTokens":5,"latencyMs":1301.0361249999842},{"questionId":"q139","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":6124,"outputTokens":5,"latencyMs":1122.7744170000078},{"questionId":"q140","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"18","actual":"16","isCorrect":false,"inputTokens":7720,"outputTokens":5,"latencyMs":1149.1232080000918},{"questionId":"q140","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"18","actual":"14","isCorrect":false,"inputTokens":5158,"outputTokens":5,"latencyMs":967.6039589999709},{"questionId":"q140","format":"toon","model":"claude-haiku-4-5-20251001","expected":"18","actual":"14","isCorrect":false,"inputTokens":6178,"outputTokens":5,"latencyMs":1290.0111660000402},{"questionId":"q140","format":"xml","model":"claude-haiku-4-5-20251001","expected":"18","actual":"20","isCorrect":false,"inputTokens":8725,"outputTokens":5,"latencyMs":1196.498915999895},{"questionId":"q140","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"18","actual":"16","isCorrect":false,"inputTokens":6124,"outputTokens":5,"latencyMs":1152.909207999939},{"questionId":"q141","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"33","actual":"34","isCorrect":false,"inputTokens":7724,"outputTokens":5,"latencyMs":1087.8998750000028},{"questionId":"q141","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"33","actual":"32","isCorrect":false,"inputTokens":5162,"outputTokens":5,"latencyMs":1113.3987089999719},{"questionId":"q141","format":"toon","model":"claude-haiku-4-5-20251001","expected":"33","actual":"37","isCorrect":false,"inputTokens":6182,"outputTokens":5,"latencyMs":1231.1717909999425},{"questionId":"q141","format":"xml","model":"claude-haiku-4-5-20251001","expected":"33","actual":"41","isCorrect":false,"inputTokens":8729,"outputTokens":5,"latencyMs":1416.8829169999808},{"questionId":"q141","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"33","actual":"31","isCorrect":false,"inputTokens":6128,"outputTokens":5,"latencyMs":1522.4662500000559},{"questionId":"q142","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"42","actual":"34","isCorrect":false,"inputTokens":7721,"outputTokens":5,"latencyMs":1328.3373340000398},{"questionId":"q142","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"42","actual":"24","isCorrect":false,"inputTokens":5159,"outputTokens":5,"latencyMs":134287.69475000002},{"questionId":"q142","format":"toon","model":"claude-haiku-4-5-20251001","expected":"42","actual":"30","isCorrect":false,"inputTokens":6179,"outputTokens":5,"latencyMs":1117.1614579999587},{"questionId":"q142","format":"xml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"40","isCorrect":false,"inputTokens":8726,"outputTokens":5,"latencyMs":1136.4989999999525},{"questionId":"q142","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"32","isCorrect":false,"inputTokens":6125,"outputTokens":5,"latencyMs":1138.7126249999274},{"questionId":"q143","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"25","actual":"28","isCorrect":false,"inputTokens":7718,"outputTokens":5,"latencyMs":1183.0647090000566},{"questionId":"q143","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"25","actual":"32","isCorrect":false,"inputTokens":5156,"outputTokens":5,"latencyMs":1412.0120420000749},{"questionId":"q143","format":"toon","model":"claude-haiku-4-5-20251001","expected":"25","actual":"34","isCorrect":false,"inputTokens":6176,"outputTokens":5,"latencyMs":1119.6773340000072},{"questionId":"q143","format":"xml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"35","isCorrect":false,"inputTokens":8723,"outputTokens":5,"latencyMs":1071.6519580000313},{"questionId":"q143","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"31","isCorrect":false,"inputTokens":6122,"outputTokens":5,"latencyMs":1391.435999999987},{"questionId":"q144","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"29","actual":"31","isCorrect":false,"inputTokens":7723,"outputTokens":5,"latencyMs":1231.0474170000525},{"questionId":"q144","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"29","actual":"29","isCorrect":true,"inputTokens":5161,"outputTokens":5,"latencyMs":1135.2230410000775},{"questionId":"q144","format":"toon","model":"claude-haiku-4-5-20251001","expected":"29","actual":"32","isCorrect":false,"inputTokens":6181,"outputTokens":5,"latencyMs":1294.7761669999454},{"questionId":"q144","format":"xml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"38","isCorrect":false,"inputTokens":8728,"outputTokens":5,"latencyMs":1584.8036249999423},{"questionId":"q144","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"34","isCorrect":false,"inputTokens":6127,"outputTokens":5,"latencyMs":1191.8048329999438},{"questionId":"q145","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"3","isCorrect":false,"inputTokens":7723,"outputTokens":5,"latencyMs":1290.0185000000056},{"questionId":"q145","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"3","isCorrect":false,"inputTokens":5161,"outputTokens":5,"latencyMs":1122.844374999986},{"questionId":"q145","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"1","isCorrect":false,"inputTokens":6181,"outputTokens":5,"latencyMs":1475.8200419999193},{"questionId":"q145","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"3","isCorrect":false,"inputTokens":8728,"outputTokens":5,"latencyMs":1173.7602919999044},{"questionId":"q145","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"0","isCorrect":false,"inputTokens":6127,"outputTokens":5,"latencyMs":881.1776660000905},{"questionId":"q146","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":7726,"outputTokens":5,"latencyMs":1632.609708999982},{"questionId":"q146","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":5164,"outputTokens":5,"latencyMs":1582.639041999937},{"questionId":"q146","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":6184,"outputTokens":5,"latencyMs":1269.1984999999404},{"questionId":"q146","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"8","isCorrect":false,"inputTokens":8731,"outputTokens":5,"latencyMs":1036.2139169999864},{"questionId":"q146","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":6130,"outputTokens":5,"latencyMs":1217.4673750000075},{"questionId":"q147","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"5","isCorrect":false,"inputTokens":7726,"outputTokens":5,"latencyMs":1237.3523329999298},{"questionId":"q147","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"5","isCorrect":false,"inputTokens":5164,"outputTokens":5,"latencyMs":1081.811207999941},{"questionId":"q147","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"3","isCorrect":false,"inputTokens":6184,"outputTokens":5,"latencyMs":1097.698999999906},{"questionId":"q147","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":8731,"outputTokens":5,"latencyMs":1493.8901250000345},{"questionId":"q147","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"3","isCorrect":false,"inputTokens":6130,"outputTokens":5,"latencyMs":5959.227459000074},{"questionId":"q148","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":7726,"outputTokens":5,"latencyMs":1081.6637079999782},{"questionId":"q148","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":5164,"outputTokens":5,"latencyMs":1492.5457500000484},{"questionId":"q148","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":6184,"outputTokens":5,"latencyMs":874.1137499999022},{"questionId":"q148","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"6","isCorrect":false,"inputTokens":8731,"outputTokens":5,"latencyMs":1358.4161669999594},{"questionId":"q148","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":6130,"outputTokens":5,"latencyMs":1165.1978750000708},{"questionId":"q149","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"6","isCorrect":false,"inputTokens":7725,"outputTokens":5,"latencyMs":1272.5105410000542},{"questionId":"q149","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"5","isCorrect":false,"inputTokens":5163,"outputTokens":5,"latencyMs":2354.076624999987},{"questionId":"q149","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"6","isCorrect":false,"inputTokens":6183,"outputTokens":5,"latencyMs":1261.2219999999506},{"questionId":"q149","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"6","isCorrect":false,"inputTokens":8730,"outputTokens":5,"latencyMs":1170.514208000037},{"questionId":"q149","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"6","isCorrect":false,"inputTokens":6129,"outputTokens":5,"latencyMs":1300.107166999951},{"questionId":"q150","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":7725,"outputTokens":5,"latencyMs":1087.340208999929},{"questionId":"q150","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":5163,"outputTokens":5,"latencyMs":1174.3553329999559},{"questionId":"q150","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":6183,"outputTokens":5,"latencyMs":1174.71949999989},{"questionId":"q150","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":8730,"outputTokens":5,"latencyMs":1559.984334000037},{"questionId":"q150","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":6129,"outputTokens":5,"latencyMs":1118.785791000002},{"questionId":"q151","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":1193,"outputTokens":4,"latencyMs":1305.3309589999262},{"questionId":"q151","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":734,"outputTokens":4,"latencyMs":1374.1964999999618},{"questionId":"q151","format":"toon","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":828,"outputTokens":4,"latencyMs":1806.83050000004},{"questionId":"q151","format":"xml","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":1263,"outputTokens":4,"latencyMs":1012.5046250000596},{"questionId":"q151","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":848,"outputTokens":4,"latencyMs":1101.1594999999506},{"questionId":"q152","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1191,"outputTokens":8,"latencyMs":1029.557541999966},{"questionId":"q152","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":732,"outputTokens":8,"latencyMs":888.0808330000145},{"questionId":"q152","format":"toon","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":826,"outputTokens":8,"latencyMs":961.3018750000047},{"questionId":"q152","format":"xml","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1261,"outputTokens":8,"latencyMs":975.6594170000171},{"questionId":"q152","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":846,"outputTokens":8,"latencyMs":1009.3876249999739},{"questionId":"q153","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1191,"outputTokens":6,"latencyMs":1242.855666999938},{"questionId":"q153","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":732,"outputTokens":6,"latencyMs":1069.828375000041},{"questionId":"q153","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":826,"outputTokens":6,"latencyMs":1983.8990830000257},{"questionId":"q153","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1261,"outputTokens":6,"latencyMs":1225.030000000028},{"questionId":"q153","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":846,"outputTokens":6,"latencyMs":879.2118749999208},{"questionId":"q154","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":1193,"outputTokens":5,"latencyMs":1353.4172089999774},{"questionId":"q154","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":734,"outputTokens":5,"latencyMs":902.1746669999557},{"questionId":"q154","format":"toon","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":828,"outputTokens":5,"latencyMs":935.2352499999106},{"questionId":"q154","format":"xml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":1263,"outputTokens":5,"latencyMs":1013.6304160000291},{"questionId":"q154","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":848,"outputTokens":5,"latencyMs":1066.8241250000428},{"questionId":"q155","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1191,"outputTokens":6,"latencyMs":853.1445829999866},{"questionId":"q155","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":732,"outputTokens":6,"latencyMs":1091.0674169999547},{"questionId":"q155","format":"toon","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":826,"outputTokens":6,"latencyMs":925.9509580000304},{"questionId":"q155","format":"xml","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1261,"outputTokens":6,"latencyMs":862.8604999999516},{"questionId":"q155","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":846,"outputTokens":6,"latencyMs":890.2595420000143},{"questionId":"q156","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1193,"outputTokens":5,"latencyMs":1059.7631670000264},{"questionId":"q156","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":734,"outputTokens":5,"latencyMs":972.4143750000512},{"questionId":"q156","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":828,"outputTokens":5,"latencyMs":1569.0905000000494},{"questionId":"q156","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1263,"outputTokens":5,"latencyMs":1005.8317500000121},{"questionId":"q156","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":848,"outputTokens":5,"latencyMs":841.4386249999516},{"questionId":"q157","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1193,"outputTokens":6,"latencyMs":1031.2155829999829},{"questionId":"q157","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":734,"outputTokens":6,"latencyMs":1158.673999999999},{"questionId":"q157","format":"toon","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":828,"outputTokens":6,"latencyMs":1014.904749999987},{"questionId":"q157","format":"xml","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1263,"outputTokens":6,"latencyMs":1238.3551670000888},{"questionId":"q157","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":848,"outputTokens":6,"latencyMs":748.4136250000447},{"questionId":"q158","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":1191,"outputTokens":4,"latencyMs":931.4844580000499},{"questionId":"q158","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":732,"outputTokens":4,"latencyMs":4516.0599999999395},{"questionId":"q158","format":"toon","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":826,"outputTokens":4,"latencyMs":825.9817500000354},{"questionId":"q158","format":"xml","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":1261,"outputTokens":4,"latencyMs":1174.4812920000404},{"questionId":"q158","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":846,"outputTokens":4,"latencyMs":1099.4896670000162},{"questionId":"q159","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1192,"outputTokens":6,"latencyMs":1363.4585840000072},{"questionId":"q159","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":733,"outputTokens":6,"latencyMs":1411.4544589999132},{"questionId":"q159","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":827,"outputTokens":6,"latencyMs":1238.5263330000453},{"questionId":"q159","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1262,"outputTokens":6,"latencyMs":1455.4916249999078},{"questionId":"q159","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":847,"outputTokens":6,"latencyMs":903.907750000013},{"questionId":"q160","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1193,"outputTokens":9,"latencyMs":1261.4720420000376},{"questionId":"q160","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":734,"outputTokens":9,"latencyMs":1111.7462079999968},{"questionId":"q160","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":828,"outputTokens":9,"latencyMs":1340.6697080000304},{"questionId":"q160","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1263,"outputTokens":9,"latencyMs":1059.558874999988},{"questionId":"q160","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":848,"outputTokens":9,"latencyMs":1120.9716669999762},{"questionId":"q161","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1193,"outputTokens":5,"latencyMs":1607.4433750000317},{"questionId":"q161","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":734,"outputTokens":5,"latencyMs":1011.5435829999624},{"questionId":"q161","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":828,"outputTokens":5,"latencyMs":1098.9950830000453},{"questionId":"q161","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1263,"outputTokens":5,"latencyMs":1010.8417080000509},{"questionId":"q161","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":848,"outputTokens":5,"latencyMs":787.2757920000004},{"questionId":"q162","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1193,"outputTokens":5,"latencyMs":1078.016958000022},{"questionId":"q162","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":734,"outputTokens":5,"latencyMs":1090.475458000088},{"questionId":"q162","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":828,"outputTokens":5,"latencyMs":1037.0881660000887},{"questionId":"q162","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1263,"outputTokens":5,"latencyMs":1157.235624999972},{"questionId":"q162","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":848,"outputTokens":5,"latencyMs":1077.465749999974},{"questionId":"q163","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1192,"outputTokens":5,"latencyMs":903.7201249999925},{"questionId":"q163","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":733,"outputTokens":5,"latencyMs":1130.392165999976},{"questionId":"q163","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":827,"outputTokens":5,"latencyMs":1237.1364579999354},{"questionId":"q163","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1262,"outputTokens":5,"latencyMs":1383.588957999833},{"questionId":"q163","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":847,"outputTokens":5,"latencyMs":835.4454580000602},{"questionId":"q164","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1192,"outputTokens":5,"latencyMs":1211.8477079998702},{"questionId":"q164","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":733,"outputTokens":5,"latencyMs":940.1050410000607},{"questionId":"q164","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":827,"outputTokens":5,"latencyMs":12776.972374999896},{"questionId":"q164","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1262,"outputTokens":5,"latencyMs":907.2967500002123},{"questionId":"q164","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":847,"outputTokens":5,"latencyMs":939.8785409999546},{"questionId":"q165","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1193,"outputTokens":5,"latencyMs":1202.0232079999987},{"questionId":"q165","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":734,"outputTokens":5,"latencyMs":1380.7811670000665},{"questionId":"q165","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":828,"outputTokens":5,"latencyMs":995.2822500001639},{"questionId":"q165","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1263,"outputTokens":5,"latencyMs":1072.2940840001684},{"questionId":"q165","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":848,"outputTokens":5,"latencyMs":1213.750750000123},{"questionId":"q166","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1196,"outputTokens":5,"latencyMs":1290.3867499998305},{"questionId":"q166","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"0","isCorrect":false,"inputTokens":737,"outputTokens":5,"latencyMs":5153.930292000063},{"questionId":"q166","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":831,"outputTokens":5,"latencyMs":771.2725420000497},{"questionId":"q166","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1266,"outputTokens":5,"latencyMs":1029.2843750000466},{"questionId":"q166","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":851,"outputTokens":5,"latencyMs":830.8779579999391},{"questionId":"q167","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1192,"outputTokens":5,"latencyMs":1000.7899160000961},{"questionId":"q167","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":733,"outputTokens":5,"latencyMs":862.6022910000756},{"questionId":"q167","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":827,"outputTokens":5,"latencyMs":800.0890420000069},{"questionId":"q167","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1262,"outputTokens":5,"latencyMs":946.1868749998976},{"questionId":"q167","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":847,"outputTokens":5,"latencyMs":1117.201124999905},{"questionId":"q168","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":1194,"outputTokens":5,"latencyMs":1204.7393749998882},{"questionId":"q168","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":735,"outputTokens":5,"latencyMs":1216.2345000000205},{"questionId":"q168","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":829,"outputTokens":5,"latencyMs":967.7403329999652},{"questionId":"q168","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":1264,"outputTokens":5,"latencyMs":809.2372079999186},{"questionId":"q168","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":849,"outputTokens":5,"latencyMs":1035.5625},{"questionId":"q169","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":1196,"outputTokens":5,"latencyMs":2165.1945829999167},{"questionId":"q169","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":737,"outputTokens":5,"latencyMs":826.672082999954},{"questionId":"q169","format":"toon","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":831,"outputTokens":5,"latencyMs":816.7095419999678},{"questionId":"q169","format":"xml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":1266,"outputTokens":5,"latencyMs":942.9736250001006},{"questionId":"q169","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"11","isCorrect":false,"inputTokens":851,"outputTokens":5,"latencyMs":876.3174590000417},{"questionId":"q170","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":1195,"outputTokens":5,"latencyMs":896.7837920000311},{"questionId":"q170","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":736,"outputTokens":5,"latencyMs":1182.9224580000155},{"questionId":"q170","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":830,"outputTokens":5,"latencyMs":846.767375000054},{"questionId":"q170","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":1265,"outputTokens":5,"latencyMs":829.7917080000043},{"questionId":"q170","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":850,"outputTokens":5,"latencyMs":1161.572166999802},{"questionId":"q171","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1197,"outputTokens":5,"latencyMs":1094.7721659999806},{"questionId":"q171","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":738,"outputTokens":5,"latencyMs":144763.1995000001},{"questionId":"q171","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":832,"outputTokens":5,"latencyMs":842.0812909998931},{"questionId":"q171","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1267,"outputTokens":5,"latencyMs":871.6582090000156},{"questionId":"q171","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":852,"outputTokens":5,"latencyMs":1651.5552499999758},{"questionId":"q172","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1198,"outputTokens":5,"latencyMs":1265.5917080000509},{"questionId":"q172","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":739,"outputTokens":5,"latencyMs":2233.4475419998635},{"questionId":"q172","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":833,"outputTokens":5,"latencyMs":970.9932909999043},{"questionId":"q172","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1268,"outputTokens":5,"latencyMs":872.0836670000572},{"questionId":"q172","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":853,"outputTokens":5,"latencyMs":950.6219170000404},{"questionId":"q173","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1199,"outputTokens":5,"latencyMs":1076.4021660001017},{"questionId":"q173","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":740,"outputTokens":5,"latencyMs":1056.1212909999304},{"questionId":"q173","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":834,"outputTokens":5,"latencyMs":1197.260416999925},{"questionId":"q173","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1269,"outputTokens":5,"latencyMs":1250.9096670001745},{"questionId":"q173","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":854,"outputTokens":5,"latencyMs":1292.780665999977},{"questionId":"q174","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1195,"outputTokens":5,"latencyMs":1365.9032089998946},{"questionId":"q174","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":736,"outputTokens":5,"latencyMs":1026.5075000000652},{"questionId":"q174","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":830,"outputTokens":5,"latencyMs":1310.348292000126},{"questionId":"q174","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1265,"outputTokens":5,"latencyMs":1062.907750000013},{"questionId":"q174","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"2","isCorrect":false,"inputTokens":850,"outputTokens":5,"latencyMs":1895.4475839999504},{"questionId":"q175","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1200,"outputTokens":5,"latencyMs":1020.9082079997752},{"questionId":"q175","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":741,"outputTokens":5,"latencyMs":698.175167000154},{"questionId":"q175","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":835,"outputTokens":5,"latencyMs":724.3631249999162},{"questionId":"q175","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1270,"outputTokens":5,"latencyMs":785.9821659999434},{"questionId":"q175","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":855,"outputTokens":5,"latencyMs":928.7626660000533},{"questionId":"q176","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1193,"outputTokens":5,"latencyMs":1458.6687499999534},{"questionId":"q176","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":734,"outputTokens":5,"latencyMs":1679.2049579999875},{"questionId":"q176","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":828,"outputTokens":5,"latencyMs":913.517583000008},{"questionId":"q176","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1263,"outputTokens":5,"latencyMs":1152.9616670000833},{"questionId":"q176","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":848,"outputTokens":5,"latencyMs":827.9927920000628},{"questionId":"q177","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1203,"outputTokens":5,"latencyMs":1160.1210000000428},{"questionId":"q177","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":744,"outputTokens":5,"latencyMs":956.6085830000229},{"questionId":"q177","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":838,"outputTokens":5,"latencyMs":1112.8240829999559},{"questionId":"q177","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1273,"outputTokens":5,"latencyMs":1460.591957999859},{"questionId":"q177","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":858,"outputTokens":5,"latencyMs":976.0302909999155},{"questionId":"q178","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1198,"outputTokens":5,"latencyMs":868.6610000000801},{"questionId":"q178","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":739,"outputTokens":5,"latencyMs":976.780834000092},{"questionId":"q178","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":833,"outputTokens":5,"latencyMs":839.9454169999808},{"questionId":"q178","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1268,"outputTokens":5,"latencyMs":1317.0982079999521},{"questionId":"q178","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":853,"outputTokens":5,"latencyMs":769.4725830000825},{"questionId":"q179","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1196,"outputTokens":5,"latencyMs":1544.8656250000931},{"questionId":"q179","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":737,"outputTokens":5,"latencyMs":999.6729170000181},{"questionId":"q179","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":831,"outputTokens":5,"latencyMs":908.8042910001241},{"questionId":"q179","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1266,"outputTokens":5,"latencyMs":1764.4774170001037},{"questionId":"q179","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":851,"outputTokens":5,"latencyMs":1012.7515829999465},{"questionId":"q180","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":7884,"outputTokens":5,"latencyMs":1023.2217090001795},{"questionId":"q180","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":4775,"outputTokens":5,"latencyMs":976.480709000025},{"questionId":"q180","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":3024,"outputTokens":5,"latencyMs":1033.1939159999602},{"questionId":"q180","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":2870,"outputTokens":5,"latencyMs":1033.219165999908},{"questionId":"q180","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9369,"outputTokens":5,"latencyMs":1624.798790999921},{"questionId":"q180","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":5775,"outputTokens":5,"latencyMs":1146.808999999892},{"questionId":"q181","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id, name, email, department, salary, yearsExperience, active","isCorrect":true,"inputTokens":7890,"outputTokens":20,"latencyMs":1493.0855829999782},{"questionId":"q181","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4781,"outputTokens":20,"latencyMs":1241.8401669999585},{"questionId":"q181","format":"toon","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":3030,"outputTokens":20,"latencyMs":1215.891667000018},{"questionId":"q181","format":"csv","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2876,"outputTokens":20,"latencyMs":1388.4194170001429},{"questionId":"q181","format":"xml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":9375,"outputTokens":20,"latencyMs":1492.4003330001142},{"questionId":"q181","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id, name, email, department, salary, yearsExperience, active","isCorrect":true,"inputTokens":5781,"outputTokens":20,"latencyMs":1253.7766250001732},{"questionId":"q182","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":7887,"outputTokens":4,"latencyMs":1609.745665999828},{"questionId":"q182","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"email","actual":"department","isCorrect":false,"inputTokens":4778,"outputTokens":4,"latencyMs":1122.415374999866},{"questionId":"q182","format":"toon","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":3027,"outputTokens":4,"latencyMs":1056.9840830001049},{"questionId":"q182","format":"csv","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":2873,"outputTokens":4,"latencyMs":1019.4697920000181},{"questionId":"q182","format":"xml","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":9372,"outputTokens":4,"latencyMs":1315.7067500001285},{"questionId":"q182","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":5778,"outputTokens":4,"latencyMs":1219.7460000000428},{"questionId":"q183","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7888,"outputTokens":4,"latencyMs":1360.387374999933},{"questionId":"q183","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4779,"outputTokens":4,"latencyMs":1274.3325409998652},{"questionId":"q183","format":"toon","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":3028,"outputTokens":4,"latencyMs":1065.140417000046},{"questionId":"q183","format":"csv","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2874,"outputTokens":4,"latencyMs":2414.950875000097},{"questionId":"q183","format":"xml","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"Finance","isCorrect":false,"inputTokens":9373,"outputTokens":4,"latencyMs":1507.3267910000868},{"questionId":"q183","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5779,"outputTokens":4,"latencyMs":1626.4114999999292},{"questionId":"q184","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7888,"outputTokens":12,"latencyMs":1317.7676669999491},{"questionId":"q184","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4779,"outputTokens":12,"latencyMs":1378.3503329998348},{"questionId":"q184","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":3028,"outputTokens":12,"latencyMs":1177.045916999923},{"questionId":"q184","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2874,"outputTokens":12,"latencyMs":1237.0911669998895},{"questionId":"q184","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":9373,"outputTokens":12,"latencyMs":1077.699208999984},{"questionId":"q184","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5779,"outputTokens":12,"latencyMs":1234.212249999866},{"questionId":"q185","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":7885,"outputTokens":5,"latencyMs":1170.173290999839},{"questionId":"q185","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":4776,"outputTokens":5,"latencyMs":1116.161500000162},{"questionId":"q185","format":"toon","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":3025,"outputTokens":5,"latencyMs":1095.7748750001192},{"questionId":"q185","format":"csv","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":2871,"outputTokens":5,"latencyMs":1158.8812500000931},{"questionId":"q185","format":"xml","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":9370,"outputTokens":5,"latencyMs":1134.904249999905},{"questionId":"q185","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"7","actual":"6","isCorrect":false,"inputTokens":5776,"outputTokens":5,"latencyMs":1243.5042499999981},{"questionId":"q186","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":13015,"outputTokens":5,"latencyMs":1678.7380000001285},{"questionId":"q186","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":7976,"outputTokens":5,"latencyMs":1384.0720420000143},{"questionId":"q186","format":"toon","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":8293,"outputTokens":5,"latencyMs":1252.6643749999348},{"questionId":"q186","format":"xml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":14520,"outputTokens":5,"latencyMs":1826.156665999908},{"questionId":"q186","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":9418,"outputTokens":5,"latencyMs":1179.1018329998478},{"questionId":"q187","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":13024,"outputTokens":21,"latencyMs":1283.0529589999933},{"questionId":"q187","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":7985,"outputTokens":21,"latencyMs":1266.3688749999274},{"questionId":"q187","format":"toon","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":8302,"outputTokens":21,"latencyMs":1158.5445000000764},{"questionId":"q187","format":"xml","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":14529,"outputTokens":21,"latencyMs":1702.550375000108},{"questionId":"q187","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":9427,"outputTokens":21,"latencyMs":1435.0252910000272},{"questionId":"q188","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":13020,"outputTokens":5,"latencyMs":1968.4701249999925},{"questionId":"q188","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":7981,"outputTokens":5,"latencyMs":1507.1979580000043},{"questionId":"q188","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":8298,"outputTokens":5,"latencyMs":1180.720457999967},{"questionId":"q188","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":14525,"outputTokens":5,"latencyMs":1502.983290999895},{"questionId":"q188","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":9423,"outputTokens":5,"latencyMs":1255.5109999999404},{"questionId":"q189","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku, name, quantity, price","isCorrect":true,"inputTokens":13024,"outputTokens":11,"latencyMs":1210.6957920000423},{"questionId":"q189","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":7985,"outputTokens":11,"latencyMs":1585.3929999999236},{"questionId":"q189","format":"toon","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":8302,"outputTokens":11,"latencyMs":1575.310999999987},{"questionId":"q189","format":"xml","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku, name, quantity, price","isCorrect":true,"inputTokens":14529,"outputTokens":11,"latencyMs":1409.9164579999633},{"questionId":"q189","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku, name, quantity, price","isCorrect":true,"inputTokens":9427,"outputTokens":11,"latencyMs":1148.3658749999013},{"questionId":"q190","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13019,"outputTokens":4,"latencyMs":1164.7564999999013},{"questionId":"q190","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7980,"outputTokens":4,"latencyMs":1463.6336660000961},{"questionId":"q190","format":"toon","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8297,"outputTokens":4,"latencyMs":933.1279580001719},{"questionId":"q190","format":"xml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"delivered","isCorrect":false,"inputTokens":14524,"outputTokens":4,"latencyMs":1203.0982910001185},{"questionId":"q190","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9422,"outputTokens":4,"latencyMs":1136.2614160000812},{"questionId":"q191","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":13025,"outputTokens":10,"latencyMs":1245.9404589999467},{"questionId":"q191","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":7986,"outputTokens":10,"latencyMs":1023.4192079999484},{"questionId":"q191","format":"toon","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":8303,"outputTokens":10,"latencyMs":1262.64629100007},{"questionId":"q191","format":"xml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":14530,"outputTokens":10,"latencyMs":1174.8674999999348},{"questionId":"q191","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":9428,"outputTokens":10,"latencyMs":1051.8848329999018},{"questionId":"q192","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":4091,"outputTokens":5,"latencyMs":1134.7446669999044},{"questionId":"q192","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":2402,"outputTokens":5,"latencyMs":1327.1704580001533},{"questionId":"q192","format":"toon","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":1548,"outputTokens":5,"latencyMs":2189.1297919999342},{"questionId":"q192","format":"csv","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":1456,"outputTokens":5,"latencyMs":1355.6505830001552},{"questionId":"q192","format":"xml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":4793,"outputTokens":5,"latencyMs":1398.5017499998212},{"questionId":"q192","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":3122,"outputTokens":5,"latencyMs":933.0404159999453},{"questionId":"q193","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date, views, clicks, conversions, revenue, bounceRate","isCorrect":true,"inputTokens":4096,"outputTokens":17,"latencyMs":1531.188084000023},{"questionId":"q193","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2407,"outputTokens":17,"latencyMs":2024.385666999966},{"questionId":"q193","format":"toon","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1553,"outputTokens":17,"latencyMs":1458.139290999854},{"questionId":"q193","format":"csv","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1461,"outputTokens":17,"latencyMs":1593.8086660001427},{"questionId":"q193","format":"xml","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4798,"outputTokens":17,"latencyMs":1313.6192910000682},{"questionId":"q193","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date, views, clicks, conversions, revenue, bounceRate","isCorrect":true,"inputTokens":3127,"outputTokens":17,"latencyMs":1642.309749999782},{"questionId":"q194","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":4094,"outputTokens":6,"latencyMs":995.8711250000633},{"questionId":"q194","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":2405,"outputTokens":6,"latencyMs":1148.8222499999683},{"questionId":"q194","format":"toon","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1551,"outputTokens":4,"latencyMs":914.0247909999453},{"questionId":"q194","format":"csv","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1459,"outputTokens":4,"latencyMs":885.8539580001961},{"questionId":"q194","format":"xml","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":4796,"outputTokens":6,"latencyMs":1090.3401669999585},{"questionId":"q194","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":3125,"outputTokens":6,"latencyMs":6191.641957999906},{"questionId":"q195","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4095,"outputTokens":10,"latencyMs":1103.72275000019},{"questionId":"q195","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2406,"outputTokens":10,"latencyMs":960.2690840000287},{"questionId":"q195","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1552,"outputTokens":10,"latencyMs":874.4887079999316},{"questionId":"q195","format":"csv","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1460,"outputTokens":10,"latencyMs":62539.49575},{"questionId":"q195","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4797,"outputTokens":10,"latencyMs":1175.3044579999987},{"questionId":"q195","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3126,"outputTokens":10,"latencyMs":1091.6320830001496},{"questionId":"q196","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":4091,"outputTokens":5,"latencyMs":1242.6800420000218},{"questionId":"q196","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":2402,"outputTokens":5,"latencyMs":1334.3684580000117},{"questionId":"q196","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":1548,"outputTokens":5,"latencyMs":1189.2047079999465},{"questionId":"q196","format":"csv","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":1456,"outputTokens":5,"latencyMs":2203.1670839998405},{"questionId":"q196","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":4793,"outputTokens":5,"latencyMs":1081.4394169999287},{"questionId":"q196","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":3122,"outputTokens":5,"latencyMs":1374.5313329999335},{"questionId":"q197","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":17422,"outputTokens":5,"latencyMs":1805.9630410000682},{"questionId":"q197","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":12557,"outputTokens":5,"latencyMs":1487.7367920000106},{"questionId":"q197","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9320,"outputTokens":5,"latencyMs":1138.2207919999491},{"questionId":"q197","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9138,"outputTokens":5,"latencyMs":1301.4219160000794},{"questionId":"q197","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":19812,"outputTokens":5,"latencyMs":1762.2343330001459},{"questionId":"q197","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":14497,"outputTokens":5,"latencyMs":1478.3664170000702},{"questionId":"q198","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id, name, repo, description, createdAt, updatedAt, pushedAt, stars, watchers, forks, defaultBranch","isCorrect":false,"inputTokens":17428,"outputTokens":35,"latencyMs":2943.5730419999454},{"questionId":"q198","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":12563,"outputTokens":35,"latencyMs":1898.279499999946},{"questionId":"q198","format":"toon","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":9326,"outputTokens":35,"latencyMs":1212.1238329999615},{"questionId":"q198","format":"csv","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":9144,"outputTokens":35,"latencyMs":1401.4154580000322},{"questionId":"q198","format":"xml","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id, name, repo, description, createdAt, updatedAt, pushedAt, stars, watchers, forks, defaultBranch","isCorrect":false,"inputTokens":19818,"outputTokens":35,"latencyMs":2067.8010420000646},{"questionId":"q198","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id, name, repo, description, createdAt, updatedAt, pushedAt, stars, watchers, forks, defaultBranch","isCorrect":false,"inputTokens":14503,"outputTokens":35,"latencyMs":1435.4114999999292},{"questionId":"q199","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":17426,"outputTokens":6,"latencyMs":1274.9157080000732},{"questionId":"q199","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"createdAt","isCorrect":false,"inputTokens":12561,"outputTokens":6,"latencyMs":1060.4352090000175},{"questionId":"q199","format":"toon","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"description","isCorrect":false,"inputTokens":9324,"outputTokens":4,"latencyMs":1256.8129170001484},{"questionId":"q199","format":"csv","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":9142,"outputTokens":6,"latencyMs":1355.0008749999106},{"questionId":"q199","format":"xml","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":19816,"outputTokens":6,"latencyMs":1593.0584160001017},{"questionId":"q199","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":14501,"outputTokens":6,"latencyMs":2208.882250000024},{"questionId":"q200","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17426,"outputTokens":7,"latencyMs":1794.4396250001155},{"questionId":"q200","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":12561,"outputTokens":7,"latencyMs":1209.313000000082},{"questionId":"q200","format":"toon","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":9324,"outputTokens":7,"latencyMs":937.376750000054},{"questionId":"q200","format":"csv","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":9142,"outputTokens":7,"latencyMs":1812.7524589998648},{"questionId":"q200","format":"xml","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":19816,"outputTokens":7,"latencyMs":1952.1014170001727},{"questionId":"q200","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":14501,"outputTokens":7,"latencyMs":1613.5063750001136},{"questionId":"q201","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":17423,"outputTokens":5,"latencyMs":1684.8589580000844},{"questionId":"q201","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":12558,"outputTokens":5,"latencyMs":2327.963083000155},{"questionId":"q201","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":9321,"outputTokens":5,"latencyMs":1801.874000000069},{"questionId":"q201","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":9139,"outputTokens":5,"latencyMs":1582.0344169999007},{"questionId":"q201","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"12","isCorrect":false,"inputTokens":19813,"outputTokens":5,"latencyMs":2354.789624999976},{"questionId":"q201","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"13","isCorrect":false,"inputTokens":14498,"outputTokens":5,"latencyMs":1156.6650000000373},{"questionId":"q202","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"75","actual":"80","isCorrect":false,"inputTokens":7716,"outputTokens":5,"latencyMs":1337.9508750000969},{"questionId":"q202","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"75","actual":"60","isCorrect":false,"inputTokens":5154,"outputTokens":5,"latencyMs":1459.3844999999274},{"questionId":"q202","format":"toon","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":6174,"outputTokens":5,"latencyMs":1836.0140839999076},{"questionId":"q202","format":"xml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"100","isCorrect":false,"inputTokens":8721,"outputTokens":5,"latencyMs":1137.3295839999337},{"questionId":"q202","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"80","isCorrect":false,"inputTokens":6120,"outputTokens":5,"latencyMs":1284.7181669999845},{"questionId":"q203","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error","isCorrect":true,"inputTokens":7726,"outputTokens":18,"latencyMs":1204.37975000008},{"questionId":"q203","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":true,"inputTokens":5164,"outputTokens":26,"latencyMs":1315.1313340000343},{"questionId":"q203","format":"toon","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error, message, stack, retryable","isCorrect":true,"inputTokens":6184,"outputTokens":26,"latencyMs":1342.030834000092},{"questionId":"q203","format":"xml","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error, message, stack, retryable","isCorrect":true,"inputTokens":8731,"outputTokens":26,"latencyMs":1421.953666999936},{"questionId":"q203","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error, message, stack, retryable","isCorrect":true,"inputTokens":6130,"outputTokens":26,"latencyMs":1152.4007500000298},{"questionId":"q204","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":7720,"outputTokens":4,"latencyMs":1808.1940840000752},{"questionId":"q204","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":5158,"outputTokens":4,"latencyMs":2826.0154579998925},{"questionId":"q204","format":"toon","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6178,"outputTokens":4,"latencyMs":1545.986625000136},{"questionId":"q204","format":"xml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":8725,"outputTokens":4,"latencyMs":1064.5447919999715},{"questionId":"q204","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6124,"outputTokens":4,"latencyMs":995.8226250000298}] \ No newline at end of file +[{"questionId":"q1","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7939,"outputTokens":6,"latencyMs":1488.7717919999996},{"questionId":"q1","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":4830,"outputTokens":6,"latencyMs":1424.8316250000007},{"questionId":"q1","format":"toon","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":3079,"outputTokens":6,"latencyMs":1200.6324999999997},{"questionId":"q1","format":"csv","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2925,"outputTokens":6,"latencyMs":1187.5327080000006},{"questionId":"q1","format":"xml","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":9424,"outputTokens":6,"latencyMs":1200.2132079999992},{"questionId":"q1","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5830,"outputTokens":6,"latencyMs":1341.1957500000008},{"questionId":"q2","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7938,"outputTokens":4,"latencyMs":1182.2189579999995},{"questionId":"q2","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":4829,"outputTokens":4,"latencyMs":1191.913125000001},{"questionId":"q2","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":3078,"outputTokens":4,"latencyMs":1393.160915999999},{"questionId":"q2","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2924,"outputTokens":4,"latencyMs":1192.7132500000007},{"questionId":"q2","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":9423,"outputTokens":4,"latencyMs":1360.8396249999987},{"questionId":"q2","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5829,"outputTokens":4,"latencyMs":1619.3704579999994},{"questionId":"q3","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7943,"outputTokens":12,"latencyMs":1131.9942499999997},{"questionId":"q3","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4834,"outputTokens":12,"latencyMs":1391.2939580000002},{"questionId":"q3","format":"toon","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":3083,"outputTokens":12,"latencyMs":1181.2237920000007},{"questionId":"q3","format":"csv","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2929,"outputTokens":12,"latencyMs":1098.3214580000003},{"questionId":"q3","format":"xml","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":9428,"outputTokens":12,"latencyMs":1468.2502499999991},{"questionId":"q3","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5834,"outputTokens":12,"latencyMs":1044.1940839999988},{"questionId":"q4","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":7941,"outputTokens":5,"latencyMs":1145.9674579999992},{"questionId":"q4","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":4832,"outputTokens":5,"latencyMs":1282.7140419999996},{"questionId":"q4","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":3081,"outputTokens":5,"latencyMs":993.7237079999995},{"questionId":"q4","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":2927,"outputTokens":5,"latencyMs":1029.5127499999999},{"questionId":"q4","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":9426,"outputTokens":5,"latencyMs":1170.4240829999999},{"questionId":"q4","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":5832,"outputTokens":5,"latencyMs":1297.9727910000001},{"questionId":"q5","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"no","actual":"false","isCorrect":true,"inputTokens":7936,"outputTokens":4,"latencyMs":1060.0110000000004},{"questionId":"q5","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"no","actual":"false","isCorrect":true,"inputTokens":4827,"outputTokens":4,"latencyMs":932.3761250000007},{"questionId":"q5","format":"toon","model":"claude-haiku-4-5-20251001","expected":"no","actual":"false","isCorrect":true,"inputTokens":3076,"outputTokens":4,"latencyMs":967.2909170000003},{"questionId":"q5","format":"csv","model":"claude-haiku-4-5-20251001","expected":"no","actual":"0","isCorrect":true,"inputTokens":2922,"outputTokens":5,"latencyMs":989.7537499999999},{"questionId":"q5","format":"xml","model":"claude-haiku-4-5-20251001","expected":"no","actual":"false","isCorrect":true,"inputTokens":9421,"outputTokens":4,"latencyMs":1163.8299580000003},{"questionId":"q5","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"no","actual":"false","isCorrect":true,"inputTokens":5827,"outputTokens":4,"latencyMs":1012.1698340000003},{"questionId":"q6","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7939,"outputTokens":6,"latencyMs":1043.6857499999987},{"questionId":"q6","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":4830,"outputTokens":6,"latencyMs":1741.9372920000005},{"questionId":"q6","format":"toon","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":3079,"outputTokens":6,"latencyMs":1061.195040999999},{"questionId":"q6","format":"csv","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2925,"outputTokens":6,"latencyMs":897.5309579999994},{"questionId":"q6","format":"xml","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":9424,"outputTokens":6,"latencyMs":1091.2617499999997},{"questionId":"q6","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5830,"outputTokens":6,"latencyMs":1198.945791},{"questionId":"q7","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7939,"outputTokens":4,"latencyMs":1124.808833000001},{"questionId":"q7","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":4830,"outputTokens":4,"latencyMs":1004.0825409999998},{"questionId":"q7","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":3079,"outputTokens":4,"latencyMs":1026.612874999999},{"questionId":"q7","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2925,"outputTokens":4,"latencyMs":900.933500000001},{"questionId":"q7","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":9424,"outputTokens":4,"latencyMs":1537.3743749999994},{"questionId":"q7","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5830,"outputTokens":4,"latencyMs":1069.091042},{"questionId":"q8","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7941,"outputTokens":12,"latencyMs":1463.4106250000004},{"questionId":"q8","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4832,"outputTokens":12,"latencyMs":1045.6618749999998},{"questionId":"q8","format":"toon","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":3081,"outputTokens":12,"latencyMs":1144.8265419999989},{"questionId":"q8","format":"csv","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2927,"outputTokens":12,"latencyMs":1266.5881250000002},{"questionId":"q8","format":"xml","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":9426,"outputTokens":12,"latencyMs":1094.5647079999999},{"questionId":"q8","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5832,"outputTokens":12,"latencyMs":1037.4817500000008},{"questionId":"q9","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":7941,"outputTokens":5,"latencyMs":1064.6803340000006},{"questionId":"q9","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":4832,"outputTokens":5,"latencyMs":1234.0882500000007},{"questionId":"q9","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":3081,"outputTokens":5,"latencyMs":1072.323041999989},{"questionId":"q9","format":"csv","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":2927,"outputTokens":5,"latencyMs":1155.7975410000072},{"questionId":"q9","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":9426,"outputTokens":5,"latencyMs":1583.6992499999906},{"questionId":"q9","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":5832,"outputTokens":5,"latencyMs":1564.031124999994},{"questionId":"q10","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"true","isCorrect":true,"inputTokens":7938,"outputTokens":4,"latencyMs":1184.0812499999884},{"questionId":"q10","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"true","isCorrect":true,"inputTokens":4829,"outputTokens":4,"latencyMs":1815.7772499999992},{"questionId":"q10","format":"toon","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"true","isCorrect":true,"inputTokens":3078,"outputTokens":4,"latencyMs":1103.1678749999992},{"questionId":"q10","format":"csv","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"1","isCorrect":true,"inputTokens":2924,"outputTokens":5,"latencyMs":1059.914082999996},{"questionId":"q10","format":"xml","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"true","isCorrect":true,"inputTokens":9423,"outputTokens":4,"latencyMs":1852.225999999995},{"questionId":"q10","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"yes","actual":"true","isCorrect":true,"inputTokens":5829,"outputTokens":4,"latencyMs":1238.4458329999907},{"questionId":"q11","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7938,"outputTokens":6,"latencyMs":1264.0889580000076},{"questionId":"q11","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":4829,"outputTokens":6,"latencyMs":984.5701249999984},{"questionId":"q11","format":"toon","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":3078,"outputTokens":6,"latencyMs":1400.2183340000047},{"questionId":"q11","format":"csv","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2924,"outputTokens":6,"latencyMs":891.1542500000069},{"questionId":"q11","format":"xml","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":9423,"outputTokens":6,"latencyMs":1176.492333000002},{"questionId":"q11","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5829,"outputTokens":6,"latencyMs":1310.523291999998},{"questionId":"q12","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7935,"outputTokens":4,"latencyMs":1027.5788330000069},{"questionId":"q12","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":4826,"outputTokens":4,"latencyMs":992.2010420000006},{"questionId":"q12","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":3075,"outputTokens":4,"latencyMs":1090.145749999996},{"questionId":"q12","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2921,"outputTokens":4,"latencyMs":867.523457999996},{"questionId":"q12","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":9420,"outputTokens":4,"latencyMs":1272.2972080000036},{"questionId":"q12","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5826,"outputTokens":4,"latencyMs":1223.0700840000063},{"questionId":"q13","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":7934,"outputTokens":5,"latencyMs":1089.1235420000012},{"questionId":"q13","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":4825,"outputTokens":5,"latencyMs":992.0438330000034},{"questionId":"q13","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":3074,"outputTokens":5,"latencyMs":1052.5001670000056},{"questionId":"q13","format":"csv","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":2920,"outputTokens":5,"latencyMs":893.6204160000052},{"questionId":"q13","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":9419,"outputTokens":5,"latencyMs":1324.1672920000128},{"questionId":"q13","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":5825,"outputTokens":5,"latencyMs":1415.1354579999897},{"questionId":"q14","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"13","isCorrect":false,"inputTokens":7934,"outputTokens":5,"latencyMs":1215.046291000006},{"questionId":"q14","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":4825,"outputTokens":5,"latencyMs":1065.2862919999898},{"questionId":"q14","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":3074,"outputTokens":5,"latencyMs":973.8248339999991},{"questionId":"q14","format":"csv","model":"claude-haiku-4-5-20251001","expected":"17","actual":"14","isCorrect":false,"inputTokens":2920,"outputTokens":5,"latencyMs":1048.0387499999924},{"questionId":"q14","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":9419,"outputTokens":5,"latencyMs":4246.3034999999945},{"questionId":"q14","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":5825,"outputTokens":5,"latencyMs":3089.0154579999944},{"questionId":"q15","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"13","isCorrect":false,"inputTokens":7934,"outputTokens":5,"latencyMs":1041.0287499999977},{"questionId":"q15","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":4825,"outputTokens":5,"latencyMs":1126.9441249999945},{"questionId":"q15","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"12","isCorrect":false,"inputTokens":3074,"outputTokens":5,"latencyMs":1174.3665829999954},{"questionId":"q15","format":"csv","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":2920,"outputTokens":5,"latencyMs":1421.4708329999994},{"questionId":"q15","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":9419,"outputTokens":5,"latencyMs":1255.9135000000097},{"questionId":"q15","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"14","isCorrect":false,"inputTokens":5825,"outputTokens":5,"latencyMs":1288.064916000003},{"questionId":"q16","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"91","actual":"85","isCorrect":false,"inputTokens":7939,"outputTokens":5,"latencyMs":1213.8851250000007},{"questionId":"q16","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"91","actual":"78","isCorrect":false,"inputTokens":4830,"outputTokens":5,"latencyMs":1034.356375000003},{"questionId":"q16","format":"toon","model":"claude-haiku-4-5-20251001","expected":"91","actual":"85","isCorrect":false,"inputTokens":3079,"outputTokens":5,"latencyMs":1150.7799589999922},{"questionId":"q16","format":"csv","model":"claude-haiku-4-5-20251001","expected":"91","actual":"76","isCorrect":false,"inputTokens":2925,"outputTokens":5,"latencyMs":927.4066250000033},{"questionId":"q16","format":"xml","model":"claude-haiku-4-5-20251001","expected":"91","actual":"92","isCorrect":false,"inputTokens":9424,"outputTokens":5,"latencyMs":989.8042080000014},{"questionId":"q16","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"91","actual":"91","isCorrect":true,"inputTokens":5830,"outputTokens":5,"latencyMs":1249.2909999999974},{"questionId":"q17","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"67","actual":"57","isCorrect":false,"inputTokens":7939,"outputTokens":5,"latencyMs":1313.7873749999999},{"questionId":"q17","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"67","actual":"60","isCorrect":false,"inputTokens":4830,"outputTokens":5,"latencyMs":1045.1794999999984},{"questionId":"q17","format":"toon","model":"claude-haiku-4-5-20251001","expected":"67","actual":"46","isCorrect":false,"inputTokens":3079,"outputTokens":5,"latencyMs":985.7277500000055},{"questionId":"q17","format":"csv","model":"claude-haiku-4-5-20251001","expected":"67","actual":"42","isCorrect":false,"inputTokens":2925,"outputTokens":5,"latencyMs":964.684500000003},{"questionId":"q17","format":"xml","model":"claude-haiku-4-5-20251001","expected":"67","actual":"58","isCorrect":false,"inputTokens":9424,"outputTokens":5,"latencyMs":1365.7662500000006},{"questionId":"q17","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"67","actual":"61","isCorrect":false,"inputTokens":5830,"outputTokens":5,"latencyMs":1559.340124999988},{"questionId":"q18","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"41","actual":"34","isCorrect":false,"inputTokens":7939,"outputTokens":5,"latencyMs":1418.1319579999981},{"questionId":"q18","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":4830,"outputTokens":5,"latencyMs":1229.3413750000036},{"questionId":"q18","format":"toon","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":3079,"outputTokens":5,"latencyMs":1237.7892920000013},{"questionId":"q18","format":"csv","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":2925,"outputTokens":5,"latencyMs":1149.4620410000061},{"questionId":"q18","format":"xml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"34","isCorrect":false,"inputTokens":9424,"outputTokens":5,"latencyMs":1498.104582999993},{"questionId":"q18","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"28","isCorrect":false,"inputTokens":5830,"outputTokens":5,"latencyMs":1437.632666999998},{"questionId":"q19","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":7935,"outputTokens":5,"latencyMs":1284.2168329999986},{"questionId":"q19","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":4826,"outputTokens":5,"latencyMs":1304.1590419999993},{"questionId":"q19","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":3075,"outputTokens":5,"latencyMs":1151.4296670000185},{"questionId":"q19","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":2921,"outputTokens":5,"latencyMs":1547.9670420000039},{"questionId":"q19","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9420,"outputTokens":5,"latencyMs":1110.2993750000169},{"questionId":"q19","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":5826,"outputTokens":5,"latencyMs":1029.0622079999885},{"questionId":"q20","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"99857.49","isCorrect":false,"inputTokens":7936,"outputTokens":8,"latencyMs":1178.934208000006},{"questionId":"q20","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"100657.68","isCorrect":false,"inputTokens":4827,"outputTokens":8,"latencyMs":1070.5215410000092},{"questionId":"q20","format":"toon","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"97474.27","isCorrect":false,"inputTokens":3076,"outputTokens":8,"latencyMs":1914.8881249999831},{"questionId":"q20","format":"csv","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"98208.5","isCorrect":false,"inputTokens":2922,"outputTokens":8,"latencyMs":1134.84874999999},{"questionId":"q20","format":"xml","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"100591.83","isCorrect":false,"inputTokens":9421,"outputTokens":8,"latencyMs":1201.0446670000092},{"questionId":"q20","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"96503","actual":"100560.31","isCorrect":false,"inputTokens":5827,"outputTokens":8,"latencyMs":1365.4649169999902},{"questionId":"q21","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"78","actual":"78","isCorrect":true,"inputTokens":7933,"outputTokens":5,"latencyMs":1057.5927919999813},{"questionId":"q21","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"78","actual":"73","isCorrect":false,"inputTokens":4824,"outputTokens":5,"latencyMs":984.6357499999867},{"questionId":"q21","format":"toon","model":"claude-haiku-4-5-20251001","expected":"78","actual":"80","isCorrect":false,"inputTokens":3073,"outputTokens":5,"latencyMs":961.1789580000041},{"questionId":"q21","format":"csv","model":"claude-haiku-4-5-20251001","expected":"78","actual":"75","isCorrect":false,"inputTokens":2919,"outputTokens":5,"latencyMs":1046.232457999984},{"questionId":"q21","format":"xml","model":"claude-haiku-4-5-20251001","expected":"78","actual":"76","isCorrect":false,"inputTokens":9418,"outputTokens":5,"latencyMs":1062.490832999989},{"questionId":"q21","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"78","actual":"76","isCorrect":false,"inputTokens":5824,"outputTokens":5,"latencyMs":1386.1599170000118},{"questionId":"q22","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":7933,"outputTokens":5,"latencyMs":1155.2419999999984},{"questionId":"q22","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":4824,"outputTokens":5,"latencyMs":984.3614579999994},{"questionId":"q22","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":3073,"outputTokens":5,"latencyMs":2339.569790999987},{"questionId":"q22","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"22","isCorrect":true,"inputTokens":2919,"outputTokens":5,"latencyMs":1645.8104999999923},{"questionId":"q22","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":9418,"outputTokens":5,"latencyMs":1110.3421669999952},{"questionId":"q22","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":5824,"outputTokens":5,"latencyMs":1158.1035830000183},{"questionId":"q23","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"12","actual":"11","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":1850.1329160000023},{"questionId":"q23","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"12","actual":"9","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1923.266666999989},{"questionId":"q23","format":"toon","model":"claude-haiku-4-5-20251001","expected":"12","actual":"8","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":1092.6574580000015},{"questionId":"q23","format":"csv","model":"claude-haiku-4-5-20251001","expected":"12","actual":"10","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":993.8929580000113},{"questionId":"q23","format":"xml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"11","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1253.2862920000043},{"questionId":"q23","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"10","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":1106.7149579999968},{"questionId":"q24","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":1006.815042000002},{"questionId":"q24","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1115.0337080000027},{"questionId":"q24","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":1048.2592920000025},{"questionId":"q24","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":910.7634580000013},{"questionId":"q24","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":119476.61179200001},{"questionId":"q24","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":982.0880420000176},{"questionId":"q25","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":1282.5784159999748},{"questionId":"q25","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1083.9630830000096},{"questionId":"q25","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":927.612374999997},{"questionId":"q25","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":954.8622909999976},{"questionId":"q25","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":4318.050334},{"questionId":"q25","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":1295.8421249999956},{"questionId":"q26","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"12","actual":"8","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":1169.7239170000248},{"questionId":"q26","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"12","actual":"8","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1160.2663750000065},{"questionId":"q26","format":"toon","model":"claude-haiku-4-5-20251001","expected":"12","actual":"6","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":1324.3053749999963},{"questionId":"q26","format":"csv","model":"claude-haiku-4-5-20251001","expected":"12","actual":"6","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":1237.4985830000078},{"questionId":"q26","format":"xml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"8","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":2209.9241660000116},{"questionId":"q26","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"7","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":1174.278707999998},{"questionId":"q27","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":1188.8322920000064},{"questionId":"q27","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1392.668875000003},{"questionId":"q27","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":1144.9836670000223},{"questionId":"q27","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":1185.1800420000218},{"questionId":"q27","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"7","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1109.5572499999835},{"questionId":"q27","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"6","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":1004.8929999999818},{"questionId":"q28","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"63","actual":"72","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":1536.55349999998},{"questionId":"q28","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"63","actual":"67","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1200.943041999999},{"questionId":"q28","format":"toon","model":"claude-haiku-4-5-20251001","expected":"63","actual":"72","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":1219.1552500000107},{"questionId":"q28","format":"csv","model":"claude-haiku-4-5-20251001","expected":"63","actual":"62","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":1005.1289589999942},{"questionId":"q28","format":"xml","model":"claude-haiku-4-5-20251001","expected":"63","actual":"62","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1041.4887500000186},{"questionId":"q28","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"63","actual":"68","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":1143.78916700001},{"questionId":"q29","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"53","actual":"54","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":1129.352041999984},{"questionId":"q29","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"53","actual":"56","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1346.812042000005},{"questionId":"q29","format":"toon","model":"claude-haiku-4-5-20251001","expected":"53","actual":"57","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":1013.5596249999944},{"questionId":"q29","format":"csv","model":"claude-haiku-4-5-20251001","expected":"53","actual":"45","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":1137.8311660000181},{"questionId":"q29","format":"xml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"57","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1281.3305829999736},{"questionId":"q29","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"62","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":1146.0332919999782},{"questionId":"q30","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"39","actual":"28","isCorrect":false,"inputTokens":7941,"outputTokens":5,"latencyMs":962.1130420000118},{"questionId":"q30","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"39","actual":"34","isCorrect":false,"inputTokens":4832,"outputTokens":5,"latencyMs":1271.9399580000027},{"questionId":"q30","format":"toon","model":"claude-haiku-4-5-20251001","expected":"39","actual":"33","isCorrect":false,"inputTokens":3081,"outputTokens":5,"latencyMs":1231.0171670000127},{"questionId":"q30","format":"csv","model":"claude-haiku-4-5-20251001","expected":"39","actual":"27","isCorrect":false,"inputTokens":2927,"outputTokens":5,"latencyMs":1907.3603749999893},{"questionId":"q30","format":"xml","model":"claude-haiku-4-5-20251001","expected":"39","actual":"32","isCorrect":false,"inputTokens":9426,"outputTokens":5,"latencyMs":1237.180583999987},{"questionId":"q30","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"39","actual":"34","isCorrect":false,"inputTokens":5832,"outputTokens":5,"latencyMs":1330.3151660000149},{"questionId":"q31","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":7942,"outputTokens":5,"latencyMs":1211.030208000011},{"questionId":"q31","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":4833,"outputTokens":5,"latencyMs":1125.4293749999779},{"questionId":"q31","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":3082,"outputTokens":5,"latencyMs":1125.0806660000235},{"questionId":"q31","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":2928,"outputTokens":5,"latencyMs":1203.0037089999823},{"questionId":"q31","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"8","isCorrect":false,"inputTokens":9427,"outputTokens":5,"latencyMs":1321.5858330000192},{"questionId":"q31","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":5833,"outputTokens":5,"latencyMs":1174.450708999997},{"questionId":"q32","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"8","actual":"7","isCorrect":false,"inputTokens":7942,"outputTokens":5,"latencyMs":1131.7181249999849},{"questionId":"q32","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"8","actual":"7","isCorrect":false,"inputTokens":4833,"outputTokens":5,"latencyMs":1292.2494589999842},{"questionId":"q32","format":"toon","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":3082,"outputTokens":5,"latencyMs":1056.2060000000056},{"questionId":"q32","format":"csv","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":2928,"outputTokens":5,"latencyMs":914.0282920000027},{"questionId":"q32","format":"xml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":9427,"outputTokens":5,"latencyMs":1058.9597080000094},{"questionId":"q32","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":5833,"outputTokens":5,"latencyMs":1138.2416660000163},{"questionId":"q33","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"15","actual":"9","isCorrect":false,"inputTokens":7942,"outputTokens":5,"latencyMs":1159.8052090000128},{"questionId":"q33","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"15","actual":"8","isCorrect":false,"inputTokens":4833,"outputTokens":5,"latencyMs":1179.3050000000221},{"questionId":"q33","format":"toon","model":"claude-haiku-4-5-20251001","expected":"15","actual":"7","isCorrect":false,"inputTokens":3082,"outputTokens":5,"latencyMs":2072.5861670000013},{"questionId":"q33","format":"csv","model":"claude-haiku-4-5-20251001","expected":"15","actual":"8","isCorrect":false,"inputTokens":2928,"outputTokens":5,"latencyMs":1516.0497909999976},{"questionId":"q33","format":"xml","model":"claude-haiku-4-5-20251001","expected":"15","actual":"9","isCorrect":false,"inputTokens":9427,"outputTokens":5,"latencyMs":1098.749375000014},{"questionId":"q33","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"15","actual":"9","isCorrect":false,"inputTokens":5833,"outputTokens":5,"latencyMs":1028.6647499999963},{"questionId":"q34","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"12","actual":"14","isCorrect":false,"inputTokens":7935,"outputTokens":5,"latencyMs":1599.851416999998},{"questionId":"q34","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"12","actual":"11","isCorrect":false,"inputTokens":4826,"outputTokens":5,"latencyMs":1247.702500000014},{"questionId":"q34","format":"toon","model":"claude-haiku-4-5-20251001","expected":"12","actual":"14","isCorrect":false,"inputTokens":3075,"outputTokens":5,"latencyMs":1222.0808750000142},{"questionId":"q34","format":"csv","model":"claude-haiku-4-5-20251001","expected":"12","actual":"13","isCorrect":false,"inputTokens":2921,"outputTokens":5,"latencyMs":1043.5218340000138},{"questionId":"q34","format":"xml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"13","isCorrect":false,"inputTokens":9420,"outputTokens":5,"latencyMs":1324.776125000004},{"questionId":"q34","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"12","actual":"13","isCorrect":false,"inputTokens":5826,"outputTokens":5,"latencyMs":1299.7890419999894},{"questionId":"q35","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"13","isCorrect":false,"inputTokens":7935,"outputTokens":5,"latencyMs":1194.0861659999937},{"questionId":"q35","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":4826,"outputTokens":5,"latencyMs":1254.9089580000145},{"questionId":"q35","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":3075,"outputTokens":5,"latencyMs":1161.8742499999935},{"questionId":"q35","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":2921,"outputTokens":5,"latencyMs":1073.0098749999888},{"questionId":"q35","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"12","isCorrect":false,"inputTokens":9420,"outputTokens":5,"latencyMs":2201.6162919999915},{"questionId":"q35","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":5826,"outputTokens":5,"latencyMs":1558.6932920000108},{"questionId":"q36","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":13071,"outputTokens":7,"latencyMs":1283.411334000004},{"questionId":"q36","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8032,"outputTokens":7,"latencyMs":1105.1879999999946},{"questionId":"q36","format":"toon","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8349,"outputTokens":7,"latencyMs":1409.0812499999884},{"questionId":"q36","format":"xml","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":14576,"outputTokens":7,"latencyMs":1401.9225830000069},{"questionId":"q36","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":9474,"outputTokens":7,"latencyMs":1155.835207999975},{"questionId":"q37","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":13071,"outputTokens":4,"latencyMs":1765.247250000015},{"questionId":"q37","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8032,"outputTokens":4,"latencyMs":1487.960958999989},{"questionId":"q37","format":"toon","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8349,"outputTokens":4,"latencyMs":1007.5285840000142},{"questionId":"q37","format":"xml","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":14576,"outputTokens":4,"latencyMs":1259.9838750000054},{"questionId":"q37","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":9474,"outputTokens":4,"latencyMs":1105.1101250000065},{"questionId":"q38","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":13071,"outputTokens":7,"latencyMs":1630.5366249999788},{"questionId":"q38","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8032,"outputTokens":7,"latencyMs":1085.6638749999984},{"questionId":"q38","format":"toon","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8349,"outputTokens":7,"latencyMs":1176.3057919999992},{"questionId":"q38","format":"xml","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":14576,"outputTokens":7,"latencyMs":1291.9421250000014},{"questionId":"q38","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":9474,"outputTokens":7,"latencyMs":1621.5995840000105},{"questionId":"q39","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":13071,"outputTokens":4,"latencyMs":1089.3430830000143},{"questionId":"q39","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8032,"outputTokens":4,"latencyMs":1120.2911670000176},{"questionId":"q39","format":"toon","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8349,"outputTokens":4,"latencyMs":1409.0022079999908},{"questionId":"q39","format":"xml","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":14576,"outputTokens":4,"latencyMs":1679.7288340000086},{"questionId":"q39","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":9474,"outputTokens":4,"latencyMs":1494.9637079999957},{"questionId":"q40","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":13071,"outputTokens":8,"latencyMs":1279.6186250000028},{"questionId":"q40","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8032,"outputTokens":8,"latencyMs":1300.0924999999988},{"questionId":"q40","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8349,"outputTokens":8,"latencyMs":1365.5876669999852},{"questionId":"q40","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":14576,"outputTokens":8,"latencyMs":1800.0563749999856},{"questionId":"q40","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":9474,"outputTokens":8,"latencyMs":1211.8039999999746},{"questionId":"q41","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":13071,"outputTokens":4,"latencyMs":1131.8700000000244},{"questionId":"q41","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8032,"outputTokens":4,"latencyMs":1012.6772080000082},{"questionId":"q41","format":"toon","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8349,"outputTokens":4,"latencyMs":1297.319542000012},{"questionId":"q41","format":"xml","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":14576,"outputTokens":4,"latencyMs":1182.1698329999927},{"questionId":"q41","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":9474,"outputTokens":4,"latencyMs":1186.7259579999954},{"questionId":"q42","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":13071,"outputTokens":8,"latencyMs":1212.6475830000127},{"questionId":"q42","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8032,"outputTokens":8,"latencyMs":1013.4181670000253},{"questionId":"q42","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8349,"outputTokens":8,"latencyMs":1341.400834},{"questionId":"q42","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":14576,"outputTokens":8,"latencyMs":1428.899792000011},{"questionId":"q42","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":9474,"outputTokens":8,"latencyMs":1146.0207080000255},{"questionId":"q43","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13071,"outputTokens":4,"latencyMs":1413.8295410000137},{"questionId":"q43","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8032,"outputTokens":4,"latencyMs":1371.2625839999819},{"questionId":"q43","format":"toon","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8349,"outputTokens":4,"latencyMs":1177.8167500000272},{"questionId":"q43","format":"xml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":14576,"outputTokens":4,"latencyMs":1314.6559169999964},{"questionId":"q43","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9474,"outputTokens":4,"latencyMs":1084.5133339999884},{"questionId":"q44","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":13072,"outputTokens":11,"latencyMs":1845.7391250000219},{"questionId":"q44","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8033,"outputTokens":11,"latencyMs":1215.7845419999794},{"questionId":"q44","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8350,"outputTokens":11,"latencyMs":1371.353415999969},{"questionId":"q44","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":14577,"outputTokens":11,"latencyMs":1405.688749999972},{"questionId":"q44","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":9475,"outputTokens":11,"latencyMs":1195.1462080000201},{"questionId":"q45","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":13072,"outputTokens":16,"latencyMs":1444.1725420000148},{"questionId":"q45","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8033,"outputTokens":16,"latencyMs":1851.9470420000143},{"questionId":"q45","format":"toon","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8350,"outputTokens":16,"latencyMs":1421.1135419999482},{"questionId":"q45","format":"xml","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":14577,"outputTokens":16,"latencyMs":2023.5860419999808},{"questionId":"q45","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":9475,"outputTokens":16,"latencyMs":1497.3693329999805},{"questionId":"q46","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":13072,"outputTokens":10,"latencyMs":1659.9631250000093},{"questionId":"q46","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8033,"outputTokens":10,"latencyMs":918.8577919999952},{"questionId":"q46","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8350,"outputTokens":10,"latencyMs":1379.7277499999618},{"questionId":"q46","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":14577,"outputTokens":10,"latencyMs":1363.199666999979},{"questionId":"q46","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":9475,"outputTokens":10,"latencyMs":1301.9265000000014},{"questionId":"q47","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":13071,"outputTokens":5,"latencyMs":1366.802708000003},{"questionId":"q47","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":8032,"outputTokens":5,"latencyMs":1218.0884169999626},{"questionId":"q47","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":8349,"outputTokens":5,"latencyMs":1183.3402910000295},{"questionId":"q47","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":14576,"outputTokens":5,"latencyMs":1063.356374999974},{"questionId":"q47","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":9474,"outputTokens":5,"latencyMs":1065.5065830000094},{"questionId":"q48","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":13072,"outputTokens":9,"latencyMs":1751.0218330000062},{"questionId":"q48","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8033,"outputTokens":9,"latencyMs":1108.9815420000232},{"questionId":"q48","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8350,"outputTokens":9,"latencyMs":1787.948665999982},{"questionId":"q48","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":14577,"outputTokens":9,"latencyMs":1295.9337500000256},{"questionId":"q48","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":9475,"outputTokens":9,"latencyMs":1736.0960830000113},{"questionId":"q49","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":13072,"outputTokens":13,"latencyMs":1199.4784590000054},{"questionId":"q49","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8033,"outputTokens":13,"latencyMs":1075.0446249999804},{"questionId":"q49","format":"toon","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8350,"outputTokens":13,"latencyMs":1981.9005830000388},{"questionId":"q49","format":"xml","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":14577,"outputTokens":13,"latencyMs":1281.1696249999804},{"questionId":"q49","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":9475,"outputTokens":13,"latencyMs":1370.431249999965},{"questionId":"q50","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":13072,"outputTokens":10,"latencyMs":1113.5716249999823},{"questionId":"q50","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":8033,"outputTokens":10,"latencyMs":1239.1278750000056},{"questionId":"q50","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":8350,"outputTokens":10,"latencyMs":1279.7286249999888},{"questionId":"q50","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":14577,"outputTokens":10,"latencyMs":1407.9125830000266},{"questionId":"q50","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":9475,"outputTokens":10,"latencyMs":1706.669125000015},{"questionId":"q51","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":13071,"outputTokens":5,"latencyMs":1270.9225829999777},{"questionId":"q51","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":8032,"outputTokens":5,"latencyMs":1130.0672090000007},{"questionId":"q51","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":8349,"outputTokens":5,"latencyMs":1345.5521249999874},{"questionId":"q51","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":14576,"outputTokens":5,"latencyMs":1226.1289170000236},{"questionId":"q51","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":9474,"outputTokens":5,"latencyMs":1119.2856669999892},{"questionId":"q52","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":13067,"outputTokens":5,"latencyMs":1196.8826250000275},{"questionId":"q52","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":8028,"outputTokens":5,"latencyMs":1126.873499999987},{"questionId":"q52","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":8345,"outputTokens":5,"latencyMs":980.0860000000102},{"questionId":"q52","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"9","isCorrect":false,"inputTokens":14572,"outputTokens":5,"latencyMs":1064.6709169999813},{"questionId":"q52","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"9","isCorrect":false,"inputTokens":9470,"outputTokens":5,"latencyMs":1166.8697909999755},{"questionId":"q53","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":13067,"outputTokens":5,"latencyMs":1113.0990410000086},{"questionId":"q53","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":8028,"outputTokens":5,"latencyMs":1303.7009589999798},{"questionId":"q53","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":8345,"outputTokens":5,"latencyMs":1238.1067499999772},{"questionId":"q53","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":14572,"outputTokens":5,"latencyMs":1333.5686250000144},{"questionId":"q53","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"6","isCorrect":false,"inputTokens":9470,"outputTokens":5,"latencyMs":1225.3654999999562},{"questionId":"q54","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"8","isCorrect":false,"inputTokens":13067,"outputTokens":5,"latencyMs":1399.3465410000063},{"questionId":"q54","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":8028,"outputTokens":5,"latencyMs":1113.1435410000267},{"questionId":"q54","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"6","isCorrect":false,"inputTokens":8345,"outputTokens":5,"latencyMs":1724.1194590000086},{"questionId":"q54","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":14572,"outputTokens":5,"latencyMs":1181.9311669999734},{"questionId":"q54","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"9","isCorrect":false,"inputTokens":9470,"outputTokens":5,"latencyMs":1049.9266670000507},{"questionId":"q55","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"42861.75","isCorrect":false,"inputTokens":13067,"outputTokens":8,"latencyMs":1439.9581250000047},{"questionId":"q55","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"40558.41","isCorrect":false,"inputTokens":8028,"outputTokens":8,"latencyMs":1148.0986250000424},{"questionId":"q55","format":"toon","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"47834.79","isCorrect":false,"inputTokens":8345,"outputTokens":8,"latencyMs":1271.2527080000145},{"questionId":"q55","format":"xml","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"38565.28","isCorrect":false,"inputTokens":14572,"outputTokens":8,"latencyMs":1461.959582999989},{"questionId":"q55","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"34904.81","actual":"41847.94","isCorrect":false,"inputTokens":9470,"outputTokens":8,"latencyMs":1480.8175409999676},{"questionId":"q56","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"756.89","isCorrect":false,"inputTokens":13065,"outputTokens":7,"latencyMs":1130.103833000001},{"questionId":"q56","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"822.94","isCorrect":false,"inputTokens":8026,"outputTokens":7,"latencyMs":1252.784875000012},{"questionId":"q56","format":"toon","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"791.84","isCorrect":false,"inputTokens":8343,"outputTokens":7,"latencyMs":1113.3430420000223},{"questionId":"q56","format":"xml","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"766.89","isCorrect":false,"inputTokens":14570,"outputTokens":7,"latencyMs":2177.307124999992},{"questionId":"q56","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"698.10","actual":"779.77","isCorrect":false,"inputTokens":9468,"outputTokens":7,"latencyMs":1105.3537500000093},{"questionId":"q57","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":13066,"outputTokens":5,"latencyMs":1392.8512499999488},{"questionId":"q57","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":8027,"outputTokens":5,"latencyMs":1126.0595420000027},{"questionId":"q57","format":"toon","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":8344,"outputTokens":5,"latencyMs":1194.2017089999863},{"questionId":"q57","format":"xml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":14571,"outputTokens":5,"latencyMs":1171.6571669999976},{"questionId":"q57","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":9469,"outputTokens":5,"latencyMs":1206.3512079999782},{"questionId":"q58","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":13065,"outputTokens":8,"latencyMs":1393.6799589999719},{"questionId":"q58","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8026,"outputTokens":8,"latencyMs":1519.2332499999902},{"questionId":"q58","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8343,"outputTokens":8,"latencyMs":2102.0287920000264},{"questionId":"q58","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":14570,"outputTokens":8,"latencyMs":1612.3956250000047},{"questionId":"q58","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":9468,"outputTokens":8,"latencyMs":1793.879041999986},{"questionId":"q59","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"43","actual":"49","isCorrect":false,"inputTokens":13069,"outputTokens":5,"latencyMs":1420.1437499999884},{"questionId":"q59","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"43","actual":"47","isCorrect":false,"inputTokens":8030,"outputTokens":5,"latencyMs":1007.740334000031},{"questionId":"q59","format":"toon","model":"claude-haiku-4-5-20251001","expected":"43","actual":"46","isCorrect":false,"inputTokens":8347,"outputTokens":5,"latencyMs":1120.1873340000166},{"questionId":"q59","format":"xml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"47","isCorrect":false,"inputTokens":14574,"outputTokens":5,"latencyMs":1570.4214159999974},{"questionId":"q59","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"49","isCorrect":false,"inputTokens":9472,"outputTokens":5,"latencyMs":1261.9378329999745},{"questionId":"q60","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"37","actual":"41","isCorrect":false,"inputTokens":13069,"outputTokens":5,"latencyMs":1163.1469999999972},{"questionId":"q60","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"37","actual":"35","isCorrect":false,"inputTokens":8030,"outputTokens":5,"latencyMs":1140.09945899999},{"questionId":"q60","format":"toon","model":"claude-haiku-4-5-20251001","expected":"37","actual":"38","isCorrect":false,"inputTokens":8347,"outputTokens":5,"latencyMs":1130.5380000000005},{"questionId":"q60","format":"xml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"38","isCorrect":false,"inputTokens":14574,"outputTokens":5,"latencyMs":1129.3633750000154},{"questionId":"q60","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"36","isCorrect":false,"inputTokens":9472,"outputTokens":5,"latencyMs":1117.113416999986},{"questionId":"q61","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"28","actual":"31","isCorrect":false,"inputTokens":13069,"outputTokens":5,"latencyMs":1169.0952919999836},{"questionId":"q61","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"28","actual":"34","isCorrect":false,"inputTokens":8030,"outputTokens":5,"latencyMs":996.319291999971},{"questionId":"q61","format":"toon","model":"claude-haiku-4-5-20251001","expected":"28","actual":"32","isCorrect":false,"inputTokens":8347,"outputTokens":5,"latencyMs":1430.8997499999823},{"questionId":"q61","format":"xml","model":"claude-haiku-4-5-20251001","expected":"28","actual":"31","isCorrect":false,"inputTokens":14574,"outputTokens":5,"latencyMs":1613.020166000002},{"questionId":"q61","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"28","actual":"34","isCorrect":false,"inputTokens":9472,"outputTokens":5,"latencyMs":1436.6957920000423},{"questionId":"q62","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":13073,"outputTokens":5,"latencyMs":1080.898833000043},{"questionId":"q62","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"8","actual":"7","isCorrect":false,"inputTokens":8034,"outputTokens":5,"latencyMs":1029.815249999985},{"questionId":"q62","format":"toon","model":"claude-haiku-4-5-20251001","expected":"8","actual":"7","isCorrect":false,"inputTokens":8351,"outputTokens":5,"latencyMs":1269.5112920000101},{"questionId":"q62","format":"xml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"7","isCorrect":false,"inputTokens":14578,"outputTokens":5,"latencyMs":2015.9492079999764},{"questionId":"q62","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":9476,"outputTokens":5,"latencyMs":1004.2100409999839},{"questionId":"q63","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6","actual":"8","isCorrect":false,"inputTokens":13073,"outputTokens":5,"latencyMs":1069.4494169999962},{"questionId":"q63","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":8034,"outputTokens":5,"latencyMs":1287.5518329999759},{"questionId":"q63","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":8351,"outputTokens":5,"latencyMs":1275.4342079999624},{"questionId":"q63","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":14578,"outputTokens":5,"latencyMs":1156.013666999992},{"questionId":"q63","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"7","isCorrect":false,"inputTokens":9476,"outputTokens":5,"latencyMs":1259.9707500000368},{"questionId":"q64","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":13073,"outputTokens":5,"latencyMs":1450.3827080000192},{"questionId":"q64","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10","actual":"6","isCorrect":false,"inputTokens":8034,"outputTokens":5,"latencyMs":1151.1852920000092},{"questionId":"q64","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10","actual":"6","isCorrect":false,"inputTokens":8351,"outputTokens":5,"latencyMs":1510.7578340000473},{"questionId":"q64","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"6","isCorrect":false,"inputTokens":14578,"outputTokens":5,"latencyMs":1430.823583999998},{"questionId":"q64","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10","actual":"7","isCorrect":false,"inputTokens":9476,"outputTokens":5,"latencyMs":1582.9399579999736},{"questionId":"q65","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":13073,"outputTokens":5,"latencyMs":1475.282125000027},{"questionId":"q65","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"9","actual":"7","isCorrect":false,"inputTokens":8034,"outputTokens":5,"latencyMs":1132.6659169999766},{"questionId":"q65","format":"toon","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":8351,"outputTokens":5,"latencyMs":1348.3977080000332},{"questionId":"q65","format":"xml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"7","isCorrect":false,"inputTokens":14578,"outputTokens":5,"latencyMs":1487.2882499999832},{"questionId":"q65","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":9476,"outputTokens":5,"latencyMs":1363.7185419999878},{"questionId":"q66","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"4","isCorrect":false,"inputTokens":13074,"outputTokens":5,"latencyMs":1189.3430410000146},{"questionId":"q66","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":8035,"outputTokens":5,"latencyMs":1351.1734999999753},{"questionId":"q66","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"4","isCorrect":false,"inputTokens":8352,"outputTokens":5,"latencyMs":1177.2700830000103},{"questionId":"q66","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":14579,"outputTokens":5,"latencyMs":1242.0754999999772},{"questionId":"q66","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"4","isCorrect":false,"inputTokens":9477,"outputTokens":5,"latencyMs":1276.0302499999525},{"questionId":"q67","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":13074,"outputTokens":5,"latencyMs":1344.6504580000183},{"questionId":"q67","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":8035,"outputTokens":5,"latencyMs":1176.157292000018},{"questionId":"q67","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":8352,"outputTokens":5,"latencyMs":20701.426457999973},{"questionId":"q67","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":14579,"outputTokens":5,"latencyMs":2367.8421249999665},{"questionId":"q67","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"6","isCorrect":false,"inputTokens":9477,"outputTokens":5,"latencyMs":1153.2113339999923},{"questionId":"q68","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":13074,"outputTokens":5,"latencyMs":1657.7776670000167},{"questionId":"q68","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"4","isCorrect":false,"inputTokens":8035,"outputTokens":5,"latencyMs":1173.5819999999949},{"questionId":"q68","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":8352,"outputTokens":5,"latencyMs":1232.1174580000225},{"questionId":"q68","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":14579,"outputTokens":5,"latencyMs":1349.014124999987},{"questionId":"q68","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":9477,"outputTokens":5,"latencyMs":1086.6965840000194},{"questionId":"q69","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"20","actual":"23","isCorrect":false,"inputTokens":13077,"outputTokens":5,"latencyMs":1153.7664579999982},{"questionId":"q69","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"20","actual":"18","isCorrect":false,"inputTokens":8038,"outputTokens":5,"latencyMs":1132.6050000000396},{"questionId":"q69","format":"toon","model":"claude-haiku-4-5-20251001","expected":"20","actual":"23","isCorrect":false,"inputTokens":8355,"outputTokens":5,"latencyMs":1340.9702089999919},{"questionId":"q69","format":"xml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"21","isCorrect":false,"inputTokens":14582,"outputTokens":5,"latencyMs":1389.9804579999764},{"questionId":"q69","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"23","isCorrect":false,"inputTokens":9480,"outputTokens":5,"latencyMs":1181.636083999998},{"questionId":"q70","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"19","actual":"15","isCorrect":false,"inputTokens":13077,"outputTokens":5,"latencyMs":1226.459166000015},{"questionId":"q70","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"19","actual":"23","isCorrect":false,"inputTokens":8038,"outputTokens":5,"latencyMs":1483.1681670000544},{"questionId":"q70","format":"toon","model":"claude-haiku-4-5-20251001","expected":"19","actual":"18","isCorrect":false,"inputTokens":8355,"outputTokens":5,"latencyMs":1059.183416999993},{"questionId":"q70","format":"xml","model":"claude-haiku-4-5-20251001","expected":"19","actual":"19","isCorrect":true,"inputTokens":14582,"outputTokens":5,"latencyMs":1345.0946669999976},{"questionId":"q70","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"19","actual":"24","isCorrect":false,"inputTokens":9480,"outputTokens":5,"latencyMs":1219.1354169999831},{"questionId":"q71","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4146,"outputTokens":6,"latencyMs":1277.316541999986},{"questionId":"q71","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2457,"outputTokens":6,"latencyMs":2851.0970419999794},{"questionId":"q71","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1603,"outputTokens":6,"latencyMs":1085.2489160000114},{"questionId":"q71","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1511,"outputTokens":6,"latencyMs":1086.2770000000019},{"questionId":"q71","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4848,"outputTokens":6,"latencyMs":1117.6914580000448},{"questionId":"q71","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3177,"outputTokens":6,"latencyMs":1230.9872919999762},{"questionId":"q72","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4146,"outputTokens":8,"latencyMs":1094.448000000033},{"questionId":"q72","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2457,"outputTokens":8,"latencyMs":1426.7915410000132},{"questionId":"q72","format":"toon","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1603,"outputTokens":8,"latencyMs":900.4749999999767},{"questionId":"q72","format":"csv","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1511,"outputTokens":8,"latencyMs":1126.0100419999799},{"questionId":"q72","format":"xml","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4848,"outputTokens":8,"latencyMs":1176.668249999988},{"questionId":"q72","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3177,"outputTokens":8,"latencyMs":999.8163330000243},{"questionId":"q73","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4147,"outputTokens":7,"latencyMs":1333.652624999988},{"questionId":"q73","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2458,"outputTokens":7,"latencyMs":1092.060541999992},{"questionId":"q73","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1604,"outputTokens":7,"latencyMs":1007.1116670000483},{"questionId":"q73","format":"csv","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1512,"outputTokens":7,"latencyMs":1013.337332999974},{"questionId":"q73","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4849,"outputTokens":7,"latencyMs":1650.107040999981},{"questionId":"q73","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3178,"outputTokens":7,"latencyMs":1439.3979579999577},{"questionId":"q74","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":4148,"outputTokens":5,"latencyMs":1417.6148329999996},{"questionId":"q74","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":2459,"outputTokens":5,"latencyMs":1242.6064170000027},{"questionId":"q74","format":"toon","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":1605,"outputTokens":5,"latencyMs":1100.431958000001},{"questionId":"q74","format":"csv","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":1513,"outputTokens":5,"latencyMs":986.0364159999881},{"questionId":"q74","format":"xml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":4850,"outputTokens":5,"latencyMs":1172.132042000012},{"questionId":"q74","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":3179,"outputTokens":5,"latencyMs":1304.2830830000457},{"questionId":"q75","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4146,"outputTokens":6,"latencyMs":1242.7824169999803},{"questionId":"q75","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2457,"outputTokens":6,"latencyMs":1163.7357910000137},{"questionId":"q75","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1603,"outputTokens":6,"latencyMs":1151.549875000026},{"questionId":"q75","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1511,"outputTokens":6,"latencyMs":1063.787249999994},{"questionId":"q75","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4848,"outputTokens":6,"latencyMs":1115.4328329999698},{"questionId":"q75","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3177,"outputTokens":6,"latencyMs":1137.960791999998},{"questionId":"q76","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4146,"outputTokens":8,"latencyMs":1201.6026249999995},{"questionId":"q76","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2457,"outputTokens":8,"latencyMs":1124.2640000000247},{"questionId":"q76","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1603,"outputTokens":8,"latencyMs":927.5135000000009},{"questionId":"q76","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1511,"outputTokens":8,"latencyMs":1195.1520000000019},{"questionId":"q76","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4848,"outputTokens":8,"latencyMs":1325.3894999999902},{"questionId":"q76","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3177,"outputTokens":8,"latencyMs":1116.2845840000082},{"questionId":"q77","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4147,"outputTokens":7,"latencyMs":1075.8688749999856},{"questionId":"q77","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2458,"outputTokens":7,"latencyMs":1041.189167000004},{"questionId":"q77","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1604,"outputTokens":7,"latencyMs":861.4979169999715},{"questionId":"q77","format":"csv","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1512,"outputTokens":7,"latencyMs":1134.7716669999645},{"questionId":"q77","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4849,"outputTokens":7,"latencyMs":1177.7597500000265},{"questionId":"q77","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3178,"outputTokens":7,"latencyMs":1119.7470000000321},{"questionId":"q78","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":4148,"outputTokens":5,"latencyMs":996.1894999999786},{"questionId":"q78","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":2459,"outputTokens":5,"latencyMs":1080.7052919999696},{"questionId":"q78","format":"toon","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":1605,"outputTokens":5,"latencyMs":935.0501249999506},{"questionId":"q78","format":"csv","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":1513,"outputTokens":5,"latencyMs":1056.5405419999734},{"questionId":"q78","format":"xml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":4850,"outputTokens":5,"latencyMs":1489.8931659999653},{"questionId":"q78","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"25","isCorrect":true,"inputTokens":3179,"outputTokens":5,"latencyMs":79487.06733300001},{"questionId":"q79","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4146,"outputTokens":6,"latencyMs":1313.9647910000058},{"questionId":"q79","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2457,"outputTokens":6,"latencyMs":1343.1313749999972},{"questionId":"q79","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1603,"outputTokens":6,"latencyMs":1334.5621249999967},{"questionId":"q79","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1511,"outputTokens":6,"latencyMs":1067.4220000000205},{"questionId":"q79","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4848,"outputTokens":6,"latencyMs":1150.077583999955},{"questionId":"q79","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3177,"outputTokens":6,"latencyMs":981.6363750000019},{"questionId":"q80","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":4143,"outputTokens":5,"latencyMs":1220.5045420000097},{"questionId":"q80","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":2454,"outputTokens":5,"latencyMs":1120.3278329999885},{"questionId":"q80","format":"toon","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":1600,"outputTokens":5,"latencyMs":1426.4953749999986},{"questionId":"q80","format":"csv","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":1508,"outputTokens":5,"latencyMs":941.414792000025},{"questionId":"q80","format":"xml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"59","isCorrect":false,"inputTokens":4845,"outputTokens":5,"latencyMs":1227.888500000001},{"questionId":"q80","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":3174,"outputTokens":5,"latencyMs":1216.2199170000385},{"questionId":"q81","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"367840","isCorrect":false,"inputTokens":4144,"outputTokens":6,"latencyMs":1537.3239579999936},{"questionId":"q81","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"340858","isCorrect":false,"inputTokens":2455,"outputTokens":6,"latencyMs":1150.2108749999898},{"questionId":"q81","format":"toon","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"326657","isCorrect":false,"inputTokens":1601,"outputTokens":6,"latencyMs":804.5737499999814},{"questionId":"q81","format":"csv","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"370775","isCorrect":false,"inputTokens":1509,"outputTokens":6,"latencyMs":840.2609170000069},{"questionId":"q81","format":"xml","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"341506","isCorrect":false,"inputTokens":4846,"outputTokens":6,"latencyMs":1314.695624999993},{"questionId":"q81","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"328320","actual":"176916","isCorrect":false,"inputTokens":3175,"outputTokens":6,"latencyMs":1555.4265829999931},{"questionId":"q82","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1434","isCorrect":false,"inputTokens":4145,"outputTokens":6,"latencyMs":1159.8954170000507},{"questionId":"q82","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1389","isCorrect":false,"inputTokens":2456,"outputTokens":6,"latencyMs":1061.4540839999681},{"questionId":"q82","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1577","isCorrect":false,"inputTokens":1602,"outputTokens":6,"latencyMs":1123.5626250000205},{"questionId":"q82","format":"csv","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1357","isCorrect":false,"inputTokens":1510,"outputTokens":6,"latencyMs":992.4902909999946},{"questionId":"q82","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1347","isCorrect":false,"inputTokens":4847,"outputTokens":6,"latencyMs":1159.3279999999795},{"questionId":"q82","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1791","actual":"1325","isCorrect":false,"inputTokens":3176,"outputTokens":6,"latencyMs":1181.7296659999993},{"questionId":"q83","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"343045.86","isCorrect":false,"inputTokens":4142,"outputTokens":8,"latencyMs":1219.903667000006},{"questionId":"q83","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"412256.91","isCorrect":false,"inputTokens":2453,"outputTokens":8,"latencyMs":1145.0830829999177},{"questionId":"q83","format":"toon","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"343945.11","isCorrect":false,"inputTokens":1599,"outputTokens":8,"latencyMs":1223.68987500004},{"questionId":"q83","format":"csv","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"390626.67","isCorrect":false,"inputTokens":1507,"outputTokens":8,"latencyMs":1040.1668749999953},{"questionId":"q83","format":"xml","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"382889.24","isCorrect":false,"inputTokens":4844,"outputTokens":8,"latencyMs":1409.154291999992},{"questionId":"q83","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"311695.88","actual":"356170.34","isCorrect":false,"inputTokens":3173,"outputTokens":8,"latencyMs":956.8994170000078},{"questionId":"q84","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4140,"outputTokens":7,"latencyMs":1256.6058330000378},{"questionId":"q84","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.537","isCorrect":false,"inputTokens":2451,"outputTokens":7,"latencyMs":1293.1294590000762},{"questionId":"q84","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.5263","isCorrect":true,"inputTokens":1597,"outputTokens":8,"latencyMs":854.5393750000512},{"questionId":"q84","format":"csv","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.5246","isCorrect":false,"inputTokens":1505,"outputTokens":8,"latencyMs":1119.7116249999963},{"questionId":"q84","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4842,"outputTokens":7,"latencyMs":1132.9079159999965},{"questionId":"q84","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3171,"outputTokens":7,"latencyMs":1521.2640829999},{"questionId":"q85","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"20","isCorrect":false,"inputTokens":4145,"outputTokens":5,"latencyMs":1171.5495420000516},{"questionId":"q85","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"24","isCorrect":false,"inputTokens":2456,"outputTokens":5,"latencyMs":1003.8035830000881},{"questionId":"q85","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"20","isCorrect":false,"inputTokens":1602,"outputTokens":5,"latencyMs":972.5325840000296},{"questionId":"q85","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"18","isCorrect":false,"inputTokens":1510,"outputTokens":5,"latencyMs":905.6272500000196},{"questionId":"q85","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"20","isCorrect":false,"inputTokens":4847,"outputTokens":5,"latencyMs":1082.7218340000836},{"questionId":"q85","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"20","isCorrect":false,"inputTokens":3176,"outputTokens":5,"latencyMs":997.8206250000512},{"questionId":"q86","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"42","actual":"25","isCorrect":false,"inputTokens":4145,"outputTokens":5,"latencyMs":1060.1338329999708},{"questionId":"q86","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"42","actual":"33","isCorrect":false,"inputTokens":2456,"outputTokens":5,"latencyMs":1412.0989999999292},{"questionId":"q86","format":"toon","model":"claude-haiku-4-5-20251001","expected":"42","actual":"42","isCorrect":true,"inputTokens":1602,"outputTokens":5,"latencyMs":980.2765409999993},{"questionId":"q86","format":"csv","model":"claude-haiku-4-5-20251001","expected":"42","actual":"41","isCorrect":false,"inputTokens":1510,"outputTokens":5,"latencyMs":943.2066659999546},{"questionId":"q86","format":"xml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"24","isCorrect":false,"inputTokens":4847,"outputTokens":5,"latencyMs":1598.437624999904},{"questionId":"q86","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"31","isCorrect":false,"inputTokens":3176,"outputTokens":5,"latencyMs":939.6132920000236},{"questionId":"q87","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"20","actual":"15","isCorrect":false,"inputTokens":4153,"outputTokens":5,"latencyMs":1164.4256670000032},{"questionId":"q87","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"20","actual":"18","isCorrect":false,"inputTokens":2464,"outputTokens":5,"latencyMs":1529.9007079999428},{"questionId":"q87","format":"toon","model":"claude-haiku-4-5-20251001","expected":"20","actual":"16","isCorrect":false,"inputTokens":1610,"outputTokens":5,"latencyMs":1043.3165000000736},{"questionId":"q87","format":"csv","model":"claude-haiku-4-5-20251001","expected":"20","actual":"17","isCorrect":false,"inputTokens":1518,"outputTokens":5,"latencyMs":1014.4737080000341},{"questionId":"q87","format":"xml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"15","isCorrect":false,"inputTokens":4855,"outputTokens":5,"latencyMs":1299.7567919999128},{"questionId":"q87","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"20","isCorrect":true,"inputTokens":3184,"outputTokens":5,"latencyMs":1036.0580000000773},{"questionId":"q88","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"14","actual":"11","isCorrect":false,"inputTokens":4153,"outputTokens":5,"latencyMs":1351.0695419999538},{"questionId":"q88","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"14","actual":"13","isCorrect":false,"inputTokens":2464,"outputTokens":5,"latencyMs":1451.123499999987},{"questionId":"q88","format":"toon","model":"claude-haiku-4-5-20251001","expected":"14","actual":"11","isCorrect":false,"inputTokens":1610,"outputTokens":5,"latencyMs":1011.8816250000382},{"questionId":"q88","format":"csv","model":"claude-haiku-4-5-20251001","expected":"14","actual":"9","isCorrect":false,"inputTokens":1518,"outputTokens":5,"latencyMs":1116.2810419999296},{"questionId":"q88","format":"xml","model":"claude-haiku-4-5-20251001","expected":"14","actual":"9","isCorrect":false,"inputTokens":4855,"outputTokens":5,"latencyMs":1202.6905839999672},{"questionId":"q88","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"14","actual":"10","isCorrect":false,"inputTokens":3184,"outputTokens":5,"latencyMs":1217.9919999999693},{"questionId":"q89","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":4151,"outputTokens":5,"latencyMs":1086.266249999986},{"questionId":"q89","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":2462,"outputTokens":5,"latencyMs":1425.2647910000524},{"questionId":"q89","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1608,"outputTokens":5,"latencyMs":1165.0251670000143},{"questionId":"q89","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":1516,"outputTokens":5,"latencyMs":1302.6017080000602},{"questionId":"q89","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":4853,"outputTokens":5,"latencyMs":1207.5639170000795},{"questionId":"q89","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"13","isCorrect":false,"inputTokens":3182,"outputTokens":5,"latencyMs":1003.2787090000929},{"questionId":"q90","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":4151,"outputTokens":5,"latencyMs":1314.3022080000956},{"questionId":"q90","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":2462,"outputTokens":5,"latencyMs":2278.123583000037},{"questionId":"q90","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"13","isCorrect":false,"inputTokens":1608,"outputTokens":5,"latencyMs":1040.3857919999864},{"questionId":"q90","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1516,"outputTokens":5,"latencyMs":1609.7861250001006},{"questionId":"q90","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":4853,"outputTokens":5,"latencyMs":1177.3617499999236},{"questionId":"q90","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"13","isCorrect":false,"inputTokens":3182,"outputTokens":5,"latencyMs":1123.888500000001},{"questionId":"q91","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"20","actual":"13","isCorrect":false,"inputTokens":4151,"outputTokens":5,"latencyMs":2513.7377080000006},{"questionId":"q91","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"20","actual":"16","isCorrect":false,"inputTokens":2462,"outputTokens":5,"latencyMs":1089.2179999999935},{"questionId":"q91","format":"toon","model":"claude-haiku-4-5-20251001","expected":"20","actual":"16","isCorrect":false,"inputTokens":1608,"outputTokens":5,"latencyMs":1062.1664580000797},{"questionId":"q91","format":"csv","model":"claude-haiku-4-5-20251001","expected":"20","actual":"18","isCorrect":false,"inputTokens":1516,"outputTokens":5,"latencyMs":1236.656958000036},{"questionId":"q91","format":"xml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"16","isCorrect":false,"inputTokens":4853,"outputTokens":5,"latencyMs":1146.2815420000115},{"questionId":"q91","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"20","actual":"14","isCorrect":false,"inputTokens":3182,"outputTokens":5,"latencyMs":1096.0875419999938},{"questionId":"q92","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"32","actual":"25","isCorrect":false,"inputTokens":4152,"outputTokens":5,"latencyMs":987.8946670000441},{"questionId":"q92","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"32","actual":"23","isCorrect":false,"inputTokens":2463,"outputTokens":5,"latencyMs":1220.6643329999642},{"questionId":"q92","format":"toon","model":"claude-haiku-4-5-20251001","expected":"32","actual":"32","isCorrect":true,"inputTokens":1609,"outputTokens":5,"latencyMs":937.5257920000004},{"questionId":"q92","format":"csv","model":"claude-haiku-4-5-20251001","expected":"32","actual":"24","isCorrect":false,"inputTokens":1517,"outputTokens":5,"latencyMs":2063.006832999992},{"questionId":"q92","format":"xml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"26","isCorrect":false,"inputTokens":4854,"outputTokens":5,"latencyMs":1468.0255830000388},{"questionId":"q92","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"32","actual":"24","isCorrect":false,"inputTokens":3183,"outputTokens":5,"latencyMs":966.7985420000041},{"questionId":"q93","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"9","actual":"11","isCorrect":false,"inputTokens":4152,"outputTokens":5,"latencyMs":1278.894708000007},{"questionId":"q93","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":2463,"outputTokens":5,"latencyMs":1107.823999999906},{"questionId":"q93","format":"toon","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":1609,"outputTokens":5,"latencyMs":907.0621670000255},{"questionId":"q93","format":"csv","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":1517,"outputTokens":5,"latencyMs":1304.2223330000415},{"questionId":"q93","format":"xml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"10","isCorrect":false,"inputTokens":4854,"outputTokens":5,"latencyMs":1006.5030419999966},{"questionId":"q93","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"9","actual":"8","isCorrect":false,"inputTokens":3183,"outputTokens":5,"latencyMs":1042.183209000039},{"questionId":"q94","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":4153,"outputTokens":5,"latencyMs":1480.0997500000522},{"questionId":"q94","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":2464,"outputTokens":5,"latencyMs":1601.9094999999506},{"questionId":"q94","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":1610,"outputTokens":5,"latencyMs":1208.4925000000512},{"questionId":"q94","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1518,"outputTokens":5,"latencyMs":1313.0127909999574},{"questionId":"q94","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"17","isCorrect":false,"inputTokens":4855,"outputTokens":5,"latencyMs":1156.5654589999467},{"questionId":"q94","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":3184,"outputTokens":5,"latencyMs":1046.6653750000987},{"questionId":"q95","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"22","actual":"15","isCorrect":false,"inputTokens":4153,"outputTokens":5,"latencyMs":1009.8270000000484},{"questionId":"q95","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":2464,"outputTokens":5,"latencyMs":1112.5791250000475},{"questionId":"q95","format":"toon","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":1610,"outputTokens":5,"latencyMs":1165.6955840000883},{"questionId":"q95","format":"csv","model":"claude-haiku-4-5-20251001","expected":"22","actual":"16","isCorrect":false,"inputTokens":1518,"outputTokens":5,"latencyMs":1050.5519169999752},{"questionId":"q95","format":"xml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":4855,"outputTokens":5,"latencyMs":1023.872166999965},{"questionId":"q95","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"22","actual":"14","isCorrect":false,"inputTokens":3184,"outputTokens":5,"latencyMs":1117.4546669999836},{"questionId":"q96","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17482,"outputTokens":6,"latencyMs":1361.8696670000209},{"questionId":"q96","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":12617,"outputTokens":6,"latencyMs":1247.9789170000004},{"questionId":"q96","format":"toon","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":9380,"outputTokens":6,"latencyMs":1211.6023749999003},{"questionId":"q96","format":"csv","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":9198,"outputTokens":6,"latencyMs":1307.9147920000833},{"questionId":"q96","format":"xml","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":19872,"outputTokens":6,"latencyMs":1437.3064170000143},{"questionId":"q96","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":14557,"outputTokens":6,"latencyMs":1455.2815420000115},{"questionId":"q97","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17481,"outputTokens":6,"latencyMs":1353.6013749999693},{"questionId":"q97","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":12616,"outputTokens":6,"latencyMs":1534.4137919999193},{"questionId":"q97","format":"toon","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":9379,"outputTokens":6,"latencyMs":2213.0383339999244},{"questionId":"q97","format":"csv","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":9197,"outputTokens":6,"latencyMs":1201.597165999934},{"questionId":"q97","format":"xml","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":19871,"outputTokens":6,"latencyMs":1513.969000000041},{"questionId":"q97","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":14556,"outputTokens":6,"latencyMs":1353.5847500000382},{"questionId":"q98","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17476,"outputTokens":6,"latencyMs":1397.7247079999652},{"questionId":"q98","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":12611,"outputTokens":6,"latencyMs":1268.1517080001067},{"questionId":"q98","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":9374,"outputTokens":6,"latencyMs":1237.637166999979},{"questionId":"q98","format":"csv","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":9192,"outputTokens":6,"latencyMs":1559.2043330000015},{"questionId":"q98","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":19866,"outputTokens":6,"latencyMs":1554.237124999985},{"questionId":"q98","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":14551,"outputTokens":6,"latencyMs":4180.1094579999335},{"questionId":"q99","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":17481,"outputTokens":4,"latencyMs":1550.9610000000102},{"questionId":"q99","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":12616,"outputTokens":4,"latencyMs":1546.9043330000713},{"questionId":"q99","format":"toon","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":9379,"outputTokens":4,"latencyMs":1625.3927080000285},{"questionId":"q99","format":"csv","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":9197,"outputTokens":4,"latencyMs":1535.6835000000428},{"questionId":"q99","format":"xml","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":19871,"outputTokens":4,"latencyMs":2670.4785830000183},{"questionId":"q99","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"master","actual":"master","isCorrect":true,"inputTokens":14556,"outputTokens":4,"latencyMs":1569.8371250000782},{"questionId":"q100","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17476,"outputTokens":6,"latencyMs":1596.1594999999506},{"questionId":"q100","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":12611,"outputTokens":6,"latencyMs":1386.254958000034},{"questionId":"q100","format":"toon","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":9374,"outputTokens":6,"latencyMs":1336.2282079999568},{"questionId":"q100","format":"csv","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":9192,"outputTokens":6,"latencyMs":1287.5360420000507},{"questionId":"q100","format":"xml","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":19866,"outputTokens":6,"latencyMs":1648.8853339999914},{"questionId":"q100","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":14551,"outputTokens":6,"latencyMs":1202.9672089999076},{"questionId":"q101","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17481,"outputTokens":6,"latencyMs":1287.3107910000253},{"questionId":"q101","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":12616,"outputTokens":6,"latencyMs":1320.3634160000365},{"questionId":"q101","format":"toon","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":9379,"outputTokens":6,"latencyMs":1191.4255419999827},{"questionId":"q101","format":"csv","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":9197,"outputTokens":6,"latencyMs":1206.390000000014},{"questionId":"q101","format":"xml","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":19871,"outputTokens":6,"latencyMs":1499.8067499999888},{"questionId":"q101","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":14556,"outputTokens":6,"latencyMs":1127.3515840000473},{"questionId":"q102","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":17483,"outputTokens":5,"latencyMs":1328.8301249999786},{"questionId":"q102","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":12618,"outputTokens":5,"latencyMs":1138.0201249999227},{"questionId":"q102","format":"toon","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":9381,"outputTokens":5,"latencyMs":980.3800830000546},{"questionId":"q102","format":"csv","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":9199,"outputTokens":5,"latencyMs":1157.8958750000456},{"questionId":"q102","format":"xml","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":19873,"outputTokens":5,"latencyMs":1360.5095420000143},{"questionId":"q102","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"678","actual":"678","isCorrect":true,"inputTokens":14558,"outputTokens":5,"latencyMs":1273.2398329999996},{"questionId":"q103","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":17477,"outputTokens":4,"latencyMs":1274.5509169999277},{"questionId":"q103","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":12612,"outputTokens":4,"latencyMs":1256.6803749999963},{"questionId":"q103","format":"toon","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":9375,"outputTokens":4,"latencyMs":1433.376416000072},{"questionId":"q103","format":"csv","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":9193,"outputTokens":4,"latencyMs":1417.8876250000903},{"questionId":"q103","format":"xml","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":19867,"outputTokens":4,"latencyMs":1639.6358750000363},{"questionId":"q103","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"main","actual":"main","isCorrect":true,"inputTokens":14552,"outputTokens":4,"latencyMs":1522.2459160001017},{"questionId":"q104","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17484,"outputTokens":6,"latencyMs":1278.353458999889},{"questionId":"q104","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":12619,"outputTokens":6,"latencyMs":1228.4452499999898},{"questionId":"q104","format":"toon","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":9382,"outputTokens":6,"latencyMs":1285.1270830000285},{"questionId":"q104","format":"csv","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":9200,"outputTokens":6,"latencyMs":1185.8712910000468},{"questionId":"q104","format":"xml","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":19874,"outputTokens":6,"latencyMs":1733.4342500000494},{"questionId":"q104","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":14559,"outputTokens":6,"latencyMs":1182.3504170000087},{"questionId":"q105","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17480,"outputTokens":6,"latencyMs":1328.0590000000084},{"questionId":"q105","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":12615,"outputTokens":6,"latencyMs":1337.2227919999277},{"questionId":"q105","format":"toon","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":9378,"outputTokens":6,"latencyMs":1245.6462499999907},{"questionId":"q105","format":"csv","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":9196,"outputTokens":6,"latencyMs":1152.3198330000741},{"questionId":"q105","format":"xml","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":19870,"outputTokens":6,"latencyMs":1417.1319579998963},{"questionId":"q105","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":14555,"outputTokens":6,"latencyMs":1587.597666000016},{"questionId":"q106","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17485,"outputTokens":6,"latencyMs":1286.3247500000289},{"questionId":"q106","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":12620,"outputTokens":6,"latencyMs":1243.2615000000224},{"questionId":"q106","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":9383,"outputTokens":6,"latencyMs":1291.9809159999713},{"questionId":"q106","format":"csv","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":9201,"outputTokens":6,"latencyMs":1398.1902080000145},{"questionId":"q106","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":19875,"outputTokens":6,"latencyMs":1624.34620800009},{"questionId":"q106","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":14560,"outputTokens":6,"latencyMs":1721.1688750000903},{"questionId":"q107","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":17473,"outputTokens":5,"latencyMs":1369.6887080000015},{"questionId":"q107","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":12608,"outputTokens":5,"latencyMs":1290.1797500000102},{"questionId":"q107","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9371,"outputTokens":5,"latencyMs":1811.6780829998897},{"questionId":"q107","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9189,"outputTokens":5,"latencyMs":1179.5881659999723},{"questionId":"q107","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":19863,"outputTokens":5,"latencyMs":2173.616832999978},{"questionId":"q107","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":14548,"outputTokens":5,"latencyMs":1352.0613330000779},{"questionId":"q108","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"11475937","isCorrect":false,"inputTokens":17476,"outputTokens":7,"latencyMs":2116.313165999949},{"questionId":"q108","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"13947885","isCorrect":false,"inputTokens":12611,"outputTokens":7,"latencyMs":1385.7955830000574},{"questionId":"q108","format":"toon","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"17348748","isCorrect":false,"inputTokens":9374,"outputTokens":7,"latencyMs":1142.456750000012},{"questionId":"q108","format":"csv","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"10737833","isCorrect":false,"inputTokens":9192,"outputTokens":7,"latencyMs":1214.3351249999832},{"questionId":"q108","format":"xml","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"10524881","isCorrect":false,"inputTokens":19866,"outputTokens":7,"latencyMs":1302.2744999999413},{"questionId":"q108","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"15413563","actual":"10959626","isCorrect":false,"inputTokens":14551,"outputTokens":7,"latencyMs":1313.938542000018},{"questionId":"q109","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1327621","isCorrect":false,"inputTokens":17477,"outputTokens":7,"latencyMs":1437.121167000034},{"questionId":"q109","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1453895","isCorrect":false,"inputTokens":12612,"outputTokens":7,"latencyMs":1452.4355000000214},{"questionId":"q109","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1429822","isCorrect":false,"inputTokens":9375,"outputTokens":7,"latencyMs":1270.1647080000257},{"questionId":"q109","format":"csv","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1359722","isCorrect":false,"inputTokens":9193,"outputTokens":7,"latencyMs":1112.9368749998976},{"questionId":"q109","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1264743","isCorrect":false,"inputTokens":19867,"outputTokens":7,"latencyMs":1466.141500000027},{"questionId":"q109","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2528243","actual":"1434047","isCorrect":false,"inputTokens":14552,"outputTokens":7,"latencyMs":1131.1252089999616},{"questionId":"q110","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"130896","isCorrect":false,"inputTokens":17475,"outputTokens":6,"latencyMs":1628.9582499999087},{"questionId":"q110","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"137824","isCorrect":false,"inputTokens":12610,"outputTokens":6,"latencyMs":1084.4367499999935},{"questionId":"q110","format":"toon","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"130658","isCorrect":false,"inputTokens":9373,"outputTokens":6,"latencyMs":1364.9022910000058},{"questionId":"q110","format":"csv","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"130686.16","isCorrect":false,"inputTokens":9191,"outputTokens":8,"latencyMs":1165.2718750000931},{"questionId":"q110","format":"xml","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"132689","isCorrect":false,"inputTokens":19865,"outputTokens":6,"latencyMs":1335.3207089999923},{"questionId":"q110","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"154136","actual":"132673","isCorrect":false,"inputTokens":14550,"outputTokens":6,"latencyMs":1393.0692500000587},{"questionId":"q111","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":17477,"outputTokens":5,"latencyMs":2100.8017080000136},{"questionId":"q111","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"41","actual":"25","isCorrect":false,"inputTokens":12612,"outputTokens":5,"latencyMs":1400.2827919999836},{"questionId":"q111","format":"toon","model":"claude-haiku-4-5-20251001","expected":"41","actual":"21","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1358.0510830000276},{"questionId":"q111","format":"csv","model":"claude-haiku-4-5-20251001","expected":"41","actual":"27","isCorrect":false,"inputTokens":9193,"outputTokens":5,"latencyMs":1191.0352500000736},{"questionId":"q111","format":"xml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"26","isCorrect":false,"inputTokens":19867,"outputTokens":5,"latencyMs":3264.8267500000075},{"questionId":"q111","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"41","actual":"27","isCorrect":false,"inputTokens":14552,"outputTokens":5,"latencyMs":1273.1958330000052},{"questionId":"q112","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"53","actual":"56","isCorrect":false,"inputTokens":17477,"outputTokens":5,"latencyMs":1519.110915999976},{"questionId":"q112","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"53","actual":"56","isCorrect":false,"inputTokens":12612,"outputTokens":5,"latencyMs":1058.4677910000319},{"questionId":"q112","format":"toon","model":"claude-haiku-4-5-20251001","expected":"53","actual":"57","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1207.4402080000145},{"questionId":"q112","format":"csv","model":"claude-haiku-4-5-20251001","expected":"53","actual":"53","isCorrect":true,"inputTokens":9193,"outputTokens":5,"latencyMs":1156.1299169999547},{"questionId":"q112","format":"xml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"57","isCorrect":false,"inputTokens":19867,"outputTokens":5,"latencyMs":1534.4429999999702},{"questionId":"q112","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"53","actual":"46","isCorrect":false,"inputTokens":14552,"outputTokens":5,"latencyMs":1102.8952910000226},{"questionId":"q113","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":17477,"outputTokens":5,"latencyMs":1363.827082999982},{"questionId":"q113","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":12612,"outputTokens":5,"latencyMs":1098.9746250000317},{"questionId":"q113","format":"toon","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1091.201000000001},{"questionId":"q113","format":"csv","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":9193,"outputTokens":5,"latencyMs":985.0388750000857},{"questionId":"q113","format":"xml","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":19867,"outputTokens":5,"latencyMs":1350.483332999982},{"questionId":"q113","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"77","actual":"100","isCorrect":false,"inputTokens":14552,"outputTokens":5,"latencyMs":1329.111082999967},{"questionId":"q114","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"37","actual":"41","isCorrect":false,"inputTokens":17477,"outputTokens":5,"latencyMs":1212.424457999994},{"questionId":"q114","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"37","actual":"50","isCorrect":false,"inputTokens":12612,"outputTokens":5,"latencyMs":1182.8672079999233},{"questionId":"q114","format":"toon","model":"claude-haiku-4-5-20251001","expected":"37","actual":"50","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1758.9869160000235},{"questionId":"q114","format":"csv","model":"claude-haiku-4-5-20251001","expected":"37","actual":"45","isCorrect":false,"inputTokens":9193,"outputTokens":5,"latencyMs":1223.081125000026},{"questionId":"q114","format":"xml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"50","isCorrect":false,"inputTokens":19867,"outputTokens":5,"latencyMs":1448.7562919999473},{"questionId":"q114","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"50","isCorrect":false,"inputTokens":14552,"outputTokens":5,"latencyMs":1141.3254169999855},{"questionId":"q115","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"16","actual":"5","isCorrect":false,"inputTokens":17477,"outputTokens":5,"latencyMs":1641.206375000067},{"questionId":"q115","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"16","actual":"31","isCorrect":false,"inputTokens":12612,"outputTokens":5,"latencyMs":1152.969541999977},{"questionId":"q115","format":"toon","model":"claude-haiku-4-5-20251001","expected":"16","actual":"15","isCorrect":false,"inputTokens":9375,"outputTokens":5,"latencyMs":1277.7170410000253},{"questionId":"q115","format":"csv","model":"claude-haiku-4-5-20251001","expected":"16","actual":"21","isCorrect":false,"inputTokens":9193,"outputTokens":5,"latencyMs":1170.6205830000108},{"questionId":"q115","format":"xml","model":"claude-haiku-4-5-20251001","expected":"16","actual":"3","isCorrect":false,"inputTokens":19867,"outputTokens":5,"latencyMs":1850.4669170000125},{"questionId":"q115","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"16","actual":"3","isCorrect":false,"inputTokens":14552,"outputTokens":5,"latencyMs":1451.0770000000484},{"questionId":"q116","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"49","actual":"42","isCorrect":false,"inputTokens":17478,"outputTokens":5,"latencyMs":1355.7432499999413},{"questionId":"q116","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"49","actual":"56","isCorrect":false,"inputTokens":12613,"outputTokens":5,"latencyMs":1275.323333999957},{"questionId":"q116","format":"toon","model":"claude-haiku-4-5-20251001","expected":"49","actual":"47","isCorrect":false,"inputTokens":9376,"outputTokens":5,"latencyMs":1957.939083000063},{"questionId":"q116","format":"csv","model":"claude-haiku-4-5-20251001","expected":"49","actual":"29","isCorrect":false,"inputTokens":9194,"outputTokens":5,"latencyMs":1257.4775420000078},{"questionId":"q116","format":"xml","model":"claude-haiku-4-5-20251001","expected":"49","actual":"25","isCorrect":false,"inputTokens":19868,"outputTokens":5,"latencyMs":1747.7625409999164},{"questionId":"q116","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"49","actual":"29","isCorrect":false,"inputTokens":14553,"outputTokens":5,"latencyMs":1300.0107079999289},{"questionId":"q117","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"23","actual":"26","isCorrect":false,"inputTokens":17478,"outputTokens":5,"latencyMs":1249.3837920000078},{"questionId":"q117","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"23","actual":"20","isCorrect":false,"inputTokens":12613,"outputTokens":5,"latencyMs":1169.6777500000317},{"questionId":"q117","format":"toon","model":"claude-haiku-4-5-20251001","expected":"23","actual":"22","isCorrect":false,"inputTokens":9376,"outputTokens":5,"latencyMs":1138.845290999976},{"questionId":"q117","format":"csv","model":"claude-haiku-4-5-20251001","expected":"23","actual":"22","isCorrect":false,"inputTokens":9194,"outputTokens":5,"latencyMs":1190.8722499998985},{"questionId":"q117","format":"xml","model":"claude-haiku-4-5-20251001","expected":"23","actual":"17","isCorrect":false,"inputTokens":19868,"outputTokens":5,"latencyMs":1297.84612500004},{"questionId":"q117","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"23","actual":"14","isCorrect":false,"inputTokens":14553,"outputTokens":5,"latencyMs":1273.5267079999903},{"questionId":"q118","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"10","isCorrect":false,"inputTokens":17478,"outputTokens":5,"latencyMs":1501.3266669999575},{"questionId":"q118","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"12","isCorrect":false,"inputTokens":12613,"outputTokens":5,"latencyMs":1335.36412500008},{"questionId":"q118","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"15","isCorrect":false,"inputTokens":9376,"outputTokens":5,"latencyMs":1168.816125000012},{"questionId":"q118","format":"csv","model":"claude-haiku-4-5-20251001","expected":"4","actual":"31","isCorrect":false,"inputTokens":9194,"outputTokens":5,"latencyMs":1292.2094170000637},{"questionId":"q118","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"16","isCorrect":false,"inputTokens":19868,"outputTokens":5,"latencyMs":1603.7261660000077},{"questionId":"q118","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"10","isCorrect":false,"inputTokens":14553,"outputTokens":5,"latencyMs":1265.9923339999514},{"questionId":"q119","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"57","actual":"8","isCorrect":false,"inputTokens":17486,"outputTokens":5,"latencyMs":1319.7199580000015},{"questionId":"q119","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"57","actual":"23","isCorrect":false,"inputTokens":12621,"outputTokens":5,"latencyMs":1675.4872079999186},{"questionId":"q119","format":"toon","model":"claude-haiku-4-5-20251001","expected":"57","actual":"12","isCorrect":false,"inputTokens":9384,"outputTokens":5,"latencyMs":1093.7844170000171},{"questionId":"q119","format":"csv","model":"claude-haiku-4-5-20251001","expected":"57","actual":"15","isCorrect":false,"inputTokens":9202,"outputTokens":5,"latencyMs":1534.8674590000883},{"questionId":"q119","format":"xml","model":"claude-haiku-4-5-20251001","expected":"57","actual":"15","isCorrect":false,"inputTokens":19876,"outputTokens":5,"latencyMs":1421.0654590000631},{"questionId":"q119","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"57","actual":"15","isCorrect":false,"inputTokens":14561,"outputTokens":5,"latencyMs":1133.124291999964},{"questionId":"q120","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"43","actual":"38","isCorrect":false,"inputTokens":17486,"outputTokens":5,"latencyMs":3100.8175000000047},{"questionId":"q120","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"43","actual":"37","isCorrect":false,"inputTokens":12621,"outputTokens":5,"latencyMs":1586.5317499999655},{"questionId":"q120","format":"toon","model":"claude-haiku-4-5-20251001","expected":"43","actual":"37","isCorrect":false,"inputTokens":9384,"outputTokens":5,"latencyMs":1087.2246670000022},{"questionId":"q120","format":"csv","model":"claude-haiku-4-5-20251001","expected":"43","actual":"32","isCorrect":false,"inputTokens":9202,"outputTokens":5,"latencyMs":1252.717082999996},{"questionId":"q120","format":"xml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"21","isCorrect":false,"inputTokens":19876,"outputTokens":5,"latencyMs":1433.415833999985},{"questionId":"q120","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"43","actual":"41","isCorrect":false,"inputTokens":14561,"outputTokens":5,"latencyMs":1497.922416999936},{"questionId":"q121","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"25","actual":"13","isCorrect":false,"inputTokens":17486,"outputTokens":5,"latencyMs":1229.641583000077},{"questionId":"q121","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"25","actual":"15","isCorrect":false,"inputTokens":12621,"outputTokens":5,"latencyMs":1479.924708000035},{"questionId":"q121","format":"toon","model":"claude-haiku-4-5-20251001","expected":"25","actual":"22","isCorrect":false,"inputTokens":9384,"outputTokens":5,"latencyMs":1331.8733749999665},{"questionId":"q121","format":"csv","model":"claude-haiku-4-5-20251001","expected":"25","actual":"18","isCorrect":false,"inputTokens":9202,"outputTokens":5,"latencyMs":1499.8951249999227},{"questionId":"q121","format":"xml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"12","isCorrect":false,"inputTokens":19876,"outputTokens":5,"latencyMs":1506.3811669999268},{"questionId":"q121","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"10","isCorrect":false,"inputTokens":14561,"outputTokens":5,"latencyMs":1207.6717090000166},{"questionId":"q122","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6","actual":"8","isCorrect":false,"inputTokens":17486,"outputTokens":5,"latencyMs":1919.7641669999575},{"questionId":"q122","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6","actual":"18","isCorrect":false,"inputTokens":12621,"outputTokens":5,"latencyMs":2294.538125000079},{"questionId":"q122","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6","actual":"15","isCorrect":false,"inputTokens":9384,"outputTokens":5,"latencyMs":1619.9265840000007},{"questionId":"q122","format":"csv","model":"claude-haiku-4-5-20251001","expected":"6","actual":"8","isCorrect":false,"inputTokens":9202,"outputTokens":5,"latencyMs":2120.3911249999655},{"questionId":"q122","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"17","isCorrect":false,"inputTokens":19876,"outputTokens":5,"latencyMs":1503.0869999999413},{"questionId":"q122","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"13","isCorrect":false,"inputTokens":14561,"outputTokens":5,"latencyMs":1234.3564580000238},{"questionId":"q123","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"8","isCorrect":false,"inputTokens":17486,"outputTokens":5,"latencyMs":1434.706542000058},{"questionId":"q123","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"3","isCorrect":false,"inputTokens":12621,"outputTokens":5,"latencyMs":1561.3982910000486},{"questionId":"q123","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"12","isCorrect":false,"inputTokens":9384,"outputTokens":5,"latencyMs":1187.2939999999944},{"questionId":"q123","format":"csv","model":"claude-haiku-4-5-20251001","expected":"1","actual":"4","isCorrect":false,"inputTokens":9202,"outputTokens":5,"latencyMs":1084.5471249999246},{"questionId":"q123","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"3","isCorrect":false,"inputTokens":19876,"outputTokens":5,"latencyMs":1870.1685000000289},{"questionId":"q123","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"4","isCorrect":false,"inputTokens":14561,"outputTokens":5,"latencyMs":1385.7719580000266},{"questionId":"q124","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":7783,"outputTokens":4,"latencyMs":1268.9927909999387},{"questionId":"q124","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":5221,"outputTokens":4,"latencyMs":1421.6205409999238},{"questionId":"q124","format":"toon","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6241,"outputTokens":4,"latencyMs":1310.4610000000102},{"questionId":"q124","format":"xml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":8788,"outputTokens":4,"latencyMs":1550.1076250000624},{"questionId":"q124","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6187,"outputTokens":4,"latencyMs":1150.0919170000125},{"questionId":"q125","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7783,"outputTokens":7,"latencyMs":1214.324041999993},{"questionId":"q125","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5221,"outputTokens":7,"latencyMs":1421.8747919999296},{"questionId":"q125","format":"toon","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6241,"outputTokens":7,"latencyMs":1067.4452499999898},{"questionId":"q125","format":"xml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8788,"outputTokens":7,"latencyMs":1218.4350420000264},{"questionId":"q125","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6187,"outputTokens":7,"latencyMs":1223.642499999958},{"questionId":"q126","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":7784,"outputTokens":5,"latencyMs":1130.6049170000479},{"questionId":"q126","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":5222,"outputTokens":5,"latencyMs":1027.4643330000108},{"questionId":"q126","format":"toon","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":6242,"outputTokens":5,"latencyMs":1244.4015420000069},{"questionId":"q126","format":"xml","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":8789,"outputTokens":5,"latencyMs":1162.2130830000388},{"questionId":"q126","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"424","actual":"424","isCorrect":true,"inputTokens":6188,"outputTokens":5,"latencyMs":1949.5525829999242},{"questionId":"q127","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7784,"outputTokens":6,"latencyMs":999.1413750000065},{"questionId":"q127","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5222,"outputTokens":6,"latencyMs":1027.876125000068},{"questionId":"q127","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6242,"outputTokens":6,"latencyMs":1985.8047080000397},{"questionId":"q127","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":8789,"outputTokens":6,"latencyMs":1117.451000000001},{"questionId":"q127","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6188,"outputTokens":6,"latencyMs":1347.6159579999512},{"questionId":"q128","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":7783,"outputTokens":4,"latencyMs":1317.608249999932},{"questionId":"q128","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":5221,"outputTokens":4,"latencyMs":1256.9298749999143},{"questionId":"q128","format":"toon","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6241,"outputTokens":4,"latencyMs":1083.470083000022},{"questionId":"q128","format":"xml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":8788,"outputTokens":4,"latencyMs":1015.1183330001077},{"questionId":"q128","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6187,"outputTokens":4,"latencyMs":955.9129999999423},{"questionId":"q129","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7783,"outputTokens":7,"latencyMs":1433.7668750000885},{"questionId":"q129","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5221,"outputTokens":7,"latencyMs":1191.4290000000037},{"questionId":"q129","format":"toon","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6241,"outputTokens":7,"latencyMs":1086.0024169998942},{"questionId":"q129","format":"xml","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":8788,"outputTokens":7,"latencyMs":1344.1925419999752},{"questionId":"q129","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6187,"outputTokens":7,"latencyMs":1327.7757920000004},{"questionId":"q130","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":7784,"outputTokens":5,"latencyMs":975.5557089999784},{"questionId":"q130","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":5222,"outputTokens":5,"latencyMs":1581.8543330000248},{"questionId":"q130","format":"toon","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":6242,"outputTokens":5,"latencyMs":1049.4542499999516},{"questionId":"q130","format":"xml","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":8789,"outputTokens":5,"latencyMs":1078.1500829999568},{"questionId":"q130","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"435","actual":"435","isCorrect":true,"inputTokens":6188,"outputTokens":5,"latencyMs":1060.7320830000099},{"questionId":"q131","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":7784,"outputTokens":5,"latencyMs":1025.7722079999512},{"questionId":"q131","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":5222,"outputTokens":5,"latencyMs":1314.0469169999706},{"questionId":"q131","format":"toon","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":6242,"outputTokens":5,"latencyMs":1069.5918330000713},{"questionId":"q131","format":"xml","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":8789,"outputTokens":5,"latencyMs":1142.9486249999609},{"questionId":"q131","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"408","actual":"408","isCorrect":true,"inputTokens":6188,"outputTokens":5,"latencyMs":992.1041250000708},{"questionId":"q132","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":7783,"outputTokens":4,"latencyMs":1161.393875000067},{"questionId":"q132","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":5221,"outputTokens":4,"latencyMs":939.8532920000143},{"questionId":"q132","format":"toon","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6241,"outputTokens":4,"latencyMs":1038.753625000012},{"questionId":"q132","format":"xml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":8788,"outputTokens":4,"latencyMs":1365.075458999956},{"questionId":"q132","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"error","actual":"error","isCorrect":true,"inputTokens":6187,"outputTokens":4,"latencyMs":1184.1347500000848},{"questionId":"q133","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7783,"outputTokens":7,"latencyMs":1443.2086669999408},{"questionId":"q133","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5221,"outputTokens":7,"latencyMs":1285.2082500000251},{"questionId":"q133","format":"toon","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6241,"outputTokens":7,"latencyMs":1422.8267079999205},{"questionId":"q133","format":"xml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8788,"outputTokens":7,"latencyMs":1320.150208999985},{"questionId":"q133","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6187,"outputTokens":7,"latencyMs":1196.3562910000328},{"questionId":"q134","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":7767,"outputTokens":5,"latencyMs":1288.936875000014},{"questionId":"q134","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"75","actual":"70","isCorrect":false,"inputTokens":5205,"outputTokens":5,"latencyMs":1696.469999999972},{"questionId":"q134","format":"toon","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":6225,"outputTokens":5,"latencyMs":1488.3868329999968},{"questionId":"q134","format":"xml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"100","isCorrect":false,"inputTokens":8772,"outputTokens":5,"latencyMs":1240.0713750000577},{"questionId":"q134","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":6171,"outputTokens":5,"latencyMs":1378.1943749999627},{"questionId":"q135","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2384.74","isCorrect":false,"inputTokens":7768,"outputTokens":8,"latencyMs":1541.0560830000322},{"questionId":"q135","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2458.48","isCorrect":false,"inputTokens":5206,"outputTokens":8,"latencyMs":1120.7529169999762},{"questionId":"q135","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2405.09","isCorrect":false,"inputTokens":6226,"outputTokens":8,"latencyMs":1157.566583000007},{"questionId":"q135","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2445.96","isCorrect":false,"inputTokens":8773,"outputTokens":8,"latencyMs":1159.1351250000298},{"questionId":"q135","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2453.41","actual":"2414.48","isCorrect":false,"inputTokens":6172,"outputTokens":8,"latencyMs":1258.330958000035},{"questionId":"q136","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"29","actual":"33","isCorrect":false,"inputTokens":7768,"outputTokens":5,"latencyMs":1139.8584170000395},{"questionId":"q136","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"29","actual":"32","isCorrect":false,"inputTokens":5206,"outputTokens":5,"latencyMs":1128.9782079999568},{"questionId":"q136","format":"toon","model":"claude-haiku-4-5-20251001","expected":"29","actual":"30","isCorrect":false,"inputTokens":6226,"outputTokens":5,"latencyMs":1076.437042000005},{"questionId":"q136","format":"xml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"31","isCorrect":false,"inputTokens":8773,"outputTokens":5,"latencyMs":1214.0489590000361},{"questionId":"q136","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"26","isCorrect":false,"inputTokens":6172,"outputTokens":5,"latencyMs":1147.1827920000069},{"questionId":"q137","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":7768,"outputTokens":5,"latencyMs":1060.8192499999423},{"questionId":"q137","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":5206,"outputTokens":5,"latencyMs":1000.7439579999773},{"questionId":"q137","format":"toon","model":"claude-haiku-4-5-20251001","expected":"17","actual":"15","isCorrect":false,"inputTokens":6226,"outputTokens":5,"latencyMs":1444.9082920000656},{"questionId":"q137","format":"xml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":8773,"outputTokens":5,"latencyMs":1364.0987090000417},{"questionId":"q137","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"17","actual":"16","isCorrect":false,"inputTokens":6172,"outputTokens":5,"latencyMs":1289.65149999992},{"questionId":"q138","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"29","actual":"24","isCorrect":false,"inputTokens":7768,"outputTokens":5,"latencyMs":1276.8952499999432},{"questionId":"q138","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"29","actual":"24","isCorrect":false,"inputTokens":5206,"outputTokens":5,"latencyMs":1270.1233340000035},{"questionId":"q138","format":"toon","model":"claude-haiku-4-5-20251001","expected":"29","actual":"24","isCorrect":false,"inputTokens":6226,"outputTokens":5,"latencyMs":1226.6909589999123},{"questionId":"q138","format":"xml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"25","isCorrect":false,"inputTokens":8773,"outputTokens":5,"latencyMs":1133.3242500000633},{"questionId":"q138","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"28","isCorrect":false,"inputTokens":6172,"outputTokens":5,"latencyMs":2179.776416000095},{"questionId":"q139","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":7771,"outputTokens":5,"latencyMs":1117.800791000016},{"questionId":"q139","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":5209,"outputTokens":5,"latencyMs":1069.6763750000391},{"questionId":"q139","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":6229,"outputTokens":5,"latencyMs":2063.5446249999804},{"questionId":"q139","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"12","isCorrect":false,"inputTokens":8776,"outputTokens":5,"latencyMs":1202.8583749999525},{"questionId":"q139","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"9","isCorrect":false,"inputTokens":6175,"outputTokens":5,"latencyMs":1061.2812919999706},{"questionId":"q140","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"18","actual":"15","isCorrect":false,"inputTokens":7771,"outputTokens":5,"latencyMs":1386.753832999966},{"questionId":"q140","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"18","actual":"13","isCorrect":false,"inputTokens":5209,"outputTokens":5,"latencyMs":1034.8489169999957},{"questionId":"q140","format":"toon","model":"claude-haiku-4-5-20251001","expected":"18","actual":"15","isCorrect":false,"inputTokens":6229,"outputTokens":5,"latencyMs":1134.6799170000013},{"questionId":"q140","format":"xml","model":"claude-haiku-4-5-20251001","expected":"18","actual":"20","isCorrect":false,"inputTokens":8776,"outputTokens":5,"latencyMs":1083.533999999985},{"questionId":"q140","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"18","actual":"15","isCorrect":false,"inputTokens":6175,"outputTokens":5,"latencyMs":1075.4867920000106},{"questionId":"q141","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"33","actual":"33","isCorrect":true,"inputTokens":7775,"outputTokens":5,"latencyMs":1596.7704580000136},{"questionId":"q141","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"33","actual":"33","isCorrect":true,"inputTokens":5213,"outputTokens":5,"latencyMs":1052.1449169999687},{"questionId":"q141","format":"toon","model":"claude-haiku-4-5-20251001","expected":"33","actual":"40","isCorrect":false,"inputTokens":6233,"outputTokens":5,"latencyMs":1162.8800829999382},{"questionId":"q141","format":"xml","model":"claude-haiku-4-5-20251001","expected":"33","actual":"37","isCorrect":false,"inputTokens":8780,"outputTokens":5,"latencyMs":1121.927708000061},{"questionId":"q141","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"33","actual":"35","isCorrect":false,"inputTokens":6179,"outputTokens":5,"latencyMs":1078.549040999962},{"questionId":"q142","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"42","actual":"27","isCorrect":false,"inputTokens":7772,"outputTokens":5,"latencyMs":1063.4963330000173},{"questionId":"q142","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"42","actual":"25","isCorrect":false,"inputTokens":5210,"outputTokens":5,"latencyMs":1157.562166999909},{"questionId":"q142","format":"toon","model":"claude-haiku-4-5-20251001","expected":"42","actual":"28","isCorrect":false,"inputTokens":6230,"outputTokens":5,"latencyMs":1122.3327499999432},{"questionId":"q142","format":"xml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"30","isCorrect":false,"inputTokens":8777,"outputTokens":5,"latencyMs":1094.998041999992},{"questionId":"q142","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"42","actual":"28","isCorrect":false,"inputTokens":6176,"outputTokens":5,"latencyMs":1123.9057500000345},{"questionId":"q143","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"25","actual":"24","isCorrect":false,"inputTokens":7769,"outputTokens":5,"latencyMs":1498.7087079999037},{"questionId":"q143","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"25","actual":"30","isCorrect":false,"inputTokens":5207,"outputTokens":5,"latencyMs":1361.3783330000006},{"questionId":"q143","format":"toon","model":"claude-haiku-4-5-20251001","expected":"25","actual":"38","isCorrect":false,"inputTokens":6227,"outputTokens":5,"latencyMs":1208.8371250000782},{"questionId":"q143","format":"xml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"39","isCorrect":false,"inputTokens":8774,"outputTokens":5,"latencyMs":1436.0557080000872},{"questionId":"q143","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"25","actual":"31","isCorrect":false,"inputTokens":6173,"outputTokens":5,"latencyMs":1439.8235000000568},{"questionId":"q144","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"29","actual":"30","isCorrect":false,"inputTokens":7774,"outputTokens":5,"latencyMs":1211.4848749999655},{"questionId":"q144","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"29","actual":"30","isCorrect":false,"inputTokens":5212,"outputTokens":5,"latencyMs":1158.1638749999693},{"questionId":"q144","format":"toon","model":"claude-haiku-4-5-20251001","expected":"29","actual":"43","isCorrect":false,"inputTokens":6232,"outputTokens":5,"latencyMs":1706.8831250000512},{"questionId":"q144","format":"xml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"38","isCorrect":false,"inputTokens":8779,"outputTokens":5,"latencyMs":1376.859208000009},{"questionId":"q144","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"29","actual":"31","isCorrect":false,"inputTokens":6178,"outputTokens":5,"latencyMs":1091.6000830000266},{"questionId":"q145","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"8","isCorrect":false,"inputTokens":7774,"outputTokens":5,"latencyMs":1214.2384999999776},{"questionId":"q145","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":5212,"outputTokens":5,"latencyMs":1169.8784999999916},{"questionId":"q145","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"3","isCorrect":false,"inputTokens":6232,"outputTokens":5,"latencyMs":1438.8369160000002},{"questionId":"q145","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"5","isCorrect":false,"inputTokens":8779,"outputTokens":5,"latencyMs":1235.8554580000928},{"questionId":"q145","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"5","isCorrect":false,"inputTokens":6178,"outputTokens":5,"latencyMs":1283.048208000022},{"questionId":"q146","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":7777,"outputTokens":5,"latencyMs":1316.5421670000069},{"questionId":"q146","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":5215,"outputTokens":5,"latencyMs":1155.4379169999156},{"questionId":"q146","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"8","isCorrect":false,"inputTokens":6235,"outputTokens":5,"latencyMs":973.9992499999935},{"questionId":"q146","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":8782,"outputTokens":5,"latencyMs":1181.0223749999423},{"questionId":"q146","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":6181,"outputTokens":5,"latencyMs":1189.6436249999097},{"questionId":"q147","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"6","isCorrect":false,"inputTokens":7777,"outputTokens":5,"latencyMs":1120.4472499999683},{"questionId":"q147","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"4","isCorrect":false,"inputTokens":5215,"outputTokens":5,"latencyMs":1058.7860420000507},{"questionId":"q147","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"5","isCorrect":false,"inputTokens":6235,"outputTokens":5,"latencyMs":1214.7113749999553},{"questionId":"q147","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"7","isCorrect":false,"inputTokens":8782,"outputTokens":5,"latencyMs":2272.7187089999206},{"questionId":"q147","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"3","isCorrect":false,"inputTokens":6181,"outputTokens":5,"latencyMs":1153.4284159999806},{"questionId":"q148","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"4","isCorrect":false,"inputTokens":7777,"outputTokens":5,"latencyMs":1250.0174589999951},{"questionId":"q148","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":5215,"outputTokens":5,"latencyMs":1667.0216249999357},{"questionId":"q148","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"4","isCorrect":false,"inputTokens":6235,"outputTokens":5,"latencyMs":1195.937874999945},{"questionId":"q148","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"5","isCorrect":false,"inputTokens":8782,"outputTokens":5,"latencyMs":1373.8929169999901},{"questionId":"q148","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"4","isCorrect":false,"inputTokens":6181,"outputTokens":5,"latencyMs":1219.8368749999208},{"questionId":"q149","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"7","isCorrect":false,"inputTokens":7776,"outputTokens":5,"latencyMs":1261.3687080000527},{"questionId":"q149","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"6","isCorrect":false,"inputTokens":5214,"outputTokens":5,"latencyMs":1096.3175000000047},{"questionId":"q149","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"7","isCorrect":false,"inputTokens":6234,"outputTokens":5,"latencyMs":1464.7897079999093},{"questionId":"q149","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"6","isCorrect":false,"inputTokens":8781,"outputTokens":5,"latencyMs":1314.154084000038},{"questionId":"q149","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"6","isCorrect":false,"inputTokens":6180,"outputTokens":5,"latencyMs":1648.5650410000235},{"questionId":"q150","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":7776,"outputTokens":5,"latencyMs":1204.6717499999795},{"questionId":"q150","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":5214,"outputTokens":5,"latencyMs":1045.0933749999385},{"questionId":"q150","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":6234,"outputTokens":5,"latencyMs":1071.8020000000251},{"questionId":"q150","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":8781,"outputTokens":5,"latencyMs":1080.7611669999314},{"questionId":"q150","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"8","isCorrect":false,"inputTokens":6180,"outputTokens":5,"latencyMs":1317.7093749999767},{"questionId":"q151","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":1244,"outputTokens":4,"latencyMs":929.0065000000177},{"questionId":"q151","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":785,"outputTokens":4,"latencyMs":1663.7153750000289},{"questionId":"q151","format":"toon","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":879,"outputTokens":4,"latencyMs":731.0117499999469},{"questionId":"q151","format":"xml","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":1314,"outputTokens":4,"latencyMs":883.2615830000723},{"questionId":"q151","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"development","actual":"development","isCorrect":true,"inputTokens":899,"outputTokens":4,"latencyMs":933.3807080000406},{"questionId":"q152","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1242,"outputTokens":8,"latencyMs":1309.4723340000492},{"questionId":"q152","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":783,"outputTokens":8,"latencyMs":1349.013917000033},{"questionId":"q152","format":"toon","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":877,"outputTokens":8,"latencyMs":2834.810291999951},{"questionId":"q152","format":"xml","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1312,"outputTokens":8,"latencyMs":1079.063041999936},{"questionId":"q152","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":897,"outputTokens":8,"latencyMs":2015.544249999919},{"questionId":"q153","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1242,"outputTokens":6,"latencyMs":1069.0238330001011},{"questionId":"q153","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":783,"outputTokens":6,"latencyMs":980.3079999999609},{"questionId":"q153","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":877,"outputTokens":6,"latencyMs":1044.9361250000075},{"questionId":"q153","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1312,"outputTokens":6,"latencyMs":2413.873292000033},{"questionId":"q153","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":897,"outputTokens":6,"latencyMs":873.3523750000168},{"questionId":"q154","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":1244,"outputTokens":5,"latencyMs":942.2453750000568},{"questionId":"q154","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":785,"outputTokens":5,"latencyMs":887.3804999999702},{"questionId":"q154","format":"toon","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":879,"outputTokens":5,"latencyMs":875.3322920000646},{"questionId":"q154","format":"xml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":1314,"outputTokens":5,"latencyMs":980.2426670000423},{"questionId":"q154","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"37","actual":"37","isCorrect":true,"inputTokens":899,"outputTokens":5,"latencyMs":1153.6150420000777},{"questionId":"q155","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1242,"outputTokens":6,"latencyMs":1241.1024170001037},{"questionId":"q155","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":783,"outputTokens":6,"latencyMs":1081.2751660000067},{"questionId":"q155","format":"toon","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":877,"outputTokens":6,"latencyMs":771.7234170000302},{"questionId":"q155","format":"xml","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1312,"outputTokens":6,"latencyMs":825.690082999994},{"questionId":"q155","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":897,"outputTokens":6,"latencyMs":752.5629590000026},{"questionId":"q156","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1244,"outputTokens":5,"latencyMs":1425.01554199995},{"questionId":"q156","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":785,"outputTokens":5,"latencyMs":1021.5733329999493},{"questionId":"q156","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":879,"outputTokens":5,"latencyMs":888.623041999992},{"questionId":"q156","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1314,"outputTokens":5,"latencyMs":920.717166999937},{"questionId":"q156","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":899,"outputTokens":5,"latencyMs":799.5221250000177},{"questionId":"q157","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1244,"outputTokens":6,"latencyMs":955.9951249998994},{"questionId":"q157","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":785,"outputTokens":6,"latencyMs":794.2132920000004},{"questionId":"q157","format":"toon","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":879,"outputTokens":6,"latencyMs":981.5377080000471},{"questionId":"q157","format":"xml","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1314,"outputTokens":6,"latencyMs":1138.1192919999594},{"questionId":"q157","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":899,"outputTokens":6,"latencyMs":856.9616249999963},{"questionId":"q158","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":1242,"outputTokens":4,"latencyMs":838.4199159999844},{"questionId":"q158","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":783,"outputTokens":4,"latencyMs":1141.6517499999609},{"questionId":"q158","format":"toon","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":877,"outputTokens":4,"latencyMs":1043.2275830000872},{"questionId":"q158","format":"xml","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":1312,"outputTokens":4,"latencyMs":1111.2398749999702},{"questionId":"q158","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"real","actual":"real","isCorrect":true,"inputTokens":897,"outputTokens":4,"latencyMs":863.2164999999804},{"questionId":"q159","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1243,"outputTokens":6,"latencyMs":880.7821669999976},{"questionId":"q159","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":784,"outputTokens":6,"latencyMs":895.2262499999488},{"questionId":"q159","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":878,"outputTokens":6,"latencyMs":1003.9393329999875},{"questionId":"q159","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1313,"outputTokens":6,"latencyMs":907.4466250000987},{"questionId":"q159","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":898,"outputTokens":6,"latencyMs":1019.0216660000151},{"questionId":"q160","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1244,"outputTokens":9,"latencyMs":1012.7389589999802},{"questionId":"q160","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":785,"outputTokens":9,"latencyMs":1003.1793339999858},{"questionId":"q160","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":879,"outputTokens":9,"latencyMs":916.7758330000797},{"questionId":"q160","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1314,"outputTokens":9,"latencyMs":1208.6454590000212},{"questionId":"q160","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":899,"outputTokens":9,"latencyMs":923.5587499999674},{"questionId":"q161","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1244,"outputTokens":5,"latencyMs":826.9563749999506},{"questionId":"q161","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":785,"outputTokens":5,"latencyMs":753.3855420000618},{"questionId":"q161","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":879,"outputTokens":5,"latencyMs":1089.936457999982},{"questionId":"q161","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1314,"outputTokens":5,"latencyMs":795.5758750000969},{"questionId":"q161","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":899,"outputTokens":5,"latencyMs":872.4575829999521},{"questionId":"q162","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1244,"outputTokens":5,"latencyMs":887.2722920000087},{"questionId":"q162","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":785,"outputTokens":5,"latencyMs":900.1268749999581},{"questionId":"q162","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":879,"outputTokens":5,"latencyMs":812.1885420000181},{"questionId":"q162","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1314,"outputTokens":5,"latencyMs":839.9153749999823},{"questionId":"q162","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":899,"outputTokens":5,"latencyMs":871.6134580000071},{"questionId":"q163","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1243,"outputTokens":5,"latencyMs":902.4298330000602},{"questionId":"q163","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":784,"outputTokens":5,"latencyMs":889.4039999999804},{"questionId":"q163","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":878,"outputTokens":5,"latencyMs":1126.9705829999875},{"questionId":"q163","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1313,"outputTokens":5,"latencyMs":958.2488329999615},{"questionId":"q163","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":898,"outputTokens":5,"latencyMs":1385.0525420000777},{"questionId":"q164","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1243,"outputTokens":5,"latencyMs":1185.5424160000402},{"questionId":"q164","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":784,"outputTokens":5,"latencyMs":883.8527500000782},{"questionId":"q164","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":878,"outputTokens":5,"latencyMs":1052.9344580001198},{"questionId":"q164","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":1313,"outputTokens":5,"latencyMs":911.154957999941},{"questionId":"q164","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2","actual":"2","isCorrect":true,"inputTokens":898,"outputTokens":5,"latencyMs":1121.9202919998206},{"questionId":"q165","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1244,"outputTokens":5,"latencyMs":879.8078749999404},{"questionId":"q165","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":785,"outputTokens":5,"latencyMs":1288.6490829999093},{"questionId":"q165","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":879,"outputTokens":5,"latencyMs":786.6004580000881},{"questionId":"q165","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1314,"outputTokens":5,"latencyMs":974.0200000000186},{"questionId":"q165","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":899,"outputTokens":5,"latencyMs":1045.2869170000777},{"questionId":"q166","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1247,"outputTokens":5,"latencyMs":824.5860000001267},{"questionId":"q166","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"0","isCorrect":false,"inputTokens":788,"outputTokens":5,"latencyMs":974.435499999905},{"questionId":"q166","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":882,"outputTokens":5,"latencyMs":804.1913339998573},{"questionId":"q166","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1317,"outputTokens":5,"latencyMs":1016.0534169999883},{"questionId":"q166","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":902,"outputTokens":5,"latencyMs":962.7041670000181},{"questionId":"q167","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1243,"outputTokens":5,"latencyMs":1005.984332999913},{"questionId":"q167","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":784,"outputTokens":5,"latencyMs":1055.2427499999758},{"questionId":"q167","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":878,"outputTokens":5,"latencyMs":945.2598330001347},{"questionId":"q167","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1313,"outputTokens":5,"latencyMs":1155.7412080001086},{"questionId":"q167","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":898,"outputTokens":5,"latencyMs":814.8985409999732},{"questionId":"q168","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":1245,"outputTokens":5,"latencyMs":1001.1619170000777},{"questionId":"q168","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":786,"outputTokens":5,"latencyMs":973.6378339999355},{"questionId":"q168","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":880,"outputTokens":5,"latencyMs":906.6011659998912},{"questionId":"q168","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":1315,"outputTokens":5,"latencyMs":5450.461332999868},{"questionId":"q168","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"5","isCorrect":true,"inputTokens":900,"outputTokens":5,"latencyMs":893.4252499998547},{"questionId":"q169","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":1247,"outputTokens":5,"latencyMs":913.8683329999913},{"questionId":"q169","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":788,"outputTokens":5,"latencyMs":1247.810541999992},{"questionId":"q169","format":"toon","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":882,"outputTokens":5,"latencyMs":927.667708999943},{"questionId":"q169","format":"xml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"9","isCorrect":false,"inputTokens":1317,"outputTokens":5,"latencyMs":1024.6397919999436},{"questionId":"q169","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"8","actual":"8","isCorrect":true,"inputTokens":902,"outputTokens":5,"latencyMs":808.8294999999925},{"questionId":"q170","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":1246,"outputTokens":5,"latencyMs":948.8401660001837},{"questionId":"q170","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":787,"outputTokens":5,"latencyMs":807.7621660002042},{"questionId":"q170","format":"toon","model":"claude-haiku-4-5-20251001","expected":"5","actual":"7","isCorrect":false,"inputTokens":881,"outputTokens":5,"latencyMs":995.4957500000019},{"questionId":"q170","format":"xml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":1316,"outputTokens":5,"latencyMs":948.4754999999423},{"questionId":"q170","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"5","actual":"6","isCorrect":false,"inputTokens":901,"outputTokens":5,"latencyMs":915.8271250000689},{"questionId":"q171","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1248,"outputTokens":5,"latencyMs":923.2111250001471},{"questionId":"q171","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":789,"outputTokens":5,"latencyMs":780.6261670000385},{"questionId":"q171","format":"toon","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":883,"outputTokens":5,"latencyMs":923.018459000159},{"questionId":"q171","format":"xml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":1318,"outputTokens":5,"latencyMs":993.0462080000434},{"questionId":"q171","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"3","actual":"3","isCorrect":true,"inputTokens":903,"outputTokens":5,"latencyMs":1245.815000000177},{"questionId":"q172","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1249,"outputTokens":5,"latencyMs":1095.3125},{"questionId":"q172","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":790,"outputTokens":5,"latencyMs":709.230874999892},{"questionId":"q172","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":884,"outputTokens":5,"latencyMs":1246.6792499998119},{"questionId":"q172","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1319,"outputTokens":5,"latencyMs":870.7104170001112},{"questionId":"q172","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":904,"outputTokens":5,"latencyMs":974.5198749999981},{"questionId":"q173","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1250,"outputTokens":5,"latencyMs":985.8480829999316},{"questionId":"q173","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":791,"outputTokens":5,"latencyMs":1002.6233339998871},{"questionId":"q173","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":885,"outputTokens":5,"latencyMs":1006.6666669999249},{"questionId":"q173","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1320,"outputTokens":5,"latencyMs":1024.3824579999782},{"questionId":"q173","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":905,"outputTokens":5,"latencyMs":2409.9704589999747},{"questionId":"q174","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1246,"outputTokens":5,"latencyMs":1009.0951249999925},{"questionId":"q174","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"2","isCorrect":false,"inputTokens":787,"outputTokens":5,"latencyMs":916.1399580000434},{"questionId":"q174","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":881,"outputTokens":5,"latencyMs":1165.979916999815},{"questionId":"q174","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1316,"outputTokens":5,"latencyMs":864.8515409999527},{"questionId":"q174","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"2","isCorrect":false,"inputTokens":901,"outputTokens":5,"latencyMs":916.4436249998398},{"questionId":"q175","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1251,"outputTokens":5,"latencyMs":1029.9872919998597},{"questionId":"q175","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":792,"outputTokens":5,"latencyMs":1144.0288329999894},{"questionId":"q175","format":"toon","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":886,"outputTokens":5,"latencyMs":822.7032080001663},{"questionId":"q175","format":"xml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":1321,"outputTokens":5,"latencyMs":1031.0812089999672},{"questionId":"q175","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"0","actual":"0","isCorrect":true,"inputTokens":906,"outputTokens":5,"latencyMs":937.500250000041},{"questionId":"q176","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1244,"outputTokens":5,"latencyMs":911.2006250000559},{"questionId":"q176","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":785,"outputTokens":5,"latencyMs":791.9767080000602},{"questionId":"q176","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":879,"outputTokens":5,"latencyMs":817.9461670001037},{"questionId":"q176","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1314,"outputTokens":5,"latencyMs":800.3201659999322},{"questionId":"q176","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":899,"outputTokens":5,"latencyMs":1166.106958999997},{"questionId":"q177","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1254,"outputTokens":5,"latencyMs":1153.844291999936},{"questionId":"q177","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":795,"outputTokens":5,"latencyMs":772.196958999848},{"questionId":"q177","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":889,"outputTokens":5,"latencyMs":761.2225830000825},{"questionId":"q177","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1324,"outputTokens":5,"latencyMs":1030.3278750001919},{"questionId":"q177","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":909,"outputTokens":5,"latencyMs":956.0868329999503},{"questionId":"q178","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1249,"outputTokens":5,"latencyMs":1061.2030420000665},{"questionId":"q178","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":790,"outputTokens":5,"latencyMs":888.081167000113},{"questionId":"q178","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":884,"outputTokens":5,"latencyMs":1051.8333749999292},{"questionId":"q178","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1319,"outputTokens":5,"latencyMs":799.0097499999683},{"questionId":"q178","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":904,"outputTokens":5,"latencyMs":905.2197910000104},{"questionId":"q179","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1247,"outputTokens":5,"latencyMs":988.6873749999795},{"questionId":"q179","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":788,"outputTokens":5,"latencyMs":723.2364169999491},{"questionId":"q179","format":"toon","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":882,"outputTokens":5,"latencyMs":728.2359160000924},{"questionId":"q179","format":"xml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":1317,"outputTokens":5,"latencyMs":977.7162500000559},{"questionId":"q179","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"1","actual":"1","isCorrect":true,"inputTokens":902,"outputTokens":5,"latencyMs":1027.893291000044},{"questionId":"q180","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":7935,"outputTokens":5,"latencyMs":1120.7492909999564},{"questionId":"q180","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":4826,"outputTokens":5,"latencyMs":1454.5087910001166},{"questionId":"q180","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":3075,"outputTokens":5,"latencyMs":1035.1915830001235},{"questionId":"q180","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":2921,"outputTokens":5,"latencyMs":931.953375000041},{"questionId":"q180","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9420,"outputTokens":5,"latencyMs":941.3573339998256},{"questionId":"q180","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":5826,"outputTokens":5,"latencyMs":986.0300420001149},{"questionId":"q181","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":7941,"outputTokens":20,"latencyMs":1113.5572500000708},{"questionId":"q181","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4832,"outputTokens":20,"latencyMs":993.531415999867},{"questionId":"q181","format":"toon","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":3081,"outputTokens":20,"latencyMs":949.0614160001278},{"questionId":"q181","format":"csv","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2927,"outputTokens":20,"latencyMs":1220.1684170002118},{"questionId":"q181","format":"xml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":9426,"outputTokens":20,"latencyMs":2677.9535000000615},{"questionId":"q181","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":5832,"outputTokens":20,"latencyMs":1150.8704589998815},{"questionId":"q182","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":7938,"outputTokens":4,"latencyMs":1143.8894159998745},{"questionId":"q182","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":4829,"outputTokens":4,"latencyMs":939.9587090001442},{"questionId":"q182","format":"toon","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":3078,"outputTokens":4,"latencyMs":1002.7564159999602},{"questionId":"q182","format":"csv","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":2924,"outputTokens":4,"latencyMs":1499.9509160001762},{"questionId":"q182","format":"xml","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":9423,"outputTokens":4,"latencyMs":1206.003415999934},{"questionId":"q182","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"email","actual":"email","isCorrect":true,"inputTokens":5829,"outputTokens":4,"latencyMs":1156.9789579999633},{"questionId":"q183","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7939,"outputTokens":4,"latencyMs":1133.3445830000564},{"questionId":"q183","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4830,"outputTokens":4,"latencyMs":1083.2715829999652},{"questionId":"q183","format":"toon","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":3079,"outputTokens":4,"latencyMs":1054.9281669999473},{"questionId":"q183","format":"csv","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2925,"outputTokens":4,"latencyMs":1052.3783750000875},{"questionId":"q183","format":"xml","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":9424,"outputTokens":4,"latencyMs":1340.8406249999534},{"questionId":"q183","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5830,"outputTokens":4,"latencyMs":1075.3891660000663},{"questionId":"q184","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7939,"outputTokens":12,"latencyMs":2124.266124999849},{"questionId":"q184","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4830,"outputTokens":12,"latencyMs":1068.5028750000056},{"questionId":"q184","format":"toon","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":3079,"outputTokens":12,"latencyMs":1167.4424999998882},{"questionId":"q184","format":"csv","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2925,"outputTokens":12,"latencyMs":1302.0248749998864},{"questionId":"q184","format":"xml","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":9424,"outputTokens":12,"latencyMs":1186.8695000000298},{"questionId":"q184","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5830,"outputTokens":12,"latencyMs":1270.5893329998944},{"questionId":"q185","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":7936,"outputTokens":5,"latencyMs":1134.64995799982},{"questionId":"q185","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":4827,"outputTokens":5,"latencyMs":1194.1974160000682},{"questionId":"q185","format":"toon","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":3076,"outputTokens":5,"latencyMs":1283.6654169999529},{"questionId":"q185","format":"csv","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":2922,"outputTokens":5,"latencyMs":1176.9219159998465},{"questionId":"q185","format":"xml","model":"claude-haiku-4-5-20251001","expected":"7","actual":"7","isCorrect":true,"inputTokens":9421,"outputTokens":5,"latencyMs":1174.4477500000503},{"questionId":"q185","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"7","actual":"6","isCorrect":false,"inputTokens":5827,"outputTokens":5,"latencyMs":1267.2275419998914},{"questionId":"q186","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":13066,"outputTokens":5,"latencyMs":1317.5971669999417},{"questionId":"q186","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":8027,"outputTokens":5,"latencyMs":1233.12991600018},{"questionId":"q186","format":"toon","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":8344,"outputTokens":5,"latencyMs":1202.2103329999372},{"questionId":"q186","format":"xml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":14571,"outputTokens":5,"latencyMs":1269.4371249999385},{"questionId":"q186","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"50","actual":"50","isCorrect":true,"inputTokens":9469,"outputTokens":5,"latencyMs":1300.4245419998188},{"questionId":"q187","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":13075,"outputTokens":21,"latencyMs":1696.230416999897},{"questionId":"q187","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":8036,"outputTokens":21,"latencyMs":1352.8969999998808},{"questionId":"q187","format":"toon","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":8353,"outputTokens":21,"latencyMs":1110.8835419998504},{"questionId":"q187","format":"xml","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":14580,"outputTokens":21,"latencyMs":1534.4549169999082},{"questionId":"q187","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":9478,"outputTokens":21,"latencyMs":1164.3688749999274},{"questionId":"q188","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":13071,"outputTokens":5,"latencyMs":1336.527957999846},{"questionId":"q188","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":8032,"outputTokens":5,"latencyMs":1103.1468749998603},{"questionId":"q188","format":"toon","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":8349,"outputTokens":5,"latencyMs":1129.3212500000373},{"questionId":"q188","format":"xml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":14576,"outputTokens":5,"latencyMs":1354.4471249999478},{"questionId":"q188","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"4","actual":"4","isCorrect":true,"inputTokens":9474,"outputTokens":5,"latencyMs":1144.939916000003},{"questionId":"q189","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":13075,"outputTokens":11,"latencyMs":1416.0433340000454},{"questionId":"q189","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":8036,"outputTokens":11,"latencyMs":1131.6960839999374},{"questionId":"q189","format":"toon","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":8353,"outputTokens":11,"latencyMs":1228.2068749999162},{"questionId":"q189","format":"xml","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":14580,"outputTokens":11,"latencyMs":1415.090707999887},{"questionId":"q189","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":9478,"outputTokens":11,"latencyMs":1161.2281250001397},{"questionId":"q190","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13070,"outputTokens":4,"latencyMs":2074.3852499998175},{"questionId":"q190","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8031,"outputTokens":4,"latencyMs":1235.8749160000589},{"questionId":"q190","format":"toon","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8348,"outputTokens":4,"latencyMs":1108.9501659998205},{"questionId":"q190","format":"xml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":14575,"outputTokens":4,"latencyMs":1624.1051250000019},{"questionId":"q190","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9473,"outputTokens":4,"latencyMs":1303.3145830000285},{"questionId":"q191","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":13076,"outputTokens":10,"latencyMs":1195.093499999959},{"questionId":"q191","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":8037,"outputTokens":10,"latencyMs":1125.0091250000987},{"questionId":"q191","format":"toon","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":8354,"outputTokens":10,"latencyMs":1002.9523330000229},{"questionId":"q191","format":"xml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":14581,"outputTokens":10,"latencyMs":1123.1422500000335},{"questionId":"q191","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":9479,"outputTokens":10,"latencyMs":1390.2729579999577},{"questionId":"q192","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":4142,"outputTokens":5,"latencyMs":1418.0205409999471},{"questionId":"q192","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":2453,"outputTokens":5,"latencyMs":1059.0376249998808},{"questionId":"q192","format":"toon","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":1599,"outputTokens":5,"latencyMs":1186.7520419999491},{"questionId":"q192","format":"csv","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":1507,"outputTokens":5,"latencyMs":1620.152291999897},{"questionId":"q192","format":"xml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":4844,"outputTokens":5,"latencyMs":1310.0510420000646},{"questionId":"q192","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"60","actual":"60","isCorrect":true,"inputTokens":3173,"outputTokens":5,"latencyMs":1221.876249999972},{"questionId":"q193","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4147,"outputTokens":17,"latencyMs":1487.5997919999063},{"questionId":"q193","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2458,"outputTokens":17,"latencyMs":1157.4452500001062},{"questionId":"q193","format":"toon","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1604,"outputTokens":17,"latencyMs":1520.7116659998428},{"questionId":"q193","format":"csv","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1512,"outputTokens":17,"latencyMs":1203.7664580000564},{"questionId":"q193","format":"xml","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4849,"outputTokens":17,"latencyMs":1226.0437910000328},{"questionId":"q193","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3178,"outputTokens":17,"latencyMs":977.1910840000492},{"questionId":"q194","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":4145,"outputTokens":6,"latencyMs":1073.1372919999994},{"questionId":"q194","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":2456,"outputTokens":6,"latencyMs":1266.1767920001876},{"questionId":"q194","format":"toon","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1602,"outputTokens":4,"latencyMs":1342.251207999885},{"questionId":"q194","format":"csv","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":1510,"outputTokens":6,"latencyMs":1179.91620899993},{"questionId":"q194","format":"xml","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":4847,"outputTokens":6,"latencyMs":1468.4425409999676},{"questionId":"q194","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":3176,"outputTokens":6,"latencyMs":900.0507499999367},{"questionId":"q195","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4146,"outputTokens":10,"latencyMs":1087.1228330000304},{"questionId":"q195","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2457,"outputTokens":10,"latencyMs":1056.3863329999149},{"questionId":"q195","format":"toon","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1603,"outputTokens":10,"latencyMs":1167.88495899993},{"questionId":"q195","format":"csv","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1511,"outputTokens":10,"latencyMs":984.4359160000458},{"questionId":"q195","format":"xml","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4848,"outputTokens":10,"latencyMs":1138.7140420000069},{"questionId":"q195","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3177,"outputTokens":10,"latencyMs":1117.2731670001522},{"questionId":"q196","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":4142,"outputTokens":5,"latencyMs":1177.2991659999825},{"questionId":"q196","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":2453,"outputTokens":5,"latencyMs":965.145041000098},{"questionId":"q196","format":"toon","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":1599,"outputTokens":5,"latencyMs":1326.4698749999516},{"questionId":"q196","format":"csv","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":1507,"outputTokens":5,"latencyMs":1118.3482919998933},{"questionId":"q196","format":"xml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":4844,"outputTokens":5,"latencyMs":1536.1617910000496},{"questionId":"q196","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"6","actual":"6","isCorrect":true,"inputTokens":3173,"outputTokens":5,"latencyMs":930.9039169999305},{"questionId":"q197","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":17473,"outputTokens":5,"latencyMs":1286.1768749998882},{"questionId":"q197","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":12608,"outputTokens":5,"latencyMs":1507.926083000144},{"questionId":"q197","format":"toon","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9371,"outputTokens":5,"latencyMs":1098.5005830000155},{"questionId":"q197","format":"csv","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":9189,"outputTokens":5,"latencyMs":11181.550583999837},{"questionId":"q197","format":"xml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":19863,"outputTokens":5,"latencyMs":1288.1963339999784},{"questionId":"q197","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"100","actual":"100","isCorrect":true,"inputTokens":14548,"outputTokens":5,"latencyMs":1512.4048330001533},{"questionId":"q198","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":17479,"outputTokens":35,"latencyMs":1934.8035840000957},{"questionId":"q198","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":12614,"outputTokens":35,"latencyMs":1396.143374999985},{"questionId":"q198","format":"toon","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":9377,"outputTokens":35,"latencyMs":1337.9257079998497},{"questionId":"q198","format":"csv","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":9195,"outputTokens":35,"latencyMs":1317.0077919999603},{"questionId":"q198","format":"xml","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":19869,"outputTokens":35,"latencyMs":2428.95404099999},{"questionId":"q198","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":14554,"outputTokens":35,"latencyMs":1332.3844579998404},{"questionId":"q199","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":17477,"outputTokens":6,"latencyMs":1435.5708749999758},{"questionId":"q199","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":12612,"outputTokens":6,"latencyMs":1323.1899170000106},{"questionId":"q199","format":"toon","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":9375,"outputTokens":6,"latencyMs":1109.1495000000577},{"questionId":"q199","format":"csv","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":9193,"outputTokens":6,"latencyMs":1622.9887500000186},{"questionId":"q199","format":"xml","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":19867,"outputTokens":6,"latencyMs":1395.366083000088},{"questionId":"q199","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":14552,"outputTokens":6,"latencyMs":1273.513874999946},{"questionId":"q200","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17477,"outputTokens":7,"latencyMs":1235.034042000072},{"questionId":"q200","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":12612,"outputTokens":7,"latencyMs":1452.7786670001224},{"questionId":"q200","format":"toon","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":9375,"outputTokens":7,"latencyMs":1009.8154169998597},{"questionId":"q200","format":"csv","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":9193,"outputTokens":7,"latencyMs":1215.5075840000063},{"questionId":"q200","format":"xml","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":19867,"outputTokens":7,"latencyMs":1331.6306249999907},{"questionId":"q200","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":14552,"outputTokens":7,"latencyMs":1234.5180420000106},{"questionId":"q201","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":17474,"outputTokens":5,"latencyMs":1365.1134999999776},{"questionId":"q201","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"11","actual":"10","isCorrect":false,"inputTokens":12609,"outputTokens":5,"latencyMs":1297.5984169999138},{"questionId":"q201","format":"toon","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":9372,"outputTokens":5,"latencyMs":1234.3033330000471},{"questionId":"q201","format":"csv","model":"claude-haiku-4-5-20251001","expected":"11","actual":"11","isCorrect":true,"inputTokens":9190,"outputTokens":5,"latencyMs":1500.0707079998683},{"questionId":"q201","format":"xml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"12","isCorrect":false,"inputTokens":19864,"outputTokens":5,"latencyMs":1370.718208000064},{"questionId":"q201","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"11","actual":"13","isCorrect":false,"inputTokens":14549,"outputTokens":5,"latencyMs":1321.9931669998914},{"questionId":"q202","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":7767,"outputTokens":5,"latencyMs":1315.0292080000509},{"questionId":"q202","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"75","actual":"70","isCorrect":false,"inputTokens":5205,"outputTokens":5,"latencyMs":918.2240830000956},{"questionId":"q202","format":"toon","model":"claude-haiku-4-5-20251001","expected":"75","actual":"75","isCorrect":true,"inputTokens":6225,"outputTokens":5,"latencyMs":993.3855000000913},{"questionId":"q202","format":"xml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"100","isCorrect":false,"inputTokens":8772,"outputTokens":5,"latencyMs":998.8321670000441},{"questionId":"q202","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"75","actual":"74","isCorrect":false,"inputTokens":6171,"outputTokens":5,"latencyMs":1022.2879580000881},{"questionId":"q203","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":7777,"outputTokens":18,"latencyMs":1319.1331670000218},{"questionId":"q203","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":false,"inputTokens":5215,"outputTokens":26,"latencyMs":1044.1837079999968},{"questionId":"q203","format":"toon","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":false,"inputTokens":6235,"outputTokens":26,"latencyMs":1229.3462499999441},{"questionId":"q203","format":"xml","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":false,"inputTokens":8782,"outputTokens":26,"latencyMs":1153.167082999833},{"questionId":"q203","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":false,"inputTokens":6181,"outputTokens":26,"latencyMs":1140.1483329997864},{"questionId":"q204","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":7771,"outputTokens":4,"latencyMs":1057.0545830000192},{"questionId":"q204","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":5209,"outputTokens":4,"latencyMs":1038.1022499999963},{"questionId":"q204","format":"toon","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6229,"outputTokens":4,"latencyMs":951.9464999998454},{"questionId":"q204","format":"xml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":8776,"outputTokens":4,"latencyMs":1045.921832999913},{"questionId":"q204","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"info","actual":"info","isCorrect":true,"inputTokens":6175,"outputTokens":4,"latencyMs":1206.6149999999907},{"questionId":"q205","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1685,"outputTokens":4,"latencyMs":781.0249159999657},{"questionId":"q205","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1057,"outputTokens":4,"latencyMs":865.4773750000168},{"questionId":"q205","format":"toon","model":"claude-haiku-4-5-20251001","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":745,"outputTokens":4,"latencyMs":847.1748330001719},{"questionId":"q205","format":"csv","model":"claude-haiku-4-5-20251001","expected":"YES","actual":"NO","isCorrect":false,"inputTokens":688,"outputTokens":4,"latencyMs":1022.7747920001857},{"questionId":"q205","format":"xml","model":"claude-haiku-4-5-20251001","expected":"YES","actual":"NO","isCorrect":false,"inputTokens":1967,"outputTokens":4,"latencyMs":788.3179999999702},{"questionId":"q205","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1257,"outputTokens":4,"latencyMs":746.1863339999691},{"questionId":"q206","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1459,"outputTokens":4,"latencyMs":1047.0943330000155},{"questionId":"q206","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":924,"outputTokens":4,"latencyMs":919.9484170000069},{"questionId":"q206","format":"toon","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":666,"outputTokens":4,"latencyMs":907.2270830001216},{"questionId":"q206","format":"csv","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":612,"outputTokens":4,"latencyMs":845.8464999999851},{"questionId":"q206","format":"xml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1696,"outputTokens":4,"latencyMs":946.0020830000285},{"questionId":"q206","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1094,"outputTokens":4,"latencyMs":920.9464169999119},{"questionId":"q207","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1925,"outputTokens":4,"latencyMs":1408.2781249999534},{"questionId":"q207","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1204,"outputTokens":4,"latencyMs":1325.2302080001682},{"questionId":"q207","format":"toon","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":838,"outputTokens":4,"latencyMs":939.0019169999287},{"questionId":"q207","format":"csv","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":778,"outputTokens":4,"latencyMs":822.6255419999361},{"questionId":"q207","format":"xml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":2255,"outputTokens":4,"latencyMs":857.5038749999367},{"questionId":"q207","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1434,"outputTokens":4,"latencyMs":850.0120409999508},{"questionId":"q208","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1676,"outputTokens":4,"latencyMs":913.5827909999061},{"questionId":"q208","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1052,"outputTokens":4,"latencyMs":1083.7319999998435},{"questionId":"q208","format":"toon","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1280,"outputTokens":4,"latencyMs":1061.7704580000136},{"questionId":"q208","format":"csv","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":685,"outputTokens":4,"latencyMs":736.6914170000236},{"questionId":"q208","format":"xml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1957,"outputTokens":4,"latencyMs":909.4639159999788},{"questionId":"q208","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1250,"outputTokens":4,"latencyMs":1399.6871670000255},{"questionId":"q209","format":"json-pretty","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1619,"outputTokens":4,"latencyMs":856.7064160001464},{"questionId":"q209","format":"json-compact","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1007,"outputTokens":4,"latencyMs":794.2346660001203},{"questionId":"q209","format":"toon","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1233,"outputTokens":4,"latencyMs":1252.9911249999423},{"questionId":"q209","format":"csv","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":472,"outputTokens":4,"latencyMs":1058.458375000162},{"questionId":"q209","format":"xml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1897,"outputTokens":4,"latencyMs":972.7352500001434},{"questionId":"q209","format":"yaml","model":"claude-haiku-4-5-20251001","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1203,"outputTokens":4,"latencyMs":910.6434579999186}] \ No newline at end of file diff --git a/benchmarks/results/accuracy/models/gemini-2.5-flash b/benchmarks/results/accuracy/models/gemini-2.5-flash index 5ebc18e..214f91f 100644 --- a/benchmarks/results/accuracy/models/gemini-2.5-flash +++ b/benchmarks/results/accuracy/models/gemini-2.5-flash @@ -1 +1 @@ -[{"questionId":"q1","format":"json-pretty","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7936,"outputTokens":5,"latencyMs":1620.1060830000001},{"questionId":"q1","format":"json-compact","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":4724,"outputTokens":5,"latencyMs":1376.159583},{"questionId":"q1","format":"toon","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":3369,"outputTokens":5,"latencyMs":1917.2902910000003},{"questionId":"q1","format":"csv","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":3220,"outputTokens":5,"latencyMs":1806.8632500000003},{"questionId":"q1","format":"xml","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":9121,"outputTokens":5,"latencyMs":2078.3292080000006},{"questionId":"q1","format":"yaml","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5770,"outputTokens":5,"latencyMs":1384.205625},{"questionId":"q2","format":"json-pretty","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7936,"outputTokens":1,"latencyMs":1588.3066249999997},{"questionId":"q2","format":"json-compact","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":4724,"outputTokens":1,"latencyMs":1639.6037500000002},{"questionId":"q2","format":"toon","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":3369,"outputTokens":1,"latencyMs":1708.695917},{"questionId":"q2","format":"csv","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":3220,"outputTokens":1,"latencyMs":1794.5588340000004},{"questionId":"q2","format":"xml","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":9121,"outputTokens":1,"latencyMs":2050.5885},{"questionId":"q2","format":"yaml","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5770,"outputTokens":1,"latencyMs":1521.7057080000004},{"questionId":"q3","format":"json-pretty","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7939,"outputTokens":10,"latencyMs":1202.385792},{"questionId":"q3","format":"json-compact","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4727,"outputTokens":10,"latencyMs":1638.112083},{"questionId":"q3","format":"toon","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":3372,"outputTokens":10,"latencyMs":2780.2663330000005},{"questionId":"q3","format":"csv","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":3223,"outputTokens":10,"latencyMs":1491.7031659999993},{"questionId":"q3","format":"xml","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":9124,"outputTokens":10,"latencyMs":1496.6606659999998},{"questionId":"q3","format":"yaml","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5773,"outputTokens":10,"latencyMs":1367.5506659999992},{"questionId":"q4","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":7939,"outputTokens":2,"latencyMs":1525.6104169999999},{"questionId":"q4","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4727,"outputTokens":2,"latencyMs":1375.4866660000007},{"questionId":"q4","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3372,"outputTokens":2,"latencyMs":1953.6631249999991},{"questionId":"q4","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3223,"outputTokens":2,"latencyMs":1794.639666000001},{"questionId":"q4","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":9124,"outputTokens":2,"latencyMs":1824.8247499999998},{"questionId":"q4","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5773,"outputTokens":2,"latencyMs":1071.3964590000014},{"questionId":"q5","format":"json-pretty","model":"gemini-2.5-flash","expected":"no","actual":"False","isCorrect":true,"inputTokens":7934,"outputTokens":1,"latencyMs":1068.6503329999996},{"questionId":"q5","format":"json-compact","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":4722,"outputTokens":1,"latencyMs":2036.3870000000024},{"questionId":"q5","format":"toon","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":3367,"outputTokens":1,"latencyMs":1787.9629580000037},{"questionId":"q5","format":"csv","model":"gemini-2.5-flash","expected":"no","actual":"No","isCorrect":true,"inputTokens":3218,"outputTokens":1,"latencyMs":1973.6712499999994},{"questionId":"q5","format":"xml","model":"gemini-2.5-flash","expected":"no","actual":"No","isCorrect":true,"inputTokens":9119,"outputTokens":1,"latencyMs":1985.177791999995},{"questionId":"q5","format":"yaml","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":5768,"outputTokens":1,"latencyMs":2499.712124999998},{"questionId":"q6","format":"json-pretty","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7935,"outputTokens":6,"latencyMs":2641.799417000002},{"questionId":"q6","format":"json-compact","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":4723,"outputTokens":6,"latencyMs":1061.836292},{"questionId":"q6","format":"toon","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":3368,"outputTokens":6,"latencyMs":1820.6608749999941},{"questionId":"q6","format":"csv","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":3219,"outputTokens":6,"latencyMs":1412.7557920000036},{"questionId":"q6","format":"xml","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":9120,"outputTokens":6,"latencyMs":1844.8989590000056},{"questionId":"q6","format":"yaml","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5769,"outputTokens":6,"latencyMs":1227.5040420000005},{"questionId":"q7","format":"json-pretty","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7935,"outputTokens":1,"latencyMs":2047.9157080000004},{"questionId":"q7","format":"json-compact","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":4723,"outputTokens":1,"latencyMs":1769.7225420000032},{"questionId":"q7","format":"toon","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":3368,"outputTokens":1,"latencyMs":2623.2365830000053},{"questionId":"q7","format":"csv","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":3219,"outputTokens":1,"latencyMs":1112.7179999999935},{"questionId":"q7","format":"xml","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":9120,"outputTokens":1,"latencyMs":1908.1761660000047},{"questionId":"q7","format":"yaml","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5769,"outputTokens":1,"latencyMs":2878.1585830000113},{"questionId":"q8","format":"json-pretty","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7938,"outputTokens":9,"latencyMs":993.8725419999973},{"questionId":"q8","format":"json-compact","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4726,"outputTokens":9,"latencyMs":1125.9782919999998},{"questionId":"q8","format":"toon","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":3371,"outputTokens":9,"latencyMs":1464.317875000008},{"questionId":"q8","format":"csv","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":3222,"outputTokens":9,"latencyMs":1619.4892499999987},{"questionId":"q8","format":"xml","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":9123,"outputTokens":9,"latencyMs":1760.0224589999998},{"questionId":"q8","format":"yaml","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5772,"outputTokens":9,"latencyMs":1012.8469580000092},{"questionId":"q9","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":7939,"outputTokens":1,"latencyMs":1302.387750000009},{"questionId":"q9","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":4727,"outputTokens":1,"latencyMs":1258.882999999987},{"questionId":"q9","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":3372,"outputTokens":1,"latencyMs":1743.854416000002},{"questionId":"q9","format":"csv","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":3223,"outputTokens":1,"latencyMs":1629.0175000000017},{"questionId":"q9","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":9124,"outputTokens":1,"latencyMs":2055.1037090000027},{"questionId":"q9","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":5773,"outputTokens":1,"latencyMs":1673.0168330000015},{"questionId":"q10","format":"json-pretty","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":7936,"outputTokens":1,"latencyMs":2424.4958329999936},{"questionId":"q10","format":"json-compact","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":4724,"outputTokens":1,"latencyMs":3073.3142090000038},{"questionId":"q10","format":"toon","model":"gemini-2.5-flash","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":3369,"outputTokens":1,"latencyMs":1846.0572919999977},{"questionId":"q10","format":"csv","model":"gemini-2.5-flash","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":3220,"outputTokens":1,"latencyMs":1509.7272499999963},{"questionId":"q10","format":"xml","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":9121,"outputTokens":1,"latencyMs":1726.9054160000087},{"questionId":"q10","format":"yaml","model":"gemini-2.5-flash","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":5770,"outputTokens":1,"latencyMs":2197.6978329999984},{"questionId":"q11","format":"json-pretty","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7937,"outputTokens":6,"latencyMs":1028.4849169999943},{"questionId":"q11","format":"json-compact","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":4725,"outputTokens":6,"latencyMs":2268.641915999993},{"questionId":"q11","format":"toon","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":3370,"outputTokens":6,"latencyMs":2226.206124999997},{"questionId":"q11","format":"csv","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":3221,"outputTokens":6,"latencyMs":1587.6684160000004},{"questionId":"q11","format":"xml","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":9122,"outputTokens":6,"latencyMs":1382.4763329999987},{"questionId":"q11","format":"yaml","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5771,"outputTokens":6,"latencyMs":1001.6328330000106},{"questionId":"q12","format":"json-pretty","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7935,"outputTokens":1,"latencyMs":1742.0453750000015},{"questionId":"q12","format":"json-compact","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":4723,"outputTokens":1,"latencyMs":597.09945899999},{"questionId":"q12","format":"toon","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":3368,"outputTokens":1,"latencyMs":1165.752959000005},{"questionId":"q12","format":"csv","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":3219,"outputTokens":1,"latencyMs":1523.6268750000017},{"questionId":"q12","format":"xml","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":9120,"outputTokens":1,"latencyMs":1517.1408749999973},{"questionId":"q12","format":"yaml","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5769,"outputTokens":1,"latencyMs":1501.5247499999823},{"questionId":"q13","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":7934,"outputTokens":2,"latencyMs":1099.4725829999952},{"questionId":"q13","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"13","isCorrect":false,"inputTokens":4722,"outputTokens":2,"latencyMs":813.8151250000083},{"questionId":"q13","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3367,"outputTokens":2,"latencyMs":2259.0083749999903},{"questionId":"q13","format":"csv","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3218,"outputTokens":2,"latencyMs":26381.103541999997},{"questionId":"q13","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":9119,"outputTokens":2,"latencyMs":2462.6522919999843},{"questionId":"q13","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":5768,"outputTokens":2,"latencyMs":2872.524375000008},{"questionId":"q14","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"20","isCorrect":false,"inputTokens":7934,"outputTokens":2,"latencyMs":1358.3752500000119},{"questionId":"q14","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":4722,"outputTokens":2,"latencyMs":1536.6222079999861},{"questionId":"q14","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3367,"outputTokens":2,"latencyMs":19015.34358299998},{"questionId":"q14","format":"csv","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3218,"outputTokens":2,"latencyMs":4555.7668330000015},{"questionId":"q14","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":9119,"outputTokens":2,"latencyMs":1317.649000000005},{"questionId":"q14","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":5768,"outputTokens":2,"latencyMs":1198.2006249999977},{"questionId":"q15","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":7934,"outputTokens":2,"latencyMs":1404.334708999988},{"questionId":"q15","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"25","isCorrect":false,"inputTokens":4722,"outputTokens":2,"latencyMs":816.1643750000221},{"questionId":"q15","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"21","isCorrect":false,"inputTokens":3367,"outputTokens":2,"latencyMs":908.4811249999912},{"questionId":"q15","format":"csv","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3218,"outputTokens":2,"latencyMs":3147.3337090000277},{"questionId":"q15","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":9119,"outputTokens":2,"latencyMs":1795.425499999983},{"questionId":"q15","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":5768,"outputTokens":2,"latencyMs":920.6291250000068},{"questionId":"q16","format":"json-pretty","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":7942,"outputTokens":2,"latencyMs":6699.309834000014},{"questionId":"q16","format":"json-compact","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":4730,"outputTokens":2,"latencyMs":12223.663375000004},{"questionId":"q16","format":"toon","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":3375,"outputTokens":2,"latencyMs":12162.302208000008},{"questionId":"q16","format":"csv","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":3226,"outputTokens":2,"latencyMs":30495.031749999995},{"questionId":"q16","format":"xml","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":9127,"outputTokens":2,"latencyMs":12720.975792000012},{"questionId":"q16","format":"yaml","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":5776,"outputTokens":2,"latencyMs":11646.67962499999},{"questionId":"q17","format":"json-pretty","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":7942,"outputTokens":2,"latencyMs":7787.500249999983},{"questionId":"q17","format":"json-compact","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":4730,"outputTokens":2,"latencyMs":34565.85833399999},{"questionId":"q17","format":"toon","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":3375,"outputTokens":2,"latencyMs":21074.722833999986},{"questionId":"q17","format":"csv","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":3226,"outputTokens":2,"latencyMs":20867.770584000013},{"questionId":"q17","format":"xml","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":9127,"outputTokens":2,"latencyMs":11772.855041999981},{"questionId":"q17","format":"yaml","model":"gemini-2.5-flash","expected":"67","actual":"51","isCorrect":false,"inputTokens":5776,"outputTokens":2,"latencyMs":8996.542249999999},{"questionId":"q18","format":"json-pretty","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":7943,"outputTokens":2,"latencyMs":7348.134874999989},{"questionId":"q18","format":"json-compact","model":"gemini-2.5-flash","expected":"41","actual":"32","isCorrect":false,"inputTokens":4731,"outputTokens":2,"latencyMs":9815.51674999998},{"questionId":"q18","format":"toon","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":3376,"outputTokens":2,"latencyMs":22804.094583000027},{"questionId":"q18","format":"csv","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":3227,"outputTokens":2,"latencyMs":20857.020582999976},{"questionId":"q18","format":"xml","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":9128,"outputTokens":2,"latencyMs":12468.095125000022},{"questionId":"q18","format":"yaml","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":5777,"outputTokens":2,"latencyMs":11821.665582999995},{"questionId":"q19","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":7935,"outputTokens":3,"latencyMs":1552.6259169999976},{"questionId":"q19","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":4723,"outputTokens":3,"latencyMs":1150.8351249999832},{"questionId":"q19","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3368,"outputTokens":3,"latencyMs":720.2055420000106},{"questionId":"q19","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3219,"outputTokens":3,"latencyMs":1196.8069160000014},{"questionId":"q19","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":9120,"outputTokens":3,"latencyMs":1334.9700829999929},{"questionId":"q19","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":5769,"outputTokens":3,"latencyMs":1702.5840409999946},{"questionId":"q20","format":"json-pretty","model":"gemini-2.5-flash","expected":"96503","actual":"98774.77","isCorrect":false,"inputTokens":7936,"outputTokens":8,"latencyMs":6484.742041999998},{"questionId":"q20","format":"json-compact","model":"gemini-2.5-flash","expected":"96503","actual":"96503.32","isCorrect":false,"inputTokens":4724,"outputTokens":8,"latencyMs":11402.229708000028},{"questionId":"q20","format":"toon","model":"gemini-2.5-flash","expected":"96503","actual":"98844.83","isCorrect":false,"inputTokens":3369,"outputTokens":8,"latencyMs":6478.344332999986},{"questionId":"q20","format":"csv","model":"gemini-2.5-flash","expected":"96503","actual":"100416.95","isCorrect":false,"inputTokens":3220,"outputTokens":9,"latencyMs":7075.777916999999},{"questionId":"q20","format":"xml","model":"gemini-2.5-flash","expected":"96503","actual":"97063.42","isCorrect":false,"inputTokens":9121,"outputTokens":8,"latencyMs":3904.3729170000006},{"questionId":"q20","format":"yaml","model":"gemini-2.5-flash","expected":"96503","actual":"97707.96","isCorrect":false,"inputTokens":5770,"outputTokens":8,"latencyMs":1255.5749999999825},{"questionId":"q21","format":"json-pretty","model":"gemini-2.5-flash","expected":"78","actual":"78","isCorrect":true,"inputTokens":7933,"outputTokens":2,"latencyMs":8717.34358299998},{"questionId":"q21","format":"json-compact","model":"gemini-2.5-flash","expected":"78","actual":"78","isCorrect":true,"inputTokens":4721,"outputTokens":2,"latencyMs":7451.16545900004},{"questionId":"q21","format":"toon","model":"gemini-2.5-flash","expected":"78","actual":"78","isCorrect":true,"inputTokens":3366,"outputTokens":2,"latencyMs":7419.215541999991},{"questionId":"q21","format":"csv","model":"gemini-2.5-flash","expected":"78","actual":"73","isCorrect":false,"inputTokens":3217,"outputTokens":2,"latencyMs":7434.191875000019},{"questionId":"q21","format":"xml","model":"gemini-2.5-flash","expected":"78","actual":"78","isCorrect":true,"inputTokens":9118,"outputTokens":2,"latencyMs":7611.284375000047},{"questionId":"q21","format":"yaml","model":"gemini-2.5-flash","expected":"78","actual":"78","isCorrect":true,"inputTokens":5767,"outputTokens":2,"latencyMs":7932.289833999996},{"questionId":"q22","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":7933,"outputTokens":2,"latencyMs":2947.1856249999837},{"questionId":"q22","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4721,"outputTokens":2,"latencyMs":2994.8342079999857},{"questionId":"q22","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3366,"outputTokens":2,"latencyMs":8035.961165999994},{"questionId":"q22","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3217,"outputTokens":2,"latencyMs":2554.591167000006},{"questionId":"q22","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":9118,"outputTokens":2,"latencyMs":6640.620041999966},{"questionId":"q22","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5767,"outputTokens":2,"latencyMs":5278.786499999987},{"questionId":"q23","format":"json-pretty","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":7944,"outputTokens":2,"latencyMs":3777.826832999999},{"questionId":"q23","format":"json-compact","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":4732,"outputTokens":2,"latencyMs":5994.621375000046},{"questionId":"q23","format":"toon","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3377,"outputTokens":2,"latencyMs":18855.594333999965},{"questionId":"q23","format":"csv","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3228,"outputTokens":2,"latencyMs":20127.151542000007},{"questionId":"q23","format":"xml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":9129,"outputTokens":2,"latencyMs":13441.331000000006},{"questionId":"q23","format":"yaml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":5778,"outputTokens":2,"latencyMs":3963.014291999978},{"questionId":"q24","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7944,"outputTokens":2,"latencyMs":11444.20845799998},{"questionId":"q24","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4732,"outputTokens":2,"latencyMs":10925.54700000002},{"questionId":"q24","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3377,"outputTokens":2,"latencyMs":20148.858791999985},{"questionId":"q24","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3228,"outputTokens":2,"latencyMs":18313.159790999955},{"questionId":"q24","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9129,"outputTokens":2,"latencyMs":9863.166041999997},{"questionId":"q24","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5778,"outputTokens":2,"latencyMs":11953.787709000055},{"questionId":"q25","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7944,"outputTokens":2,"latencyMs":3091.559416000033},{"questionId":"q25","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4732,"outputTokens":2,"latencyMs":7484.88708299998},{"questionId":"q25","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3377,"outputTokens":2,"latencyMs":20345.743917000014},{"questionId":"q25","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3228,"outputTokens":2,"latencyMs":4942.241916999978},{"questionId":"q25","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9129,"outputTokens":2,"latencyMs":2934.06258300005},{"questionId":"q25","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5778,"outputTokens":2,"latencyMs":7035.912041999982},{"questionId":"q26","format":"json-pretty","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":7944,"outputTokens":2,"latencyMs":3585.4487090000184},{"questionId":"q26","format":"json-compact","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":4732,"outputTokens":2,"latencyMs":4897.935292000009},{"questionId":"q26","format":"toon","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3377,"outputTokens":2,"latencyMs":6487.981250000012},{"questionId":"q26","format":"csv","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3228,"outputTokens":2,"latencyMs":4266.286415999988},{"questionId":"q26","format":"xml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":9129,"outputTokens":2,"latencyMs":5545.42633300001},{"questionId":"q26","format":"yaml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":5778,"outputTokens":2,"latencyMs":3959.1063330000034},{"questionId":"q27","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7944,"outputTokens":2,"latencyMs":3170.1490840000333},{"questionId":"q27","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4732,"outputTokens":2,"latencyMs":3644.3847500000265},{"questionId":"q27","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3377,"outputTokens":2,"latencyMs":6269.288583000016},{"questionId":"q27","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3228,"outputTokens":2,"latencyMs":18181.298332999984},{"questionId":"q27","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9129,"outputTokens":2,"latencyMs":8128.769541000016},{"questionId":"q27","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5778,"outputTokens":2,"latencyMs":3106.8657080000266},{"questionId":"q28","format":"json-pretty","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":7940,"outputTokens":2,"latencyMs":9777.213958000008},{"questionId":"q28","format":"json-compact","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":4728,"outputTokens":2,"latencyMs":13901.586416999984},{"questionId":"q28","format":"toon","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":3373,"outputTokens":2,"latencyMs":24501.584124999994},{"questionId":"q28","format":"csv","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":3224,"outputTokens":2,"latencyMs":9195.251874999958},{"questionId":"q28","format":"xml","model":"gemini-2.5-flash","expected":"63","actual":"51","isCorrect":false,"inputTokens":9125,"outputTokens":2,"latencyMs":15374.286458000017},{"questionId":"q28","format":"yaml","model":"gemini-2.5-flash","expected":"63","actual":"62","isCorrect":false,"inputTokens":5774,"outputTokens":2,"latencyMs":9895.215457999962},{"questionId":"q29","format":"json-pretty","model":"gemini-2.5-flash","expected":"53","actual":"52","isCorrect":false,"inputTokens":7941,"outputTokens":2,"latencyMs":14667.157915999996},{"questionId":"q29","format":"json-compact","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":4729,"outputTokens":2,"latencyMs":26762.58929100004},{"questionId":"q29","format":"toon","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":3374,"outputTokens":2,"latencyMs":18114.293999999994},{"questionId":"q29","format":"csv","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":3225,"outputTokens":2,"latencyMs":10658.97870799998},{"questionId":"q29","format":"xml","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":9126,"outputTokens":2,"latencyMs":18207.187124999997},{"questionId":"q29","format":"yaml","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":5775,"outputTokens":2,"latencyMs":14167.809834000014},{"questionId":"q30","format":"json-pretty","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":7941,"outputTokens":2,"latencyMs":10284.415208999999},{"questionId":"q30","format":"json-compact","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":4729,"outputTokens":2,"latencyMs":8957.151916000003},{"questionId":"q30","format":"toon","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":3374,"outputTokens":2,"latencyMs":26708.699167000013},{"questionId":"q30","format":"csv","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":3225,"outputTokens":2,"latencyMs":17063.996915999975},{"questionId":"q30","format":"xml","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":9126,"outputTokens":2,"latencyMs":13823.362665999972},{"questionId":"q30","format":"yaml","model":"gemini-2.5-flash","expected":"39","actual":"36","isCorrect":false,"inputTokens":5775,"outputTokens":2,"latencyMs":1441.6552919999813},{"questionId":"q31","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7942,"outputTokens":2,"latencyMs":3174.6300420000334},{"questionId":"q31","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4730,"outputTokens":2,"latencyMs":3280.776165999996},{"questionId":"q31","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3375,"outputTokens":2,"latencyMs":6520.769374999974},{"questionId":"q31","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3226,"outputTokens":2,"latencyMs":14226.484125000017},{"questionId":"q31","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9127,"outputTokens":2,"latencyMs":3187.221499999985},{"questionId":"q31","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5776,"outputTokens":2,"latencyMs":6762.0557499999995},{"questionId":"q32","format":"json-pretty","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":7942,"outputTokens":1,"latencyMs":5678.026957999973},{"questionId":"q32","format":"json-compact","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":4730,"outputTokens":1,"latencyMs":10535.883249999955},{"questionId":"q32","format":"toon","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":3375,"outputTokens":1,"latencyMs":16208.797583999985},{"questionId":"q32","format":"csv","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":3226,"outputTokens":1,"latencyMs":4340.415333000012},{"questionId":"q32","format":"xml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":9127,"outputTokens":1,"latencyMs":8750.609624999983},{"questionId":"q32","format":"yaml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":5776,"outputTokens":1,"latencyMs":2937.6979160000337},{"questionId":"q33","format":"json-pretty","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":7942,"outputTokens":2,"latencyMs":2444.4974590000347},{"questionId":"q33","format":"json-compact","model":"gemini-2.5-flash","expected":"15","actual":"14","isCorrect":false,"inputTokens":4730,"outputTokens":2,"latencyMs":2481.1167920000153},{"questionId":"q33","format":"toon","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":3375,"outputTokens":2,"latencyMs":20550.77204099996},{"questionId":"q33","format":"csv","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":3226,"outputTokens":2,"latencyMs":4036.605209000001},{"questionId":"q33","format":"xml","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":9127,"outputTokens":2,"latencyMs":5092.871207999997},{"questionId":"q33","format":"yaml","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":5776,"outputTokens":2,"latencyMs":5412.286916000012},{"questionId":"q34","format":"json-pretty","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":7935,"outputTokens":2,"latencyMs":7722.417749999964},{"questionId":"q34","format":"json-compact","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":4723,"outputTokens":2,"latencyMs":2283.7882079999545},{"questionId":"q34","format":"toon","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3368,"outputTokens":2,"latencyMs":8765.946875000023},{"questionId":"q34","format":"csv","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3219,"outputTokens":2,"latencyMs":3853.5071669999743},{"questionId":"q34","format":"xml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":9120,"outputTokens":2,"latencyMs":5799.792083999957},{"questionId":"q34","format":"yaml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":5769,"outputTokens":2,"latencyMs":2828.8265419999952},{"questionId":"q35","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7935,"outputTokens":2,"latencyMs":7051.2432499999995},{"questionId":"q35","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4723,"outputTokens":2,"latencyMs":6523.064624999999},{"questionId":"q35","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3368,"outputTokens":2,"latencyMs":16626.74254200002},{"questionId":"q35","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3219,"outputTokens":2,"latencyMs":7578.612000000023},{"questionId":"q35","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9120,"outputTokens":2,"latencyMs":9761.287207999965},{"questionId":"q35","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5769,"outputTokens":2,"latencyMs":10367.98099999997},{"questionId":"q36","format":"json-pretty","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":13820,"outputTokens":6,"latencyMs":815.4214999999967},{"questionId":"q36","format":"json-compact","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8462,"outputTokens":6,"latencyMs":622.0727920000209},{"questionId":"q36","format":"toon","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":9080,"outputTokens":6,"latencyMs":820.865374999994},{"questionId":"q36","format":"xml","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":15208,"outputTokens":6,"latencyMs":934.5148330000229},{"questionId":"q36","format":"yaml","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":10073,"outputTokens":6,"latencyMs":1449.8975829999545},{"questionId":"q37","format":"json-pretty","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":13820,"outputTokens":2,"latencyMs":1144.4068750000442},{"questionId":"q37","format":"json-compact","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8462,"outputTokens":2,"latencyMs":2035.6813750000438},{"questionId":"q37","format":"toon","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":9080,"outputTokens":2,"latencyMs":1208.724291999999},{"questionId":"q37","format":"xml","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":15208,"outputTokens":2,"latencyMs":2654.840790999995},{"questionId":"q37","format":"yaml","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":10073,"outputTokens":2,"latencyMs":1333.1426249999786},{"questionId":"q38","format":"json-pretty","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":13820,"outputTokens":5,"latencyMs":795.5492499999818},{"questionId":"q38","format":"json-compact","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8462,"outputTokens":5,"latencyMs":999.8318329999456},{"questionId":"q38","format":"toon","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":9080,"outputTokens":5,"latencyMs":1066.8681670000078},{"questionId":"q38","format":"xml","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":15208,"outputTokens":5,"latencyMs":1233.380541999999},{"questionId":"q38","format":"yaml","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":10073,"outputTokens":5,"latencyMs":2250.163459000003},{"questionId":"q39","format":"json-pretty","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":13820,"outputTokens":1,"latencyMs":1779.981792000006},{"questionId":"q39","format":"json-compact","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8462,"outputTokens":1,"latencyMs":1373.357750000083},{"questionId":"q39","format":"toon","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":9080,"outputTokens":1,"latencyMs":1363.2912910000887},{"questionId":"q39","format":"xml","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":15208,"outputTokens":1,"latencyMs":3187.0674160000635},{"questionId":"q39","format":"yaml","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":10073,"outputTokens":1,"latencyMs":3345.267041999963},{"questionId":"q40","format":"json-pretty","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":13820,"outputTokens":7,"latencyMs":2599.3716669999994},{"questionId":"q40","format":"json-compact","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8462,"outputTokens":7,"latencyMs":2719.128374999971},{"questionId":"q40","format":"toon","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":9080,"outputTokens":7,"latencyMs":1029.6424170000246},{"questionId":"q40","format":"xml","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":15208,"outputTokens":7,"latencyMs":3222.0798330000835},{"questionId":"q40","format":"yaml","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":10073,"outputTokens":7,"latencyMs":2971.143584000063},{"questionId":"q41","format":"json-pretty","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":13820,"outputTokens":1,"latencyMs":1305.46866699995},{"questionId":"q41","format":"json-compact","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8462,"outputTokens":1,"latencyMs":1838.065750000067},{"questionId":"q41","format":"toon","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":9080,"outputTokens":1,"latencyMs":2252.3830000000307},{"questionId":"q41","format":"xml","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":15208,"outputTokens":1,"latencyMs":2241.6963329999708},{"questionId":"q41","format":"yaml","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":10073,"outputTokens":1,"latencyMs":981.8932920000516},{"questionId":"q42","format":"json-pretty","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":13820,"outputTokens":7,"latencyMs":1312.829792000004},{"questionId":"q42","format":"json-compact","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8462,"outputTokens":7,"latencyMs":916.0978749999776},{"questionId":"q42","format":"toon","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":9080,"outputTokens":7,"latencyMs":770.876707999967},{"questionId":"q42","format":"xml","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":15208,"outputTokens":7,"latencyMs":1216.4797079999698},{"questionId":"q42","format":"yaml","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":10073,"outputTokens":7,"latencyMs":1619.8707920000888},{"questionId":"q43","format":"json-pretty","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13820,"outputTokens":1,"latencyMs":2104.589500000002},{"questionId":"q43","format":"json-compact","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8462,"outputTokens":1,"latencyMs":1463.918917000061},{"questionId":"q43","format":"toon","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9080,"outputTokens":1,"latencyMs":1824.6936249999562},{"questionId":"q43","format":"xml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":15208,"outputTokens":1,"latencyMs":1250.2023749999935},{"questionId":"q43","format":"yaml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10073,"outputTokens":1,"latencyMs":2023.1199580000248},{"questionId":"q44","format":"json-pretty","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":13821,"outputTokens":5,"latencyMs":1631.9162500000093},{"questionId":"q44","format":"json-compact","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8463,"outputTokens":5,"latencyMs":1441.6492079999298},{"questionId":"q44","format":"toon","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":9081,"outputTokens":5,"latencyMs":1685.2262500000652},{"questionId":"q44","format":"xml","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":15209,"outputTokens":5,"latencyMs":1996.9154580000322},{"questionId":"q44","format":"yaml","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":10074,"outputTokens":5,"latencyMs":1368.9887919999892},{"questionId":"q45","format":"json-pretty","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":13821,"outputTokens":13,"latencyMs":1137.5090420000488},{"questionId":"q45","format":"json-compact","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8463,"outputTokens":13,"latencyMs":1192.4489580000518},{"questionId":"q45","format":"toon","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":9081,"outputTokens":13,"latencyMs":1157.3325000000186},{"questionId":"q45","format":"xml","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":15209,"outputTokens":13,"latencyMs":1731.6961669999873},{"questionId":"q45","format":"yaml","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":10074,"outputTokens":13,"latencyMs":1598.575959000038},{"questionId":"q46","format":"json-pretty","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":13821,"outputTokens":10,"latencyMs":1133.9154160000617},{"questionId":"q46","format":"json-compact","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8463,"outputTokens":10,"latencyMs":942.4959169999929},{"questionId":"q46","format":"toon","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":9081,"outputTokens":10,"latencyMs":1240.8249580000993},{"questionId":"q46","format":"xml","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":15209,"outputTokens":10,"latencyMs":925.7205420000246},{"questionId":"q46","format":"yaml","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":10074,"outputTokens":10,"latencyMs":903.28125},{"questionId":"q47","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":13820,"outputTokens":1,"latencyMs":2520.162000000011},{"questionId":"q47","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8462,"outputTokens":1,"latencyMs":3669.8919999999925},{"questionId":"q47","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9080,"outputTokens":1,"latencyMs":2297.32820800005},{"questionId":"q47","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":15208,"outputTokens":1,"latencyMs":2497.794666999951},{"questionId":"q47","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":10073,"outputTokens":1,"latencyMs":2109.137875000015},{"questionId":"q48","format":"json-pretty","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":13821,"outputTokens":4,"latencyMs":1603.156290999963},{"questionId":"q48","format":"json-compact","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8463,"outputTokens":4,"latencyMs":3629.2671669999836},{"questionId":"q48","format":"toon","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":9081,"outputTokens":4,"latencyMs":1788.1930410000496},{"questionId":"q48","format":"xml","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":15209,"outputTokens":4,"latencyMs":1369.5576669999864},{"questionId":"q48","format":"yaml","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":10074,"outputTokens":4,"latencyMs":1006.8878750000149},{"questionId":"q49","format":"json-pretty","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":13821,"outputTokens":10,"latencyMs":2547.933667000034},{"questionId":"q49","format":"json-compact","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8463,"outputTokens":10,"latencyMs":1743.4449170000153},{"questionId":"q49","format":"toon","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":9081,"outputTokens":10,"latencyMs":2186.7593750000233},{"questionId":"q49","format":"xml","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":15209,"outputTokens":10,"latencyMs":1187.7785420001019},{"questionId":"q49","format":"yaml","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":10074,"outputTokens":10,"latencyMs":1683.6239999999525},{"questionId":"q50","format":"json-pretty","model":"gemini-2.5-flash","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":13821,"outputTokens":10,"latencyMs":1721.3944170000032},{"questionId":"q50","format":"json-compact","model":"gemini-2.5-flash","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":8463,"outputTokens":10,"latencyMs":1562.199459000025},{"questionId":"q50","format":"toon","model":"gemini-2.5-flash","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":9081,"outputTokens":10,"latencyMs":1105.3498750000726},{"questionId":"q50","format":"xml","model":"gemini-2.5-flash","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":15209,"outputTokens":10,"latencyMs":1500.1487500000512},{"questionId":"q50","format":"yaml","model":"gemini-2.5-flash","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":10074,"outputTokens":10,"latencyMs":942.3254159999778},{"questionId":"q51","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":13820,"outputTokens":1,"latencyMs":898.7797920000739},{"questionId":"q51","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":8462,"outputTokens":1,"latencyMs":1900.8208330000052},{"questionId":"q51","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":9080,"outputTokens":1,"latencyMs":1916.4359580000164},{"questionId":"q51","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":15208,"outputTokens":1,"latencyMs":2384.3469999999506},{"questionId":"q51","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":10073,"outputTokens":1,"latencyMs":2070.591708000051},{"questionId":"q52","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":13815,"outputTokens":2,"latencyMs":3981.7847920000786},{"questionId":"q52","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":8457,"outputTokens":2,"latencyMs":1068.609875000082},{"questionId":"q52","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9075,"outputTokens":2,"latencyMs":4271.996707999962},{"questionId":"q52","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":15203,"outputTokens":2,"latencyMs":957.0320409999695},{"questionId":"q52","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":10068,"outputTokens":2,"latencyMs":3856.816332999966},{"questionId":"q53","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":13815,"outputTokens":2,"latencyMs":1777.1473329999717},{"questionId":"q53","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":8457,"outputTokens":2,"latencyMs":1940.5147080000024},{"questionId":"q53","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9075,"outputTokens":2,"latencyMs":2952.5018329999875},{"questionId":"q53","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":15203,"outputTokens":2,"latencyMs":4358.311832999927},{"questionId":"q53","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":10068,"outputTokens":2,"latencyMs":4204.48758299998},{"questionId":"q54","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":13816,"outputTokens":2,"latencyMs":1617.643374999985},{"questionId":"q54","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":8458,"outputTokens":2,"latencyMs":3751.965375000029},{"questionId":"q54","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9076,"outputTokens":2,"latencyMs":7950.504417000106},{"questionId":"q54","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":15204,"outputTokens":2,"latencyMs":1697.780708000064},{"questionId":"q54","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":10069,"outputTokens":2,"latencyMs":4053.4824159999844},{"questionId":"q55","format":"json-pretty","model":"gemini-2.5-flash","expected":"34904.81","actual":"34164.29","isCorrect":false,"inputTokens":13816,"outputTokens":8,"latencyMs":6604.489624999929},{"questionId":"q55","format":"json-compact","model":"gemini-2.5-flash","expected":"34904.81","actual":"34503.77","isCorrect":false,"inputTokens":8458,"outputTokens":8,"latencyMs":7712.289875000017},{"questionId":"q55","format":"toon","model":"gemini-2.5-flash","expected":"34904.81","actual":"34904.81","isCorrect":true,"inputTokens":9076,"outputTokens":8,"latencyMs":12020.321541999932},{"questionId":"q55","format":"xml","model":"gemini-2.5-flash","expected":"34904.81","actual":"33816.03","isCorrect":false,"inputTokens":15204,"outputTokens":8,"latencyMs":10766.89387499995},{"questionId":"q55","format":"yaml","model":"gemini-2.5-flash","expected":"34904.81","actual":"32662.63","isCorrect":false,"inputTokens":10069,"outputTokens":8,"latencyMs":5277.383291999926},{"questionId":"q56","format":"json-pretty","model":"gemini-2.5-flash","expected":"698.10","actual":"690.09","isCorrect":false,"inputTokens":13814,"outputTokens":6,"latencyMs":6390.6391669999575},{"questionId":"q56","format":"json-compact","model":"gemini-2.5-flash","expected":"698.10","actual":"690.07","isCorrect":false,"inputTokens":8456,"outputTokens":6,"latencyMs":5243.891291999957},{"questionId":"q56","format":"toon","model":"gemini-2.5-flash","expected":"698.10","actual":"700.56","isCorrect":false,"inputTokens":9074,"outputTokens":6,"latencyMs":5147.067500000005},{"questionId":"q56","format":"xml","model":"gemini-2.5-flash","expected":"698.10","actual":"716.95","isCorrect":false,"inputTokens":15202,"outputTokens":6,"latencyMs":6168.09291700006},{"questionId":"q56","format":"yaml","model":"gemini-2.5-flash","expected":"698.10","actual":"717.75","isCorrect":false,"inputTokens":10067,"outputTokens":6,"latencyMs":5408.747457999969},{"questionId":"q57","format":"json-pretty","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":13815,"outputTokens":2,"latencyMs":2226.9870420000516},{"questionId":"q57","format":"json-compact","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":8457,"outputTokens":2,"latencyMs":2529.5987500000047},{"questionId":"q57","format":"toon","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":9075,"outputTokens":2,"latencyMs":1754.7252089999383},{"questionId":"q57","format":"xml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":15203,"outputTokens":2,"latencyMs":3065.6008750000037},{"questionId":"q57","format":"yaml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":10068,"outputTokens":2,"latencyMs":3902.0060829999857},{"questionId":"q58","format":"json-pretty","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":13814,"outputTokens":7,"latencyMs":5868.485749999993},{"questionId":"q58","format":"json-compact","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8456,"outputTokens":7,"latencyMs":6146.208875000011},{"questionId":"q58","format":"toon","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":9074,"outputTokens":7,"latencyMs":4994.1254999999655},{"questionId":"q58","format":"xml","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":15202,"outputTokens":7,"latencyMs":5159.582500000019},{"questionId":"q58","format":"yaml","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":10067,"outputTokens":7,"latencyMs":1334.4466669999529},{"questionId":"q59","format":"json-pretty","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":13820,"outputTokens":2,"latencyMs":7055.729875000077},{"questionId":"q59","format":"json-compact","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":8462,"outputTokens":2,"latencyMs":6951.658458000049},{"questionId":"q59","format":"toon","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":9080,"outputTokens":2,"latencyMs":8721.288458999945},{"questionId":"q59","format":"xml","model":"gemini-2.5-flash","expected":"43","actual":"46","isCorrect":false,"inputTokens":15208,"outputTokens":2,"latencyMs":1506.6962920000078},{"questionId":"q59","format":"yaml","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":10073,"outputTokens":2,"latencyMs":8111.610249999911},{"questionId":"q60","format":"json-pretty","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":13820,"outputTokens":2,"latencyMs":6377.179708999931},{"questionId":"q60","format":"json-compact","model":"gemini-2.5-flash","expected":"37","actual":"32","isCorrect":false,"inputTokens":8462,"outputTokens":2,"latencyMs":5946.993207999971},{"questionId":"q60","format":"toon","model":"gemini-2.5-flash","expected":"37","actual":"32","isCorrect":false,"inputTokens":9080,"outputTokens":2,"latencyMs":6766.23758299998},{"questionId":"q60","format":"xml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":15208,"outputTokens":2,"latencyMs":7115.59149999998},{"questionId":"q60","format":"yaml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":10073,"outputTokens":2,"latencyMs":7670.933415999985},{"questionId":"q61","format":"json-pretty","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":13820,"outputTokens":2,"latencyMs":6518.408749999944},{"questionId":"q61","format":"json-compact","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":8462,"outputTokens":2,"latencyMs":6657.696125000017},{"questionId":"q61","format":"toon","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":9080,"outputTokens":2,"latencyMs":7043.327458999935},{"questionId":"q61","format":"xml","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":15208,"outputTokens":2,"latencyMs":7240.095583999995},{"questionId":"q61","format":"yaml","model":"gemini-2.5-flash","expected":"28","actual":"30","isCorrect":false,"inputTokens":10073,"outputTokens":2,"latencyMs":862.8539170000004},{"questionId":"q62","format":"json-pretty","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":13824,"outputTokens":1,"latencyMs":5657.20179099997},{"questionId":"q62","format":"json-compact","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":8466,"outputTokens":1,"latencyMs":5021.635375000071},{"questionId":"q62","format":"toon","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":9084,"outputTokens":1,"latencyMs":7374.494624999934},{"questionId":"q62","format":"xml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":15212,"outputTokens":1,"latencyMs":6728.89283399994},{"questionId":"q62","format":"yaml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":10077,"outputTokens":1,"latencyMs":4183.434583000024},{"questionId":"q63","format":"json-pretty","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":13824,"outputTokens":1,"latencyMs":6732.950040999916},{"questionId":"q63","format":"json-compact","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":8466,"outputTokens":1,"latencyMs":5283.612165999948},{"questionId":"q63","format":"toon","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":9084,"outputTokens":1,"latencyMs":6928.49229100009},{"questionId":"q63","format":"xml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":15212,"outputTokens":1,"latencyMs":7497.820084000006},{"questionId":"q63","format":"yaml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":10077,"outputTokens":1,"latencyMs":6564.30462499999},{"questionId":"q64","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":13825,"outputTokens":2,"latencyMs":4867.906916000065},{"questionId":"q64","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":8467,"outputTokens":2,"latencyMs":6219.108541000052},{"questionId":"q64","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9085,"outputTokens":2,"latencyMs":6718.410832999973},{"questionId":"q64","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":15213,"outputTokens":2,"latencyMs":7313.5782090000575},{"questionId":"q64","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":10078,"outputTokens":2,"latencyMs":6688.668999999994},{"questionId":"q65","format":"json-pretty","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":13824,"outputTokens":1,"latencyMs":6911.2190420000115},{"questionId":"q65","format":"json-compact","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":8466,"outputTokens":1,"latencyMs":6382.689665999962},{"questionId":"q65","format":"toon","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":9084,"outputTokens":1,"latencyMs":8500.886583000072},{"questionId":"q65","format":"xml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":15212,"outputTokens":1,"latencyMs":6628.892166000092},{"questionId":"q65","format":"yaml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":10077,"outputTokens":1,"latencyMs":6511.817916000029},{"questionId":"q66","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":13822,"outputTokens":1,"latencyMs":5523.528667000006},{"questionId":"q66","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8464,"outputTokens":1,"latencyMs":5269.448041999945},{"questionId":"q66","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9082,"outputTokens":1,"latencyMs":8937.705042000045},{"questionId":"q66","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":15210,"outputTokens":1,"latencyMs":6981.282999999938},{"questionId":"q66","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":10075,"outputTokens":1,"latencyMs":2075.2573340000818},{"questionId":"q67","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":13822,"outputTokens":1,"latencyMs":5845.610250000027},{"questionId":"q67","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8464,"outputTokens":1,"latencyMs":5101.5918749999255},{"questionId":"q67","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9082,"outputTokens":1,"latencyMs":8682.392708000029},{"questionId":"q67","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":15210,"outputTokens":1,"latencyMs":6556.906958000036},{"questionId":"q67","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":10075,"outputTokens":1,"latencyMs":5522.142749999999},{"questionId":"q68","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":13823,"outputTokens":1,"latencyMs":6619.074915999896},{"questionId":"q68","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":8465,"outputTokens":1,"latencyMs":4986.16174999997},{"questionId":"q68","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":9083,"outputTokens":1,"latencyMs":8596.609582999954},{"questionId":"q68","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":15211,"outputTokens":1,"latencyMs":5516.122542000026},{"questionId":"q68","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":10076,"outputTokens":1,"latencyMs":4959.057666000095},{"questionId":"q69","format":"json-pretty","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":13826,"outputTokens":2,"latencyMs":8746.21187499992},{"questionId":"q69","format":"json-compact","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":8468,"outputTokens":2,"latencyMs":9277.466958999983},{"questionId":"q69","format":"toon","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":9086,"outputTokens":2,"latencyMs":13220.56008299999},{"questionId":"q69","format":"xml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":15214,"outputTokens":2,"latencyMs":12523.984916000045},{"questionId":"q69","format":"yaml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":10079,"outputTokens":2,"latencyMs":12558.967292000074},{"questionId":"q70","format":"json-pretty","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":13826,"outputTokens":2,"latencyMs":8663.695250000106},{"questionId":"q70","format":"json-compact","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":8468,"outputTokens":2,"latencyMs":9034.074750000029},{"questionId":"q70","format":"toon","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":9086,"outputTokens":2,"latencyMs":12445.53362500004},{"questionId":"q70","format":"xml","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":15214,"outputTokens":2,"latencyMs":11390.757332999958},{"questionId":"q70","format":"yaml","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":10079,"outputTokens":2,"latencyMs":7945.0461249999935},{"questionId":"q71","format":"json-pretty","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4794,"outputTokens":4,"latencyMs":1101.8675419999054},{"questionId":"q71","format":"json-compact","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3042,"outputTokens":4,"latencyMs":853.0512500000186},{"questionId":"q71","format":"toon","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2305,"outputTokens":4,"latencyMs":1713.6020410000347},{"questionId":"q71","format":"csv","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2219,"outputTokens":4,"latencyMs":1572.8806249999907},{"questionId":"q71","format":"xml","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":5437,"outputTokens":4,"latencyMs":1101.8969999999972},{"questionId":"q71","format":"yaml","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3823,"outputTokens":4,"latencyMs":1370.4896670000162},{"questionId":"q72","format":"json-pretty","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4794,"outputTokens":8,"latencyMs":1766.2064580000006},{"questionId":"q72","format":"json-compact","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3042,"outputTokens":8,"latencyMs":1488.0073750000447},{"questionId":"q72","format":"toon","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2305,"outputTokens":8,"latencyMs":2421.3104590000585},{"questionId":"q72","format":"csv","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2219,"outputTokens":8,"latencyMs":1665.9193330000853},{"questionId":"q72","format":"xml","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":5437,"outputTokens":8,"latencyMs":1045.875457999995},{"questionId":"q72","format":"yaml","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3823,"outputTokens":8,"latencyMs":1187.8712090000045},{"questionId":"q73","format":"json-pretty","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4795,"outputTokens":4,"latencyMs":1978.9810409999918},{"questionId":"q73","format":"json-compact","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3043,"outputTokens":4,"latencyMs":1853.4133329999167},{"questionId":"q73","format":"toon","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2306,"outputTokens":4,"latencyMs":1835.8217079999158},{"questionId":"q73","format":"csv","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2220,"outputTokens":4,"latencyMs":2342.813084000023},{"questionId":"q73","format":"xml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":5438,"outputTokens":4,"latencyMs":2666.0796669999836},{"questionId":"q73","format":"yaml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3824,"outputTokens":4,"latencyMs":2087.578749999986},{"questionId":"q74","format":"json-pretty","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":4795,"outputTokens":2,"latencyMs":1439.0207080000546},{"questionId":"q74","format":"json-compact","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3043,"outputTokens":2,"latencyMs":1511.642165999976},{"questionId":"q74","format":"toon","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2306,"outputTokens":2,"latencyMs":2226.9555420000106},{"questionId":"q74","format":"csv","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2220,"outputTokens":2,"latencyMs":1534.452540999977},{"questionId":"q74","format":"xml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":5438,"outputTokens":2,"latencyMs":3028.6882079999195},{"questionId":"q74","format":"yaml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3824,"outputTokens":2,"latencyMs":2781.0632079999195},{"questionId":"q75","format":"json-pretty","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4794,"outputTokens":4,"latencyMs":1744.635833999957},{"questionId":"q75","format":"json-compact","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3042,"outputTokens":4,"latencyMs":2465.6241669999436},{"questionId":"q75","format":"toon","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2305,"outputTokens":4,"latencyMs":1470.0111670000479},{"questionId":"q75","format":"csv","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2219,"outputTokens":4,"latencyMs":1710.8115830000024},{"questionId":"q75","format":"xml","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":5437,"outputTokens":4,"latencyMs":1434},{"questionId":"q75","format":"yaml","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3823,"outputTokens":4,"latencyMs":936.0790829999605},{"questionId":"q76","format":"json-pretty","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4794,"outputTokens":6,"latencyMs":939.5597080000443},{"questionId":"q76","format":"json-compact","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3042,"outputTokens":6,"latencyMs":926.0800840000156},{"questionId":"q76","format":"toon","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2305,"outputTokens":6,"latencyMs":1528.9299999999348},{"questionId":"q76","format":"csv","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2219,"outputTokens":6,"latencyMs":1778.4568749999162},{"questionId":"q76","format":"xml","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":5437,"outputTokens":6,"latencyMs":2679.2514170000795},{"questionId":"q76","format":"yaml","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3823,"outputTokens":6,"latencyMs":1804.234749999945},{"questionId":"q77","format":"json-pretty","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4795,"outputTokens":4,"latencyMs":976.7687910000095},{"questionId":"q77","format":"json-compact","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3043,"outputTokens":4,"latencyMs":1624.5794999999925},{"questionId":"q77","format":"toon","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2306,"outputTokens":4,"latencyMs":2053.7377080000006},{"questionId":"q77","format":"csv","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2220,"outputTokens":4,"latencyMs":1512.628875000053},{"questionId":"q77","format":"xml","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":5438,"outputTokens":4,"latencyMs":2267.344374999986},{"questionId":"q77","format":"yaml","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3824,"outputTokens":4,"latencyMs":1777.6101250000065},{"questionId":"q78","format":"json-pretty","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":4795,"outputTokens":2,"latencyMs":2256.0689160000766},{"questionId":"q78","format":"json-compact","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":3043,"outputTokens":2,"latencyMs":1668.527834000066},{"questionId":"q78","format":"toon","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":2306,"outputTokens":2,"latencyMs":2199.315874999971},{"questionId":"q78","format":"csv","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":2220,"outputTokens":2,"latencyMs":2015.2797919999575},{"questionId":"q78","format":"xml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":5438,"outputTokens":2,"latencyMs":2091.531707999995},{"questionId":"q78","format":"yaml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":3824,"outputTokens":2,"latencyMs":1665.2967909999425},{"questionId":"q79","format":"json-pretty","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4794,"outputTokens":4,"latencyMs":1844.547750000027},{"questionId":"q79","format":"json-compact","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3042,"outputTokens":4,"latencyMs":1618.6494580000872},{"questionId":"q79","format":"toon","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2305,"outputTokens":4,"latencyMs":1702.673666000017},{"questionId":"q79","format":"csv","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2219,"outputTokens":4,"latencyMs":1774.4572499999776},{"questionId":"q79","format":"xml","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":5437,"outputTokens":4,"latencyMs":2189.773125000065},{"questionId":"q79","format":"yaml","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3823,"outputTokens":4,"latencyMs":1763.8889999999665},{"questionId":"q80","format":"json-pretty","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":4787,"outputTokens":2,"latencyMs":5315.679374999949},{"questionId":"q80","format":"json-compact","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3035,"outputTokens":2,"latencyMs":1975.0052920000162},{"questionId":"q80","format":"toon","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2298,"outputTokens":2,"latencyMs":10189.952415999956},{"questionId":"q80","format":"csv","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2212,"outputTokens":2,"latencyMs":1827.8344580000266},{"questionId":"q80","format":"xml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":5430,"outputTokens":2,"latencyMs":3697.638666999992},{"questionId":"q80","format":"yaml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3816,"outputTokens":2,"latencyMs":1110.1782080000266},{"questionId":"q81","format":"json-pretty","model":"gemini-2.5-flash","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":4788,"outputTokens":6,"latencyMs":6776.055083000101},{"questionId":"q81","format":"json-compact","model":"gemini-2.5-flash","expected":"328320","actual":"384592","isCorrect":false,"inputTokens":3036,"outputTokens":6,"latencyMs":5078.634749999968},{"questionId":"q81","format":"toon","model":"gemini-2.5-flash","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":2299,"outputTokens":6,"latencyMs":4330.306833000039},{"questionId":"q81","format":"csv","model":"gemini-2.5-flash","expected":"328320","actual":"328813","isCorrect":false,"inputTokens":2213,"outputTokens":6,"latencyMs":4490.076209000079},{"questionId":"q81","format":"xml","model":"gemini-2.5-flash","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":5431,"outputTokens":6,"latencyMs":6545.347792000044},{"questionId":"q81","format":"yaml","model":"gemini-2.5-flash","expected":"328320","actual":"333068","isCorrect":false,"inputTokens":3817,"outputTokens":6,"latencyMs":5087.304250000045},{"questionId":"q82","format":"json-pretty","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":4788,"outputTokens":4,"latencyMs":5225.209416999947},{"questionId":"q82","format":"json-compact","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":3036,"outputTokens":4,"latencyMs":16061.115874999901},{"questionId":"q82","format":"toon","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":2299,"outputTokens":4,"latencyMs":5231.7384160000365},{"questionId":"q82","format":"csv","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":2213,"outputTokens":4,"latencyMs":5526.608083000174},{"questionId":"q82","format":"xml","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":5431,"outputTokens":4,"latencyMs":5976.9340829998255},{"questionId":"q82","format":"yaml","model":"gemini-2.5-flash","expected":"1791","actual":"1939","isCorrect":false,"inputTokens":3817,"outputTokens":4,"latencyMs":1738.1159169999883},{"questionId":"q83","format":"json-pretty","model":"gemini-2.5-flash","expected":"311695.88","actual":"306694.88","isCorrect":false,"inputTokens":4786,"outputTokens":9,"latencyMs":10304.099625000032},{"questionId":"q83","format":"json-compact","model":"gemini-2.5-flash","expected":"311695.88","actual":"377017.34","isCorrect":false,"inputTokens":3034,"outputTokens":9,"latencyMs":9507.895291999914},{"questionId":"q83","format":"toon","model":"gemini-2.5-flash","expected":"311695.88","actual":"303746.47","isCorrect":false,"inputTokens":2297,"outputTokens":9,"latencyMs":7738.845709000016},{"questionId":"q83","format":"csv","model":"gemini-2.5-flash","expected":"311695.88","actual":"342202.93","isCorrect":false,"inputTokens":2211,"outputTokens":9,"latencyMs":6813.100875000004},{"questionId":"q83","format":"xml","model":"gemini-2.5-flash","expected":"311695.88","actual":"338166.75","isCorrect":false,"inputTokens":5429,"outputTokens":9,"latencyMs":10666.665041999891},{"questionId":"q83","format":"yaml","model":"gemini-2.5-flash","expected":"311695.88","actual":"301490.73","isCorrect":false,"inputTokens":3815,"outputTokens":9,"latencyMs":7287.290541999973},{"questionId":"q84","format":"json-pretty","model":"gemini-2.5-flash","expected":"0.53","actual":"0.5297","isCorrect":true,"inputTokens":4784,"outputTokens":6,"latencyMs":21744.219833999872},{"questionId":"q84","format":"json-compact","model":"gemini-2.5-flash","expected":"0.53","actual":"0.528","isCorrect":true,"inputTokens":3032,"outputTokens":5,"latencyMs":10048.218040999956},{"questionId":"q84","format":"toon","model":"gemini-2.5-flash","expected":"0.53","actual":"0.511","isCorrect":false,"inputTokens":2295,"outputTokens":5,"latencyMs":5817.3007080000825},{"questionId":"q84","format":"csv","model":"gemini-2.5-flash","expected":"0.53","actual":"0.5388","isCorrect":false,"inputTokens":2209,"outputTokens":6,"latencyMs":4071.8981659999117},{"questionId":"q84","format":"xml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.47","isCorrect":false,"inputTokens":5427,"outputTokens":4,"latencyMs":3877.2279579997994},{"questionId":"q84","format":"yaml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.566","isCorrect":false,"inputTokens":3813,"outputTokens":5,"latencyMs":4401.772082999814},{"questionId":"q85","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4790,"outputTokens":2,"latencyMs":7841.177082999842},{"questionId":"q85","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3038,"outputTokens":2,"latencyMs":6594.83591699996},{"questionId":"q85","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2301,"outputTokens":2,"latencyMs":14989.136167000048},{"questionId":"q85","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2215,"outputTokens":2,"latencyMs":22286.44337500003},{"questionId":"q85","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5433,"outputTokens":2,"latencyMs":8889.93366599991},{"questionId":"q85","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3819,"outputTokens":2,"latencyMs":4296.61400000006},{"questionId":"q86","format":"json-pretty","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":4788,"outputTokens":2,"latencyMs":7879.6445000001695},{"questionId":"q86","format":"json-compact","model":"gemini-2.5-flash","expected":"42","actual":"38","isCorrect":false,"inputTokens":3036,"outputTokens":2,"latencyMs":14905.026874999981},{"questionId":"q86","format":"toon","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":2299,"outputTokens":2,"latencyMs":14178.267125000013},{"questionId":"q86","format":"csv","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":2213,"outputTokens":2,"latencyMs":7867.299750000006},{"questionId":"q86","format":"xml","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":5431,"outputTokens":2,"latencyMs":7064.391459000064},{"questionId":"q86","format":"yaml","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":3817,"outputTokens":2,"latencyMs":8687.054207999958},{"questionId":"q87","format":"json-pretty","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":4797,"outputTokens":2,"latencyMs":12501.875249999808},{"questionId":"q87","format":"json-compact","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3045,"outputTokens":2,"latencyMs":8787.602416999871},{"questionId":"q87","format":"toon","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2308,"outputTokens":2,"latencyMs":14900.868499999866},{"questionId":"q87","format":"csv","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2222,"outputTokens":2,"latencyMs":17227.210250000004},{"questionId":"q87","format":"xml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":5440,"outputTokens":2,"latencyMs":10448.573667000048},{"questionId":"q87","format":"yaml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3826,"outputTokens":2,"latencyMs":11173.918750000186},{"questionId":"q88","format":"json-pretty","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":4797,"outputTokens":2,"latencyMs":12390.112124999985},{"questionId":"q88","format":"json-compact","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":3045,"outputTokens":2,"latencyMs":19404.431333000073},{"questionId":"q88","format":"toon","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":2308,"outputTokens":2,"latencyMs":16014.096666000085},{"questionId":"q88","format":"csv","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":2222,"outputTokens":2,"latencyMs":17551.891541999998},{"questionId":"q88","format":"xml","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":5440,"outputTokens":2,"latencyMs":10584.149915999966},{"questionId":"q88","format":"yaml","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":3826,"outputTokens":2,"latencyMs":6875.552832999965},{"questionId":"q89","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4798,"outputTokens":2,"latencyMs":7751.852916999953},{"questionId":"q89","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3046,"outputTokens":2,"latencyMs":22503.38000000012},{"questionId":"q89","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2309,"outputTokens":2,"latencyMs":17454.401416999986},{"questionId":"q89","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2223,"outputTokens":2,"latencyMs":14442.165999999968},{"questionId":"q89","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5441,"outputTokens":2,"latencyMs":24619.457124999957},{"questionId":"q89","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3827,"outputTokens":2,"latencyMs":10507.275249999948},{"questionId":"q90","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4798,"outputTokens":2,"latencyMs":20256.532916999888},{"questionId":"q90","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3046,"outputTokens":2,"latencyMs":20173.710999999894},{"questionId":"q90","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2309,"outputTokens":2,"latencyMs":16398.499040999915},{"questionId":"q90","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2223,"outputTokens":2,"latencyMs":16167.04625000013},{"questionId":"q90","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5441,"outputTokens":2,"latencyMs":15138.672250000061},{"questionId":"q90","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3827,"outputTokens":2,"latencyMs":10029.309957999969},{"questionId":"q91","format":"json-pretty","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":4798,"outputTokens":2,"latencyMs":9550.991792000132},{"questionId":"q91","format":"json-compact","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3046,"outputTokens":2,"latencyMs":17190.87474999996},{"questionId":"q91","format":"toon","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2309,"outputTokens":2,"latencyMs":15229.16020799987},{"questionId":"q91","format":"csv","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2223,"outputTokens":2,"latencyMs":23082.90812499984},{"questionId":"q91","format":"xml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":5441,"outputTokens":2,"latencyMs":26555.38383299997},{"questionId":"q91","format":"yaml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3827,"outputTokens":2,"latencyMs":9781.933750000084},{"questionId":"q92","format":"json-pretty","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":4796,"outputTokens":2,"latencyMs":12433.872082999907},{"questionId":"q92","format":"json-compact","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3044,"outputTokens":2,"latencyMs":21259.824458000017},{"questionId":"q92","format":"toon","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2307,"outputTokens":2,"latencyMs":14788.983291000128},{"questionId":"q92","format":"csv","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2221,"outputTokens":2,"latencyMs":20083.6594169999},{"questionId":"q92","format":"xml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":5439,"outputTokens":2,"latencyMs":11762.891209000023},{"questionId":"q92","format":"yaml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3825,"outputTokens":2,"latencyMs":9649.968958000187},{"questionId":"q93","format":"json-pretty","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":4796,"outputTokens":1,"latencyMs":8159.669375000056},{"questionId":"q93","format":"json-compact","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":3044,"outputTokens":1,"latencyMs":17700.48974999995},{"questionId":"q93","format":"toon","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":2307,"outputTokens":1,"latencyMs":17174.231499999994},{"questionId":"q93","format":"csv","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":2221,"outputTokens":1,"latencyMs":18453.296667000046},{"questionId":"q93","format":"xml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":5439,"outputTokens":1,"latencyMs":10520.70066699991},{"questionId":"q93","format":"yaml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":3825,"outputTokens":1,"latencyMs":6200.701666999841},{"questionId":"q94","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4798,"outputTokens":2,"latencyMs":31263.397750000004},{"questionId":"q94","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3046,"outputTokens":2,"latencyMs":27911.912208999973},{"questionId":"q94","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2309,"outputTokens":2,"latencyMs":18668.751084000105},{"questionId":"q94","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2223,"outputTokens":2,"latencyMs":18332.33825000003},{"questionId":"q94","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5441,"outputTokens":2,"latencyMs":18221.784458999988},{"questionId":"q94","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3827,"outputTokens":2,"latencyMs":11501.188624999952},{"questionId":"q95","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4798,"outputTokens":2,"latencyMs":12168.62733300007},{"questionId":"q95","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3046,"outputTokens":2,"latencyMs":18283.335084000137},{"questionId":"q95","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2309,"outputTokens":2,"latencyMs":20068.500292000128},{"questionId":"q95","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2223,"outputTokens":2,"latencyMs":17943.502290999983},{"questionId":"q95","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5441,"outputTokens":2,"latencyMs":14015.277915999992},{"questionId":"q95","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3827,"outputTokens":2,"latencyMs":12597.466791999992},{"questionId":"q96","format":"json-pretty","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":20011,"outputTokens":6,"latencyMs":2211.3141250000335},{"questionId":"q96","format":"json-compact","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":15049,"outputTokens":6,"latencyMs":1644.998792000115},{"questionId":"q96","format":"toon","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":12381,"outputTokens":6,"latencyMs":2336.6362079998944},{"questionId":"q96","format":"csv","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":12228,"outputTokens":6,"latencyMs":2207.3656659999397},{"questionId":"q96","format":"xml","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":21897,"outputTokens":6,"latencyMs":1280.1559170000255},{"questionId":"q96","format":"yaml","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17095,"outputTokens":6,"latencyMs":1050.6362499999814},{"questionId":"q97","format":"json-pretty","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":20014,"outputTokens":5,"latencyMs":1562.2919159999583},{"questionId":"q97","format":"json-compact","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":15052,"outputTokens":5,"latencyMs":1106.6092920000665},{"questionId":"q97","format":"toon","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":12384,"outputTokens":5,"latencyMs":3056.3536250002217},{"questionId":"q97","format":"csv","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":12231,"outputTokens":5,"latencyMs":5364.961082999827},{"questionId":"q97","format":"xml","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":21900,"outputTokens":5,"latencyMs":1327.876707999967},{"questionId":"q97","format":"yaml","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17098,"outputTokens":5,"latencyMs":1008.9911249999423},{"questionId":"q98","format":"json-pretty","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":20008,"outputTokens":4,"latencyMs":1441.7123750001192},{"questionId":"q98","format":"json-compact","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":15046,"outputTokens":4,"latencyMs":1975.5891660000198},{"questionId":"q98","format":"toon","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":12378,"outputTokens":4,"latencyMs":3208.3119169999845},{"questionId":"q98","format":"csv","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":12225,"outputTokens":4,"latencyMs":2171.4729170000646},{"questionId":"q98","format":"xml","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":21894,"outputTokens":4,"latencyMs":1465.2379580000415},{"questionId":"q98","format":"yaml","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17092,"outputTokens":4,"latencyMs":2112.2501250000205},{"questionId":"q99","format":"json-pretty","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":20015,"outputTokens":1,"latencyMs":1402.308291999856},{"questionId":"q99","format":"json-compact","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":15053,"outputTokens":1,"latencyMs":1666.5179159999825},{"questionId":"q99","format":"toon","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":12385,"outputTokens":1,"latencyMs":5125.7323340000585},{"questionId":"q99","format":"csv","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":12232,"outputTokens":1,"latencyMs":1189.7219589999877},{"questionId":"q99","format":"xml","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":21901,"outputTokens":1,"latencyMs":1129.217750000069},{"questionId":"q99","format":"yaml","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":17099,"outputTokens":1,"latencyMs":1653.3000000000466},{"questionId":"q100","format":"json-pretty","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":20007,"outputTokens":6,"latencyMs":1401.7360839999747},{"questionId":"q100","format":"json-compact","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":15045,"outputTokens":6,"latencyMs":1115.8533329998609},{"questionId":"q100","format":"toon","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":12377,"outputTokens":6,"latencyMs":2909.888125000056},{"questionId":"q100","format":"csv","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":12224,"outputTokens":6,"latencyMs":1939.7250000000931},{"questionId":"q100","format":"xml","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":21893,"outputTokens":6,"latencyMs":1366.182541999966},{"questionId":"q100","format":"yaml","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17091,"outputTokens":6,"latencyMs":1966.4656250001863},{"questionId":"q101","format":"json-pretty","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":20013,"outputTokens":5,"latencyMs":1422.1554170001764},{"questionId":"q101","format":"json-compact","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":15051,"outputTokens":5,"latencyMs":1447.3659579998348},{"questionId":"q101","format":"toon","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":12383,"outputTokens":5,"latencyMs":2951.293124999851},{"questionId":"q101","format":"csv","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":12230,"outputTokens":5,"latencyMs":1743.5361669999547},{"questionId":"q101","format":"xml","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":21899,"outputTokens":5,"latencyMs":2728.970958000049},{"questionId":"q101","format":"yaml","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17097,"outputTokens":5,"latencyMs":2645.9682089998387},{"questionId":"q102","format":"json-pretty","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":20013,"outputTokens":3,"latencyMs":1191.8668339999858},{"questionId":"q102","format":"json-compact","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":15051,"outputTokens":3,"latencyMs":998.8467090001795},{"questionId":"q102","format":"toon","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":12383,"outputTokens":3,"latencyMs":2757.990875000134},{"questionId":"q102","format":"csv","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":12230,"outputTokens":3,"latencyMs":2029.8067499999888},{"questionId":"q102","format":"xml","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":21899,"outputTokens":3,"latencyMs":1506.327459000051},{"questionId":"q102","format":"yaml","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":17097,"outputTokens":3,"latencyMs":1936.9512079998385},{"questionId":"q103","format":"json-pretty","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":20009,"outputTokens":1,"latencyMs":1516.0164580000564},{"questionId":"q103","format":"json-compact","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":15047,"outputTokens":1,"latencyMs":2294.416416000109},{"questionId":"q103","format":"toon","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":12379,"outputTokens":1,"latencyMs":2849.68429200002},{"questionId":"q103","format":"csv","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":12226,"outputTokens":1,"latencyMs":1401.626958000008},{"questionId":"q103","format":"xml","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":21895,"outputTokens":1,"latencyMs":1381.8995420001447},{"questionId":"q103","format":"yaml","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":17093,"outputTokens":1,"latencyMs":1657.8283330001868},{"questionId":"q104","format":"json-pretty","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":20016,"outputTokens":6,"latencyMs":1387.4827919998206},{"questionId":"q104","format":"json-compact","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":15054,"outputTokens":6,"latencyMs":1175.5420419999864},{"questionId":"q104","format":"toon","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":12386,"outputTokens":6,"latencyMs":3027.7397080000956},{"questionId":"q104","format":"csv","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":12233,"outputTokens":6,"latencyMs":3088.3325000000186},{"questionId":"q104","format":"xml","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":21902,"outputTokens":6,"latencyMs":1291.0158749998081},{"questionId":"q104","format":"yaml","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17100,"outputTokens":6,"latencyMs":3673.9894159999676},{"questionId":"q105","format":"json-pretty","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":20011,"outputTokens":5,"latencyMs":1349.0667499999981},{"questionId":"q105","format":"json-compact","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":15049,"outputTokens":5,"latencyMs":1157.9330829998944},{"questionId":"q105","format":"toon","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":12381,"outputTokens":5,"latencyMs":2871.0199160000775},{"questionId":"q105","format":"csv","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":12228,"outputTokens":5,"latencyMs":2589.363791999873},{"questionId":"q105","format":"xml","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":21897,"outputTokens":5,"latencyMs":2238.4021249997895},{"questionId":"q105","format":"yaml","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17095,"outputTokens":5,"latencyMs":2021.4101670000236},{"questionId":"q106","format":"json-pretty","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":20014,"outputTokens":4,"latencyMs":1291.0011670000385},{"questionId":"q106","format":"json-compact","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":15052,"outputTokens":4,"latencyMs":2353.214125000173},{"questionId":"q106","format":"toon","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":12384,"outputTokens":4,"latencyMs":3064.0125420000404},{"questionId":"q106","format":"csv","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":12231,"outputTokens":4,"latencyMs":3329.2382500001695},{"questionId":"q106","format":"xml","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":21900,"outputTokens":4,"latencyMs":2356.347291999962},{"questionId":"q106","format":"yaml","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17098,"outputTokens":4,"latencyMs":2086.360624999972},{"questionId":"q107","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"90","isCorrect":false,"inputTokens":20006,"outputTokens":2,"latencyMs":5701.683209000155},{"questionId":"q107","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"64","isCorrect":false,"inputTokens":15044,"outputTokens":2,"latencyMs":1196.967083999887},{"questionId":"q107","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":12376,"outputTokens":3,"latencyMs":8496.278332999907},{"questionId":"q107","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"60","isCorrect":false,"inputTokens":12223,"outputTokens":2,"latencyMs":1352.2553749999497},{"questionId":"q107","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"60","isCorrect":false,"inputTokens":21892,"outputTokens":2,"latencyMs":1251.8892919998616},{"questionId":"q107","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":17090,"outputTokens":3,"latencyMs":8314.54454199993},{"questionId":"q108","format":"json-pretty","model":"gemini-2.5-flash","expected":"15413563","actual":"13508608","isCorrect":false,"inputTokens":20009,"outputTokens":8,"latencyMs":8380.613124999916},{"questionId":"q108","format":"json-compact","model":"gemini-2.5-flash","expected":"15413563","actual":"11234796","isCorrect":false,"inputTokens":15047,"outputTokens":8,"latencyMs":2146.2014159997925},{"questionId":"q108","format":"toon","model":"gemini-2.5-flash","expected":"15413563","actual":"12793798","isCorrect":false,"inputTokens":12379,"outputTokens":8,"latencyMs":7147.5953750000335},{"questionId":"q108","format":"csv","model":"gemini-2.5-flash","expected":"15413563","actual":"18196163","isCorrect":false,"inputTokens":12226,"outputTokens":8,"latencyMs":9613.253124999814},{"questionId":"q108","format":"xml","model":"gemini-2.5-flash","expected":"15413563","actual":"12791599","isCorrect":false,"inputTokens":21895,"outputTokens":8,"latencyMs":9019.025749999797},{"questionId":"q108","format":"yaml","model":"gemini-2.5-flash","expected":"15413563","actual":"12920406","isCorrect":false,"inputTokens":17093,"outputTokens":8,"latencyMs":77547.83733400004},{"questionId":"q109","format":"json-pretty","model":"gemini-2.5-flash","expected":"2528243","actual":"2757984","isCorrect":false,"inputTokens":20009,"outputTokens":7,"latencyMs":12165.649916999973},{"questionId":"q109","format":"json-compact","model":"gemini-2.5-flash","expected":"2528243","actual":"2674984","isCorrect":false,"inputTokens":15047,"outputTokens":7,"latencyMs":8778.673999999883},{"questionId":"q109","format":"toon","model":"gemini-2.5-flash","expected":"2528243","actual":"3012173","isCorrect":false,"inputTokens":12379,"outputTokens":7,"latencyMs":12954.33424999984},{"questionId":"q109","format":"csv","model":"gemini-2.5-flash","expected":"2528243","actual":"1464010","isCorrect":false,"inputTokens":12226,"outputTokens":7,"latencyMs":1306.0217079999857},{"questionId":"q109","format":"xml","model":"gemini-2.5-flash","expected":"2528243","actual":"2795908","isCorrect":false,"inputTokens":21895,"outputTokens":7,"latencyMs":12849.799542000052},{"questionId":"q109","format":"yaml","model":"gemini-2.5-flash","expected":"2528243","actual":"1861962","isCorrect":false,"inputTokens":17093,"outputTokens":7,"latencyMs":1189.123584000161},{"questionId":"q110","format":"json-pretty","model":"gemini-2.5-flash","expected":"154136","actual":"205012.51","isCorrect":false,"inputTokens":20008,"outputTokens":9,"latencyMs":85507.77341600019},{"questionId":"q110","format":"json-compact","model":"gemini-2.5-flash","expected":"154136","actual":"168868.22","isCorrect":false,"inputTokens":15046,"outputTokens":9,"latencyMs":857.5727079999633},{"questionId":"q110","format":"toon","model":"gemini-2.5-flash","expected":"154136","actual":"168686.7","isCorrect":false,"inputTokens":12378,"outputTokens":8,"latencyMs":33603.38358299993},{"questionId":"q110","format":"csv","model":"gemini-2.5-flash","expected":"154136","actual":"189191.06","isCorrect":false,"inputTokens":12225,"outputTokens":9,"latencyMs":10578.574166999897},{"questionId":"q110","format":"xml","model":"gemini-2.5-flash","expected":"154136","actual":"164929.16","isCorrect":false,"inputTokens":21894,"outputTokens":9,"latencyMs":13948.815542000113},{"questionId":"q110","format":"yaml","model":"gemini-2.5-flash","expected":"154136","actual":"157771.68","isCorrect":false,"inputTokens":17092,"outputTokens":9,"latencyMs":1213.3114169999026},{"questionId":"q111","format":"json-pretty","model":"gemini-2.5-flash","expected":"41","actual":"34","isCorrect":false,"inputTokens":20010,"outputTokens":2,"latencyMs":1175.5665420000441},{"questionId":"q111","format":"json-compact","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":15048,"outputTokens":2,"latencyMs":1331.0354169998318},{"questionId":"q111","format":"toon","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":12380,"outputTokens":2,"latencyMs":52398.45104199997},{"questionId":"q111","format":"csv","model":"gemini-2.5-flash","expected":"41","actual":"32","isCorrect":false,"inputTokens":12227,"outputTokens":2,"latencyMs":1756.012667000061},{"questionId":"q111","format":"xml","model":"gemini-2.5-flash","expected":"41","actual":"32","isCorrect":false,"inputTokens":21896,"outputTokens":2,"latencyMs":1230.2739999999758},{"questionId":"q111","format":"yaml","model":"gemini-2.5-flash","expected":"41","actual":"42","isCorrect":false,"inputTokens":17094,"outputTokens":2,"latencyMs":1164.9205000000075},{"questionId":"q112","format":"json-pretty","model":"gemini-2.5-flash","expected":"53","actual":"30","isCorrect":false,"inputTokens":20010,"outputTokens":2,"latencyMs":1371.9680829998106},{"questionId":"q112","format":"json-compact","model":"gemini-2.5-flash","expected":"53","actual":"41","isCorrect":false,"inputTokens":15048,"outputTokens":2,"latencyMs":1389.3990829999093},{"questionId":"q112","format":"toon","model":"gemini-2.5-flash","expected":"53","actual":"48","isCorrect":false,"inputTokens":12380,"outputTokens":2,"latencyMs":51797.823458000086},{"questionId":"q112","format":"csv","model":"gemini-2.5-flash","expected":"53","actual":"54","isCorrect":false,"inputTokens":12227,"outputTokens":2,"latencyMs":5937.101417000173},{"questionId":"q112","format":"xml","model":"gemini-2.5-flash","expected":"53","actual":"46","isCorrect":false,"inputTokens":21896,"outputTokens":2,"latencyMs":1850.1518329998944},{"questionId":"q112","format":"yaml","model":"gemini-2.5-flash","expected":"53","actual":"42","isCorrect":false,"inputTokens":17094,"outputTokens":2,"latencyMs":1271.803291000193},{"questionId":"q113","format":"json-pretty","model":"gemini-2.5-flash","expected":"77","actual":"77","isCorrect":true,"inputTokens":20013,"outputTokens":2,"latencyMs":12139.733791999985},{"questionId":"q113","format":"json-compact","model":"gemini-2.5-flash","expected":"77","actual":"52","isCorrect":false,"inputTokens":15051,"outputTokens":2,"latencyMs":1563.9794580000453},{"questionId":"q113","format":"toon","model":"gemini-2.5-flash","expected":"77","actual":"77","isCorrect":true,"inputTokens":12383,"outputTokens":2,"latencyMs":60018.49533299985},{"questionId":"q113","format":"csv","model":"gemini-2.5-flash","expected":"77","actual":"53","isCorrect":false,"inputTokens":12230,"outputTokens":2,"latencyMs":2886.4936659999657},{"questionId":"q113","format":"xml","model":"gemini-2.5-flash","expected":"77","actual":"68","isCorrect":false,"inputTokens":21899,"outputTokens":2,"latencyMs":7701.858624999877},{"questionId":"q113","format":"yaml","model":"gemini-2.5-flash","expected":"77","actual":"50","isCorrect":false,"inputTokens":17097,"outputTokens":2,"latencyMs":1499.8272080000024},{"questionId":"q114","format":"json-pretty","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":20013,"outputTokens":2,"latencyMs":10663.782916999888},{"questionId":"q114","format":"json-compact","model":"gemini-2.5-flash","expected":"37","actual":"48","isCorrect":false,"inputTokens":15051,"outputTokens":2,"latencyMs":827.0181249999441},{"questionId":"q114","format":"toon","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":12383,"outputTokens":2,"latencyMs":30728.82379200007},{"questionId":"q114","format":"csv","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":12230,"outputTokens":2,"latencyMs":57617.62024999992},{"questionId":"q114","format":"xml","model":"gemini-2.5-flash","expected":"37","actual":"38","isCorrect":false,"inputTokens":21899,"outputTokens":2,"latencyMs":7654.3029160001315},{"questionId":"q114","format":"yaml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":17097,"outputTokens":2,"latencyMs":5690.2677080000285},{"questionId":"q115","format":"json-pretty","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":20013,"outputTokens":2,"latencyMs":7403.075249999994},{"questionId":"q115","format":"json-compact","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":15051,"outputTokens":2,"latencyMs":11672.22716699983},{"questionId":"q115","format":"toon","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":12383,"outputTokens":2,"latencyMs":60024.06812499999},{"questionId":"q115","format":"csv","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":12230,"outputTokens":2,"latencyMs":16827.86266699992},{"questionId":"q115","format":"xml","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":21899,"outputTokens":2,"latencyMs":14204.877040999942},{"questionId":"q115","format":"yaml","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":17097,"outputTokens":2,"latencyMs":10882.716291000135},{"questionId":"q116","format":"json-pretty","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":20012,"outputTokens":2,"latencyMs":10126.601749999914},{"questionId":"q116","format":"json-compact","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":15050,"outputTokens":2,"latencyMs":11377.47104199999},{"questionId":"q116","format":"toon","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":12382,"outputTokens":2,"latencyMs":66874.20062500006},{"questionId":"q116","format":"csv","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":12229,"outputTokens":2,"latencyMs":55503.976083999965},{"questionId":"q116","format":"xml","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":21898,"outputTokens":2,"latencyMs":22246.931790999835},{"questionId":"q116","format":"yaml","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":17096,"outputTokens":2,"latencyMs":11075.051124999998},{"questionId":"q117","format":"json-pretty","model":"gemini-2.5-flash","expected":"23","actual":"21","isCorrect":false,"inputTokens":20012,"outputTokens":2,"latencyMs":1228.845166000072},{"questionId":"q117","format":"json-compact","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":15050,"outputTokens":2,"latencyMs":8741.270292000147},{"questionId":"q117","format":"toon","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":12382,"outputTokens":2,"latencyMs":121814.02916599996},{"questionId":"q117","format":"csv","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":12229,"outputTokens":2,"latencyMs":61989.02254200005},{"questionId":"q117","format":"xml","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":21898,"outputTokens":2,"latencyMs":11036.310292000184},{"questionId":"q117","format":"yaml","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":17096,"outputTokens":2,"latencyMs":9939.378125000047},{"questionId":"q118","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":20011,"outputTokens":1,"latencyMs":7667.9341249999125},{"questionId":"q118","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":15049,"outputTokens":1,"latencyMs":12107.239041999914},{"questionId":"q118","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":12381,"outputTokens":1,"latencyMs":57362.15020799986},{"questionId":"q118","format":"csv","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":12228,"outputTokens":1,"latencyMs":9611.33104199986},{"questionId":"q118","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":21897,"outputTokens":1,"latencyMs":12881.183583999984},{"questionId":"q118","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":17095,"outputTokens":1,"latencyMs":10873.08916600002},{"questionId":"q119","format":"json-pretty","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":20022,"outputTokens":2,"latencyMs":20178.348249999806},{"questionId":"q119","format":"json-compact","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":15060,"outputTokens":2,"latencyMs":18216.745832999935},{"questionId":"q119","format":"toon","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":12392,"outputTokens":2,"latencyMs":66507.61320799985},{"questionId":"q119","format":"csv","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":12239,"outputTokens":2,"latencyMs":83228.03312499984},{"questionId":"q119","format":"xml","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":21908,"outputTokens":2,"latencyMs":24800.78345800005},{"questionId":"q119","format":"yaml","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":17106,"outputTokens":2,"latencyMs":34368.74825000018},{"questionId":"q120","format":"json-pretty","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":20023,"outputTokens":2,"latencyMs":12939.749624999939},{"questionId":"q120","format":"json-compact","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":15061,"outputTokens":2,"latencyMs":70179.32116599986},{"questionId":"q120","format":"toon","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":12393,"outputTokens":2,"latencyMs":38629.9513340001},{"questionId":"q120","format":"csv","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":12240,"outputTokens":2,"latencyMs":25283.877875000006},{"questionId":"q120","format":"xml","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":21909,"outputTokens":2,"latencyMs":22682.039416000014},{"questionId":"q120","format":"yaml","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":17107,"outputTokens":2,"latencyMs":18730.286957999924},{"questionId":"q121","format":"json-pretty","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":20023,"outputTokens":2,"latencyMs":10387.956000000006},{"questionId":"q121","format":"json-compact","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":15061,"outputTokens":2,"latencyMs":13692.476250000065},{"questionId":"q121","format":"toon","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":12393,"outputTokens":2,"latencyMs":79759.93679199996},{"questionId":"q121","format":"csv","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":12240,"outputTokens":2,"latencyMs":35631.86233299994},{"questionId":"q121","format":"xml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":21909,"outputTokens":2,"latencyMs":14524.548749999842},{"questionId":"q121","format":"yaml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":17107,"outputTokens":2,"latencyMs":10477.33529200009},{"questionId":"q122","format":"json-pretty","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":20022,"outputTokens":1,"latencyMs":17417.211249999935},{"questionId":"q122","format":"json-compact","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":15060,"outputTokens":1,"latencyMs":21743.355708000017},{"questionId":"q122","format":"toon","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":12392,"outputTokens":1,"latencyMs":69987.69104199996},{"questionId":"q122","format":"csv","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":12239,"outputTokens":1,"latencyMs":73348.87191600003},{"questionId":"q122","format":"xml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":21908,"outputTokens":1,"latencyMs":21004.25129100005},{"questionId":"q122","format":"yaml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":17106,"outputTokens":1,"latencyMs":15259.87562499987},{"questionId":"q123","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":20022,"outputTokens":1,"latencyMs":5461.979041999904},{"questionId":"q123","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":15060,"outputTokens":1,"latencyMs":70539.96841700003},{"questionId":"q123","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":12392,"outputTokens":1,"latencyMs":31870.766082999995},{"questionId":"q123","format":"csv","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":12239,"outputTokens":1,"latencyMs":37310.19433300011},{"questionId":"q123","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":21908,"outputTokens":1,"latencyMs":10856.934333999874},{"questionId":"q123","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":17106,"outputTokens":1,"latencyMs":6666.342249999987},{"questionId":"q124","format":"json-pretty","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":8539,"outputTokens":1,"latencyMs":1878.8176249999087},{"questionId":"q124","format":"json-compact","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":5866,"outputTokens":1,"latencyMs":1409.4439170002006},{"questionId":"q124","format":"toon","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":6993,"outputTokens":1,"latencyMs":1369.5112499999814},{"questionId":"q124","format":"xml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":9362,"outputTokens":1,"latencyMs":1790.701708999928},{"questionId":"q124","format":"yaml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":6920,"outputTokens":1,"latencyMs":1836.0749999999534},{"questionId":"q125","format":"json-pretty","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8539,"outputTokens":4,"latencyMs":2251.6957499999553},{"questionId":"q125","format":"json-compact","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5866,"outputTokens":4,"latencyMs":1370.18120799982},{"questionId":"q125","format":"toon","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6993,"outputTokens":4,"latencyMs":2900.4670410000253},{"questionId":"q125","format":"xml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":9362,"outputTokens":4,"latencyMs":1347.2522919999901},{"questionId":"q125","format":"yaml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6920,"outputTokens":4,"latencyMs":1399.5693329998758},{"questionId":"q126","format":"json-pretty","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":8540,"outputTokens":3,"latencyMs":629.7614999997895},{"questionId":"q126","format":"json-compact","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":5867,"outputTokens":3,"latencyMs":3635.8279579998925},{"questionId":"q126","format":"toon","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":6994,"outputTokens":3,"latencyMs":3848.690999999875},{"questionId":"q126","format":"xml","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":9363,"outputTokens":3,"latencyMs":1416.82720900001},{"questionId":"q126","format":"yaml","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":6921,"outputTokens":3,"latencyMs":3025.1147080000956},{"questionId":"q127","format":"json-pretty","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":8540,"outputTokens":4,"latencyMs":1309.6894170001615},{"questionId":"q127","format":"json-compact","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5867,"outputTokens":4,"latencyMs":1731.4591250000522},{"questionId":"q127","format":"toon","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6994,"outputTokens":4,"latencyMs":2646.668166999938},{"questionId":"q127","format":"xml","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":9363,"outputTokens":4,"latencyMs":2428.78829200007},{"questionId":"q127","format":"yaml","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6921,"outputTokens":4,"latencyMs":2095.0494580001105},{"questionId":"q128","format":"json-pretty","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":8539,"outputTokens":1,"latencyMs":1550.8666669998784},{"questionId":"q128","format":"json-compact","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":5866,"outputTokens":1,"latencyMs":2002.3954999998678},{"questionId":"q128","format":"toon","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":6993,"outputTokens":1,"latencyMs":1840.2402500000317},{"questionId":"q128","format":"xml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":9362,"outputTokens":1,"latencyMs":1800.7229169998318},{"questionId":"q128","format":"yaml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":6920,"outputTokens":1,"latencyMs":1992.9220419998746},{"questionId":"q129","format":"json-pretty","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":8539,"outputTokens":4,"latencyMs":1808.3219159999862},{"questionId":"q129","format":"json-compact","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5866,"outputTokens":4,"latencyMs":1745.299541000044},{"questionId":"q129","format":"toon","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6993,"outputTokens":4,"latencyMs":2423.28370800009},{"questionId":"q129","format":"xml","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":9362,"outputTokens":4,"latencyMs":2074.5135000001173},{"questionId":"q129","format":"yaml","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6920,"outputTokens":4,"latencyMs":1317.3766250000335},{"questionId":"q130","format":"json-pretty","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":8540,"outputTokens":3,"latencyMs":1010.6947500000242},{"questionId":"q130","format":"json-compact","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":5867,"outputTokens":3,"latencyMs":1228.840749999974},{"questionId":"q130","format":"toon","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":6994,"outputTokens":3,"latencyMs":2259.1699999999255},{"questionId":"q130","format":"xml","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":9363,"outputTokens":3,"latencyMs":1966.4926670000423},{"questionId":"q130","format":"yaml","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":6921,"outputTokens":3,"latencyMs":2354.1357499998994},{"questionId":"q131","format":"json-pretty","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":8540,"outputTokens":3,"latencyMs":888.1196249998175},{"questionId":"q131","format":"json-compact","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":5867,"outputTokens":3,"latencyMs":1894.6611670001876},{"questionId":"q131","format":"toon","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":6994,"outputTokens":3,"latencyMs":1911.845208000159},{"questionId":"q131","format":"xml","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":9363,"outputTokens":3,"latencyMs":1935.7601250000298},{"questionId":"q131","format":"yaml","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":6921,"outputTokens":3,"latencyMs":1843.8479169998318},{"questionId":"q132","format":"json-pretty","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":8539,"outputTokens":1,"latencyMs":3363.752249999903},{"questionId":"q132","format":"json-compact","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":5866,"outputTokens":1,"latencyMs":2269.923167000059},{"questionId":"q132","format":"toon","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":6993,"outputTokens":1,"latencyMs":1724.937750000041},{"questionId":"q132","format":"xml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":9362,"outputTokens":1,"latencyMs":1993.0314999998081},{"questionId":"q132","format":"yaml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":6920,"outputTokens":1,"latencyMs":1220.3704999999609},{"questionId":"q133","format":"json-pretty","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8539,"outputTokens":4,"latencyMs":1217.0715000000782},{"questionId":"q133","format":"json-compact","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5866,"outputTokens":4,"latencyMs":1927.4073330000974},{"questionId":"q133","format":"toon","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6993,"outputTokens":4,"latencyMs":1874.252708000131},{"questionId":"q133","format":"xml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":9362,"outputTokens":4,"latencyMs":2147.5908750002272},{"questionId":"q133","format":"yaml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6920,"outputTokens":4,"latencyMs":2515.3095410000533},{"questionId":"q134","format":"json-pretty","model":"gemini-2.5-flash","expected":"75","actual":"64","isCorrect":false,"inputTokens":8514,"outputTokens":2,"latencyMs":3175.3592499999795},{"questionId":"q134","format":"json-compact","model":"gemini-2.5-flash","expected":"75","actual":"67","isCorrect":false,"inputTokens":5841,"outputTokens":2,"latencyMs":976.6002090000547},{"questionId":"q134","format":"toon","model":"gemini-2.5-flash","expected":"75","actual":"75","isCorrect":true,"inputTokens":6968,"outputTokens":2,"latencyMs":11622.03495799983},{"questionId":"q134","format":"xml","model":"gemini-2.5-flash","expected":"75","actual":"60","isCorrect":false,"inputTokens":9337,"outputTokens":2,"latencyMs":2764.29366700002},{"questionId":"q134","format":"yaml","model":"gemini-2.5-flash","expected":"75","actual":"75","isCorrect":true,"inputTokens":6895,"outputTokens":2,"latencyMs":10543.151874999981},{"questionId":"q135","format":"json-pretty","model":"gemini-2.5-flash","expected":"2453.41","actual":"2446.5","isCorrect":false,"inputTokens":8515,"outputTokens":6,"latencyMs":4752.668125000084},{"questionId":"q135","format":"json-compact","model":"gemini-2.5-flash","expected":"2453.41","actual":"2317.86","isCorrect":false,"inputTokens":5842,"outputTokens":7,"latencyMs":4809.121500000125},{"questionId":"q135","format":"toon","model":"gemini-2.5-flash","expected":"2453.41","actual":"2121.9466666666665","isCorrect":false,"inputTokens":6969,"outputTokens":18,"latencyMs":5130.499125000089},{"questionId":"q135","format":"xml","model":"gemini-2.5-flash","expected":"2453.41","actual":"2409.8","isCorrect":false,"inputTokens":9338,"outputTokens":6,"latencyMs":6959.698749999981},{"questionId":"q135","format":"yaml","model":"gemini-2.5-flash","expected":"2453.41","actual":"2381.61","isCorrect":false,"inputTokens":6896,"outputTokens":7,"latencyMs":5282.854708000086},{"questionId":"q136","format":"json-pretty","model":"gemini-2.5-flash","expected":"29","actual":"28","isCorrect":false,"inputTokens":8514,"outputTokens":2,"latencyMs":825.8032919999678},{"questionId":"q136","format":"json-compact","model":"gemini-2.5-flash","expected":"29","actual":"28","isCorrect":false,"inputTokens":5841,"outputTokens":2,"latencyMs":713.9450840000063},{"questionId":"q136","format":"toon","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":6968,"outputTokens":2,"latencyMs":24505.540041000117},{"questionId":"q136","format":"xml","model":"gemini-2.5-flash","expected":"29","actual":"27","isCorrect":false,"inputTokens":9337,"outputTokens":2,"latencyMs":2090.9554999999236},{"questionId":"q136","format":"yaml","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":6895,"outputTokens":2,"latencyMs":15048.97654199996},{"questionId":"q137","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":8514,"outputTokens":2,"latencyMs":5170.641334000044},{"questionId":"q137","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":5841,"outputTokens":2,"latencyMs":6473.029875000007},{"questionId":"q137","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":6968,"outputTokens":2,"latencyMs":4246.861290999921},{"questionId":"q137","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"16","isCorrect":false,"inputTokens":9337,"outputTokens":2,"latencyMs":1270.3035000001546},{"questionId":"q137","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":6895,"outputTokens":2,"latencyMs":8520.89433400007},{"questionId":"q138","format":"json-pretty","model":"gemini-2.5-flash","expected":"29","actual":"20","isCorrect":false,"inputTokens":8514,"outputTokens":2,"latencyMs":1850.1505420000758},{"questionId":"q138","format":"json-compact","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":5841,"outputTokens":2,"latencyMs":9030.687209000112},{"questionId":"q138","format":"toon","model":"gemini-2.5-flash","expected":"29","actual":"28","isCorrect":false,"inputTokens":6968,"outputTokens":2,"latencyMs":4828.5370410000905},{"questionId":"q138","format":"xml","model":"gemini-2.5-flash","expected":"29","actual":"20","isCorrect":false,"inputTokens":9337,"outputTokens":2,"latencyMs":1184.6289590001106},{"questionId":"q138","format":"yaml","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":6895,"outputTokens":2,"latencyMs":5919.808792000171},{"questionId":"q139","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":8517,"outputTokens":2,"latencyMs":5852.9212919999845},{"questionId":"q139","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5844,"outputTokens":2,"latencyMs":13428.725749999983},{"questionId":"q139","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"12","isCorrect":false,"inputTokens":6971,"outputTokens":2,"latencyMs":10008.94108400005},{"questionId":"q139","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"10","isCorrect":false,"inputTokens":9340,"outputTokens":2,"latencyMs":9106.528457999928},{"questionId":"q139","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":6898,"outputTokens":2,"latencyMs":31064.059250000166},{"questionId":"q140","format":"json-pretty","model":"gemini-2.5-flash","expected":"18","actual":"13","isCorrect":false,"inputTokens":8517,"outputTokens":2,"latencyMs":2124.0804999999236},{"questionId":"q140","format":"json-compact","model":"gemini-2.5-flash","expected":"18","actual":"18","isCorrect":true,"inputTokens":5844,"outputTokens":2,"latencyMs":14951.041833000025},{"questionId":"q140","format":"toon","model":"gemini-2.5-flash","expected":"18","actual":"17","isCorrect":false,"inputTokens":6971,"outputTokens":2,"latencyMs":6033.8783750000875},{"questionId":"q140","format":"xml","model":"gemini-2.5-flash","expected":"18","actual":"18","isCorrect":true,"inputTokens":9340,"outputTokens":2,"latencyMs":3825.774333999958},{"questionId":"q140","format":"yaml","model":"gemini-2.5-flash","expected":"18","actual":"18","isCorrect":true,"inputTokens":6898,"outputTokens":2,"latencyMs":15656.88370799995},{"questionId":"q141","format":"json-pretty","model":"gemini-2.5-flash","expected":"33","actual":"30","isCorrect":false,"inputTokens":8523,"outputTokens":2,"latencyMs":920.4534169998951},{"questionId":"q141","format":"json-compact","model":"gemini-2.5-flash","expected":"33","actual":"33","isCorrect":true,"inputTokens":5850,"outputTokens":2,"latencyMs":8256.13604100002},{"questionId":"q141","format":"toon","model":"gemini-2.5-flash","expected":"33","actual":"33","isCorrect":true,"inputTokens":6977,"outputTokens":2,"latencyMs":8657.864458000055},{"questionId":"q141","format":"xml","model":"gemini-2.5-flash","expected":"33","actual":"33","isCorrect":true,"inputTokens":9346,"outputTokens":2,"latencyMs":8993.88512500003},{"questionId":"q141","format":"yaml","model":"gemini-2.5-flash","expected":"33","actual":"33","isCorrect":true,"inputTokens":6904,"outputTokens":2,"latencyMs":6527.931417000014},{"questionId":"q142","format":"json-pretty","model":"gemini-2.5-flash","expected":"42","actual":"44","isCorrect":false,"inputTokens":8523,"outputTokens":2,"latencyMs":1363.3203749998938},{"questionId":"q142","format":"json-compact","model":"gemini-2.5-flash","expected":"42","actual":"30","isCorrect":false,"inputTokens":5850,"outputTokens":2,"latencyMs":6156.330874999985},{"questionId":"q142","format":"toon","model":"gemini-2.5-flash","expected":"42","actual":"34","isCorrect":false,"inputTokens":6977,"outputTokens":2,"latencyMs":1357.8627090000082},{"questionId":"q142","format":"xml","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":9346,"outputTokens":2,"latencyMs":8131.651750000194},{"questionId":"q142","format":"yaml","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":6904,"outputTokens":2,"latencyMs":24600.726624999894},{"questionId":"q143","format":"json-pretty","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":8515,"outputTokens":2,"latencyMs":13955.858000000007},{"questionId":"q143","format":"json-compact","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":5842,"outputTokens":2,"latencyMs":28712.087249999866},{"questionId":"q143","format":"toon","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":6969,"outputTokens":2,"latencyMs":22100.538207999896},{"questionId":"q143","format":"xml","model":"gemini-2.5-flash","expected":"25","actual":"20","isCorrect":false,"inputTokens":9338,"outputTokens":2,"latencyMs":7590.498083000071},{"questionId":"q143","format":"yaml","model":"gemini-2.5-flash","expected":"25","actual":"21","isCorrect":false,"inputTokens":6896,"outputTokens":2,"latencyMs":5638.855457999976},{"questionId":"q144","format":"json-pretty","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":8523,"outputTokens":2,"latencyMs":7354.892666999949},{"questionId":"q144","format":"json-compact","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":5850,"outputTokens":2,"latencyMs":31292.425041000126},{"questionId":"q144","format":"toon","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":6977,"outputTokens":2,"latencyMs":8738.58424999984},{"questionId":"q144","format":"xml","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":9346,"outputTokens":2,"latencyMs":8489.522292000009},{"questionId":"q144","format":"yaml","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":6904,"outputTokens":2,"latencyMs":12565.038208000129},{"questionId":"q145","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":8523,"outputTokens":1,"latencyMs":6862.408624999924},{"questionId":"q145","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":5850,"outputTokens":1,"latencyMs":7379.369916999945},{"questionId":"q145","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":6977,"outputTokens":1,"latencyMs":5361.5800000000745},{"questionId":"q145","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":9346,"outputTokens":1,"latencyMs":8726.69533300004},{"questionId":"q145","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":6904,"outputTokens":1,"latencyMs":7306.31495800009},{"questionId":"q146","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":8526,"outputTokens":1,"latencyMs":23603.895541999955},{"questionId":"q146","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":5853,"outputTokens":1,"latencyMs":30334.129915999947},{"questionId":"q146","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":6980,"outputTokens":1,"latencyMs":10467.511208000127},{"questionId":"q146","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":9349,"outputTokens":1,"latencyMs":33134.07887499989},{"questionId":"q146","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":6907,"outputTokens":1,"latencyMs":14736.566666999832},{"questionId":"q147","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":8526,"outputTokens":1,"latencyMs":20741.889624999836},{"questionId":"q147","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":5853,"outputTokens":1,"latencyMs":34321.42762500001},{"questionId":"q147","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":6980,"outputTokens":1,"latencyMs":23117.92879200005},{"questionId":"q147","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":9349,"outputTokens":1,"latencyMs":10977.787250000052},{"questionId":"q147","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":6907,"outputTokens":1,"latencyMs":8634.03341699997},{"questionId":"q148","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8526,"outputTokens":1,"latencyMs":27021.10258299997},{"questionId":"q148","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":5853,"outputTokens":1,"latencyMs":19140.245125000132},{"questionId":"q148","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":6980,"outputTokens":1,"latencyMs":19195.624416999985},{"questionId":"q148","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9349,"outputTokens":1,"latencyMs":11410.610291999998},{"questionId":"q148","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":6907,"outputTokens":1,"latencyMs":17015.364916999824},{"questionId":"q149","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":8522,"outputTokens":1,"latencyMs":13242.74987499998},{"questionId":"q149","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":5849,"outputTokens":1,"latencyMs":30915.795083999867},{"questionId":"q149","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":6976,"outputTokens":1,"latencyMs":23221.64691700018},{"questionId":"q149","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":9345,"outputTokens":1,"latencyMs":26706.130709000165},{"questionId":"q149","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":6903,"outputTokens":1,"latencyMs":15082.205499999924},{"questionId":"q150","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":8522,"outputTokens":1,"latencyMs":9463.296958999941},{"questionId":"q150","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":5849,"outputTokens":1,"latencyMs":29378.66441700002},{"questionId":"q150","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":6976,"outputTokens":1,"latencyMs":9360.268625000026},{"questionId":"q150","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":9345,"outputTokens":1,"latencyMs":12332.108042000094},{"questionId":"q150","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":6903,"outputTokens":1,"latencyMs":6255.294665999943},{"questionId":"q151","format":"json-pretty","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":1187,"outputTokens":1,"latencyMs":591.1827920000069},{"questionId":"q151","format":"json-compact","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":670,"outputTokens":1,"latencyMs":612.2220830000006},{"questionId":"q151","format":"toon","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":814,"outputTokens":1,"latencyMs":604.3744590000715},{"questionId":"q151","format":"xml","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":1246,"outputTokens":1,"latencyMs":565.8472909999546},{"questionId":"q151","format":"yaml","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":836,"outputTokens":1,"latencyMs":541.5233749998733},{"questionId":"q152","format":"json-pretty","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1185,"outputTokens":6,"latencyMs":969.4197499998845},{"questionId":"q152","format":"json-compact","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":668,"outputTokens":6,"latencyMs":632.0847499999218},{"questionId":"q152","format":"toon","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":812,"outputTokens":6,"latencyMs":795.3438750000205},{"questionId":"q152","format":"xml","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1244,"outputTokens":6,"latencyMs":2274.6209169998765},{"questionId":"q152","format":"yaml","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":834,"outputTokens":6,"latencyMs":1200.3158750000875},{"questionId":"q153","format":"json-pretty","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1185,"outputTokens":4,"latencyMs":848.6990410001017},{"questionId":"q153","format":"json-compact","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":668,"outputTokens":4,"latencyMs":788.9229170000181},{"questionId":"q153","format":"toon","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":812,"outputTokens":4,"latencyMs":959.1193750000093},{"questionId":"q153","format":"xml","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1244,"outputTokens":4,"latencyMs":2285.088915999979},{"questionId":"q153","format":"yaml","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":834,"outputTokens":4,"latencyMs":1031.71875},{"questionId":"q154","format":"json-pretty","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":1187,"outputTokens":2,"latencyMs":757.0324579998851},{"questionId":"q154","format":"json-compact","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":670,"outputTokens":2,"latencyMs":1198.8829590000678},{"questionId":"q154","format":"toon","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":814,"outputTokens":2,"latencyMs":1052.9379590000026},{"questionId":"q154","format":"xml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":1246,"outputTokens":2,"latencyMs":915.5204159999266},{"questionId":"q154","format":"yaml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":836,"outputTokens":2,"latencyMs":785.371041999897},{"questionId":"q155","format":"json-pretty","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1185,"outputTokens":5,"latencyMs":966.9602920000907},{"questionId":"q155","format":"json-compact","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":668,"outputTokens":5,"latencyMs":791.5538329998963},{"questionId":"q155","format":"toon","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":812,"outputTokens":5,"latencyMs":651.5699169998989},{"questionId":"q155","format":"xml","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1244,"outputTokens":5,"latencyMs":839.2252500001341},{"questionId":"q155","format":"yaml","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":834,"outputTokens":5,"latencyMs":930.6539170001633},{"questionId":"q156","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1187,"outputTokens":1,"latencyMs":1171.0769579999615},{"questionId":"q156","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":670,"outputTokens":1,"latencyMs":1025.3227500000503},{"questionId":"q156","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":814,"outputTokens":1,"latencyMs":987.5835829998832},{"questionId":"q156","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1246,"outputTokens":1,"latencyMs":941.9106250000186},{"questionId":"q156","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":836,"outputTokens":1,"latencyMs":909.7709999999497},{"questionId":"q157","format":"json-pretty","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1187,"outputTokens":5,"latencyMs":1042.7451670002192},{"questionId":"q157","format":"json-compact","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":670,"outputTokens":5,"latencyMs":916.3139590001665},{"questionId":"q157","format":"toon","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":814,"outputTokens":5,"latencyMs":1150.8895419999026},{"questionId":"q157","format":"xml","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1246,"outputTokens":5,"latencyMs":1076.97637499962},{"questionId":"q157","format":"yaml","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":836,"outputTokens":5,"latencyMs":811.0715830000117},{"questionId":"q158","format":"json-pretty","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":1185,"outputTokens":1,"latencyMs":918.6858329996467},{"questionId":"q158","format":"json-compact","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":668,"outputTokens":1,"latencyMs":1415.4067499996163},{"questionId":"q158","format":"toon","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":812,"outputTokens":1,"latencyMs":772.3880000002682},{"questionId":"q158","format":"xml","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":1244,"outputTokens":1,"latencyMs":1148.560541999992},{"questionId":"q158","format":"yaml","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":834,"outputTokens":1,"latencyMs":991.7529170000926},{"questionId":"q159","format":"json-pretty","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1186,"outputTokens":4,"latencyMs":1039.5265000001527},{"questionId":"q159","format":"json-compact","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":669,"outputTokens":4,"latencyMs":986.9138329997659},{"questionId":"q159","format":"toon","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":813,"outputTokens":4,"latencyMs":879.3160830000415},{"questionId":"q159","format":"xml","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1245,"outputTokens":4,"latencyMs":489.81733300024644},{"questionId":"q159","format":"yaml","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":835,"outputTokens":4,"latencyMs":733.0474580000155},{"questionId":"q160","format":"json-pretty","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1187,"outputTokens":5,"latencyMs":675.8573339995928},{"questionId":"q160","format":"json-compact","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":670,"outputTokens":5,"latencyMs":964.5507499999367},{"questionId":"q160","format":"toon","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":814,"outputTokens":5,"latencyMs":1027.5942080002278},{"questionId":"q160","format":"xml","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1246,"outputTokens":5,"latencyMs":609.8585419999436},{"questionId":"q160","format":"yaml","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":836,"outputTokens":5,"latencyMs":743.0652499999851},{"questionId":"q161","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1187,"outputTokens":1,"latencyMs":948.5502920001745},{"questionId":"q161","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":670,"outputTokens":1,"latencyMs":1972.8965420001186},{"questionId":"q161","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":814,"outputTokens":1,"latencyMs":807.8485420001671},{"questionId":"q161","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1246,"outputTokens":1,"latencyMs":1116.9069170001894},{"questionId":"q161","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":836,"outputTokens":1,"latencyMs":949.3885419997387},{"questionId":"q162","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1187,"outputTokens":1,"latencyMs":2159.51712499978},{"questionId":"q162","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":670,"outputTokens":1,"latencyMs":1081.4400410000235},{"questionId":"q162","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":814,"outputTokens":1,"latencyMs":1460.2627920000814},{"questionId":"q162","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1246,"outputTokens":1,"latencyMs":2162.5616249996237},{"questionId":"q162","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":836,"outputTokens":1,"latencyMs":1409.9183330000378},{"questionId":"q163","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1186,"outputTokens":1,"latencyMs":2093.9573749997653},{"questionId":"q163","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":669,"outputTokens":1,"latencyMs":1716.838541999925},{"questionId":"q163","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":813,"outputTokens":1,"latencyMs":1341.624334000051},{"questionId":"q163","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":1513.9824999999255},{"questionId":"q163","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":835,"outputTokens":1,"latencyMs":1172.6995000001043},{"questionId":"q164","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1186,"outputTokens":1,"latencyMs":1462.171083000023},{"questionId":"q164","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":669,"outputTokens":1,"latencyMs":1192.5067079998553},{"questionId":"q164","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":813,"outputTokens":1,"latencyMs":1271.7925839996897},{"questionId":"q164","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":1166.698790999595},{"questionId":"q164","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":835,"outputTokens":1,"latencyMs":2312.9520839997567},{"questionId":"q165","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1186,"outputTokens":1,"latencyMs":1973.9130409997888},{"questionId":"q165","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":669,"outputTokens":1,"latencyMs":2381.2769590001553},{"questionId":"q165","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":813,"outputTokens":1,"latencyMs":1579.131750000175},{"questionId":"q165","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":1286.0622090003453},{"questionId":"q165","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":835,"outputTokens":1,"latencyMs":1799.2124580000527},{"questionId":"q166","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1190,"outputTokens":1,"latencyMs":1500.3542919997126},{"questionId":"q166","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":673,"outputTokens":1,"latencyMs":2298.299667000305},{"questionId":"q166","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":817,"outputTokens":1,"latencyMs":2150.4938749996945},{"questionId":"q166","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1249,"outputTokens":1,"latencyMs":2658.724958999548},{"questionId":"q166","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":839,"outputTokens":1,"latencyMs":2749.8499170001596},{"questionId":"q167","format":"json-pretty","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1186,"outputTokens":1,"latencyMs":1472.6816249997355},{"questionId":"q167","format":"json-compact","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":669,"outputTokens":1,"latencyMs":2015.28525000019},{"questionId":"q167","format":"toon","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":813,"outputTokens":1,"latencyMs":1763.8271250003017},{"questionId":"q167","format":"xml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":1690.4910000003874},{"questionId":"q167","format":"yaml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":835,"outputTokens":1,"latencyMs":1108.0051669999957},{"questionId":"q168","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1188,"outputTokens":1,"latencyMs":1360.5312919998541},{"questionId":"q168","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":671,"outputTokens":1,"latencyMs":1903.3250000001863},{"questionId":"q168","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":815,"outputTokens":1,"latencyMs":933.7392909997143},{"questionId":"q168","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1247,"outputTokens":1,"latencyMs":1741.5702909999527},{"questionId":"q168","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":837,"outputTokens":1,"latencyMs":1468.5960000003688},{"questionId":"q169","format":"json-pretty","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":1190,"outputTokens":1,"latencyMs":1182.110375000164},{"questionId":"q169","format":"json-compact","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":673,"outputTokens":1,"latencyMs":1541.7142920000479},{"questionId":"q169","format":"toon","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":817,"outputTokens":1,"latencyMs":1253.5442079999484},{"questionId":"q169","format":"xml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":1249,"outputTokens":1,"latencyMs":1070.9946669996716},{"questionId":"q169","format":"yaml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":839,"outputTokens":1,"latencyMs":1010.012124999892},{"questionId":"q170","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1189,"outputTokens":1,"latencyMs":1369.9096669997089},{"questionId":"q170","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":672,"outputTokens":1,"latencyMs":1759.5700419996865},{"questionId":"q170","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":816,"outputTokens":1,"latencyMs":1816.319708000403},{"questionId":"q170","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1248,"outputTokens":1,"latencyMs":1520.5940419998951},{"questionId":"q170","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":838,"outputTokens":1,"latencyMs":1498.6745409998111},{"questionId":"q171","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1191,"outputTokens":1,"latencyMs":1525.3791250004433},{"questionId":"q171","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":674,"outputTokens":1,"latencyMs":1338.7996669998392},{"questionId":"q171","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":818,"outputTokens":1,"latencyMs":1138.5583340004086},{"questionId":"q171","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1250,"outputTokens":1,"latencyMs":1376.096584000159},{"questionId":"q171","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":840,"outputTokens":1,"latencyMs":1314.8401669999585},{"questionId":"q172","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1191,"outputTokens":1,"latencyMs":1574.2900419998914},{"questionId":"q172","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":674,"outputTokens":1,"latencyMs":2473.918708000332},{"questionId":"q172","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":818,"outputTokens":1,"latencyMs":2170.27329199994},{"questionId":"q172","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1250,"outputTokens":1,"latencyMs":2295.8110830001533},{"questionId":"q172","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":840,"outputTokens":1,"latencyMs":1623.1959170000628},{"questionId":"q173","format":"json-pretty","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1193,"outputTokens":1,"latencyMs":1461.3517920002341},{"questionId":"q173","format":"json-compact","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":676,"outputTokens":1,"latencyMs":2231.2799169998616},{"questionId":"q173","format":"toon","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":820,"outputTokens":1,"latencyMs":2229.0055410000496},{"questionId":"q173","format":"xml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1252,"outputTokens":1,"latencyMs":3159.4098330000415},{"questionId":"q173","format":"yaml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":842,"outputTokens":1,"latencyMs":2170.447500000242},{"questionId":"q174","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1189,"outputTokens":1,"latencyMs":1548.6582500003278},{"questionId":"q174","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":672,"outputTokens":1,"latencyMs":1888.0097079998814},{"questionId":"q174","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":816,"outputTokens":1,"latencyMs":2156.2941660000943},{"questionId":"q174","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1248,"outputTokens":1,"latencyMs":2344.32354200026},{"questionId":"q174","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":838,"outputTokens":1,"latencyMs":1481.4086250001565},{"questionId":"q175","format":"json-pretty","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1194,"outputTokens":1,"latencyMs":2554.567374999635},{"questionId":"q175","format":"json-compact","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":677,"outputTokens":1,"latencyMs":2061.3967920001596},{"questionId":"q175","format":"toon","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":821,"outputTokens":1,"latencyMs":2012.9174999999814},{"questionId":"q175","format":"xml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1253,"outputTokens":1,"latencyMs":2284.4985419996083},{"questionId":"q175","format":"yaml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":843,"outputTokens":1,"latencyMs":2713.7318330002017},{"questionId":"q176","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1187,"outputTokens":1,"latencyMs":788.006042000372},{"questionId":"q176","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":670,"outputTokens":1,"latencyMs":1397.4356669997796},{"questionId":"q176","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":814,"outputTokens":1,"latencyMs":1883.1750839999877},{"questionId":"q176","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1246,"outputTokens":1,"latencyMs":3641.527958000079},{"questionId":"q176","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":836,"outputTokens":1,"latencyMs":1312.123292000033},{"questionId":"q177","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1197,"outputTokens":1,"latencyMs":2362.46879200032},{"questionId":"q177","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":680,"outputTokens":1,"latencyMs":2401.209290999919},{"questionId":"q177","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":824,"outputTokens":1,"latencyMs":2620.8036670000292},{"questionId":"q177","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1256,"outputTokens":1,"latencyMs":2614.5357499998063},{"questionId":"q177","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":846,"outputTokens":1,"latencyMs":2068.564207999967},{"questionId":"q178","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1190,"outputTokens":1,"latencyMs":2713.30145799974},{"questionId":"q178","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":673,"outputTokens":1,"latencyMs":2629.350916999858},{"questionId":"q178","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":817,"outputTokens":1,"latencyMs":2680.834209000226},{"questionId":"q178","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1249,"outputTokens":1,"latencyMs":2505.900541000068},{"questionId":"q178","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":839,"outputTokens":1,"latencyMs":2552.773792000022},{"questionId":"q179","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1189,"outputTokens":1,"latencyMs":2907.059915999882},{"questionId":"q179","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":672,"outputTokens":1,"latencyMs":1820.1195840002038},{"questionId":"q179","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":816,"outputTokens":1,"latencyMs":1407.497791999951},{"questionId":"q179","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1248,"outputTokens":1,"latencyMs":1846.6843750001863},{"questionId":"q179","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":838,"outputTokens":1,"latencyMs":2180.3003340000287},{"questionId":"q180","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":7935,"outputTokens":3,"latencyMs":1212.9160830001347},{"questionId":"q180","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":4723,"outputTokens":3,"latencyMs":1416.3876660000533},{"questionId":"q180","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3368,"outputTokens":3,"latencyMs":1474.308665999677},{"questionId":"q180","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3219,"outputTokens":3,"latencyMs":1068.3357090000063},{"questionId":"q180","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":9120,"outputTokens":3,"latencyMs":1028.4487499999814},{"questionId":"q180","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":5769,"outputTokens":3,"latencyMs":1085.0066249999218},{"questionId":"q181","format":"json-pretty","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":7941,"outputTokens":14,"latencyMs":907.0155420000665},{"questionId":"q181","format":"json-compact","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4729,"outputTokens":14,"latencyMs":1321.8303749999031},{"questionId":"q181","format":"toon","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":3374,"outputTokens":14,"latencyMs":1004.1324999998324},{"questionId":"q181","format":"csv","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":3225,"outputTokens":14,"latencyMs":858.8057090002112},{"questionId":"q181","format":"xml","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":9126,"outputTokens":14,"latencyMs":1852.3972499999218},{"questionId":"q181","format":"yaml","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id, name, email, department, salary, yearsExperience, active","isCorrect":true,"inputTokens":5775,"outputTokens":14,"latencyMs":1557.4962920001708},{"questionId":"q182","format":"json-pretty","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":7938,"outputTokens":1,"latencyMs":1794.7828750000335},{"questionId":"q182","format":"json-compact","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":4726,"outputTokens":1,"latencyMs":1725.9866250003688},{"questionId":"q182","format":"toon","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":3371,"outputTokens":1,"latencyMs":1722.0005839997903},{"questionId":"q182","format":"csv","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":3222,"outputTokens":1,"latencyMs":1028.5410830001347},{"questionId":"q182","format":"xml","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":9123,"outputTokens":1,"latencyMs":1794.104082999751},{"questionId":"q182","format":"yaml","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":5772,"outputTokens":1,"latencyMs":2306.409458000213},{"questionId":"q183","format":"json-pretty","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7939,"outputTokens":1,"latencyMs":2420.0017090002075},{"questionId":"q183","format":"json-compact","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4727,"outputTokens":1,"latencyMs":1487.8882079999894},{"questionId":"q183","format":"toon","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":3372,"outputTokens":1,"latencyMs":2413.9310000003316},{"questionId":"q183","format":"csv","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":3223,"outputTokens":1,"latencyMs":1909.1462090001442},{"questionId":"q183","format":"xml","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":9124,"outputTokens":1,"latencyMs":1290.9736250001006},{"questionId":"q183","format":"yaml","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5773,"outputTokens":1,"latencyMs":1331.7375420001335},{"questionId":"q184","format":"json-pretty","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7939,"outputTokens":5,"latencyMs":1760.4049160000868},{"questionId":"q184","format":"json-compact","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4727,"outputTokens":5,"latencyMs":1648.2857079999521},{"questionId":"q184","format":"toon","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":3372,"outputTokens":5,"latencyMs":1505.4954590001144},{"questionId":"q184","format":"csv","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":3223,"outputTokens":5,"latencyMs":1306.1098330002278},{"questionId":"q184","format":"xml","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":9124,"outputTokens":5,"latencyMs":1157.970042000059},{"questionId":"q184","format":"yaml","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5773,"outputTokens":5,"latencyMs":1380.3544580000453},{"questionId":"q185","format":"json-pretty","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":7936,"outputTokens":1,"latencyMs":1531.8517499999143},{"questionId":"q185","format":"json-compact","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":4724,"outputTokens":1,"latencyMs":1155.6744160000235},{"questionId":"q185","format":"toon","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":3369,"outputTokens":1,"latencyMs":1061.6985830003396},{"questionId":"q185","format":"csv","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":3220,"outputTokens":1,"latencyMs":1593.3009170000441},{"questionId":"q185","format":"xml","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":9121,"outputTokens":1,"latencyMs":1476.6880830000155},{"questionId":"q185","format":"yaml","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":5770,"outputTokens":1,"latencyMs":2076.828875000123},{"questionId":"q186","format":"json-pretty","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":13815,"outputTokens":2,"latencyMs":3474.00549999997},{"questionId":"q186","format":"json-compact","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":8457,"outputTokens":2,"latencyMs":1974.5938329999335},{"questionId":"q186","format":"toon","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":9075,"outputTokens":2,"latencyMs":4834.293082999997},{"questionId":"q186","format":"xml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":15203,"outputTokens":2,"latencyMs":4401.235042000189},{"questionId":"q186","format":"yaml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":10068,"outputTokens":2,"latencyMs":3442.832832999993},{"questionId":"q187","format":"json-pretty","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":13824,"outputTokens":18,"latencyMs":1844.223000000231},{"questionId":"q187","format":"json-compact","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":8466,"outputTokens":18,"latencyMs":2060.5529169999063},{"questionId":"q187","format":"toon","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":9084,"outputTokens":18,"latencyMs":2620.188792000059},{"questionId":"q187","format":"xml","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":15212,"outputTokens":18,"latencyMs":2616.6397499996237},{"questionId":"q187","format":"yaml","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":10077,"outputTokens":18,"latencyMs":2009.7512499997392},{"questionId":"q188","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":13820,"outputTokens":1,"latencyMs":2406.472666999791},{"questionId":"q188","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":8462,"outputTokens":1,"latencyMs":2702.165959000122},{"questionId":"q188","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":9080,"outputTokens":1,"latencyMs":1155.4453329998069},{"questionId":"q188","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":15208,"outputTokens":1,"latencyMs":2955.3977080001496},{"questionId":"q188","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":10073,"outputTokens":1,"latencyMs":970.4963330002502},{"questionId":"q189","format":"json-pretty","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku, name, quantity, price","isCorrect":true,"inputTokens":13824,"outputTokens":7,"latencyMs":917.7760830000043},{"questionId":"q189","format":"json-compact","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":8466,"outputTokens":7,"latencyMs":1141.300125000067},{"questionId":"q189","format":"toon","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":9084,"outputTokens":7,"latencyMs":1438.8795830002055},{"questionId":"q189","format":"xml","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":15212,"outputTokens":7,"latencyMs":1478.7789580002427},{"questionId":"q189","format":"yaml","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku, name, quantity, price","isCorrect":true,"inputTokens":10077,"outputTokens":7,"latencyMs":1917.4187079998665},{"questionId":"q190","format":"json-pretty","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13819,"outputTokens":1,"latencyMs":1082.9091670000926},{"questionId":"q190","format":"json-compact","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8461,"outputTokens":1,"latencyMs":1000.3527910001576},{"questionId":"q190","format":"toon","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9079,"outputTokens":1,"latencyMs":1352.1447499999776},{"questionId":"q190","format":"xml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":15207,"outputTokens":1,"latencyMs":1106.2289579999633},{"questionId":"q190","format":"yaml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10072,"outputTokens":1,"latencyMs":2854.573832999915},{"questionId":"q191","format":"json-pretty","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":13825,"outputTokens":7,"latencyMs":2415.387416999787},{"questionId":"q191","format":"json-compact","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":8467,"outputTokens":7,"latencyMs":2137.6247500004247},{"questionId":"q191","format":"toon","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":9085,"outputTokens":7,"latencyMs":2239.8890420002863},{"questionId":"q191","format":"xml","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":15213,"outputTokens":7,"latencyMs":2401.039417000022},{"questionId":"q191","format":"yaml","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":10078,"outputTokens":7,"latencyMs":2252.2938339998946},{"questionId":"q192","format":"json-pretty","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":4786,"outputTokens":2,"latencyMs":1696.7272079996765},{"questionId":"q192","format":"json-compact","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3034,"outputTokens":2,"latencyMs":1500.7553749997169},{"questionId":"q192","format":"toon","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2297,"outputTokens":2,"latencyMs":3044.6805000002496},{"questionId":"q192","format":"csv","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2211,"outputTokens":2,"latencyMs":5834.432500000112},{"questionId":"q192","format":"xml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":5429,"outputTokens":2,"latencyMs":4872.553334000055},{"questionId":"q192","format":"yaml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3815,"outputTokens":2,"latencyMs":5954.5800000000745},{"questionId":"q193","format":"json-pretty","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4791,"outputTokens":13,"latencyMs":1352.6990840001963},{"questionId":"q193","format":"json-compact","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3039,"outputTokens":13,"latencyMs":981.6302089998499},{"questionId":"q193","format":"toon","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2302,"outputTokens":13,"latencyMs":1188.740167000331},{"questionId":"q193","format":"csv","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2216,"outputTokens":13,"latencyMs":1259.0512500000186},{"questionId":"q193","format":"xml","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":5434,"outputTokens":13,"latencyMs":700.9146659998223},{"questionId":"q193","format":"yaml","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3820,"outputTokens":13,"latencyMs":1130.6056670001708},{"questionId":"q194","format":"json-pretty","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":4789,"outputTokens":1,"latencyMs":1332.6502910000272},{"questionId":"q194","format":"json-compact","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3037,"outputTokens":1,"latencyMs":1124.1449580001645},{"questionId":"q194","format":"toon","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2300,"outputTokens":1,"latencyMs":1335.3164999997243},{"questionId":"q194","format":"csv","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2214,"outputTokens":1,"latencyMs":1323.4112919997424},{"questionId":"q194","format":"xml","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":5432,"outputTokens":1,"latencyMs":2700.08116699988},{"questionId":"q194","format":"yaml","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3818,"outputTokens":1,"latencyMs":1186.6253329999745},{"questionId":"q195","format":"json-pretty","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4790,"outputTokens":10,"latencyMs":984.6150420000777},{"questionId":"q195","format":"json-compact","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3038,"outputTokens":10,"latencyMs":1821.8455419996753},{"questionId":"q195","format":"toon","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2301,"outputTokens":10,"latencyMs":2350.780958000105},{"questionId":"q195","format":"csv","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2215,"outputTokens":10,"latencyMs":1587.6732499999925},{"questionId":"q195","format":"xml","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":5433,"outputTokens":10,"latencyMs":1420.3786249998957},{"questionId":"q195","format":"yaml","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3819,"outputTokens":10,"latencyMs":3651.4653339996003},{"questionId":"q196","format":"json-pretty","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":4786,"outputTokens":1,"latencyMs":1254.4762920001522},{"questionId":"q196","format":"json-compact","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":3034,"outputTokens":1,"latencyMs":1484.8886659997515},{"questionId":"q196","format":"toon","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":2297,"outputTokens":1,"latencyMs":1758.2165840002708},{"questionId":"q196","format":"csv","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":2211,"outputTokens":1,"latencyMs":1691.111917000264},{"questionId":"q196","format":"xml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":5429,"outputTokens":1,"latencyMs":1848.8265419998206},{"questionId":"q196","format":"yaml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":3815,"outputTokens":1,"latencyMs":1692.0818749996834},{"questionId":"q197","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"64","isCorrect":false,"inputTokens":20006,"outputTokens":2,"latencyMs":1461.7679169997573},{"questionId":"q197","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"71","isCorrect":false,"inputTokens":15044,"outputTokens":2,"latencyMs":1209.0277500003576},{"questionId":"q197","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":12376,"outputTokens":3,"latencyMs":51040.35479200026},{"questionId":"q197","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"47","isCorrect":false,"inputTokens":12223,"outputTokens":2,"latencyMs":1551.7953329999},{"questionId":"q197","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"50","isCorrect":false,"inputTokens":21892,"outputTokens":2,"latencyMs":965.8400829997845},{"questionId":"q197","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":17090,"outputTokens":3,"latencyMs":7417.507125000004},{"questionId":"q198","format":"json-pretty","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":20012,"outputTokens":27,"latencyMs":1154.3140420001},{"questionId":"q198","format":"json-compact","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":15050,"outputTokens":27,"latencyMs":1266.092792000156},{"questionId":"q198","format":"toon","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":12382,"outputTokens":27,"latencyMs":1173.6531249997206},{"questionId":"q198","format":"csv","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":12229,"outputTokens":27,"latencyMs":1641.626000000164},{"questionId":"q198","format":"xml","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":21898,"outputTokens":27,"latencyMs":1404.4440409997478},{"questionId":"q198","format":"yaml","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":17096,"outputTokens":27,"latencyMs":961.2164590000175},{"questionId":"q199","format":"json-pretty","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":20010,"outputTokens":3,"latencyMs":1519.7402499997988},{"questionId":"q199","format":"json-compact","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":15048,"outputTokens":3,"latencyMs":1453.4793750001118},{"questionId":"q199","format":"toon","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":12380,"outputTokens":3,"latencyMs":1862.0968329999596},{"questionId":"q199","format":"csv","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":12227,"outputTokens":3,"latencyMs":1967.816416000016},{"questionId":"q199","format":"xml","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":21896,"outputTokens":3,"latencyMs":3684.9720409996808},{"questionId":"q199","format":"yaml","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":17094,"outputTokens":3,"latencyMs":2438.554749999661},{"questionId":"q200","format":"json-pretty","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":20010,"outputTokens":1,"latencyMs":1556.8056669998914},{"questionId":"q200","format":"json-compact","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":15048,"outputTokens":1,"latencyMs":1722.4028329998255},{"questionId":"q200","format":"toon","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":12380,"outputTokens":1,"latencyMs":2272.2974580000155},{"questionId":"q200","format":"csv","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":12227,"outputTokens":1,"latencyMs":3092.4380830000155},{"questionId":"q200","format":"xml","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":21896,"outputTokens":1,"latencyMs":2527.161958000157},{"questionId":"q200","format":"yaml","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17094,"outputTokens":1,"latencyMs":2655.2592499996535},{"questionId":"q201","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":20007,"outputTokens":2,"latencyMs":3692.654957999941},{"questionId":"q201","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":15045,"outputTokens":2,"latencyMs":2381.5062090000138},{"questionId":"q201","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":12377,"outputTokens":2,"latencyMs":1285.6337910001166},{"questionId":"q201","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":12224,"outputTokens":2,"latencyMs":1767.9712919997983},{"questionId":"q201","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":21893,"outputTokens":2,"latencyMs":3908.708417000249},{"questionId":"q201","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":17091,"outputTokens":2,"latencyMs":2645.853124999907},{"questionId":"q202","format":"json-pretty","model":"gemini-2.5-flash","expected":"75","actual":"60","isCorrect":false,"inputTokens":8514,"outputTokens":2,"latencyMs":885.8707079999149},{"questionId":"q202","format":"json-compact","model":"gemini-2.5-flash","expected":"75","actual":"75","isCorrect":true,"inputTokens":5841,"outputTokens":2,"latencyMs":11676.807790999766},{"questionId":"q202","format":"toon","model":"gemini-2.5-flash","expected":"75","actual":"61","isCorrect":false,"inputTokens":6968,"outputTokens":2,"latencyMs":3322.3034589998424},{"questionId":"q202","format":"xml","model":"gemini-2.5-flash","expected":"75","actual":"58","isCorrect":false,"inputTokens":9337,"outputTokens":2,"latencyMs":867.7002920000814},{"questionId":"q202","format":"yaml","model":"gemini-2.5-flash","expected":"75","actual":"75","isCorrect":true,"inputTokens":6895,"outputTokens":2,"latencyMs":10668.92804200016},{"questionId":"q203","format":"json-pretty","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error.message,error.stack,error.retryable","isCorrect":false,"inputTokens":8524,"outputTokens":24,"latencyMs":5134.065167000052},{"questionId":"q203","format":"json-compact","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":true,"inputTokens":5851,"outputTokens":20,"latencyMs":2453.064499999862},{"questionId":"q203","format":"toon","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":true,"inputTokens":6978,"outputTokens":20,"latencyMs":2511.5249580000527},{"questionId":"q203","format":"xml","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":true,"inputTokens":9347,"outputTokens":20,"latencyMs":3595.069500000216},{"questionId":"q203","format":"yaml","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error","isCorrect":true,"inputTokens":6905,"outputTokens":14,"latencyMs":3624.207583999727},{"questionId":"q204","format":"json-pretty","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":8518,"outputTokens":1,"latencyMs":2568.4681669999845},{"questionId":"q204","format":"json-compact","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":5845,"outputTokens":1,"latencyMs":1242.1411659996957},{"questionId":"q204","format":"toon","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":6972,"outputTokens":1,"latencyMs":1972.8034999999218},{"questionId":"q204","format":"xml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":9341,"outputTokens":1,"latencyMs":1536.450375000015},{"questionId":"q204","format":"yaml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":6899,"outputTokens":1,"latencyMs":2199.339999999851}] \ No newline at end of file +[{"questionId":"q1","format":"json-pretty","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7986,"outputTokens":5,"latencyMs":1656.011875},{"questionId":"q1","format":"json-compact","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":4774,"outputTokens":5,"latencyMs":1836.582625},{"questionId":"q1","format":"toon","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":3419,"outputTokens":5,"latencyMs":1990.5067499999998},{"questionId":"q1","format":"csv","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":3270,"outputTokens":5,"latencyMs":2153.538541},{"questionId":"q1","format":"xml","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":9171,"outputTokens":5,"latencyMs":1564.4797500000002},{"questionId":"q1","format":"yaml","model":"gemini-2.5-flash","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5820,"outputTokens":5,"latencyMs":1666.6015000000002},{"questionId":"q2","format":"json-pretty","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7986,"outputTokens":1,"latencyMs":1613.088416},{"questionId":"q2","format":"json-compact","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":4774,"outputTokens":1,"latencyMs":1776.978792},{"questionId":"q2","format":"toon","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":3419,"outputTokens":1,"latencyMs":2641.761792},{"questionId":"q2","format":"csv","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":3270,"outputTokens":1,"latencyMs":1456.5100419999999},{"questionId":"q2","format":"xml","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":9171,"outputTokens":1,"latencyMs":807.640625},{"questionId":"q2","format":"yaml","model":"gemini-2.5-flash","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5820,"outputTokens":1,"latencyMs":1162.4529589999997},{"questionId":"q3","format":"json-pretty","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7989,"outputTokens":10,"latencyMs":1202.8930829999995},{"questionId":"q3","format":"json-compact","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4777,"outputTokens":10,"latencyMs":1251.4219999999996},{"questionId":"q3","format":"toon","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":3422,"outputTokens":10,"latencyMs":1941.3606670000004},{"questionId":"q3","format":"csv","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":3273,"outputTokens":10,"latencyMs":1868.6544170000006},{"questionId":"q3","format":"xml","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":9174,"outputTokens":10,"latencyMs":1990.0292080000004},{"questionId":"q3","format":"yaml","model":"gemini-2.5-flash","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5823,"outputTokens":10,"latencyMs":1382.726666},{"questionId":"q4","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":7989,"outputTokens":2,"latencyMs":1211.5154159999997},{"questionId":"q4","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4777,"outputTokens":2,"latencyMs":1284.2250419999996},{"questionId":"q4","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3422,"outputTokens":2,"latencyMs":1484.9246250000006},{"questionId":"q4","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3273,"outputTokens":2,"latencyMs":1365.4948329999997},{"questionId":"q4","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":9174,"outputTokens":2,"latencyMs":1420.2346659999994},{"questionId":"q4","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5823,"outputTokens":2,"latencyMs":871.4260000000004},{"questionId":"q5","format":"json-pretty","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":7984,"outputTokens":1,"latencyMs":1855.9904999999999},{"questionId":"q5","format":"json-compact","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":4772,"outputTokens":1,"latencyMs":1611.3014170000024},{"questionId":"q5","format":"toon","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":3417,"outputTokens":1,"latencyMs":1902.3068340000027},{"questionId":"q5","format":"csv","model":"gemini-2.5-flash","expected":"no","actual":"No","isCorrect":true,"inputTokens":3268,"outputTokens":1,"latencyMs":5743.748124999998},{"questionId":"q5","format":"xml","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":9169,"outputTokens":1,"latencyMs":1834.2512500000012},{"questionId":"q5","format":"yaml","model":"gemini-2.5-flash","expected":"no","actual":"false","isCorrect":true,"inputTokens":5818,"outputTokens":1,"latencyMs":1439.1392500000002},{"questionId":"q6","format":"json-pretty","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7985,"outputTokens":6,"latencyMs":2210.3305000000037},{"questionId":"q6","format":"json-compact","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":4773,"outputTokens":6,"latencyMs":1498.5363750000033},{"questionId":"q6","format":"toon","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":3418,"outputTokens":6,"latencyMs":1485.365417000001},{"questionId":"q6","format":"csv","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":3269,"outputTokens":6,"latencyMs":1691.1132079999952},{"questionId":"q6","format":"xml","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":9170,"outputTokens":6,"latencyMs":1697.1664170000004},{"questionId":"q6","format":"yaml","model":"gemini-2.5-flash","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5819,"outputTokens":6,"latencyMs":1321.5843749999985},{"questionId":"q7","format":"json-pretty","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7985,"outputTokens":1,"latencyMs":1226.601042000002},{"questionId":"q7","format":"json-compact","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":4773,"outputTokens":1,"latencyMs":990.0980420000051},{"questionId":"q7","format":"toon","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":3418,"outputTokens":1,"latencyMs":2441.042042000001},{"questionId":"q7","format":"csv","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":3269,"outputTokens":1,"latencyMs":2131.032417000002},{"questionId":"q7","format":"xml","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":9170,"outputTokens":1,"latencyMs":1561.1451659999948},{"questionId":"q7","format":"yaml","model":"gemini-2.5-flash","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5819,"outputTokens":1,"latencyMs":2547.7474159999983},{"questionId":"q8","format":"json-pretty","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7988,"outputTokens":9,"latencyMs":1626.634957999995},{"questionId":"q8","format":"json-compact","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4776,"outputTokens":9,"latencyMs":2167.277374999998},{"questionId":"q8","format":"toon","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":3421,"outputTokens":9,"latencyMs":1904.266791999995},{"questionId":"q8","format":"csv","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":3272,"outputTokens":9,"latencyMs":1679.4006250000093},{"questionId":"q8","format":"xml","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":9173,"outputTokens":9,"latencyMs":1618.3975839999985},{"questionId":"q8","format":"yaml","model":"gemini-2.5-flash","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5822,"outputTokens":9,"latencyMs":1114.6244999999908},{"questionId":"q9","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":7989,"outputTokens":1,"latencyMs":1092.0109160000065},{"questionId":"q9","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":4777,"outputTokens":1,"latencyMs":1581.2634999999937},{"questionId":"q9","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":3422,"outputTokens":1,"latencyMs":1663.1554579999938},{"questionId":"q9","format":"csv","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":3273,"outputTokens":1,"latencyMs":1654.9675410000054},{"questionId":"q9","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":9174,"outputTokens":1,"latencyMs":1647.651582999999},{"questionId":"q9","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":5823,"outputTokens":1,"latencyMs":2846.678500000009},{"questionId":"q10","format":"json-pretty","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":7986,"outputTokens":1,"latencyMs":1785.317750000002},{"questionId":"q10","format":"json-compact","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":4774,"outputTokens":1,"latencyMs":1635.0255000000034},{"questionId":"q10","format":"toon","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":3419,"outputTokens":1,"latencyMs":1641.1004169999942},{"questionId":"q10","format":"csv","model":"gemini-2.5-flash","expected":"yes","actual":"1","isCorrect":true,"inputTokens":3270,"outputTokens":1,"latencyMs":2245.7938330000034},{"questionId":"q10","format":"xml","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":9171,"outputTokens":1,"latencyMs":1768.680416999996},{"questionId":"q10","format":"yaml","model":"gemini-2.5-flash","expected":"yes","actual":"true","isCorrect":true,"inputTokens":5820,"outputTokens":1,"latencyMs":1190.7126250000001},{"questionId":"q11","format":"json-pretty","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7987,"outputTokens":6,"latencyMs":1142.570583000008},{"questionId":"q11","format":"json-compact","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":4775,"outputTokens":6,"latencyMs":1397.6600840000028},{"questionId":"q11","format":"toon","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":3420,"outputTokens":6,"latencyMs":2107.9766669999954},{"questionId":"q11","format":"csv","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":3271,"outputTokens":6,"latencyMs":1493.9762499999924},{"questionId":"q11","format":"xml","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":9172,"outputTokens":6,"latencyMs":1495.257375000001},{"questionId":"q11","format":"yaml","model":"gemini-2.5-flash","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5821,"outputTokens":6,"latencyMs":1624.1958339999983},{"questionId":"q12","format":"json-pretty","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7985,"outputTokens":1,"latencyMs":1831.3699999999953},{"questionId":"q12","format":"json-compact","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":4773,"outputTokens":1,"latencyMs":1353.909083000006},{"questionId":"q12","format":"toon","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":3418,"outputTokens":1,"latencyMs":1516.1869589999988},{"questionId":"q12","format":"csv","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":3269,"outputTokens":1,"latencyMs":1187.5398749999877},{"questionId":"q12","format":"xml","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":9170,"outputTokens":1,"latencyMs":1111.8902499999967},{"questionId":"q12","format":"yaml","model":"gemini-2.5-flash","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5819,"outputTokens":1,"latencyMs":1508.4381670000002},{"questionId":"q13","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":7984,"outputTokens":2,"latencyMs":2812.382124999989},{"questionId":"q13","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"14","isCorrect":false,"inputTokens":4772,"outputTokens":2,"latencyMs":1029.4419159999961},{"questionId":"q13","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3417,"outputTokens":2,"latencyMs":3800.1013339999918},{"questionId":"q13","format":"csv","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3268,"outputTokens":2,"latencyMs":2610.6463329999824},{"questionId":"q13","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":9169,"outputTokens":2,"latencyMs":6352.012625000003},{"questionId":"q13","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"13","isCorrect":false,"inputTokens":5818,"outputTokens":2,"latencyMs":1283.252999999997},{"questionId":"q14","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"18","isCorrect":false,"inputTokens":7984,"outputTokens":2,"latencyMs":1847.9974999999977},{"questionId":"q14","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"14","isCorrect":false,"inputTokens":4772,"outputTokens":2,"latencyMs":1387.544790999993},{"questionId":"q14","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3417,"outputTokens":2,"latencyMs":14960.001749999996},{"questionId":"q14","format":"csv","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3268,"outputTokens":2,"latencyMs":4283.818834000005},{"questionId":"q14","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"15","isCorrect":false,"inputTokens":9169,"outputTokens":2,"latencyMs":1330.6379999999772},{"questionId":"q14","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":5818,"outputTokens":2,"latencyMs":1432.2300420000101},{"questionId":"q15","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"15","isCorrect":false,"inputTokens":7984,"outputTokens":2,"latencyMs":1851.5004160000244},{"questionId":"q15","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"14","isCorrect":false,"inputTokens":4772,"outputTokens":2,"latencyMs":955.9562500000175},{"questionId":"q15","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3417,"outputTokens":2,"latencyMs":14948.072749999992},{"questionId":"q15","format":"csv","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":3268,"outputTokens":2,"latencyMs":1051.2674169999955},{"questionId":"q15","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"15","isCorrect":false,"inputTokens":9169,"outputTokens":2,"latencyMs":1000.2221669999999},{"questionId":"q15","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"17","isCorrect":true,"inputTokens":5818,"outputTokens":2,"latencyMs":791.3987079999933},{"questionId":"q16","format":"json-pretty","model":"gemini-2.5-flash","expected":"91","actual":"89","isCorrect":false,"inputTokens":7992,"outputTokens":2,"latencyMs":1048.6172499999811},{"questionId":"q16","format":"json-compact","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":4780,"outputTokens":2,"latencyMs":11066.713958000008},{"questionId":"q16","format":"toon","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":3425,"outputTokens":2,"latencyMs":24111.375916999998},{"questionId":"q16","format":"csv","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":3276,"outputTokens":2,"latencyMs":10385.846791999997},{"questionId":"q16","format":"xml","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":9177,"outputTokens":2,"latencyMs":13736.646999999997},{"questionId":"q16","format":"yaml","model":"gemini-2.5-flash","expected":"91","actual":"91","isCorrect":true,"inputTokens":5826,"outputTokens":2,"latencyMs":55293.06004099999},{"questionId":"q17","format":"json-pretty","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":7992,"outputTokens":2,"latencyMs":9229.357541999983},{"questionId":"q17","format":"json-compact","model":"gemini-2.5-flash","expected":"67","actual":"82","isCorrect":false,"inputTokens":4780,"outputTokens":2,"latencyMs":997.8179579999996},{"questionId":"q17","format":"toon","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":3425,"outputTokens":2,"latencyMs":28663.185832999996},{"questionId":"q17","format":"csv","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":3276,"outputTokens":2,"latencyMs":36499.74437499998},{"questionId":"q17","format":"xml","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":9177,"outputTokens":2,"latencyMs":11665.665500000003},{"questionId":"q17","format":"yaml","model":"gemini-2.5-flash","expected":"67","actual":"67","isCorrect":true,"inputTokens":5826,"outputTokens":2,"latencyMs":9416.106792000006},{"questionId":"q18","format":"json-pretty","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":7993,"outputTokens":2,"latencyMs":13151.061041999987},{"questionId":"q18","format":"json-compact","model":"gemini-2.5-flash","expected":"41","actual":"32","isCorrect":false,"inputTokens":4781,"outputTokens":2,"latencyMs":9203.242832999997},{"questionId":"q18","format":"toon","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":3426,"outputTokens":2,"latencyMs":26275.780416000023},{"questionId":"q18","format":"csv","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":3277,"outputTokens":2,"latencyMs":15187.939042000013},{"questionId":"q18","format":"xml","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":9178,"outputTokens":2,"latencyMs":9186.644291000004},{"questionId":"q18","format":"yaml","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":5827,"outputTokens":2,"latencyMs":9817.314708999998},{"questionId":"q19","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":7985,"outputTokens":3,"latencyMs":1320.543708000012},{"questionId":"q19","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":4773,"outputTokens":3,"latencyMs":2034.6812079999945},{"questionId":"q19","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3418,"outputTokens":3,"latencyMs":3272.146875000006},{"questionId":"q19","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3269,"outputTokens":3,"latencyMs":1796.5952920000127},{"questionId":"q19","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":9170,"outputTokens":3,"latencyMs":1156.749582999997},{"questionId":"q19","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":5819,"outputTokens":3,"latencyMs":1236.8168329999899},{"questionId":"q20","format":"json-pretty","model":"gemini-2.5-flash","expected":"96503","actual":"107871.63","isCorrect":false,"inputTokens":7986,"outputTokens":9,"latencyMs":7464.087333000003},{"questionId":"q20","format":"json-compact","model":"gemini-2.5-flash","expected":"96503","actual":"98852.36","isCorrect":false,"inputTokens":4774,"outputTokens":8,"latencyMs":7041.502083999978},{"questionId":"q20","format":"toon","model":"gemini-2.5-flash","expected":"96503","actual":"96472.48","isCorrect":false,"inputTokens":3419,"outputTokens":8,"latencyMs":11081.598916999996},{"questionId":"q20","format":"csv","model":"gemini-2.5-flash","expected":"96503","actual":"95204.66","isCorrect":false,"inputTokens":3270,"outputTokens":8,"latencyMs":1045.7930840000045},{"questionId":"q20","format":"xml","model":"gemini-2.5-flash","expected":"96503","actual":"96426.37","isCorrect":false,"inputTokens":9171,"outputTokens":8,"latencyMs":8985.494000000006},{"questionId":"q20","format":"yaml","model":"gemini-2.5-flash","expected":"96503","actual":"98154","isCorrect":false,"inputTokens":5820,"outputTokens":5,"latencyMs":7470.7623329999915},{"questionId":"q21","format":"json-pretty","model":"gemini-2.5-flash","expected":"78","actual":"76","isCorrect":false,"inputTokens":7983,"outputTokens":2,"latencyMs":1150.2314999999944},{"questionId":"q21","format":"json-compact","model":"gemini-2.5-flash","expected":"78","actual":"76","isCorrect":false,"inputTokens":4771,"outputTokens":2,"latencyMs":1013.1046250000072},{"questionId":"q21","format":"toon","model":"gemini-2.5-flash","expected":"78","actual":"78","isCorrect":true,"inputTokens":3416,"outputTokens":2,"latencyMs":6890.548999999999},{"questionId":"q21","format":"csv","model":"gemini-2.5-flash","expected":"78","actual":"78","isCorrect":true,"inputTokens":3267,"outputTokens":2,"latencyMs":5611.0597919999855},{"questionId":"q21","format":"xml","model":"gemini-2.5-flash","expected":"78","actual":"83","isCorrect":false,"inputTokens":9168,"outputTokens":2,"latencyMs":1223.384665999998},{"questionId":"q21","format":"yaml","model":"gemini-2.5-flash","expected":"78","actual":"86","isCorrect":false,"inputTokens":5817,"outputTokens":2,"latencyMs":1372.0235840000096},{"questionId":"q22","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"17","isCorrect":false,"inputTokens":7983,"outputTokens":2,"latencyMs":1376.9362090000068},{"questionId":"q22","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4771,"outputTokens":2,"latencyMs":7622.860374999989},{"questionId":"q22","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3416,"outputTokens":2,"latencyMs":4612.219999999972},{"questionId":"q22","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3267,"outputTokens":2,"latencyMs":3386.404750000045},{"questionId":"q22","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":9168,"outputTokens":2,"latencyMs":2973.6921250000014},{"questionId":"q22","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5817,"outputTokens":2,"latencyMs":5995.901874999981},{"questionId":"q23","format":"json-pretty","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":7994,"outputTokens":2,"latencyMs":3474.671291000035},{"questionId":"q23","format":"json-compact","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":4782,"outputTokens":2,"latencyMs":7617.479875000019},{"questionId":"q23","format":"toon","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3427,"outputTokens":2,"latencyMs":8030.814083000005},{"questionId":"q23","format":"csv","model":"gemini-2.5-flash","expected":"12","actual":"13","isCorrect":false,"inputTokens":3278,"outputTokens":2,"latencyMs":6762.8348750000005},{"questionId":"q23","format":"xml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":9179,"outputTokens":2,"latencyMs":10368.729374999995},{"questionId":"q23","format":"yaml","model":"gemini-2.5-flash","expected":"12","actual":"13","isCorrect":false,"inputTokens":5828,"outputTokens":2,"latencyMs":3185.3557080000173},{"questionId":"q24","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7994,"outputTokens":2,"latencyMs":3394.8552079999936},{"questionId":"q24","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4782,"outputTokens":2,"latencyMs":9425.736166999966},{"questionId":"q24","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3427,"outputTokens":2,"latencyMs":17100.70212500001},{"questionId":"q24","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3278,"outputTokens":2,"latencyMs":18132.481333000003},{"questionId":"q24","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"10","isCorrect":false,"inputTokens":9179,"outputTokens":2,"latencyMs":2952.896833000006},{"questionId":"q24","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5828,"outputTokens":2,"latencyMs":10975.772167000046},{"questionId":"q25","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7994,"outputTokens":2,"latencyMs":3117.427291000029},{"questionId":"q25","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4782,"outputTokens":2,"latencyMs":12603.213208000001},{"questionId":"q25","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3427,"outputTokens":2,"latencyMs":5332.389332999999},{"questionId":"q25","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3278,"outputTokens":2,"latencyMs":5728.379917000013},{"questionId":"q25","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9179,"outputTokens":2,"latencyMs":4594.727542000008},{"questionId":"q25","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"10","isCorrect":false,"inputTokens":5828,"outputTokens":2,"latencyMs":1278.7732909999904},{"questionId":"q26","format":"json-pretty","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":7994,"outputTokens":2,"latencyMs":4840.823583999998},{"questionId":"q26","format":"json-compact","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":4782,"outputTokens":2,"latencyMs":3303.833459000045},{"questionId":"q26","format":"toon","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3427,"outputTokens":2,"latencyMs":12179.259167000011},{"questionId":"q26","format":"csv","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3278,"outputTokens":2,"latencyMs":5701.590749999974},{"questionId":"q26","format":"xml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":9179,"outputTokens":2,"latencyMs":11773.724583000003},{"questionId":"q26","format":"yaml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":5828,"outputTokens":2,"latencyMs":10865.65741699998},{"questionId":"q27","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7994,"outputTokens":2,"latencyMs":5065.205957999977},{"questionId":"q27","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4782,"outputTokens":2,"latencyMs":3724.4394999999786},{"questionId":"q27","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3427,"outputTokens":2,"latencyMs":19634.759624999948},{"questionId":"q27","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3278,"outputTokens":2,"latencyMs":6665.829791000055},{"questionId":"q27","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9179,"outputTokens":2,"latencyMs":11677.283333999978},{"questionId":"q27","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5828,"outputTokens":2,"latencyMs":2939.5036669999827},{"questionId":"q28","format":"json-pretty","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":7990,"outputTokens":2,"latencyMs":22423.697958000004},{"questionId":"q28","format":"json-compact","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":4778,"outputTokens":2,"latencyMs":16059.056417000014},{"questionId":"q28","format":"toon","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":3423,"outputTokens":2,"latencyMs":33466.33620900003},{"questionId":"q28","format":"csv","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":3274,"outputTokens":2,"latencyMs":19364.76795900002},{"questionId":"q28","format":"xml","model":"gemini-2.5-flash","expected":"63","actual":"63","isCorrect":true,"inputTokens":9175,"outputTokens":2,"latencyMs":18327.188874999993},{"questionId":"q28","format":"yaml","model":"gemini-2.5-flash","expected":"63","actual":"81","isCorrect":false,"inputTokens":5824,"outputTokens":2,"latencyMs":954.987833000021},{"questionId":"q29","format":"json-pretty","model":"gemini-2.5-flash","expected":"53","actual":"52","isCorrect":false,"inputTokens":7991,"outputTokens":2,"latencyMs":10784.046957999992},{"questionId":"q29","format":"json-compact","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":4779,"outputTokens":2,"latencyMs":10300.17745800002},{"questionId":"q29","format":"toon","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":3424,"outputTokens":2,"latencyMs":27243.58712500002},{"questionId":"q29","format":"csv","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":3275,"outputTokens":2,"latencyMs":18767.33391699998},{"questionId":"q29","format":"xml","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":9176,"outputTokens":2,"latencyMs":13746.828375000041},{"questionId":"q29","format":"yaml","model":"gemini-2.5-flash","expected":"53","actual":"53","isCorrect":true,"inputTokens":5825,"outputTokens":2,"latencyMs":18417.737000000023},{"questionId":"q30","format":"json-pretty","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":7991,"outputTokens":2,"latencyMs":15577.846417000052},{"questionId":"q30","format":"json-compact","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":4779,"outputTokens":2,"latencyMs":14562.397292000009},{"questionId":"q30","format":"toon","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":3424,"outputTokens":2,"latencyMs":21743.736875000002},{"questionId":"q30","format":"csv","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":3275,"outputTokens":2,"latencyMs":12514.55441599997},{"questionId":"q30","format":"xml","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":9176,"outputTokens":2,"latencyMs":11103.440208000015},{"questionId":"q30","format":"yaml","model":"gemini-2.5-flash","expected":"39","actual":"39","isCorrect":true,"inputTokens":5825,"outputTokens":2,"latencyMs":9557.580541000003},{"questionId":"q31","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":7992,"outputTokens":2,"latencyMs":3189.6394590000273},{"questionId":"q31","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4780,"outputTokens":2,"latencyMs":3602.683958999987},{"questionId":"q31","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3425,"outputTokens":2,"latencyMs":4698.791334000009},{"questionId":"q31","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3276,"outputTokens":2,"latencyMs":20750.543124999967},{"questionId":"q31","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9177,"outputTokens":2,"latencyMs":3688.8777080000145},{"questionId":"q31","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":5826,"outputTokens":2,"latencyMs":3180.416292000038},{"questionId":"q32","format":"json-pretty","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":7992,"outputTokens":1,"latencyMs":6653.546875},{"questionId":"q32","format":"json-compact","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":4780,"outputTokens":1,"latencyMs":4528.003459000029},{"questionId":"q32","format":"toon","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":3425,"outputTokens":1,"latencyMs":18510.533415999962},{"questionId":"q32","format":"csv","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":3276,"outputTokens":1,"latencyMs":4844.818957999989},{"questionId":"q32","format":"xml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":9177,"outputTokens":1,"latencyMs":7291.041833000025},{"questionId":"q32","format":"yaml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":5826,"outputTokens":1,"latencyMs":7180.915791000007},{"questionId":"q33","format":"json-pretty","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":7992,"outputTokens":2,"latencyMs":7553.364500000025},{"questionId":"q33","format":"json-compact","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":4780,"outputTokens":2,"latencyMs":6148.510416999983},{"questionId":"q33","format":"toon","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":3425,"outputTokens":2,"latencyMs":5060.540749999986},{"questionId":"q33","format":"csv","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":3276,"outputTokens":2,"latencyMs":19560.352874999982},{"questionId":"q33","format":"xml","model":"gemini-2.5-flash","expected":"15","actual":"11","isCorrect":false,"inputTokens":9177,"outputTokens":2,"latencyMs":1009.3627080000006},{"questionId":"q33","format":"yaml","model":"gemini-2.5-flash","expected":"15","actual":"15","isCorrect":true,"inputTokens":5826,"outputTokens":2,"latencyMs":7510.184833000007},{"questionId":"q34","format":"json-pretty","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":7985,"outputTokens":2,"latencyMs":8230.049332999974},{"questionId":"q34","format":"json-compact","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":4773,"outputTokens":2,"latencyMs":3204.524166999967},{"questionId":"q34","format":"toon","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3418,"outputTokens":2,"latencyMs":14734.818499999994},{"questionId":"q34","format":"csv","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":3269,"outputTokens":2,"latencyMs":8128.005250000046},{"questionId":"q34","format":"xml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":9170,"outputTokens":2,"latencyMs":8189.758208000043},{"questionId":"q34","format":"yaml","model":"gemini-2.5-flash","expected":"12","actual":"12","isCorrect":true,"inputTokens":5819,"outputTokens":2,"latencyMs":1721.1036249999888},{"questionId":"q35","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"15","isCorrect":false,"inputTokens":7985,"outputTokens":2,"latencyMs":1512.4792080000043},{"questionId":"q35","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":4773,"outputTokens":2,"latencyMs":6352.312875000003},{"questionId":"q35","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3418,"outputTokens":2,"latencyMs":6859.721625000006},{"questionId":"q35","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":3269,"outputTokens":2,"latencyMs":9141.87937499996},{"questionId":"q35","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":9170,"outputTokens":2,"latencyMs":7320.847083000001},{"questionId":"q35","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"12","isCorrect":false,"inputTokens":5819,"outputTokens":2,"latencyMs":1744.2551669999957},{"questionId":"q36","format":"json-pretty","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":13870,"outputTokens":6,"latencyMs":911.3584169999813},{"questionId":"q36","format":"json-compact","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8512,"outputTokens":6,"latencyMs":1251.7307920000167},{"questionId":"q36","format":"toon","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":9130,"outputTokens":6,"latencyMs":1039.0112920000101},{"questionId":"q36","format":"xml","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":15258,"outputTokens":6,"latencyMs":877.2927090000012},{"questionId":"q36","format":"yaml","model":"gemini-2.5-flash","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":10123,"outputTokens":6,"latencyMs":1009.1390409999876},{"questionId":"q37","format":"json-pretty","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":13870,"outputTokens":2,"latencyMs":864.3567500000354},{"questionId":"q37","format":"json-compact","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8512,"outputTokens":2,"latencyMs":1043.0689159999602},{"questionId":"q37","format":"toon","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":9130,"outputTokens":2,"latencyMs":1443.2314580000238},{"questionId":"q37","format":"xml","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":15258,"outputTokens":2,"latencyMs":879.7593750000233},{"questionId":"q37","format":"yaml","model":"gemini-2.5-flash","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":10123,"outputTokens":2,"latencyMs":1212.7902919999906},{"questionId":"q38","format":"json-pretty","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":13870,"outputTokens":5,"latencyMs":1020.4506669999682},{"questionId":"q38","format":"json-compact","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8512,"outputTokens":5,"latencyMs":1437.9718329999596},{"questionId":"q38","format":"toon","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":9130,"outputTokens":5,"latencyMs":985.947583000001},{"questionId":"q38","format":"xml","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":15258,"outputTokens":5,"latencyMs":2481.873500000045},{"questionId":"q38","format":"yaml","model":"gemini-2.5-flash","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":10123,"outputTokens":5,"latencyMs":1156.137208},{"questionId":"q39","format":"json-pretty","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":13870,"outputTokens":1,"latencyMs":1803.2302080000518},{"questionId":"q39","format":"json-compact","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8512,"outputTokens":1,"latencyMs":1235.3869159999304},{"questionId":"q39","format":"toon","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":9130,"outputTokens":1,"latencyMs":2971.1143750000047},{"questionId":"q39","format":"xml","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":15258,"outputTokens":1,"latencyMs":1187.5562910001026},{"questionId":"q39","format":"yaml","model":"gemini-2.5-flash","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":10123,"outputTokens":1,"latencyMs":1447.1040420000209},{"questionId":"q40","format":"json-pretty","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":13870,"outputTokens":7,"latencyMs":1299.890167000005},{"questionId":"q40","format":"json-compact","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8512,"outputTokens":7,"latencyMs":1348.858332999982},{"questionId":"q40","format":"toon","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":9130,"outputTokens":7,"latencyMs":1591.6766669999342},{"questionId":"q40","format":"xml","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":15258,"outputTokens":7,"latencyMs":1306.8049579999642},{"questionId":"q40","format":"yaml","model":"gemini-2.5-flash","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":10123,"outputTokens":7,"latencyMs":1690.2915420000209},{"questionId":"q41","format":"json-pretty","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":13870,"outputTokens":1,"latencyMs":1279.3167920000851},{"questionId":"q41","format":"json-compact","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8512,"outputTokens":1,"latencyMs":1154.5741670000134},{"questionId":"q41","format":"toon","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":9130,"outputTokens":1,"latencyMs":1253.4435829999857},{"questionId":"q41","format":"xml","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":15258,"outputTokens":1,"latencyMs":1238.2858339999802},{"questionId":"q41","format":"yaml","model":"gemini-2.5-flash","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":10123,"outputTokens":1,"latencyMs":1097.853333999985},{"questionId":"q42","format":"json-pretty","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":13870,"outputTokens":7,"latencyMs":865.839583999943},{"questionId":"q42","format":"json-compact","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8512,"outputTokens":7,"latencyMs":1098.3405410000123},{"questionId":"q42","format":"toon","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":9130,"outputTokens":7,"latencyMs":940.1808749999618},{"questionId":"q42","format":"xml","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":15258,"outputTokens":7,"latencyMs":974.1252499999246},{"questionId":"q42","format":"yaml","model":"gemini-2.5-flash","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":10123,"outputTokens":7,"latencyMs":988.7236669999547},{"questionId":"q43","format":"json-pretty","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13870,"outputTokens":1,"latencyMs":1438.7290000000503},{"questionId":"q43","format":"json-compact","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8512,"outputTokens":1,"latencyMs":1406.2638329999754},{"questionId":"q43","format":"toon","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9130,"outputTokens":1,"latencyMs":1141.3426250000484},{"questionId":"q43","format":"xml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":15258,"outputTokens":1,"latencyMs":1028.0277500000084},{"questionId":"q43","format":"yaml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10123,"outputTokens":1,"latencyMs":1182.3112919999985},{"questionId":"q44","format":"json-pretty","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":13871,"outputTokens":5,"latencyMs":1463.574874999933},{"questionId":"q44","format":"json-compact","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8513,"outputTokens":5,"latencyMs":1269.9729580000276},{"questionId":"q44","format":"toon","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":9131,"outputTokens":5,"latencyMs":5270.121959000011},{"questionId":"q44","format":"xml","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":15259,"outputTokens":5,"latencyMs":1153.3834579999093},{"questionId":"q44","format":"yaml","model":"gemini-2.5-flash","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":10124,"outputTokens":5,"latencyMs":1689.9718339999672},{"questionId":"q45","format":"json-pretty","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":13871,"outputTokens":13,"latencyMs":1655.905625000014},{"questionId":"q45","format":"json-compact","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8513,"outputTokens":13,"latencyMs":1298.904458999983},{"questionId":"q45","format":"toon","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":9131,"outputTokens":13,"latencyMs":4728.3269169999985},{"questionId":"q45","format":"xml","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":15259,"outputTokens":13,"latencyMs":1348.2243749999907},{"questionId":"q45","format":"yaml","model":"gemini-2.5-flash","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":10124,"outputTokens":13,"latencyMs":1353.6142919999547},{"questionId":"q46","format":"json-pretty","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":13871,"outputTokens":10,"latencyMs":1210.6754589999327},{"questionId":"q46","format":"json-compact","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8513,"outputTokens":10,"latencyMs":1100.8445830000564},{"questionId":"q46","format":"toon","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":9131,"outputTokens":10,"latencyMs":893.8447089999681},{"questionId":"q46","format":"xml","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":15259,"outputTokens":10,"latencyMs":976.953499999945},{"questionId":"q46","format":"yaml","model":"gemini-2.5-flash","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":10124,"outputTokens":10,"latencyMs":1370.1751249999506},{"questionId":"q47","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":13870,"outputTokens":1,"latencyMs":1156.325959000038},{"questionId":"q47","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8512,"outputTokens":1,"latencyMs":1278.2277079999913},{"questionId":"q47","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9130,"outputTokens":1,"latencyMs":1263.0402499999618},{"questionId":"q47","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":15258,"outputTokens":1,"latencyMs":779.7844160000095},{"questionId":"q47","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":10123,"outputTokens":1,"latencyMs":2318.2522500000196},{"questionId":"q48","format":"json-pretty","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":13871,"outputTokens":4,"latencyMs":1809.798375000013},{"questionId":"q48","format":"json-compact","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8513,"outputTokens":4,"latencyMs":1151.6029999998864},{"questionId":"q48","format":"toon","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":9131,"outputTokens":4,"latencyMs":2730.975332999951},{"questionId":"q48","format":"xml","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":15259,"outputTokens":4,"latencyMs":1037.6277919999557},{"questionId":"q48","format":"yaml","model":"gemini-2.5-flash","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":10124,"outputTokens":4,"latencyMs":1490.3517500000307},{"questionId":"q49","format":"json-pretty","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":13871,"outputTokens":10,"latencyMs":1690.90945799998},{"questionId":"q49","format":"json-compact","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8513,"outputTokens":10,"latencyMs":1574.7915000000503},{"questionId":"q49","format":"toon","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":9131,"outputTokens":10,"latencyMs":1566.4823750000214},{"questionId":"q49","format":"xml","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":15259,"outputTokens":10,"latencyMs":1358.9633749999339},{"questionId":"q49","format":"yaml","model":"gemini-2.5-flash","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":10124,"outputTokens":10,"latencyMs":2230.063124999986},{"questionId":"q50","format":"json-pretty","model":"gemini-2.5-flash","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":13871,"outputTokens":10,"latencyMs":1447.5517910000635},{"questionId":"q50","format":"json-compact","model":"gemini-2.5-flash","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":8513,"outputTokens":10,"latencyMs":1545.0617499999935},{"questionId":"q50","format":"toon","model":"gemini-2.5-flash","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":9131,"outputTokens":10,"latencyMs":1648.3489590000827},{"questionId":"q50","format":"xml","model":"gemini-2.5-flash","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":15259,"outputTokens":10,"latencyMs":1442.81258300005},{"questionId":"q50","format":"yaml","model":"gemini-2.5-flash","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":10124,"outputTokens":10,"latencyMs":2586.512374999933},{"questionId":"q51","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":13870,"outputTokens":1,"latencyMs":1217.031791999936},{"questionId":"q51","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":8512,"outputTokens":1,"latencyMs":1203.1166250000242},{"questionId":"q51","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":9130,"outputTokens":1,"latencyMs":2419.716667000088},{"questionId":"q51","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":15258,"outputTokens":1,"latencyMs":2423.585166999954},{"questionId":"q51","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":10123,"outputTokens":1,"latencyMs":2469.8865000000224},{"questionId":"q52","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"7","isCorrect":false,"inputTokens":13865,"outputTokens":1,"latencyMs":1052.8387090000324},{"questionId":"q52","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"9","isCorrect":false,"inputTokens":8507,"outputTokens":1,"latencyMs":949.4661250000354},{"questionId":"q52","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9125,"outputTokens":2,"latencyMs":3603.015500000096},{"questionId":"q52","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":15253,"outputTokens":2,"latencyMs":874.3775410000235},{"questionId":"q52","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"9","isCorrect":false,"inputTokens":10118,"outputTokens":1,"latencyMs":1293.1589170000516},{"questionId":"q53","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"7","isCorrect":false,"inputTokens":13865,"outputTokens":1,"latencyMs":1174.9120409999741},{"questionId":"q53","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"9","isCorrect":false,"inputTokens":8507,"outputTokens":1,"latencyMs":1137.143041000003},{"questionId":"q53","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9125,"outputTokens":2,"latencyMs":4107.412666000077},{"questionId":"q53","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":15253,"outputTokens":2,"latencyMs":3064.9035829999484},{"questionId":"q53","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":10118,"outputTokens":2,"latencyMs":1649.7403749999357},{"questionId":"q54","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":13866,"outputTokens":2,"latencyMs":4111.767458999995},{"questionId":"q54","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":8508,"outputTokens":2,"latencyMs":3723.5099580000388},{"questionId":"q54","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9126,"outputTokens":2,"latencyMs":3989.5645409999415},{"questionId":"q54","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"8","isCorrect":false,"inputTokens":15254,"outputTokens":1,"latencyMs":1192.0494160000235},{"questionId":"q54","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"8","isCorrect":false,"inputTokens":10119,"outputTokens":1,"latencyMs":1023.8075419999659},{"questionId":"q55","format":"json-pretty","model":"gemini-2.5-flash","expected":"34904.81","actual":"34503.74","isCorrect":false,"inputTokens":13866,"outputTokens":8,"latencyMs":6049.468458999996},{"questionId":"q55","format":"json-compact","model":"gemini-2.5-flash","expected":"34904.81","actual":"33887.89","isCorrect":false,"inputTokens":8508,"outputTokens":8,"latencyMs":6102.079792000004},{"questionId":"q55","format":"toon","model":"gemini-2.5-flash","expected":"34904.81","actual":"33814.77","isCorrect":false,"inputTokens":9126,"outputTokens":8,"latencyMs":4674.433749999967},{"questionId":"q55","format":"xml","model":"gemini-2.5-flash","expected":"34904.81","actual":"30894.47","isCorrect":false,"inputTokens":15254,"outputTokens":8,"latencyMs":9416.530167000019},{"questionId":"q55","format":"yaml","model":"gemini-2.5-flash","expected":"34904.81","actual":"34105.74","isCorrect":false,"inputTokens":10119,"outputTokens":8,"latencyMs":5880.628083000076},{"questionId":"q56","format":"json-pretty","model":"gemini-2.5-flash","expected":"698.10","actual":"690.08","isCorrect":false,"inputTokens":13864,"outputTokens":6,"latencyMs":6194.131208000006},{"questionId":"q56","format":"json-compact","model":"gemini-2.5-flash","expected":"698.10","actual":"638.4992","isCorrect":false,"inputTokens":8506,"outputTokens":8,"latencyMs":37997.802250000066},{"questionId":"q56","format":"toon","model":"gemini-2.5-flash","expected":"698.10","actual":"690.1342","isCorrect":false,"inputTokens":9124,"outputTokens":8,"latencyMs":7192.714541999972},{"questionId":"q56","format":"xml","model":"gemini-2.5-flash","expected":"698.10","actual":"699.60","isCorrect":false,"inputTokens":15252,"outputTokens":6,"latencyMs":7964.745666999952},{"questionId":"q56","format":"yaml","model":"gemini-2.5-flash","expected":"698.10","actual":"603.3146","isCorrect":false,"inputTokens":10117,"outputTokens":8,"latencyMs":5946.955583999981},{"questionId":"q57","format":"json-pretty","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":13865,"outputTokens":2,"latencyMs":1228.188291999977},{"questionId":"q57","format":"json-compact","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":8507,"outputTokens":2,"latencyMs":4206.823084000032},{"questionId":"q57","format":"toon","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":9125,"outputTokens":2,"latencyMs":3108.2494169999845},{"questionId":"q57","format":"xml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":15253,"outputTokens":2,"latencyMs":2689.737916999962},{"questionId":"q57","format":"yaml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":10118,"outputTokens":2,"latencyMs":3949.609000000055},{"questionId":"q58","format":"json-pretty","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":13864,"outputTokens":7,"latencyMs":1092.4530409999425},{"questionId":"q58","format":"json-compact","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8506,"outputTokens":7,"latencyMs":1385.6622920000227},{"questionId":"q58","format":"toon","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":9124,"outputTokens":7,"latencyMs":5924.611707999953},{"questionId":"q58","format":"xml","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":15252,"outputTokens":7,"latencyMs":1260.2550409999676},{"questionId":"q58","format":"yaml","model":"gemini-2.5-flash","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":10117,"outputTokens":7,"latencyMs":1205.030541999964},{"questionId":"q59","format":"json-pretty","model":"gemini-2.5-flash","expected":"43","actual":"42","isCorrect":false,"inputTokens":13870,"outputTokens":2,"latencyMs":1681.3931250000605},{"questionId":"q59","format":"json-compact","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":8512,"outputTokens":2,"latencyMs":6966.428666000022},{"questionId":"q59","format":"toon","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":9130,"outputTokens":2,"latencyMs":9269.288791000028},{"questionId":"q59","format":"xml","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":15258,"outputTokens":2,"latencyMs":6981.577208000002},{"questionId":"q59","format":"yaml","model":"gemini-2.5-flash","expected":"43","actual":"44","isCorrect":false,"inputTokens":10123,"outputTokens":2,"latencyMs":1509.5580840000184},{"questionId":"q60","format":"json-pretty","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":13870,"outputTokens":2,"latencyMs":7463.431749999989},{"questionId":"q60","format":"json-compact","model":"gemini-2.5-flash","expected":"37","actual":"42","isCorrect":false,"inputTokens":8512,"outputTokens":2,"latencyMs":1369.4598750000587},{"questionId":"q60","format":"toon","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":9130,"outputTokens":2,"latencyMs":10805.439291000017},{"questionId":"q60","format":"xml","model":"gemini-2.5-flash","expected":"37","actual":"35","isCorrect":false,"inputTokens":15258,"outputTokens":2,"latencyMs":6297.579999999958},{"questionId":"q60","format":"yaml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":10123,"outputTokens":2,"latencyMs":7174.834082999965},{"questionId":"q61","format":"json-pretty","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":13870,"outputTokens":2,"latencyMs":6910.065500000026},{"questionId":"q61","format":"json-compact","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":8512,"outputTokens":2,"latencyMs":6562.695833000005},{"questionId":"q61","format":"toon","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":9130,"outputTokens":2,"latencyMs":6849.546417000005},{"questionId":"q61","format":"xml","model":"gemini-2.5-flash","expected":"28","actual":"33","isCorrect":false,"inputTokens":15258,"outputTokens":2,"latencyMs":973.6099580000155},{"questionId":"q61","format":"yaml","model":"gemini-2.5-flash","expected":"28","actual":"28","isCorrect":true,"inputTokens":10123,"outputTokens":2,"latencyMs":7529.520625000005},{"questionId":"q62","format":"json-pretty","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":13874,"outputTokens":1,"latencyMs":6359.261583000072},{"questionId":"q62","format":"json-compact","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":8516,"outputTokens":1,"latencyMs":6613.465250000008},{"questionId":"q62","format":"toon","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":9134,"outputTokens":1,"latencyMs":11535.92629199999},{"questionId":"q62","format":"xml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":15262,"outputTokens":1,"latencyMs":8995.139166999958},{"questionId":"q62","format":"yaml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":10127,"outputTokens":1,"latencyMs":7840.574999999953},{"questionId":"q63","format":"json-pretty","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":13874,"outputTokens":1,"latencyMs":6306.841958999983},{"questionId":"q63","format":"json-compact","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":8516,"outputTokens":1,"latencyMs":8349.116041000001},{"questionId":"q63","format":"toon","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":9134,"outputTokens":1,"latencyMs":8002.142749999999},{"questionId":"q63","format":"xml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":15262,"outputTokens":1,"latencyMs":7971.087000000058},{"questionId":"q63","format":"yaml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":10127,"outputTokens":1,"latencyMs":6188.31241699995},{"questionId":"q64","format":"json-pretty","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":13875,"outputTokens":2,"latencyMs":6594.39620800002},{"questionId":"q64","format":"json-compact","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":8517,"outputTokens":2,"latencyMs":6782.7014999999665},{"questionId":"q64","format":"toon","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":9135,"outputTokens":2,"latencyMs":7458.658124999958},{"questionId":"q64","format":"xml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":15263,"outputTokens":2,"latencyMs":7929.491832999978},{"questionId":"q64","format":"yaml","model":"gemini-2.5-flash","expected":"10","actual":"10","isCorrect":true,"inputTokens":10128,"outputTokens":2,"latencyMs":7440.645749999909},{"questionId":"q65","format":"json-pretty","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":13874,"outputTokens":1,"latencyMs":6689.368250000058},{"questionId":"q65","format":"json-compact","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":8516,"outputTokens":1,"latencyMs":6597.672708000056},{"questionId":"q65","format":"toon","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":9134,"outputTokens":1,"latencyMs":5332.369125000085},{"questionId":"q65","format":"xml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":15262,"outputTokens":1,"latencyMs":7524.719584000064},{"questionId":"q65","format":"yaml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":10127,"outputTokens":1,"latencyMs":8792.515040999977},{"questionId":"q66","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":13872,"outputTokens":1,"latencyMs":4640.436208999949},{"questionId":"q66","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8514,"outputTokens":1,"latencyMs":5104.22895800008},{"questionId":"q66","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9132,"outputTokens":1,"latencyMs":13748.536542000016},{"questionId":"q66","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":15260,"outputTokens":1,"latencyMs":10128.556125000003},{"questionId":"q66","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":10125,"outputTokens":1,"latencyMs":6913.737209000043},{"questionId":"q67","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":13872,"outputTokens":1,"latencyMs":5353.086625000113},{"questionId":"q67","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8514,"outputTokens":1,"latencyMs":5773.756124999956},{"questionId":"q67","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9132,"outputTokens":1,"latencyMs":8621.003000000026},{"questionId":"q67","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":15260,"outputTokens":1,"latencyMs":5489.957416999969},{"questionId":"q67","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":10125,"outputTokens":1,"latencyMs":5483.381583999959},{"questionId":"q68","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":13873,"outputTokens":1,"latencyMs":5374.589750000043},{"questionId":"q68","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":8515,"outputTokens":1,"latencyMs":5635.895540999947},{"questionId":"q68","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":9133,"outputTokens":1,"latencyMs":8578.707333000028},{"questionId":"q68","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":15261,"outputTokens":1,"latencyMs":5653.253333000001},{"questionId":"q68","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"8","isCorrect":false,"inputTokens":10126,"outputTokens":1,"latencyMs":1619.4710419999901},{"questionId":"q69","format":"json-pretty","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":13876,"outputTokens":2,"latencyMs":8430.429166999995},{"questionId":"q69","format":"json-compact","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":8518,"outputTokens":2,"latencyMs":8781.718583000009},{"questionId":"q69","format":"toon","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":9136,"outputTokens":2,"latencyMs":15556.881834},{"questionId":"q69","format":"xml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":15264,"outputTokens":2,"latencyMs":8488.195915999939},{"questionId":"q69","format":"yaml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":10129,"outputTokens":2,"latencyMs":12721.021667000023},{"questionId":"q70","format":"json-pretty","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":13876,"outputTokens":2,"latencyMs":8316.785499999998},{"questionId":"q70","format":"json-compact","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":8518,"outputTokens":2,"latencyMs":8320.387292},{"questionId":"q70","format":"toon","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":9136,"outputTokens":2,"latencyMs":17950.370666999952},{"questionId":"q70","format":"xml","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":15264,"outputTokens":2,"latencyMs":13579.097124999971},{"questionId":"q70","format":"yaml","model":"gemini-2.5-flash","expected":"19","actual":"19","isCorrect":true,"inputTokens":10129,"outputTokens":2,"latencyMs":8877.398750000051},{"questionId":"q71","format":"json-pretty","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4844,"outputTokens":4,"latencyMs":1273.6484169999603},{"questionId":"q71","format":"json-compact","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3092,"outputTokens":4,"latencyMs":1687.1027079999913},{"questionId":"q71","format":"toon","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2355,"outputTokens":4,"latencyMs":1947.8026250000112},{"questionId":"q71","format":"csv","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2269,"outputTokens":4,"latencyMs":1641.0701249999693},{"questionId":"q71","format":"xml","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":5487,"outputTokens":4,"latencyMs":1254.951667000074},{"questionId":"q71","format":"yaml","model":"gemini-2.5-flash","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3873,"outputTokens":4,"latencyMs":1636.3882079999894},{"questionId":"q72","format":"json-pretty","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4844,"outputTokens":8,"latencyMs":819.9340000000084},{"questionId":"q72","format":"json-compact","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3092,"outputTokens":8,"latencyMs":1942.2550419999752},{"questionId":"q72","format":"toon","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2355,"outputTokens":8,"latencyMs":2923.626500000013},{"questionId":"q72","format":"csv","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2269,"outputTokens":8,"latencyMs":909.6921670000302},{"questionId":"q72","format":"xml","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":5487,"outputTokens":8,"latencyMs":1155.7246659999946},{"questionId":"q72","format":"yaml","model":"gemini-2.5-flash","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3873,"outputTokens":8,"latencyMs":1184.3948749999981},{"questionId":"q73","format":"json-pretty","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4845,"outputTokens":4,"latencyMs":893.2712499999907},{"questionId":"q73","format":"json-compact","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3093,"outputTokens":4,"latencyMs":1858.1940419999883},{"questionId":"q73","format":"toon","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2356,"outputTokens":4,"latencyMs":2700.344708999968},{"questionId":"q73","format":"csv","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2270,"outputTokens":4,"latencyMs":1464.15058399993},{"questionId":"q73","format":"xml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":5488,"outputTokens":4,"latencyMs":1671.998874999932},{"questionId":"q73","format":"yaml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3874,"outputTokens":4,"latencyMs":2177.4862079999875},{"questionId":"q74","format":"json-pretty","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":4845,"outputTokens":2,"latencyMs":1024.5641250000335},{"questionId":"q74","format":"json-compact","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3093,"outputTokens":2,"latencyMs":2246.8999169999734},{"questionId":"q74","format":"toon","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2356,"outputTokens":2,"latencyMs":2018.9257919999072},{"questionId":"q74","format":"csv","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2270,"outputTokens":2,"latencyMs":1694.2114580000052},{"questionId":"q74","format":"xml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":5488,"outputTokens":2,"latencyMs":1499.1982080000453},{"questionId":"q74","format":"yaml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3874,"outputTokens":2,"latencyMs":1431.4549589999951},{"questionId":"q75","format":"json-pretty","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4844,"outputTokens":4,"latencyMs":1404.309666999965},{"questionId":"q75","format":"json-compact","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3092,"outputTokens":4,"latencyMs":2519.6857910000253},{"questionId":"q75","format":"toon","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2355,"outputTokens":4,"latencyMs":2677.5994159999536},{"questionId":"q75","format":"csv","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2269,"outputTokens":4,"latencyMs":1738.2083750000456},{"questionId":"q75","format":"xml","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":5487,"outputTokens":4,"latencyMs":1464.0473749999655},{"questionId":"q75","format":"yaml","model":"gemini-2.5-flash","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3873,"outputTokens":4,"latencyMs":2000.5473340000026},{"questionId":"q76","format":"json-pretty","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4844,"outputTokens":6,"latencyMs":1094.7897499999963},{"questionId":"q76","format":"json-compact","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3092,"outputTokens":6,"latencyMs":1339.9942499999888},{"questionId":"q76","format":"toon","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2355,"outputTokens":6,"latencyMs":2807.9394160000375},{"questionId":"q76","format":"csv","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2269,"outputTokens":6,"latencyMs":1792.1295000000391},{"questionId":"q76","format":"xml","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":5487,"outputTokens":6,"latencyMs":1296.422583000036},{"questionId":"q76","format":"yaml","model":"gemini-2.5-flash","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3873,"outputTokens":6,"latencyMs":1311.4097919999622},{"questionId":"q77","format":"json-pretty","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4845,"outputTokens":4,"latencyMs":853.0527079999447},{"questionId":"q77","format":"json-compact","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3093,"outputTokens":4,"latencyMs":2155.1386250000214},{"questionId":"q77","format":"toon","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2356,"outputTokens":4,"latencyMs":1895.4962920000544},{"questionId":"q77","format":"csv","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2270,"outputTokens":4,"latencyMs":1312.9975410000188},{"questionId":"q77","format":"xml","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":5488,"outputTokens":4,"latencyMs":1530.621374999988},{"questionId":"q77","format":"yaml","model":"gemini-2.5-flash","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3874,"outputTokens":4,"latencyMs":1959.6304170000367},{"questionId":"q78","format":"json-pretty","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":4845,"outputTokens":2,"latencyMs":1429.8989170000423},{"questionId":"q78","format":"json-compact","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":3093,"outputTokens":2,"latencyMs":1891.5067079999717},{"questionId":"q78","format":"toon","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":2356,"outputTokens":2,"latencyMs":2545.275874999934},{"questionId":"q78","format":"csv","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":2270,"outputTokens":2,"latencyMs":1011.4208329999819},{"questionId":"q78","format":"xml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":5488,"outputTokens":2,"latencyMs":1727.6587500000605},{"questionId":"q78","format":"yaml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":3874,"outputTokens":2,"latencyMs":2105.6512499999953},{"questionId":"q79","format":"json-pretty","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4844,"outputTokens":4,"latencyMs":1617.4391659999965},{"questionId":"q79","format":"json-compact","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3092,"outputTokens":4,"latencyMs":1474.4968329999829},{"questionId":"q79","format":"toon","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2355,"outputTokens":4,"latencyMs":2458.9222919999156},{"questionId":"q79","format":"csv","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2269,"outputTokens":4,"latencyMs":1466.3002080000006},{"questionId":"q79","format":"xml","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":5487,"outputTokens":4,"latencyMs":2600.9231250000885},{"questionId":"q79","format":"yaml","model":"gemini-2.5-flash","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3873,"outputTokens":4,"latencyMs":1171.1017090000678},{"questionId":"q80","format":"json-pretty","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":4837,"outputTokens":2,"latencyMs":1320.9948749999749},{"questionId":"q80","format":"json-compact","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3085,"outputTokens":2,"latencyMs":2588.623624999891},{"questionId":"q80","format":"toon","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2348,"outputTokens":2,"latencyMs":976.7281659999862},{"questionId":"q80","format":"csv","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2262,"outputTokens":2,"latencyMs":4159.593792000087},{"questionId":"q80","format":"xml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":5480,"outputTokens":2,"latencyMs":1415.9411669999827},{"questionId":"q80","format":"yaml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3866,"outputTokens":2,"latencyMs":5140.633916000021},{"questionId":"q81","format":"json-pretty","model":"gemini-2.5-flash","expected":"328320","actual":"301217","isCorrect":false,"inputTokens":4838,"outputTokens":6,"latencyMs":3688.0807499999646},{"questionId":"q81","format":"json-compact","model":"gemini-2.5-flash","expected":"328320","actual":"276852","isCorrect":false,"inputTokens":3086,"outputTokens":6,"latencyMs":1251.472916000057},{"questionId":"q81","format":"toon","model":"gemini-2.5-flash","expected":"328320","actual":"301262","isCorrect":false,"inputTokens":2349,"outputTokens":6,"latencyMs":4495.195999999996},{"questionId":"q81","format":"csv","model":"gemini-2.5-flash","expected":"328320","actual":"314059","isCorrect":false,"inputTokens":2263,"outputTokens":6,"latencyMs":4342.571833999944},{"questionId":"q81","format":"xml","model":"gemini-2.5-flash","expected":"328320","actual":"328720","isCorrect":false,"inputTokens":5481,"outputTokens":6,"latencyMs":4129.76337499998},{"questionId":"q81","format":"yaml","model":"gemini-2.5-flash","expected":"328320","actual":"333334","isCorrect":false,"inputTokens":3867,"outputTokens":6,"latencyMs":6819.509582999861},{"questionId":"q82","format":"json-pretty","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":4838,"outputTokens":4,"latencyMs":5424.899709000019},{"questionId":"q82","format":"json-compact","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":3086,"outputTokens":4,"latencyMs":6037.405417000176},{"questionId":"q82","format":"toon","model":"gemini-2.5-flash","expected":"1791","actual":"1779","isCorrect":false,"inputTokens":2349,"outputTokens":4,"latencyMs":1787.0482079999056},{"questionId":"q82","format":"csv","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":2263,"outputTokens":4,"latencyMs":5114.281542000128},{"questionId":"q82","format":"xml","model":"gemini-2.5-flash","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":5481,"outputTokens":4,"latencyMs":5510.192416999955},{"questionId":"q82","format":"yaml","model":"gemini-2.5-flash","expected":"1791","actual":"1870","isCorrect":false,"inputTokens":3867,"outputTokens":4,"latencyMs":2520.570708999876},{"questionId":"q83","format":"json-pretty","model":"gemini-2.5-flash","expected":"311695.88","actual":"330232.08","isCorrect":false,"inputTokens":4836,"outputTokens":9,"latencyMs":23542.202959000133},{"questionId":"q83","format":"json-compact","model":"gemini-2.5-flash","expected":"311695.88","actual":"426685.06","isCorrect":false,"inputTokens":3084,"outputTokens":9,"latencyMs":1623.9747920001391},{"questionId":"q83","format":"toon","model":"gemini-2.5-flash","expected":"311695.88","actual":"345700.31","isCorrect":false,"inputTokens":2347,"outputTokens":9,"latencyMs":15935.997500000056},{"questionId":"q83","format":"csv","model":"gemini-2.5-flash","expected":"311695.88","actual":"311696.88","isCorrect":false,"inputTokens":2261,"outputTokens":9,"latencyMs":14888.816417000024},{"questionId":"q83","format":"xml","model":"gemini-2.5-flash","expected":"311695.88","actual":"290117.84","isCorrect":false,"inputTokens":5479,"outputTokens":9,"latencyMs":7061.407750000013},{"questionId":"q83","format":"yaml","model":"gemini-2.5-flash","expected":"311695.88","actual":"508103.53","isCorrect":false,"inputTokens":3865,"outputTokens":9,"latencyMs":853.9489160000812},{"questionId":"q84","format":"json-pretty","model":"gemini-2.5-flash","expected":"0.53","actual":"0.5057377049180328","isCorrect":false,"inputTokens":4834,"outputTokens":18,"latencyMs":1081.7927080001682},{"questionId":"q84","format":"json-compact","model":"gemini-2.5-flash","expected":"0.53","actual":"0.502","isCorrect":false,"inputTokens":3082,"outputTokens":5,"latencyMs":18153.151792000048},{"questionId":"q84","format":"toon","model":"gemini-2.5-flash","expected":"0.53","actual":"0.5091666666666667","isCorrect":false,"inputTokens":2345,"outputTokens":18,"latencyMs":3576.615375000052},{"questionId":"q84","format":"csv","model":"gemini-2.5-flash","expected":"0.53","actual":"0.530","isCorrect":true,"inputTokens":2259,"outputTokens":5,"latencyMs":6600.331583000021},{"questionId":"q84","format":"xml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.5177","isCorrect":false,"inputTokens":5477,"outputTokens":6,"latencyMs":29811.653041000012},{"questionId":"q84","format":"yaml","model":"gemini-2.5-flash","expected":"0.53","actual":"0.518","isCorrect":false,"inputTokens":3863,"outputTokens":5,"latencyMs":4018.6596250000875},{"questionId":"q85","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"26","isCorrect":false,"inputTokens":4840,"outputTokens":2,"latencyMs":674.6431249999441},{"questionId":"q85","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3088,"outputTokens":2,"latencyMs":620.3347920000087},{"questionId":"q85","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2351,"outputTokens":2,"latencyMs":14085.80466699996},{"questionId":"q85","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2265,"outputTokens":2,"latencyMs":9628.03366700001},{"questionId":"q85","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5483,"outputTokens":2,"latencyMs":8394.644374999916},{"questionId":"q85","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3869,"outputTokens":2,"latencyMs":7079.810167000163},{"questionId":"q86","format":"json-pretty","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":4838,"outputTokens":2,"latencyMs":1324.622207999928},{"questionId":"q86","format":"json-compact","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":3086,"outputTokens":2,"latencyMs":16803.79891699995},{"questionId":"q86","format":"toon","model":"gemini-2.5-flash","expected":"42","actual":"36","isCorrect":false,"inputTokens":2349,"outputTokens":2,"latencyMs":4542.796332999831},{"questionId":"q86","format":"csv","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":2263,"outputTokens":2,"latencyMs":12295.852999999886},{"questionId":"q86","format":"xml","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":5481,"outputTokens":2,"latencyMs":31190.132584000006},{"questionId":"q86","format":"yaml","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":3867,"outputTokens":2,"latencyMs":8732.081707999809},{"questionId":"q87","format":"json-pretty","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":4847,"outputTokens":2,"latencyMs":9138.61474999995},{"questionId":"q87","format":"json-compact","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3095,"outputTokens":2,"latencyMs":20417.680374999996},{"questionId":"q87","format":"toon","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2358,"outputTokens":2,"latencyMs":29283.702082999982},{"questionId":"q87","format":"csv","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2272,"outputTokens":2,"latencyMs":15461.12820799998},{"questionId":"q87","format":"xml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":5490,"outputTokens":2,"latencyMs":11028.183750000084},{"questionId":"q87","format":"yaml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3876,"outputTokens":2,"latencyMs":8260.692540999968},{"questionId":"q88","format":"json-pretty","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":4847,"outputTokens":2,"latencyMs":8077.278416999849},{"questionId":"q88","format":"json-compact","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":3095,"outputTokens":2,"latencyMs":16375.601625000127},{"questionId":"q88","format":"toon","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":2358,"outputTokens":2,"latencyMs":15985.696083999937},{"questionId":"q88","format":"csv","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":2272,"outputTokens":2,"latencyMs":17140.661458000075},{"questionId":"q88","format":"xml","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":5490,"outputTokens":2,"latencyMs":10237.676207999932},{"questionId":"q88","format":"yaml","model":"gemini-2.5-flash","expected":"14","actual":"14","isCorrect":true,"inputTokens":3876,"outputTokens":2,"latencyMs":9312.99612499983},{"questionId":"q89","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4848,"outputTokens":2,"latencyMs":12897.602916999953},{"questionId":"q89","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3096,"outputTokens":2,"latencyMs":23420.403542000102},{"questionId":"q89","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2359,"outputTokens":2,"latencyMs":17705.816791999852},{"questionId":"q89","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2273,"outputTokens":2,"latencyMs":17042.223374999827},{"questionId":"q89","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5491,"outputTokens":2,"latencyMs":12185.241374999983},{"questionId":"q89","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3877,"outputTokens":2,"latencyMs":10423.526250000112},{"questionId":"q90","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4848,"outputTokens":2,"latencyMs":9316.858208999969},{"questionId":"q90","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3096,"outputTokens":2,"latencyMs":22347.77441699989},{"questionId":"q90","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2359,"outputTokens":2,"latencyMs":16534.988499999978},{"questionId":"q90","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2273,"outputTokens":2,"latencyMs":15626.632250000024},{"questionId":"q90","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5491,"outputTokens":2,"latencyMs":11505.264582999982},{"questionId":"q90","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3877,"outputTokens":2,"latencyMs":10128.77108399989},{"questionId":"q91","format":"json-pretty","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":4848,"outputTokens":2,"latencyMs":9835.889332999941},{"questionId":"q91","format":"json-compact","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3096,"outputTokens":2,"latencyMs":17367.953791999957},{"questionId":"q91","format":"toon","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2359,"outputTokens":2,"latencyMs":16737.145041999873},{"questionId":"q91","format":"csv","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":2273,"outputTokens":2,"latencyMs":15347.96233400004},{"questionId":"q91","format":"xml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":5491,"outputTokens":2,"latencyMs":12565.465917000081},{"questionId":"q91","format":"yaml","model":"gemini-2.5-flash","expected":"20","actual":"20","isCorrect":true,"inputTokens":3877,"outputTokens":2,"latencyMs":9497.484791999916},{"questionId":"q92","format":"json-pretty","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":4846,"outputTokens":2,"latencyMs":15599.572584000183},{"questionId":"q92","format":"json-compact","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3094,"outputTokens":2,"latencyMs":22413.92575000017},{"questionId":"q92","format":"toon","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2357,"outputTokens":2,"latencyMs":14316.69287499995},{"questionId":"q92","format":"csv","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":2271,"outputTokens":2,"latencyMs":17951.869458000176},{"questionId":"q92","format":"xml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":5489,"outputTokens":2,"latencyMs":13678.618583000032},{"questionId":"q92","format":"yaml","model":"gemini-2.5-flash","expected":"32","actual":"32","isCorrect":true,"inputTokens":3875,"outputTokens":2,"latencyMs":7204.374749999959},{"questionId":"q93","format":"json-pretty","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":4846,"outputTokens":1,"latencyMs":7385.514708000002},{"questionId":"q93","format":"json-compact","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":3094,"outputTokens":1,"latencyMs":20300.001374999993},{"questionId":"q93","format":"toon","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":2357,"outputTokens":1,"latencyMs":15480.577459000051},{"questionId":"q93","format":"csv","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":2271,"outputTokens":1,"latencyMs":15965.906750000082},{"questionId":"q93","format":"xml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":5489,"outputTokens":1,"latencyMs":11860.461624999996},{"questionId":"q93","format":"yaml","model":"gemini-2.5-flash","expected":"9","actual":"9","isCorrect":true,"inputTokens":3875,"outputTokens":1,"latencyMs":7374.475208999822},{"questionId":"q94","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4848,"outputTokens":2,"latencyMs":17802.565916000167},{"questionId":"q94","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3096,"outputTokens":2,"latencyMs":25006.38162499992},{"questionId":"q94","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2359,"outputTokens":2,"latencyMs":23761.986709000077},{"questionId":"q94","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2273,"outputTokens":2,"latencyMs":21649.962957999902},{"questionId":"q94","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"20","isCorrect":false,"inputTokens":5491,"outputTokens":2,"latencyMs":12137.00624999986},{"questionId":"q94","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3877,"outputTokens":2,"latencyMs":12139.743708000053},{"questionId":"q95","format":"json-pretty","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":4848,"outputTokens":2,"latencyMs":13314.50537499995},{"questionId":"q95","format":"json-compact","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3096,"outputTokens":2,"latencyMs":21887.974333000137},{"questionId":"q95","format":"toon","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2359,"outputTokens":2,"latencyMs":18226.03562500002},{"questionId":"q95","format":"csv","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":2273,"outputTokens":2,"latencyMs":19160.339625000022},{"questionId":"q95","format":"xml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":5491,"outputTokens":2,"latencyMs":13405.59733400005},{"questionId":"q95","format":"yaml","model":"gemini-2.5-flash","expected":"22","actual":"22","isCorrect":true,"inputTokens":3877,"outputTokens":2,"latencyMs":12594.141999999993},{"questionId":"q96","format":"json-pretty","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":20061,"outputTokens":6,"latencyMs":1379.76654099999},{"questionId":"q96","format":"json-compact","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":15099,"outputTokens":6,"latencyMs":1182.8872499999125},{"questionId":"q96","format":"toon","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":12431,"outputTokens":6,"latencyMs":2095.627792000072},{"questionId":"q96","format":"csv","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":12278,"outputTokens":6,"latencyMs":1863.0246669999324},{"questionId":"q96","format":"xml","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":21947,"outputTokens":6,"latencyMs":1416.5412079999223},{"questionId":"q96","format":"yaml","model":"gemini-2.5-flash","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17145,"outputTokens":6,"latencyMs":1603.1592919998802},{"questionId":"q97","format":"json-pretty","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":20064,"outputTokens":5,"latencyMs":1243.3246250001248},{"questionId":"q97","format":"json-compact","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":15102,"outputTokens":5,"latencyMs":936.1264170000795},{"questionId":"q97","format":"toon","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":12434,"outputTokens":5,"latencyMs":3282.2452909999993},{"questionId":"q97","format":"csv","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":12281,"outputTokens":5,"latencyMs":2333.9549160001334},{"questionId":"q97","format":"xml","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":21950,"outputTokens":5,"latencyMs":1567.2955000000075},{"questionId":"q97","format":"yaml","model":"gemini-2.5-flash","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17148,"outputTokens":5,"latencyMs":1820.2603750000708},{"questionId":"q98","format":"json-pretty","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":20058,"outputTokens":4,"latencyMs":1617.1906669999007},{"questionId":"q98","format":"json-compact","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":15096,"outputTokens":4,"latencyMs":1708.3548330001067},{"questionId":"q98","format":"toon","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":12428,"outputTokens":4,"latencyMs":3380.612874999875},{"questionId":"q98","format":"csv","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":12275,"outputTokens":4,"latencyMs":2624.5580409998074},{"questionId":"q98","format":"xml","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":21944,"outputTokens":4,"latencyMs":1540.8079160000198},{"questionId":"q98","format":"yaml","model":"gemini-2.5-flash","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17142,"outputTokens":4,"latencyMs":1469.3807089999318},{"questionId":"q99","format":"json-pretty","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":20065,"outputTokens":1,"latencyMs":1616.8342089999933},{"questionId":"q99","format":"json-compact","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":15103,"outputTokens":1,"latencyMs":1247.2139169999864},{"questionId":"q99","format":"toon","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":12435,"outputTokens":1,"latencyMs":3588.829125000164},{"questionId":"q99","format":"csv","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":12282,"outputTokens":1,"latencyMs":2294.2163749998435},{"questionId":"q99","format":"xml","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":21951,"outputTokens":1,"latencyMs":1307.8530000001192},{"questionId":"q99","format":"yaml","model":"gemini-2.5-flash","expected":"master","actual":"master","isCorrect":true,"inputTokens":17149,"outputTokens":1,"latencyMs":1813.3706249999814},{"questionId":"q100","format":"json-pretty","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":20057,"outputTokens":6,"latencyMs":1705.2275420001242},{"questionId":"q100","format":"json-compact","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":15095,"outputTokens":6,"latencyMs":994.2192909999285},{"questionId":"q100","format":"toon","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":12427,"outputTokens":6,"latencyMs":2034.7620840000454},{"questionId":"q100","format":"csv","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":12274,"outputTokens":6,"latencyMs":1778.57337499992},{"questionId":"q100","format":"xml","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":21943,"outputTokens":6,"latencyMs":979.6049999999814},{"questionId":"q100","format":"yaml","model":"gemini-2.5-flash","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17141,"outputTokens":6,"latencyMs":1405.680749999825},{"questionId":"q101","format":"json-pretty","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":20063,"outputTokens":5,"latencyMs":1257.4502080001403},{"questionId":"q101","format":"json-compact","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":15101,"outputTokens":5,"latencyMs":1256.9422090000007},{"questionId":"q101","format":"toon","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":12433,"outputTokens":5,"latencyMs":2984.801417000126},{"questionId":"q101","format":"csv","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":12280,"outputTokens":5,"latencyMs":1378.7828750000335},{"questionId":"q101","format":"xml","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":21949,"outputTokens":5,"latencyMs":1447.0383750000037},{"questionId":"q101","format":"yaml","model":"gemini-2.5-flash","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17147,"outputTokens":5,"latencyMs":1483.5686669999268},{"questionId":"q102","format":"json-pretty","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":20063,"outputTokens":3,"latencyMs":1266.189000000013},{"questionId":"q102","format":"json-compact","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":15101,"outputTokens":3,"latencyMs":1373.611125000054},{"questionId":"q102","format":"toon","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":12433,"outputTokens":3,"latencyMs":3330.522375000175},{"questionId":"q102","format":"csv","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":12280,"outputTokens":3,"latencyMs":1150.4383749999106},{"questionId":"q102","format":"xml","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":21949,"outputTokens":3,"latencyMs":1752.7039999999106},{"questionId":"q102","format":"yaml","model":"gemini-2.5-flash","expected":"678","actual":"678","isCorrect":true,"inputTokens":17147,"outputTokens":3,"latencyMs":1895.0212499999907},{"questionId":"q103","format":"json-pretty","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":20059,"outputTokens":1,"latencyMs":1266.8927500001155},{"questionId":"q103","format":"json-compact","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":15097,"outputTokens":1,"latencyMs":1219.6347090001218},{"questionId":"q103","format":"toon","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":12429,"outputTokens":1,"latencyMs":1369.9991250000894},{"questionId":"q103","format":"csv","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":12276,"outputTokens":1,"latencyMs":2684.0490000001155},{"questionId":"q103","format":"xml","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":21945,"outputTokens":1,"latencyMs":1392.3160419999622},{"questionId":"q103","format":"yaml","model":"gemini-2.5-flash","expected":"main","actual":"main","isCorrect":true,"inputTokens":17143,"outputTokens":1,"latencyMs":1218.4339159999508},{"questionId":"q104","format":"json-pretty","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":20066,"outputTokens":6,"latencyMs":1583.7229169998318},{"questionId":"q104","format":"json-compact","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":15104,"outputTokens":6,"latencyMs":1572.4323750000913},{"questionId":"q104","format":"toon","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":12436,"outputTokens":6,"latencyMs":3083.38758300012},{"questionId":"q104","format":"csv","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":12283,"outputTokens":6,"latencyMs":2550.107874999987},{"questionId":"q104","format":"xml","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":21952,"outputTokens":6,"latencyMs":1007.4006250000093},{"questionId":"q104","format":"yaml","model":"gemini-2.5-flash","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17150,"outputTokens":6,"latencyMs":1423.5112499999814},{"questionId":"q105","format":"json-pretty","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":20061,"outputTokens":5,"latencyMs":1308.844957999885},{"questionId":"q105","format":"json-compact","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":15099,"outputTokens":5,"latencyMs":1470.284415999893},{"questionId":"q105","format":"toon","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":12431,"outputTokens":5,"latencyMs":2931.722542000003},{"questionId":"q105","format":"csv","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":12278,"outputTokens":5,"latencyMs":2242.2194590000436},{"questionId":"q105","format":"xml","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":21947,"outputTokens":5,"latencyMs":2733.12104200013},{"questionId":"q105","format":"yaml","model":"gemini-2.5-flash","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17145,"outputTokens":5,"latencyMs":1418.5506670000032},{"questionId":"q106","format":"json-pretty","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":20064,"outputTokens":4,"latencyMs":1407.6134170000441},{"questionId":"q106","format":"json-compact","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":15102,"outputTokens":4,"latencyMs":1388.8776249999646},{"questionId":"q106","format":"toon","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":12434,"outputTokens":4,"latencyMs":3467.4795830000658},{"questionId":"q106","format":"csv","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":12281,"outputTokens":4,"latencyMs":2293.3367500000168},{"questionId":"q106","format":"xml","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":21950,"outputTokens":4,"latencyMs":4215.568374999799},{"questionId":"q106","format":"yaml","model":"gemini-2.5-flash","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17148,"outputTokens":4,"latencyMs":1464.9945420001168},{"questionId":"q107","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"60","isCorrect":false,"inputTokens":20056,"outputTokens":2,"latencyMs":1355.9782499999274},{"questionId":"q107","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"60","isCorrect":false,"inputTokens":15094,"outputTokens":2,"latencyMs":1299.0179159999825},{"questionId":"q107","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":12426,"outputTokens":3,"latencyMs":5658.16570799984},{"questionId":"q107","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"56","isCorrect":false,"inputTokens":12273,"outputTokens":2,"latencyMs":2862.1560829998925},{"questionId":"q107","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"60","isCorrect":false,"inputTokens":21942,"outputTokens":2,"latencyMs":1303.4911249999423},{"questionId":"q107","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"50","isCorrect":false,"inputTokens":17140,"outputTokens":2,"latencyMs":1140.744791999925},{"questionId":"q108","format":"json-pretty","model":"gemini-2.5-flash","expected":"15413563","actual":"11099187","isCorrect":false,"inputTokens":20059,"outputTokens":8,"latencyMs":10316.176875000121},{"questionId":"q108","format":"json-compact","model":"gemini-2.5-flash","expected":"15413563","actual":"9801867","isCorrect":false,"inputTokens":15097,"outputTokens":7,"latencyMs":1323.7321250000969},{"questionId":"q108","format":"toon","model":"gemini-2.5-flash","expected":"15413563","actual":"16335345","isCorrect":false,"inputTokens":12429,"outputTokens":8,"latencyMs":54495.21329200012},{"questionId":"q108","format":"csv","model":"gemini-2.5-flash","expected":"15413563","actual":"11221768","isCorrect":false,"inputTokens":12276,"outputTokens":8,"latencyMs":1396.3895420001354},{"questionId":"q108","format":"xml","model":"gemini-2.5-flash","expected":"15413563","actual":"11234479","isCorrect":false,"inputTokens":21945,"outputTokens":8,"latencyMs":1366.1082500000484},{"questionId":"q108","format":"yaml","model":"gemini-2.5-flash","expected":"15413563","actual":"10034298","isCorrect":false,"inputTokens":17143,"outputTokens":8,"latencyMs":1383.4054169999436},{"questionId":"q109","format":"json-pretty","model":"gemini-2.5-flash","expected":"2528243","actual":"1321798","isCorrect":false,"inputTokens":20059,"outputTokens":7,"latencyMs":1058.299083000049},{"questionId":"q109","format":"json-compact","model":"gemini-2.5-flash","expected":"2528243","actual":"2752360","isCorrect":false,"inputTokens":15097,"outputTokens":7,"latencyMs":9235.554709000047},{"questionId":"q109","format":"toon","model":"gemini-2.5-flash","expected":"2528243","actual":"2174301","isCorrect":false,"inputTokens":12429,"outputTokens":7,"latencyMs":58476.24766700016},{"questionId":"q109","format":"csv","model":"gemini-2.5-flash","expected":"2528243","actual":"1650893","isCorrect":false,"inputTokens":12276,"outputTokens":7,"latencyMs":1108.1446670000441},{"questionId":"q109","format":"xml","model":"gemini-2.5-flash","expected":"2528243","actual":"1831818","isCorrect":false,"inputTokens":21945,"outputTokens":7,"latencyMs":1106.5948330000974},{"questionId":"q109","format":"yaml","model":"gemini-2.5-flash","expected":"2528243","actual":"1585806","isCorrect":false,"inputTokens":17143,"outputTokens":7,"latencyMs":1114.0953750000335},{"questionId":"q110","format":"json-pretty","model":"gemini-2.5-flash","expected":"154136","actual":"187403","isCorrect":false,"inputTokens":20058,"outputTokens":6,"latencyMs":16162.972415999975},{"questionId":"q110","format":"json-compact","model":"gemini-2.5-flash","expected":"154136","actual":"197538.18","isCorrect":false,"inputTokens":15096,"outputTokens":9,"latencyMs":8160.90933299996},{"questionId":"q110","format":"toon","model":"gemini-2.5-flash","expected":"154136","actual":"207436.06","isCorrect":false,"inputTokens":12428,"outputTokens":9,"latencyMs":29813.758125000168},{"questionId":"q110","format":"csv","model":"gemini-2.5-flash","expected":"154136","actual":"209791.78","isCorrect":false,"inputTokens":12275,"outputTokens":9,"latencyMs":107709.256208},{"questionId":"q110","format":"xml","model":"gemini-2.5-flash","expected":"154136","actual":"219585.9534883721","isCorrect":false,"inputTokens":21944,"outputTokens":17,"latencyMs":99790.41591700003},{"questionId":"q110","format":"yaml","model":"gemini-2.5-flash","expected":"154136","actual":"157777","isCorrect":false,"inputTokens":17142,"outputTokens":6,"latencyMs":1126.0601669999305},{"questionId":"q111","format":"json-pretty","model":"gemini-2.5-flash","expected":"41","actual":"35","isCorrect":false,"inputTokens":20060,"outputTokens":2,"latencyMs":1181.1095419998746},{"questionId":"q111","format":"json-compact","model":"gemini-2.5-flash","expected":"41","actual":"35","isCorrect":false,"inputTokens":15098,"outputTokens":2,"latencyMs":1258.5384159998503},{"questionId":"q111","format":"toon","model":"gemini-2.5-flash","expected":"41","actual":"41","isCorrect":true,"inputTokens":12430,"outputTokens":2,"latencyMs":102561.9534169999},{"questionId":"q111","format":"csv","model":"gemini-2.5-flash","expected":"41","actual":"32","isCorrect":false,"inputTokens":12277,"outputTokens":2,"latencyMs":1528.503916000016},{"questionId":"q111","format":"xml","model":"gemini-2.5-flash","expected":"41","actual":"29","isCorrect":false,"inputTokens":21946,"outputTokens":2,"latencyMs":1197.9323330000043},{"questionId":"q111","format":"yaml","model":"gemini-2.5-flash","expected":"41","actual":"29","isCorrect":false,"inputTokens":17144,"outputTokens":2,"latencyMs":1155.7112080000807},{"questionId":"q112","format":"json-pretty","model":"gemini-2.5-flash","expected":"53","actual":"33","isCorrect":false,"inputTokens":20060,"outputTokens":2,"latencyMs":2073.3257920001633},{"questionId":"q112","format":"json-compact","model":"gemini-2.5-flash","expected":"53","actual":"35","isCorrect":false,"inputTokens":15098,"outputTokens":2,"latencyMs":1400.4217080001254},{"questionId":"q112","format":"toon","model":"gemini-2.5-flash","expected":"53","actual":"46","isCorrect":false,"inputTokens":12430,"outputTokens":2,"latencyMs":79038.8575840001},{"questionId":"q112","format":"csv","model":"gemini-2.5-flash","expected":"53","actual":"38","isCorrect":false,"inputTokens":12277,"outputTokens":2,"latencyMs":1558.280207999982},{"questionId":"q112","format":"xml","model":"gemini-2.5-flash","expected":"53","actual":"30","isCorrect":false,"inputTokens":21946,"outputTokens":2,"latencyMs":889.4184580000583},{"questionId":"q112","format":"yaml","model":"gemini-2.5-flash","expected":"53","actual":"39","isCorrect":false,"inputTokens":17144,"outputTokens":2,"latencyMs":1251.1912920000032},{"questionId":"q113","format":"json-pretty","model":"gemini-2.5-flash","expected":"77","actual":"53","isCorrect":false,"inputTokens":20063,"outputTokens":2,"latencyMs":1203.5173339999747},{"questionId":"q113","format":"json-compact","model":"gemini-2.5-flash","expected":"77","actual":"77","isCorrect":true,"inputTokens":15101,"outputTokens":2,"latencyMs":11259.004542000126},{"questionId":"q113","format":"toon","model":"gemini-2.5-flash","expected":"77","actual":"77","isCorrect":true,"inputTokens":12433,"outputTokens":2,"latencyMs":58597.390417000046},{"questionId":"q113","format":"csv","model":"gemini-2.5-flash","expected":"77","actual":"47","isCorrect":false,"inputTokens":12280,"outputTokens":2,"latencyMs":1852.2271249999758},{"questionId":"q113","format":"xml","model":"gemini-2.5-flash","expected":"77","actual":"48","isCorrect":false,"inputTokens":21949,"outputTokens":2,"latencyMs":905.06650000019},{"questionId":"q113","format":"yaml","model":"gemini-2.5-flash","expected":"77","actual":"50","isCorrect":false,"inputTokens":17147,"outputTokens":2,"latencyMs":1085.7499160000589},{"questionId":"q114","format":"json-pretty","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":20063,"outputTokens":2,"latencyMs":10916.724542000098},{"questionId":"q114","format":"json-compact","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":15101,"outputTokens":2,"latencyMs":21519.140124999918},{"questionId":"q114","format":"toon","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":12433,"outputTokens":2,"latencyMs":4866.667957999976},{"questionId":"q114","format":"csv","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":12280,"outputTokens":2,"latencyMs":5938.537083999952},{"questionId":"q114","format":"xml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":21949,"outputTokens":2,"latencyMs":13699.162125000032},{"questionId":"q114","format":"yaml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":17147,"outputTokens":2,"latencyMs":12652.14512500004},{"questionId":"q115","format":"json-pretty","model":"gemini-2.5-flash","expected":"16","actual":"17","isCorrect":false,"inputTokens":20063,"outputTokens":2,"latencyMs":2171.710749999853},{"questionId":"q115","format":"json-compact","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":15101,"outputTokens":2,"latencyMs":1386.298124999972},{"questionId":"q115","format":"toon","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":12433,"outputTokens":2,"latencyMs":59226.840209000045},{"questionId":"q115","format":"csv","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":12280,"outputTokens":2,"latencyMs":56847.61570800003},{"questionId":"q115","format":"xml","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":21949,"outputTokens":2,"latencyMs":11761.753291000146},{"questionId":"q115","format":"yaml","model":"gemini-2.5-flash","expected":"16","actual":"16","isCorrect":true,"inputTokens":17147,"outputTokens":2,"latencyMs":11814.380709000165},{"questionId":"q116","format":"json-pretty","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":20062,"outputTokens":2,"latencyMs":12099.77179099992},{"questionId":"q116","format":"json-compact","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":15100,"outputTokens":2,"latencyMs":20180.179083999945},{"questionId":"q116","format":"toon","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":12432,"outputTokens":2,"latencyMs":56740.78904200019},{"questionId":"q116","format":"csv","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":12279,"outputTokens":2,"latencyMs":60982.6934580002},{"questionId":"q116","format":"xml","model":"gemini-2.5-flash","expected":"49","actual":"45","isCorrect":false,"inputTokens":21948,"outputTokens":2,"latencyMs":1288.1656669999938},{"questionId":"q116","format":"yaml","model":"gemini-2.5-flash","expected":"49","actual":"49","isCorrect":true,"inputTokens":17146,"outputTokens":2,"latencyMs":10750.120041999966},{"questionId":"q117","format":"json-pretty","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":20062,"outputTokens":2,"latencyMs":10168.367874999996},{"questionId":"q117","format":"json-compact","model":"gemini-2.5-flash","expected":"23","actual":"20","isCorrect":false,"inputTokens":15100,"outputTokens":2,"latencyMs":2832.8600839998107},{"questionId":"q117","format":"toon","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":12432,"outputTokens":2,"latencyMs":64044.87024999992},{"questionId":"q117","format":"csv","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":12279,"outputTokens":2,"latencyMs":53365.00216699997},{"questionId":"q117","format":"xml","model":"gemini-2.5-flash","expected":"23","actual":"34","isCorrect":false,"inputTokens":21948,"outputTokens":2,"latencyMs":1048.6273339998443},{"questionId":"q117","format":"yaml","model":"gemini-2.5-flash","expected":"23","actual":"23","isCorrect":true,"inputTokens":17146,"outputTokens":2,"latencyMs":12963.301457999973},{"questionId":"q118","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"9","isCorrect":false,"inputTokens":20061,"outputTokens":1,"latencyMs":1109.1874160000589},{"questionId":"q118","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":15099,"outputTokens":1,"latencyMs":11571.37041600002},{"questionId":"q118","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":12431,"outputTokens":1,"latencyMs":10794.831416999921},{"questionId":"q118","format":"csv","model":"gemini-2.5-flash","expected":"4","actual":"10","isCorrect":false,"inputTokens":12278,"outputTokens":2,"latencyMs":1337.2851249999367},{"questionId":"q118","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"11","isCorrect":false,"inputTokens":21947,"outputTokens":2,"latencyMs":922.6695000000764},{"questionId":"q118","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":17145,"outputTokens":1,"latencyMs":9968.47016700008},{"questionId":"q119","format":"json-pretty","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":20072,"outputTokens":2,"latencyMs":32988.18366700015},{"questionId":"q119","format":"json-compact","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":15110,"outputTokens":2,"latencyMs":18829.703750000102},{"questionId":"q119","format":"toon","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":12442,"outputTokens":2,"latencyMs":66461.61770799989},{"questionId":"q119","format":"csv","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":12289,"outputTokens":2,"latencyMs":130304.78175000008},{"questionId":"q119","format":"xml","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":21958,"outputTokens":2,"latencyMs":15508.21516599995},{"questionId":"q119","format":"yaml","model":"gemini-2.5-flash","expected":"57","actual":"57","isCorrect":true,"inputTokens":17156,"outputTokens":2,"latencyMs":17170.537833999842},{"questionId":"q120","format":"json-pretty","model":"gemini-2.5-flash","expected":"43","actual":"33","isCorrect":false,"inputTokens":20073,"outputTokens":2,"latencyMs":16782.957707999973},{"questionId":"q120","format":"json-compact","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":15111,"outputTokens":2,"latencyMs":25629.525040999986},{"questionId":"q120","format":"toon","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":12443,"outputTokens":2,"latencyMs":56493.19550000015},{"questionId":"q120","format":"csv","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":12290,"outputTokens":2,"latencyMs":63294.836208999855},{"questionId":"q120","format":"xml","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":21959,"outputTokens":2,"latencyMs":17340.792417000048},{"questionId":"q120","format":"yaml","model":"gemini-2.5-flash","expected":"43","actual":"43","isCorrect":true,"inputTokens":17157,"outputTokens":2,"latencyMs":17521.762375000166},{"questionId":"q121","format":"json-pretty","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":20073,"outputTokens":2,"latencyMs":15417.791875000112},{"questionId":"q121","format":"json-compact","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":15111,"outputTokens":2,"latencyMs":8789.133875000058},{"questionId":"q121","format":"toon","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":12443,"outputTokens":2,"latencyMs":62909.714707999956},{"questionId":"q121","format":"csv","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":12290,"outputTokens":2,"latencyMs":27998.544916999992},{"questionId":"q121","format":"xml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":21959,"outputTokens":2,"latencyMs":13607.38291699998},{"questionId":"q121","format":"yaml","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":17157,"outputTokens":2,"latencyMs":15246.032208999852},{"questionId":"q122","format":"json-pretty","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":20072,"outputTokens":1,"latencyMs":14739.216499999864},{"questionId":"q122","format":"json-compact","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":15110,"outputTokens":1,"latencyMs":15880.542499999981},{"questionId":"q122","format":"toon","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":12442,"outputTokens":1,"latencyMs":87034.686583},{"questionId":"q122","format":"csv","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":12289,"outputTokens":1,"latencyMs":50978.31733300001},{"questionId":"q122","format":"xml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":21958,"outputTokens":1,"latencyMs":11955.919374999823},{"questionId":"q122","format":"yaml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":17156,"outputTokens":1,"latencyMs":18304.14858300006},{"questionId":"q123","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":20072,"outputTokens":1,"latencyMs":6977.884208999807},{"questionId":"q123","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":15110,"outputTokens":1,"latencyMs":12594.359375},{"questionId":"q123","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":12442,"outputTokens":1,"latencyMs":82760.21416700003},{"questionId":"q123","format":"csv","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":12289,"outputTokens":1,"latencyMs":38447.406541999895},{"questionId":"q123","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":21958,"outputTokens":1,"latencyMs":33861.31433299999},{"questionId":"q123","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":17156,"outputTokens":1,"latencyMs":8677.22870899993},{"questionId":"q124","format":"json-pretty","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":8589,"outputTokens":1,"latencyMs":1067.0424160000402},{"questionId":"q124","format":"json-compact","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":5916,"outputTokens":1,"latencyMs":1422.5429159998894},{"questionId":"q124","format":"toon","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":7043,"outputTokens":1,"latencyMs":1386.483375000069},{"questionId":"q124","format":"xml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":9412,"outputTokens":1,"latencyMs":1326.9962090000045},{"questionId":"q124","format":"yaml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":6970,"outputTokens":1,"latencyMs":1281.7155829998665},{"questionId":"q125","format":"json-pretty","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8589,"outputTokens":4,"latencyMs":818.738791000098},{"questionId":"q125","format":"json-compact","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5916,"outputTokens":4,"latencyMs":2007.2522090000566},{"questionId":"q125","format":"toon","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7043,"outputTokens":4,"latencyMs":2646.8927089998033},{"questionId":"q125","format":"xml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":9412,"outputTokens":4,"latencyMs":2005.3849579999223},{"questionId":"q125","format":"yaml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6970,"outputTokens":4,"latencyMs":2113.35554099991},{"questionId":"q126","format":"json-pretty","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":8590,"outputTokens":3,"latencyMs":2044.4919579999987},{"questionId":"q126","format":"json-compact","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":5917,"outputTokens":3,"latencyMs":2609.145082999952},{"questionId":"q126","format":"toon","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":7044,"outputTokens":3,"latencyMs":1301.0319590000436},{"questionId":"q126","format":"xml","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":9413,"outputTokens":3,"latencyMs":1536.7617079999764},{"questionId":"q126","format":"yaml","model":"gemini-2.5-flash","expected":"424","actual":"424","isCorrect":true,"inputTokens":6971,"outputTokens":3,"latencyMs":1829.378750000149},{"questionId":"q127","format":"json-pretty","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":8590,"outputTokens":4,"latencyMs":1221.6261249999516},{"questionId":"q127","format":"json-compact","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5917,"outputTokens":4,"latencyMs":1433.4795409999788},{"questionId":"q127","format":"toon","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7044,"outputTokens":4,"latencyMs":1638.4302090001293},{"questionId":"q127","format":"xml","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":9413,"outputTokens":4,"latencyMs":3023.75766699994},{"questionId":"q127","format":"yaml","model":"gemini-2.5-flash","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6971,"outputTokens":4,"latencyMs":1013.5391669999808},{"questionId":"q128","format":"json-pretty","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":8589,"outputTokens":1,"latencyMs":1389.7364999998827},{"questionId":"q128","format":"json-compact","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":5916,"outputTokens":1,"latencyMs":1313.0048750001006},{"questionId":"q128","format":"toon","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":7043,"outputTokens":1,"latencyMs":2430.599707999965},{"questionId":"q128","format":"xml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":9412,"outputTokens":1,"latencyMs":2389.8567919998895},{"questionId":"q128","format":"yaml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":6970,"outputTokens":1,"latencyMs":1699.6213750001043},{"questionId":"q129","format":"json-pretty","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":8589,"outputTokens":4,"latencyMs":3999.8988749999553},{"questionId":"q129","format":"json-compact","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5916,"outputTokens":4,"latencyMs":1966.3846670000348},{"questionId":"q129","format":"toon","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7043,"outputTokens":4,"latencyMs":2553.3473749998957},{"questionId":"q129","format":"xml","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":9412,"outputTokens":4,"latencyMs":2558.1586250001565},{"questionId":"q129","format":"yaml","model":"gemini-2.5-flash","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6970,"outputTokens":4,"latencyMs":2536.7762090000324},{"questionId":"q130","format":"json-pretty","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":8590,"outputTokens":3,"latencyMs":1060.3891659998335},{"questionId":"q130","format":"json-compact","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":5917,"outputTokens":3,"latencyMs":1334.6286250001285},{"questionId":"q130","format":"toon","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":7044,"outputTokens":3,"latencyMs":2516.798916999949},{"questionId":"q130","format":"xml","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":9413,"outputTokens":3,"latencyMs":1110.3052499999758},{"questionId":"q130","format":"yaml","model":"gemini-2.5-flash","expected":"435","actual":"435","isCorrect":true,"inputTokens":6971,"outputTokens":3,"latencyMs":1782.2730419998989},{"questionId":"q131","format":"json-pretty","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":8590,"outputTokens":3,"latencyMs":1135.8688749999274},{"questionId":"q131","format":"json-compact","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":5917,"outputTokens":3,"latencyMs":1453.549624999985},{"questionId":"q131","format":"toon","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":7044,"outputTokens":3,"latencyMs":1791.537958999863},{"questionId":"q131","format":"xml","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":9413,"outputTokens":3,"latencyMs":1893.4191659998614},{"questionId":"q131","format":"yaml","model":"gemini-2.5-flash","expected":"408","actual":"408","isCorrect":true,"inputTokens":6971,"outputTokens":3,"latencyMs":1324.530499999877},{"questionId":"q132","format":"json-pretty","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":8589,"outputTokens":1,"latencyMs":1403.2711249999702},{"questionId":"q132","format":"json-compact","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":5916,"outputTokens":1,"latencyMs":1547.476416000165},{"questionId":"q132","format":"toon","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":7043,"outputTokens":1,"latencyMs":2020.4890000000596},{"questionId":"q132","format":"xml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":9412,"outputTokens":1,"latencyMs":1207.1847910000943},{"questionId":"q132","format":"yaml","model":"gemini-2.5-flash","expected":"error","actual":"error","isCorrect":true,"inputTokens":6970,"outputTokens":1,"latencyMs":975.5986669999547},{"questionId":"q133","format":"json-pretty","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":8589,"outputTokens":4,"latencyMs":1582.2352499999106},{"questionId":"q133","format":"json-compact","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5916,"outputTokens":4,"latencyMs":1356.5891250001732},{"questionId":"q133","format":"toon","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7043,"outputTokens":4,"latencyMs":1940.3270839999896},{"questionId":"q133","format":"xml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":9412,"outputTokens":4,"latencyMs":2231.642125000013},{"questionId":"q133","format":"yaml","model":"gemini-2.5-flash","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6970,"outputTokens":4,"latencyMs":1732.2569170000497},{"questionId":"q134","format":"json-pretty","model":"gemini-2.5-flash","expected":"75","actual":"64","isCorrect":false,"inputTokens":8564,"outputTokens":2,"latencyMs":1193.1987499999814},{"questionId":"q134","format":"json-compact","model":"gemini-2.5-flash","expected":"75","actual":"68","isCorrect":false,"inputTokens":5891,"outputTokens":2,"latencyMs":1198.6124579999596},{"questionId":"q134","format":"toon","model":"gemini-2.5-flash","expected":"75","actual":"63","isCorrect":false,"inputTokens":7018,"outputTokens":2,"latencyMs":1257.1860829999205},{"questionId":"q134","format":"xml","model":"gemini-2.5-flash","expected":"75","actual":"59","isCorrect":false,"inputTokens":9387,"outputTokens":2,"latencyMs":1389.6678750000428},{"questionId":"q134","format":"yaml","model":"gemini-2.5-flash","expected":"75","actual":"58","isCorrect":false,"inputTokens":6945,"outputTokens":2,"latencyMs":1282.8915000001434},{"questionId":"q135","format":"json-pretty","model":"gemini-2.5-flash","expected":"2453.41","actual":"2501.8666666666663","isCorrect":false,"inputTokens":8565,"outputTokens":18,"latencyMs":40920.58108299994},{"questionId":"q135","format":"json-compact","model":"gemini-2.5-flash","expected":"2453.41","actual":"2408","isCorrect":false,"inputTokens":5892,"outputTokens":4,"latencyMs":1728.709625000134},{"questionId":"q135","format":"toon","model":"gemini-2.5-flash","expected":"2453.41","actual":"2447","isCorrect":false,"inputTokens":7019,"outputTokens":4,"latencyMs":16688.119791999925},{"questionId":"q135","format":"xml","model":"gemini-2.5-flash","expected":"2453.41","actual":"2100","isCorrect":false,"inputTokens":9388,"outputTokens":4,"latencyMs":4804.8305000001565},{"questionId":"q135","format":"yaml","model":"gemini-2.5-flash","expected":"2453.41","actual":"2453","isCorrect":false,"inputTokens":6946,"outputTokens":4,"latencyMs":36221.412832999835},{"questionId":"q136","format":"json-pretty","model":"gemini-2.5-flash","expected":"29","actual":"30","isCorrect":false,"inputTokens":8564,"outputTokens":2,"latencyMs":828.0814580000006},{"questionId":"q136","format":"json-compact","model":"gemini-2.5-flash","expected":"29","actual":"30","isCorrect":false,"inputTokens":5891,"outputTokens":2,"latencyMs":1107.8331669999752},{"questionId":"q136","format":"toon","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":7018,"outputTokens":2,"latencyMs":12779.795999999857},{"questionId":"q136","format":"xml","model":"gemini-2.5-flash","expected":"29","actual":"30","isCorrect":false,"inputTokens":9387,"outputTokens":2,"latencyMs":2270.392875000136},{"questionId":"q136","format":"yaml","model":"gemini-2.5-flash","expected":"29","actual":"28","isCorrect":false,"inputTokens":6945,"outputTokens":2,"latencyMs":4683.715624999953},{"questionId":"q137","format":"json-pretty","model":"gemini-2.5-flash","expected":"17","actual":"14","isCorrect":false,"inputTokens":8564,"outputTokens":2,"latencyMs":916.3514170001727},{"questionId":"q137","format":"json-compact","model":"gemini-2.5-flash","expected":"17","actual":"13","isCorrect":false,"inputTokens":5891,"outputTokens":2,"latencyMs":1543.7150830000173},{"questionId":"q137","format":"toon","model":"gemini-2.5-flash","expected":"17","actual":"13","isCorrect":false,"inputTokens":7018,"outputTokens":2,"latencyMs":1275.6650829999708},{"questionId":"q137","format":"xml","model":"gemini-2.5-flash","expected":"17","actual":"15","isCorrect":false,"inputTokens":9387,"outputTokens":2,"latencyMs":1444.6372079998255},{"questionId":"q137","format":"yaml","model":"gemini-2.5-flash","expected":"17","actual":"16","isCorrect":false,"inputTokens":6945,"outputTokens":2,"latencyMs":708.771791000152},{"questionId":"q138","format":"json-pretty","model":"gemini-2.5-flash","expected":"29","actual":"28","isCorrect":false,"inputTokens":8564,"outputTokens":2,"latencyMs":5278.935290999943},{"questionId":"q138","format":"json-compact","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":5891,"outputTokens":2,"latencyMs":12964.67979199998},{"questionId":"q138","format":"toon","model":"gemini-2.5-flash","expected":"29","actual":"26","isCorrect":false,"inputTokens":7018,"outputTokens":2,"latencyMs":3055.3002500000875},{"questionId":"q138","format":"xml","model":"gemini-2.5-flash","expected":"29","actual":"20","isCorrect":false,"inputTokens":9387,"outputTokens":2,"latencyMs":1519.7010419999715},{"questionId":"q138","format":"yaml","model":"gemini-2.5-flash","expected":"29","actual":"20","isCorrect":false,"inputTokens":6945,"outputTokens":2,"latencyMs":1103.3929580000695},{"questionId":"q139","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"10","isCorrect":false,"inputTokens":8567,"outputTokens":2,"latencyMs":1919.766290999949},{"questionId":"q139","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"10","isCorrect":false,"inputTokens":5894,"outputTokens":2,"latencyMs":1519.2419169999193},{"questionId":"q139","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"10","isCorrect":false,"inputTokens":7021,"outputTokens":2,"latencyMs":7955.886208000127},{"questionId":"q139","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"7","isCorrect":false,"inputTokens":9390,"outputTokens":1,"latencyMs":1516.8090000001248},{"questionId":"q139","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"8","isCorrect":false,"inputTokens":6948,"outputTokens":1,"latencyMs":1109.528249999974},{"questionId":"q140","format":"json-pretty","model":"gemini-2.5-flash","expected":"18","actual":"13","isCorrect":false,"inputTokens":8567,"outputTokens":2,"latencyMs":1943.87820799998},{"questionId":"q140","format":"json-compact","model":"gemini-2.5-flash","expected":"18","actual":"18","isCorrect":true,"inputTokens":5894,"outputTokens":2,"latencyMs":5151.926541000139},{"questionId":"q140","format":"toon","model":"gemini-2.5-flash","expected":"18","actual":"15","isCorrect":false,"inputTokens":7021,"outputTokens":2,"latencyMs":958.3163750001695},{"questionId":"q140","format":"xml","model":"gemini-2.5-flash","expected":"18","actual":"15","isCorrect":false,"inputTokens":9390,"outputTokens":2,"latencyMs":974.9349590002093},{"questionId":"q140","format":"yaml","model":"gemini-2.5-flash","expected":"18","actual":"14","isCorrect":false,"inputTokens":6948,"outputTokens":2,"latencyMs":3796.676167000085},{"questionId":"q141","format":"json-pretty","model":"gemini-2.5-flash","expected":"33","actual":"35","isCorrect":false,"inputTokens":8573,"outputTokens":2,"latencyMs":773.2098330000881},{"questionId":"q141","format":"json-compact","model":"gemini-2.5-flash","expected":"33","actual":"36","isCorrect":false,"inputTokens":5900,"outputTokens":2,"latencyMs":1219.797833000077},{"questionId":"q141","format":"toon","model":"gemini-2.5-flash","expected":"33","actual":"30","isCorrect":false,"inputTokens":7027,"outputTokens":2,"latencyMs":1298.999707999872},{"questionId":"q141","format":"xml","model":"gemini-2.5-flash","expected":"33","actual":"28","isCorrect":false,"inputTokens":9396,"outputTokens":2,"latencyMs":1206.6860830001533},{"questionId":"q141","format":"yaml","model":"gemini-2.5-flash","expected":"33","actual":"28","isCorrect":false,"inputTokens":6954,"outputTokens":2,"latencyMs":1047.3714169999585},{"questionId":"q142","format":"json-pretty","model":"gemini-2.5-flash","expected":"42","actual":"39","isCorrect":false,"inputTokens":8573,"outputTokens":2,"latencyMs":1078.253250000067},{"questionId":"q142","format":"json-compact","model":"gemini-2.5-flash","expected":"42","actual":"36","isCorrect":false,"inputTokens":5900,"outputTokens":2,"latencyMs":1654.7375000000466},{"questionId":"q142","format":"toon","model":"gemini-2.5-flash","expected":"42","actual":"42","isCorrect":true,"inputTokens":7027,"outputTokens":2,"latencyMs":29096.239583000075},{"questionId":"q142","format":"xml","model":"gemini-2.5-flash","expected":"42","actual":"45","isCorrect":false,"inputTokens":9396,"outputTokens":2,"latencyMs":1043.4383330000564},{"questionId":"q142","format":"yaml","model":"gemini-2.5-flash","expected":"42","actual":"30","isCorrect":false,"inputTokens":6954,"outputTokens":2,"latencyMs":880.1657920000143},{"questionId":"q143","format":"json-pretty","model":"gemini-2.5-flash","expected":"25","actual":"24","isCorrect":false,"inputTokens":8565,"outputTokens":2,"latencyMs":1222.5377919999883},{"questionId":"q143","format":"json-compact","model":"gemini-2.5-flash","expected":"25","actual":"25","isCorrect":true,"inputTokens":5892,"outputTokens":2,"latencyMs":26826.012707999907},{"questionId":"q143","format":"toon","model":"gemini-2.5-flash","expected":"25","actual":"23","isCorrect":false,"inputTokens":7019,"outputTokens":2,"latencyMs":8472.798624999821},{"questionId":"q143","format":"xml","model":"gemini-2.5-flash","expected":"25","actual":"24","isCorrect":false,"inputTokens":9388,"outputTokens":2,"latencyMs":1977.0973749998957},{"questionId":"q143","format":"yaml","model":"gemini-2.5-flash","expected":"25","actual":"23","isCorrect":false,"inputTokens":6946,"outputTokens":2,"latencyMs":6226.0799579999875},{"questionId":"q144","format":"json-pretty","model":"gemini-2.5-flash","expected":"29","actual":"40","isCorrect":false,"inputTokens":8573,"outputTokens":2,"latencyMs":1166.8498750000726},{"questionId":"q144","format":"json-compact","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":5900,"outputTokens":2,"latencyMs":33054.69249999989},{"questionId":"q144","format":"toon","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":7027,"outputTokens":2,"latencyMs":7581.747792000184},{"questionId":"q144","format":"xml","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":9396,"outputTokens":2,"latencyMs":12077.431208000053},{"questionId":"q144","format":"yaml","model":"gemini-2.5-flash","expected":"29","actual":"29","isCorrect":true,"inputTokens":6954,"outputTokens":2,"latencyMs":9126.843499999959},{"questionId":"q145","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":8573,"outputTokens":1,"latencyMs":5426.647708999924},{"questionId":"q145","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":5900,"outputTokens":1,"latencyMs":5517.758708000183},{"questionId":"q145","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":7027,"outputTokens":1,"latencyMs":5683.845791999949},{"questionId":"q145","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":9396,"outputTokens":1,"latencyMs":12242.952416999964},{"questionId":"q145","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":6954,"outputTokens":1,"latencyMs":25925.259124999866},{"questionId":"q146","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":8576,"outputTokens":1,"latencyMs":9966.068208999932},{"questionId":"q146","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":5903,"outputTokens":1,"latencyMs":28817.112250000006},{"questionId":"q146","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":7030,"outputTokens":1,"latencyMs":8321.378334000008},{"questionId":"q146","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":9399,"outputTokens":1,"latencyMs":10415.615125000011},{"questionId":"q146","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":6957,"outputTokens":1,"latencyMs":7776.882166999858},{"questionId":"q147","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":8576,"outputTokens":1,"latencyMs":10874.438000000082},{"questionId":"q147","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":5903,"outputTokens":1,"latencyMs":30615.237790999934},{"questionId":"q147","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":7030,"outputTokens":1,"latencyMs":21920.779249999905},{"questionId":"q147","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":9399,"outputTokens":1,"latencyMs":12014.872500000056},{"questionId":"q147","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":6957,"outputTokens":1,"latencyMs":9544.11945899995},{"questionId":"q148","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":8576,"outputTokens":1,"latencyMs":7593.520833000075},{"questionId":"q148","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":5903,"outputTokens":1,"latencyMs":31547.131583000068},{"questionId":"q148","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":7030,"outputTokens":1,"latencyMs":10815.48499999987},{"questionId":"q148","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":9399,"outputTokens":1,"latencyMs":15457.432041999884},{"questionId":"q148","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":6957,"outputTokens":1,"latencyMs":7902.872541000135},{"questionId":"q149","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":8572,"outputTokens":1,"latencyMs":20869.27370800008},{"questionId":"q149","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":5899,"outputTokens":1,"latencyMs":29980.081125000026},{"questionId":"q149","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":7026,"outputTokens":1,"latencyMs":13085.466917000012},{"questionId":"q149","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"9","isCorrect":false,"inputTokens":9395,"outputTokens":1,"latencyMs":2662.11012499989},{"questionId":"q149","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":6953,"outputTokens":1,"latencyMs":7577.170167000033},{"questionId":"q150","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"4","isCorrect":false,"inputTokens":8572,"outputTokens":1,"latencyMs":10007.99770800001},{"questionId":"q150","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":5899,"outputTokens":1,"latencyMs":31311.905458000023},{"questionId":"q150","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":7026,"outputTokens":1,"latencyMs":23302.89304200001},{"questionId":"q150","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"7","isCorrect":false,"inputTokens":9395,"outputTokens":1,"latencyMs":9480.15674999985},{"questionId":"q150","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"4","isCorrect":false,"inputTokens":6953,"outputTokens":1,"latencyMs":6279.272334000096},{"questionId":"q151","format":"json-pretty","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":1237,"outputTokens":1,"latencyMs":714.2743750000373},{"questionId":"q151","format":"json-compact","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":720,"outputTokens":1,"latencyMs":629.9168750001118},{"questionId":"q151","format":"toon","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":864,"outputTokens":1,"latencyMs":655.0294170000125},{"questionId":"q151","format":"xml","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":1296,"outputTokens":1,"latencyMs":635.811999999918},{"questionId":"q151","format":"yaml","model":"gemini-2.5-flash","expected":"development","actual":"development","isCorrect":true,"inputTokens":886,"outputTokens":1,"latencyMs":588.2579159999732},{"questionId":"q152","format":"json-pretty","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1235,"outputTokens":6,"latencyMs":710.5584160001017},{"questionId":"q152","format":"json-compact","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":718,"outputTokens":6,"latencyMs":713.5436670000199},{"questionId":"q152","format":"toon","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":862,"outputTokens":6,"latencyMs":942.6960410000756},{"questionId":"q152","format":"xml","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1294,"outputTokens":6,"latencyMs":701.4752919999883},{"questionId":"q152","format":"yaml","model":"gemini-2.5-flash","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":884,"outputTokens":6,"latencyMs":551.4979590000585},{"questionId":"q153","format":"json-pretty","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1235,"outputTokens":4,"latencyMs":627.7900000000373},{"questionId":"q153","format":"json-compact","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":718,"outputTokens":4,"latencyMs":540.4054580000229},{"questionId":"q153","format":"toon","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":862,"outputTokens":4,"latencyMs":1035.3336660000496},{"questionId":"q153","format":"xml","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1294,"outputTokens":4,"latencyMs":1038.0611250000075},{"questionId":"q153","format":"yaml","model":"gemini-2.5-flash","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":884,"outputTokens":4,"latencyMs":1141.808083999902},{"questionId":"q154","format":"json-pretty","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":1237,"outputTokens":2,"latencyMs":1196.8311669998802},{"questionId":"q154","format":"json-compact","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":720,"outputTokens":2,"latencyMs":1027.9143750001676},{"questionId":"q154","format":"toon","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":864,"outputTokens":2,"latencyMs":1237.6204580001067},{"questionId":"q154","format":"xml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":1296,"outputTokens":2,"latencyMs":1536.2729579999577},{"questionId":"q154","format":"yaml","model":"gemini-2.5-flash","expected":"37","actual":"37","isCorrect":true,"inputTokens":886,"outputTokens":2,"latencyMs":1053.627000000095},{"questionId":"q155","format":"json-pretty","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1235,"outputTokens":5,"latencyMs":1546.0707080001011},{"questionId":"q155","format":"json-compact","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":718,"outputTokens":5,"latencyMs":1239.732166999951},{"questionId":"q155","format":"toon","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":862,"outputTokens":5,"latencyMs":611.3902080000844},{"questionId":"q155","format":"xml","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1294,"outputTokens":5,"latencyMs":726.5870000000577},{"questionId":"q155","format":"yaml","model":"gemini-2.5-flash","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":884,"outputTokens":5,"latencyMs":546.1228750001173},{"questionId":"q156","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1237,"outputTokens":1,"latencyMs":714.8755419999361},{"questionId":"q156","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":720,"outputTokens":1,"latencyMs":623.8084170001093},{"questionId":"q156","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":864,"outputTokens":1,"latencyMs":1037.0977910000365},{"questionId":"q156","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1296,"outputTokens":1,"latencyMs":993.644583000103},{"questionId":"q156","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":886,"outputTokens":1,"latencyMs":807.9565000000875},{"questionId":"q157","format":"json-pretty","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1237,"outputTokens":5,"latencyMs":789.1144169999752},{"questionId":"q157","format":"json-compact","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":720,"outputTokens":5,"latencyMs":580.3932499999646},{"questionId":"q157","format":"toon","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":864,"outputTokens":5,"latencyMs":663.5290839998052},{"questionId":"q157","format":"xml","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1296,"outputTokens":5,"latencyMs":968.2126670000143},{"questionId":"q157","format":"yaml","model":"gemini-2.5-flash","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":886,"outputTokens":5,"latencyMs":813.9502499999944},{"questionId":"q158","format":"json-pretty","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":1235,"outputTokens":1,"latencyMs":650.6129160001874},{"questionId":"q158","format":"json-compact","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":718,"outputTokens":1,"latencyMs":555.185749999946},{"questionId":"q158","format":"toon","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":862,"outputTokens":1,"latencyMs":1190.2554999999702},{"questionId":"q158","format":"xml","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":1294,"outputTokens":1,"latencyMs":1328.9656659998},{"questionId":"q158","format":"yaml","model":"gemini-2.5-flash","expected":"real","actual":"real","isCorrect":true,"inputTokens":884,"outputTokens":1,"latencyMs":987.4156249999069},{"questionId":"q159","format":"json-pretty","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1236,"outputTokens":4,"latencyMs":3030.4746670001186},{"questionId":"q159","format":"json-compact","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":719,"outputTokens":4,"latencyMs":926.8707079999149},{"questionId":"q159","format":"toon","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":863,"outputTokens":4,"latencyMs":1090.5767080001533},{"questionId":"q159","format":"xml","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1295,"outputTokens":4,"latencyMs":980.5242909998633},{"questionId":"q159","format":"yaml","model":"gemini-2.5-flash","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":885,"outputTokens":4,"latencyMs":1105.1357909999788},{"questionId":"q160","format":"json-pretty","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1237,"outputTokens":5,"latencyMs":2836.101708999835},{"questionId":"q160","format":"json-compact","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":720,"outputTokens":5,"latencyMs":1143.893875000067},{"questionId":"q160","format":"toon","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":864,"outputTokens":5,"latencyMs":689.4268339998089},{"questionId":"q160","format":"xml","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1296,"outputTokens":5,"latencyMs":632.33574999962},{"questionId":"q160","format":"yaml","model":"gemini-2.5-flash","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":886,"outputTokens":5,"latencyMs":671.2921659997664},{"questionId":"q161","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1237,"outputTokens":1,"latencyMs":1052.6630420000292},{"questionId":"q161","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":720,"outputTokens":1,"latencyMs":1086.7699159998447},{"questionId":"q161","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":864,"outputTokens":1,"latencyMs":1079.8444580002688},{"questionId":"q161","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1296,"outputTokens":1,"latencyMs":1044.9725839998573},{"questionId":"q161","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":886,"outputTokens":1,"latencyMs":976.4435000000522},{"questionId":"q162","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1237,"outputTokens":1,"latencyMs":1337.0680419998243},{"questionId":"q162","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":720,"outputTokens":1,"latencyMs":1334.8939169999212},{"questionId":"q162","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":864,"outputTokens":1,"latencyMs":1001.1695419996977},{"questionId":"q162","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1296,"outputTokens":1,"latencyMs":950.9347500000149},{"questionId":"q162","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":886,"outputTokens":1,"latencyMs":898.8835840001702},{"questionId":"q163","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1236,"outputTokens":1,"latencyMs":1177.4634170001373},{"questionId":"q163","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":719,"outputTokens":1,"latencyMs":1334.1764579997398},{"questionId":"q163","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":863,"outputTokens":1,"latencyMs":1148.38300000038},{"questionId":"q163","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1295,"outputTokens":1,"latencyMs":2282.0497499997728},{"questionId":"q163","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":885,"outputTokens":1,"latencyMs":3237.1732080001384},{"questionId":"q164","format":"json-pretty","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1236,"outputTokens":1,"latencyMs":1329.3487909999676},{"questionId":"q164","format":"json-compact","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":719,"outputTokens":1,"latencyMs":1184.8616249999031},{"questionId":"q164","format":"toon","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":863,"outputTokens":1,"latencyMs":1649.8752919998951},{"questionId":"q164","format":"xml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":1295,"outputTokens":1,"latencyMs":1179.0983339999802},{"questionId":"q164","format":"yaml","model":"gemini-2.5-flash","expected":"2","actual":"2","isCorrect":true,"inputTokens":885,"outputTokens":1,"latencyMs":2753.481666999869},{"questionId":"q165","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1236,"outputTokens":1,"latencyMs":1750.9820840000175},{"questionId":"q165","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":719,"outputTokens":1,"latencyMs":1686.8586669997312},{"questionId":"q165","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":863,"outputTokens":1,"latencyMs":1428.7210000003688},{"questionId":"q165","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1295,"outputTokens":1,"latencyMs":650.6599579998292},{"questionId":"q165","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":885,"outputTokens":1,"latencyMs":1572.7508339998312},{"questionId":"q166","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1240,"outputTokens":1,"latencyMs":1199.4317089999095},{"questionId":"q166","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":723,"outputTokens":1,"latencyMs":2643.249499999918},{"questionId":"q166","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":867,"outputTokens":1,"latencyMs":1925.100666999817},{"questionId":"q166","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1299,"outputTokens":1,"latencyMs":1735.2946250000969},{"questionId":"q166","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":889,"outputTokens":1,"latencyMs":2291.729416999966},{"questionId":"q167","format":"json-pretty","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1236,"outputTokens":1,"latencyMs":1165.5019999998622},{"questionId":"q167","format":"json-compact","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":719,"outputTokens":1,"latencyMs":1602.4723749998957},{"questionId":"q167","format":"toon","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":863,"outputTokens":1,"latencyMs":2674.9296249998733},{"questionId":"q167","format":"xml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1295,"outputTokens":1,"latencyMs":1566.2657920001075},{"questionId":"q167","format":"yaml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":885,"outputTokens":1,"latencyMs":1006.7567079998553},{"questionId":"q168","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1238,"outputTokens":1,"latencyMs":1587.6670409999788},{"questionId":"q168","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":721,"outputTokens":1,"latencyMs":1424.9540419997647},{"questionId":"q168","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":865,"outputTokens":1,"latencyMs":1259.1735419998877},{"questionId":"q168","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1297,"outputTokens":1,"latencyMs":2154.8523750002496},{"questionId":"q168","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":887,"outputTokens":1,"latencyMs":1571.5424170000479},{"questionId":"q169","format":"json-pretty","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":1240,"outputTokens":1,"latencyMs":3195.747207999695},{"questionId":"q169","format":"json-compact","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":723,"outputTokens":1,"latencyMs":2006.9592500003055},{"questionId":"q169","format":"toon","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":867,"outputTokens":1,"latencyMs":1555.6437920001335},{"questionId":"q169","format":"xml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":1299,"outputTokens":1,"latencyMs":2224.92799999984},{"questionId":"q169","format":"yaml","model":"gemini-2.5-flash","expected":"8","actual":"8","isCorrect":true,"inputTokens":889,"outputTokens":1,"latencyMs":1248.4440419999883},{"questionId":"q170","format":"json-pretty","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1239,"outputTokens":1,"latencyMs":3982.0419580000453},{"questionId":"q170","format":"json-compact","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":722,"outputTokens":1,"latencyMs":2576.5991670000367},{"questionId":"q170","format":"toon","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":866,"outputTokens":1,"latencyMs":4060.126999999862},{"questionId":"q170","format":"xml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":1298,"outputTokens":1,"latencyMs":1937.491291999817},{"questionId":"q170","format":"yaml","model":"gemini-2.5-flash","expected":"5","actual":"5","isCorrect":true,"inputTokens":888,"outputTokens":1,"latencyMs":1705.1981250001118},{"questionId":"q171","format":"json-pretty","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1241,"outputTokens":1,"latencyMs":1217.6796249998733},{"questionId":"q171","format":"json-compact","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":724,"outputTokens":1,"latencyMs":1606.7131659998558},{"questionId":"q171","format":"toon","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":868,"outputTokens":1,"latencyMs":1774.6648750002496},{"questionId":"q171","format":"xml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":1300,"outputTokens":1,"latencyMs":1195.7557910000905},{"questionId":"q171","format":"yaml","model":"gemini-2.5-flash","expected":"3","actual":"3","isCorrect":true,"inputTokens":890,"outputTokens":1,"latencyMs":1956.6881250003353},{"questionId":"q172","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1241,"outputTokens":1,"latencyMs":835.3650409998372},{"questionId":"q172","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":724,"outputTokens":1,"latencyMs":1510.3631670000032},{"questionId":"q172","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":868,"outputTokens":1,"latencyMs":1780.3312919996679},{"questionId":"q172","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1300,"outputTokens":1,"latencyMs":2000.1072080000304},{"questionId":"q172","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":890,"outputTokens":1,"latencyMs":1098.8821670003235},{"questionId":"q173","format":"json-pretty","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1243,"outputTokens":1,"latencyMs":2118.300083000213},{"questionId":"q173","format":"json-compact","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":726,"outputTokens":1,"latencyMs":2103.217958000023},{"questionId":"q173","format":"toon","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":870,"outputTokens":1,"latencyMs":3209.9628329998814},{"questionId":"q173","format":"xml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1302,"outputTokens":1,"latencyMs":2693.650875000283},{"questionId":"q173","format":"yaml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":892,"outputTokens":1,"latencyMs":1793.4279169999063},{"questionId":"q174","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1239,"outputTokens":1,"latencyMs":2179.5218340000138},{"questionId":"q174","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":722,"outputTokens":1,"latencyMs":2181.4001660002396},{"questionId":"q174","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":866,"outputTokens":1,"latencyMs":1788.8398750000633},{"questionId":"q174","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1298,"outputTokens":1,"latencyMs":3043.868249999825},{"questionId":"q174","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":888,"outputTokens":1,"latencyMs":1799.4413749999367},{"questionId":"q175","format":"json-pretty","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1244,"outputTokens":1,"latencyMs":1856.9236249998212},{"questionId":"q175","format":"json-compact","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":727,"outputTokens":1,"latencyMs":2470.588165999856},{"questionId":"q175","format":"toon","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":871,"outputTokens":1,"latencyMs":2116.788166999817},{"questionId":"q175","format":"xml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":1303,"outputTokens":1,"latencyMs":2093.4396670004353},{"questionId":"q175","format":"yaml","model":"gemini-2.5-flash","expected":"0","actual":"0","isCorrect":true,"inputTokens":893,"outputTokens":1,"latencyMs":1927.34620800009},{"questionId":"q176","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1237,"outputTokens":1,"latencyMs":1133.6035830001347},{"questionId":"q176","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":720,"outputTokens":1,"latencyMs":1728.5193330002949},{"questionId":"q176","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":864,"outputTokens":1,"latencyMs":1444.868833000306},{"questionId":"q176","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1296,"outputTokens":1,"latencyMs":3402.543167000171},{"questionId":"q176","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":886,"outputTokens":1,"latencyMs":1406.3881660001352},{"questionId":"q177","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1247,"outputTokens":1,"latencyMs":2466.386582999956},{"questionId":"q177","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":730,"outputTokens":1,"latencyMs":2060.1465829997323},{"questionId":"q177","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":874,"outputTokens":1,"latencyMs":2656.8424169998616},{"questionId":"q177","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1306,"outputTokens":1,"latencyMs":2637.5038339998573},{"questionId":"q177","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":896,"outputTokens":1,"latencyMs":3448.6969579998404},{"questionId":"q178","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1240,"outputTokens":1,"latencyMs":4023.9037080002017},{"questionId":"q178","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":723,"outputTokens":1,"latencyMs":2369.979000000283},{"questionId":"q178","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":867,"outputTokens":1,"latencyMs":3023.4400419997983},{"questionId":"q178","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"2","isCorrect":false,"inputTokens":1299,"outputTokens":1,"latencyMs":1269.8315830002539},{"questionId":"q178","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":889,"outputTokens":1,"latencyMs":1606.4947499996051},{"questionId":"q179","format":"json-pretty","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1239,"outputTokens":1,"latencyMs":2168.672083000187},{"questionId":"q179","format":"json-compact","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":722,"outputTokens":1,"latencyMs":2064.1299999998882},{"questionId":"q179","format":"toon","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":866,"outputTokens":1,"latencyMs":2011.0787499998696},{"questionId":"q179","format":"xml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":1298,"outputTokens":1,"latencyMs":3021.5313329999335},{"questionId":"q179","format":"yaml","model":"gemini-2.5-flash","expected":"1","actual":"1","isCorrect":true,"inputTokens":888,"outputTokens":1,"latencyMs":1747.6599169997498},{"questionId":"q180","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":7985,"outputTokens":3,"latencyMs":1446.862083000131},{"questionId":"q180","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":4773,"outputTokens":3,"latencyMs":1767.1606250000186},{"questionId":"q180","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3418,"outputTokens":3,"latencyMs":708.0473329997621},{"questionId":"q180","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":3269,"outputTokens":3,"latencyMs":1554.0443339999765},{"questionId":"q180","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":9170,"outputTokens":3,"latencyMs":6119.1349159996025},{"questionId":"q180","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":5819,"outputTokens":3,"latencyMs":1263.5255829999223},{"questionId":"q181","format":"json-pretty","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":7991,"outputTokens":14,"latencyMs":4622.160707999952},{"questionId":"q181","format":"json-compact","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4779,"outputTokens":14,"latencyMs":1652.5232920004055},{"questionId":"q181","format":"toon","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":3424,"outputTokens":14,"latencyMs":1267.4767499999143},{"questionId":"q181","format":"csv","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":3275,"outputTokens":14,"latencyMs":955.1592500000261},{"questionId":"q181","format":"xml","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":9176,"outputTokens":14,"latencyMs":1751.6956659997813},{"questionId":"q181","format":"yaml","model":"gemini-2.5-flash","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":5825,"outputTokens":14,"latencyMs":1520.0058329999447},{"questionId":"q182","format":"json-pretty","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":7988,"outputTokens":1,"latencyMs":1944.3507920000702},{"questionId":"q182","format":"json-compact","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":4776,"outputTokens":1,"latencyMs":1344.7699579996988},{"questionId":"q182","format":"toon","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":3421,"outputTokens":1,"latencyMs":1090.3308339999057},{"questionId":"q182","format":"csv","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":3272,"outputTokens":1,"latencyMs":1246.2287920000963},{"questionId":"q182","format":"xml","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":9173,"outputTokens":1,"latencyMs":1681.2632500003092},{"questionId":"q182","format":"yaml","model":"gemini-2.5-flash","expected":"email","actual":"email","isCorrect":true,"inputTokens":5822,"outputTokens":1,"latencyMs":2097.8243339997716},{"questionId":"q183","format":"json-pretty","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7989,"outputTokens":1,"latencyMs":1373.8630830002949},{"questionId":"q183","format":"json-compact","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4777,"outputTokens":1,"latencyMs":1371.1412500003353},{"questionId":"q183","format":"toon","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":3422,"outputTokens":1,"latencyMs":1571.0131660001352},{"questionId":"q183","format":"csv","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":3273,"outputTokens":1,"latencyMs":1382.7117920001037},{"questionId":"q183","format":"xml","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":9174,"outputTokens":1,"latencyMs":1624.1318749999627},{"questionId":"q183","format":"yaml","model":"gemini-2.5-flash","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5823,"outputTokens":1,"latencyMs":1883.3100410001352},{"questionId":"q184","format":"json-pretty","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7989,"outputTokens":5,"latencyMs":1720.2970000002533},{"questionId":"q184","format":"json-compact","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4777,"outputTokens":5,"latencyMs":2262.5714579997584},{"questionId":"q184","format":"toon","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":3422,"outputTokens":5,"latencyMs":1357.559084000066},{"questionId":"q184","format":"csv","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":3273,"outputTokens":5,"latencyMs":1239.9162090001628},{"questionId":"q184","format":"xml","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":9174,"outputTokens":5,"latencyMs":3165.6596670001745},{"questionId":"q184","format":"yaml","model":"gemini-2.5-flash","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5823,"outputTokens":5,"latencyMs":1777.2916660001501},{"questionId":"q185","format":"json-pretty","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":7986,"outputTokens":1,"latencyMs":2076.745957999956},{"questionId":"q185","format":"json-compact","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":4774,"outputTokens":1,"latencyMs":1548.473792000208},{"questionId":"q185","format":"toon","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":3419,"outputTokens":1,"latencyMs":2909.85570800025},{"questionId":"q185","format":"csv","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":3270,"outputTokens":1,"latencyMs":1385.9492079997435},{"questionId":"q185","format":"xml","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":9171,"outputTokens":1,"latencyMs":1949.480624999851},{"questionId":"q185","format":"yaml","model":"gemini-2.5-flash","expected":"7","actual":"7","isCorrect":true,"inputTokens":5820,"outputTokens":1,"latencyMs":1824.6168329999782},{"questionId":"q186","format":"json-pretty","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":13865,"outputTokens":2,"latencyMs":1103.0194580000825},{"questionId":"q186","format":"json-compact","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":8507,"outputTokens":2,"latencyMs":961.7970419996418},{"questionId":"q186","format":"toon","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":9125,"outputTokens":2,"latencyMs":768.455417000223},{"questionId":"q186","format":"xml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":15253,"outputTokens":2,"latencyMs":967.9823340000585},{"questionId":"q186","format":"yaml","model":"gemini-2.5-flash","expected":"50","actual":"50","isCorrect":true,"inputTokens":10118,"outputTokens":2,"latencyMs":3138.2025839998387},{"questionId":"q187","format":"json-pretty","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":13874,"outputTokens":18,"latencyMs":2083.487625000067},{"questionId":"q187","format":"json-compact","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":8516,"outputTokens":18,"latencyMs":1389.503500000108},{"questionId":"q187","format":"toon","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":9134,"outputTokens":18,"latencyMs":2578.829582999926},{"questionId":"q187","format":"xml","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":15262,"outputTokens":18,"latencyMs":2163.259042000398},{"questionId":"q187","format":"yaml","model":"gemini-2.5-flash","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":10127,"outputTokens":18,"latencyMs":2421.821874999907},{"questionId":"q188","format":"json-pretty","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":13870,"outputTokens":1,"latencyMs":1735.4886670000851},{"questionId":"q188","format":"json-compact","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":8512,"outputTokens":1,"latencyMs":1492.0317910001613},{"questionId":"q188","format":"toon","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":9130,"outputTokens":1,"latencyMs":1938.577124999836},{"questionId":"q188","format":"xml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":15258,"outputTokens":1,"latencyMs":1526.4801250002347},{"questionId":"q188","format":"yaml","model":"gemini-2.5-flash","expected":"4","actual":"4","isCorrect":true,"inputTokens":10123,"outputTokens":1,"latencyMs":1310.831709000282},{"questionId":"q189","format":"json-pretty","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":13874,"outputTokens":7,"latencyMs":2396.463041999843},{"questionId":"q189","format":"json-compact","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":8516,"outputTokens":7,"latencyMs":1256.0764999999665},{"questionId":"q189","format":"toon","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":9134,"outputTokens":7,"latencyMs":1141.192916000262},{"questionId":"q189","format":"xml","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":15262,"outputTokens":7,"latencyMs":865.6484580002725},{"questionId":"q189","format":"yaml","model":"gemini-2.5-flash","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":10127,"outputTokens":7,"latencyMs":1471.272999999579},{"questionId":"q190","format":"json-pretty","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":13869,"outputTokens":1,"latencyMs":1294.996083999984},{"questionId":"q190","format":"json-compact","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8511,"outputTokens":1,"latencyMs":1216.7760830000043},{"questionId":"q190","format":"toon","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":9129,"outputTokens":1,"latencyMs":2036.0644999998622},{"questionId":"q190","format":"xml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":15257,"outputTokens":1,"latencyMs":2198.97424999997},{"questionId":"q190","format":"yaml","model":"gemini-2.5-flash","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10122,"outputTokens":1,"latencyMs":2003.7483330001123},{"questionId":"q191","format":"json-pretty","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":13875,"outputTokens":7,"latencyMs":1674.174624999985},{"questionId":"q191","format":"json-compact","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":8517,"outputTokens":7,"latencyMs":1204.0699169998989},{"questionId":"q191","format":"toon","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":9135,"outputTokens":7,"latencyMs":1726.0945420004427},{"questionId":"q191","format":"xml","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":15263,"outputTokens":7,"latencyMs":1140.9084999999031},{"questionId":"q191","format":"yaml","model":"gemini-2.5-flash","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":10128,"outputTokens":7,"latencyMs":2343.766459000297},{"questionId":"q192","format":"json-pretty","model":"gemini-2.5-flash","expected":"60","actual":"59","isCorrect":false,"inputTokens":4836,"outputTokens":2,"latencyMs":965.0458329999819},{"questionId":"q192","format":"json-compact","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3084,"outputTokens":2,"latencyMs":3846.6419159998186},{"questionId":"q192","format":"toon","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2347,"outputTokens":2,"latencyMs":5973.953041999601},{"questionId":"q192","format":"csv","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":2261,"outputTokens":2,"latencyMs":1743.6736249998212},{"questionId":"q192","format":"xml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":5479,"outputTokens":2,"latencyMs":4332.852584000211},{"questionId":"q192","format":"yaml","model":"gemini-2.5-flash","expected":"60","actual":"60","isCorrect":true,"inputTokens":3865,"outputTokens":2,"latencyMs":5812.806791999843},{"questionId":"q193","format":"json-pretty","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4841,"outputTokens":13,"latencyMs":1686.3909579999745},{"questionId":"q193","format":"json-compact","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3089,"outputTokens":13,"latencyMs":1012.8692499999888},{"questionId":"q193","format":"toon","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2352,"outputTokens":13,"latencyMs":1796.8359169997275},{"questionId":"q193","format":"csv","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2266,"outputTokens":13,"latencyMs":1341.453040999826},{"questionId":"q193","format":"xml","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":5484,"outputTokens":13,"latencyMs":1171.129708999768},{"questionId":"q193","format":"yaml","model":"gemini-2.5-flash","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3870,"outputTokens":13,"latencyMs":1392.670582999941},{"questionId":"q194","format":"json-pretty","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":4839,"outputTokens":1,"latencyMs":1229.5054999999702},{"questionId":"q194","format":"json-compact","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3087,"outputTokens":1,"latencyMs":1365.973958000075},{"questionId":"q194","format":"toon","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2350,"outputTokens":1,"latencyMs":1316.475458000321},{"questionId":"q194","format":"csv","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2264,"outputTokens":1,"latencyMs":1109.0172920003533},{"questionId":"q194","format":"xml","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":5482,"outputTokens":1,"latencyMs":2092.274499999825},{"questionId":"q194","format":"yaml","model":"gemini-2.5-flash","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3868,"outputTokens":1,"latencyMs":1707.313084000256},{"questionId":"q195","format":"json-pretty","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4840,"outputTokens":10,"latencyMs":773.2220830000006},{"questionId":"q195","format":"json-compact","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3088,"outputTokens":10,"latencyMs":1242.5294160000049},{"questionId":"q195","format":"toon","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2351,"outputTokens":10,"latencyMs":1906.6233749999665},{"questionId":"q195","format":"csv","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2265,"outputTokens":10,"latencyMs":1189.7229579999112},{"questionId":"q195","format":"xml","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":5483,"outputTokens":10,"latencyMs":2315.4849160001613},{"questionId":"q195","format":"yaml","model":"gemini-2.5-flash","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3869,"outputTokens":10,"latencyMs":1170.6192499999888},{"questionId":"q196","format":"json-pretty","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":4836,"outputTokens":1,"latencyMs":1493.891375000123},{"questionId":"q196","format":"json-compact","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":3084,"outputTokens":1,"latencyMs":1494.020917000249},{"questionId":"q196","format":"toon","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":2347,"outputTokens":1,"latencyMs":1595.266209000256},{"questionId":"q196","format":"csv","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":2261,"outputTokens":1,"latencyMs":1682.9736250001006},{"questionId":"q196","format":"xml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":5479,"outputTokens":1,"latencyMs":1971.1372919999994},{"questionId":"q196","format":"yaml","model":"gemini-2.5-flash","expected":"6","actual":"6","isCorrect":true,"inputTokens":3865,"outputTokens":1,"latencyMs":1747.547416999936},{"questionId":"q197","format":"json-pretty","model":"gemini-2.5-flash","expected":"100","actual":"64","isCorrect":false,"inputTokens":20056,"outputTokens":2,"latencyMs":1280.5209169997834},{"questionId":"q197","format":"json-compact","model":"gemini-2.5-flash","expected":"100","actual":"60","isCorrect":false,"inputTokens":15094,"outputTokens":2,"latencyMs":1799.125},{"questionId":"q197","format":"toon","model":"gemini-2.5-flash","expected":"100","actual":"59","isCorrect":false,"inputTokens":12426,"outputTokens":2,"latencyMs":1655.565750000067},{"questionId":"q197","format":"csv","model":"gemini-2.5-flash","expected":"100","actual":"100","isCorrect":true,"inputTokens":12273,"outputTokens":3,"latencyMs":4215.504958000034},{"questionId":"q197","format":"xml","model":"gemini-2.5-flash","expected":"100","actual":"60","isCorrect":false,"inputTokens":21942,"outputTokens":2,"latencyMs":1543.4917500000447},{"questionId":"q197","format":"yaml","model":"gemini-2.5-flash","expected":"100","actual":"50","isCorrect":false,"inputTokens":17140,"outputTokens":2,"latencyMs":1411.948166999966},{"questionId":"q198","format":"json-pretty","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":20062,"outputTokens":27,"latencyMs":1880.8194579998963},{"questionId":"q198","format":"json-compact","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":15100,"outputTokens":27,"latencyMs":2417.0248749996535},{"questionId":"q198","format":"toon","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":12432,"outputTokens":27,"latencyMs":1455.520291000139},{"questionId":"q198","format":"csv","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":12279,"outputTokens":27,"latencyMs":1544.5812920001335},{"questionId":"q198","format":"xml","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":21948,"outputTokens":27,"latencyMs":1272.468249999918},{"questionId":"q198","format":"yaml","model":"gemini-2.5-flash","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":17146,"outputTokens":27,"latencyMs":1432.5256670000963},{"questionId":"q199","format":"json-pretty","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":20060,"outputTokens":3,"latencyMs":2211.931791000068},{"questionId":"q199","format":"json-compact","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":15098,"outputTokens":3,"latencyMs":2367.136917000171},{"questionId":"q199","format":"toon","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":12430,"outputTokens":3,"latencyMs":1879.897125000134},{"questionId":"q199","format":"csv","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":12277,"outputTokens":3,"latencyMs":1440.2126670000143},{"questionId":"q199","format":"xml","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":21946,"outputTokens":3,"latencyMs":1997.3791249999776},{"questionId":"q199","format":"yaml","model":"gemini-2.5-flash","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":17144,"outputTokens":3,"latencyMs":1965.4168330002576},{"questionId":"q200","format":"json-pretty","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":20060,"outputTokens":1,"latencyMs":2462.445624999702},{"questionId":"q200","format":"json-compact","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":15098,"outputTokens":1,"latencyMs":1158.14708300028},{"questionId":"q200","format":"toon","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":12430,"outputTokens":1,"latencyMs":2571.0614170003682},{"questionId":"q200","format":"csv","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":12277,"outputTokens":1,"latencyMs":2518.3229590002447},{"questionId":"q200","format":"xml","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":21946,"outputTokens":1,"latencyMs":2650.1444160002284},{"questionId":"q200","format":"yaml","model":"gemini-2.5-flash","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17144,"outputTokens":1,"latencyMs":1687.7891669999808},{"questionId":"q201","format":"json-pretty","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":20057,"outputTokens":2,"latencyMs":3629.072374999989},{"questionId":"q201","format":"json-compact","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":15095,"outputTokens":2,"latencyMs":2676.048208999913},{"questionId":"q201","format":"toon","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":12427,"outputTokens":2,"latencyMs":2040.828625000082},{"questionId":"q201","format":"csv","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":12274,"outputTokens":2,"latencyMs":1990.3549580001272},{"questionId":"q201","format":"xml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":21943,"outputTokens":2,"latencyMs":2825.825124999974},{"questionId":"q201","format":"yaml","model":"gemini-2.5-flash","expected":"11","actual":"11","isCorrect":true,"inputTokens":17141,"outputTokens":2,"latencyMs":2873.598708000034},{"questionId":"q202","format":"json-pretty","model":"gemini-2.5-flash","expected":"75","actual":"75","isCorrect":true,"inputTokens":8564,"outputTokens":2,"latencyMs":10687.280999999959},{"questionId":"q202","format":"json-compact","model":"gemini-2.5-flash","expected":"75","actual":"67","isCorrect":false,"inputTokens":5891,"outputTokens":2,"latencyMs":991.2583750002086},{"questionId":"q202","format":"toon","model":"gemini-2.5-flash","expected":"75","actual":"75","isCorrect":true,"inputTokens":7018,"outputTokens":2,"latencyMs":11300.983666999731},{"questionId":"q202","format":"xml","model":"gemini-2.5-flash","expected":"75","actual":"60","isCorrect":false,"inputTokens":9387,"outputTokens":2,"latencyMs":1059.8835829999298},{"questionId":"q202","format":"yaml","model":"gemini-2.5-flash","expected":"75","actual":"58","isCorrect":false,"inputTokens":6945,"outputTokens":2,"latencyMs":1021.2741660000756},{"questionId":"q203","format":"json-pretty","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,error.message,error.stack,error.retryable","isCorrect":false,"inputTokens":8574,"outputTokens":26,"latencyMs":2541.763125000056},{"questionId":"q203","format":"json-compact","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":5901,"outputTokens":13,"latencyMs":2903.7585419998504},{"questionId":"q203","format":"toon","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":false,"inputTokens":7028,"outputTokens":20,"latencyMs":3423.2478329995647},{"questionId":"q203","format":"xml","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":9397,"outputTokens":13,"latencyMs":4180.648209000006},{"questionId":"q203","format":"yaml","model":"gemini-2.5-flash","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error,message,stack,retryable","isCorrect":false,"inputTokens":6955,"outputTokens":20,"latencyMs":5466.100667000283},{"questionId":"q204","format":"json-pretty","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":8568,"outputTokens":1,"latencyMs":2111.21875},{"questionId":"q204","format":"json-compact","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":5895,"outputTokens":1,"latencyMs":1022.1352909998968},{"questionId":"q204","format":"toon","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":7022,"outputTokens":1,"latencyMs":1100.4959169998765},{"questionId":"q204","format":"xml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":9391,"outputTokens":1,"latencyMs":1614.9553750003688},{"questionId":"q204","format":"yaml","model":"gemini-2.5-flash","expected":"info","actual":"info","isCorrect":true,"inputTokens":6949,"outputTokens":1,"latencyMs":1259.7537920000032},{"questionId":"q205","format":"json-pretty","model":"gemini-2.5-flash","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1678,"outputTokens":1,"latencyMs":3173.756290999707},{"questionId":"q205","format":"json-compact","model":"gemini-2.5-flash","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1027,"outputTokens":1,"latencyMs":7402.774416999891},{"questionId":"q205","format":"toon","model":"gemini-2.5-flash","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":790,"outputTokens":1,"latencyMs":4736.116041000001},{"questionId":"q205","format":"csv","model":"gemini-2.5-flash","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":739,"outputTokens":1,"latencyMs":5931.817209000234},{"questionId":"q205","format":"xml","model":"gemini-2.5-flash","expected":"YES","actual":"NO","isCorrect":false,"inputTokens":1901,"outputTokens":1,"latencyMs":2020.544666999951},{"questionId":"q205","format":"yaml","model":"gemini-2.5-flash","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1240,"outputTokens":1,"latencyMs":3512.2852499997243},{"questionId":"q206","format":"json-pretty","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1450,"outputTokens":1,"latencyMs":5513.346541999839},{"questionId":"q206","format":"json-compact","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":895,"outputTokens":1,"latencyMs":2014.1796249998733},{"questionId":"q206","format":"toon","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":700,"outputTokens":1,"latencyMs":2948.0258329999633},{"questionId":"q206","format":"csv","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":652,"outputTokens":1,"latencyMs":5390.936832999811},{"questionId":"q206","format":"xml","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1637,"outputTokens":1,"latencyMs":1604.2141249999404},{"questionId":"q206","format":"yaml","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1076,"outputTokens":1,"latencyMs":5025.293584000319},{"questionId":"q207","format":"json-pretty","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1916,"outputTokens":1,"latencyMs":9530.83704100037},{"questionId":"q207","format":"json-compact","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1169,"outputTokens":1,"latencyMs":2656.4764999998733},{"questionId":"q207","format":"toon","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":890,"outputTokens":1,"latencyMs":4081.831625000108},{"questionId":"q207","format":"csv","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":836,"outputTokens":1,"latencyMs":13257.7953329999},{"questionId":"q207","format":"xml","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":2177,"outputTokens":1,"latencyMs":2043.736082999967},{"questionId":"q207","format":"yaml","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1414,"outputTokens":1,"latencyMs":2348.742291999981},{"questionId":"q208","format":"json-pretty","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1666,"outputTokens":1,"latencyMs":23475.330124999862},{"questionId":"q208","format":"json-compact","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1019,"outputTokens":1,"latencyMs":3639.9307079999708},{"questionId":"q208","format":"toon","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1258,"outputTokens":1,"latencyMs":11545.254791000392},{"questionId":"q208","format":"csv","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":733,"outputTokens":1,"latencyMs":8341.815500000026},{"questionId":"q208","format":"xml","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1888,"outputTokens":1,"latencyMs":2059.0587500003166},{"questionId":"q208","format":"yaml","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1230,"outputTokens":1,"latencyMs":5425.482917000074},{"questionId":"q209","format":"json-pretty","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1616,"outputTokens":1,"latencyMs":13257.379040999804},{"questionId":"q209","format":"json-compact","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":981,"outputTokens":1,"latencyMs":7621.506374999881},{"questionId":"q209","format":"toon","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1218,"outputTokens":1,"latencyMs":3775.1144159999676},{"questionId":"q209","format":"csv","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":520,"outputTokens":1,"latencyMs":8472.738208999857},{"questionId":"q209","format":"xml","model":"gemini-2.5-flash","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1835,"outputTokens":1,"latencyMs":1492.8847090001218},{"questionId":"q209","format":"yaml","model":"gemini-2.5-flash","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1190,"outputTokens":1,"latencyMs":2429.2907079998404}] \ No newline at end of file diff --git a/benchmarks/results/accuracy/models/gpt-5-nano b/benchmarks/results/accuracy/models/gpt-5-nano index 7aee973..0744aca 100644 --- a/benchmarks/results/accuracy/models/gpt-5-nano +++ b/benchmarks/results/accuracy/models/gpt-5-nano @@ -1 +1 @@ -[{"questionId":"q1","format":"json-pretty","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":6406,"outputTokens":72,"latencyMs":2795.7537920000004},{"questionId":"q1","format":"json-compact","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":3999,"outputTokens":136,"latencyMs":2731.920209},{"questionId":"q1","format":"toon","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2563,"outputTokens":136,"latencyMs":2043.561292},{"questionId":"q1","format":"csv","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2397,"outputTokens":72,"latencyMs":3079.960125},{"questionId":"q1","format":"xml","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7368,"outputTokens":72,"latencyMs":2146.722875},{"questionId":"q1","format":"yaml","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5026,"outputTokens":72,"latencyMs":1723.30575},{"questionId":"q2","format":"json-pretty","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":6406,"outputTokens":135,"latencyMs":2380.766125},{"questionId":"q2","format":"json-compact","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":3999,"outputTokens":135,"latencyMs":2692.873125},{"questionId":"q2","format":"toon","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2563,"outputTokens":135,"latencyMs":2005.906459},{"questionId":"q2","format":"csv","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2397,"outputTokens":135,"latencyMs":2132.547084},{"questionId":"q2","format":"xml","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7368,"outputTokens":135,"latencyMs":2446.40625},{"questionId":"q2","format":"yaml","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5026,"outputTokens":135,"latencyMs":2234.044917},{"questionId":"q3","format":"json-pretty","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":6408,"outputTokens":140,"latencyMs":2156.7097919999997},{"questionId":"q3","format":"json-compact","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4001,"outputTokens":140,"latencyMs":2113.4089169999997},{"questionId":"q3","format":"toon","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2565,"outputTokens":204,"latencyMs":3955.9496670000008},{"questionId":"q3","format":"csv","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2399,"outputTokens":140,"latencyMs":2025.2626250000003},{"questionId":"q3","format":"xml","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7370,"outputTokens":204,"latencyMs":3569.3472919999995},{"questionId":"q3","format":"yaml","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5028,"outputTokens":140,"latencyMs":3380.634500000001},{"questionId":"q4","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":6408,"outputTokens":71,"latencyMs":4386.4415},{"questionId":"q4","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4001,"outputTokens":71,"latencyMs":2326.8963750000003},{"questionId":"q4","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2565,"outputTokens":135,"latencyMs":3054.547041},{"questionId":"q4","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2399,"outputTokens":199,"latencyMs":2742.175000000001},{"questionId":"q4","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":7370,"outputTokens":199,"latencyMs":2664.4997089999997},{"questionId":"q4","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":5028,"outputTokens":135,"latencyMs":2353.598750000001},{"questionId":"q5","format":"json-pretty","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":6404,"outputTokens":391,"latencyMs":4494.445208000001},{"questionId":"q5","format":"json-compact","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":3997,"outputTokens":135,"latencyMs":2237.265708000001},{"questionId":"q5","format":"toon","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":2561,"outputTokens":199,"latencyMs":5053.545375},{"questionId":"q5","format":"csv","model":"gpt-5-nano","expected":"no","actual":"No","isCorrect":true,"inputTokens":2395,"outputTokens":263,"latencyMs":4431.87125},{"questionId":"q5","format":"xml","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":7366,"outputTokens":199,"latencyMs":2658.2782499999994},{"questionId":"q5","format":"yaml","model":"gpt-5-nano","expected":"no","actual":"No","isCorrect":true,"inputTokens":5024,"outputTokens":327,"latencyMs":4898.068541999999},{"questionId":"q6","format":"json-pretty","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":6405,"outputTokens":136,"latencyMs":3223.991291000002},{"questionId":"q6","format":"json-compact","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":3998,"outputTokens":136,"latencyMs":2550.9398330000004},{"questionId":"q6","format":"toon","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2562,"outputTokens":456,"latencyMs":4873.721000000001},{"questionId":"q6","format":"csv","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2396,"outputTokens":136,"latencyMs":2422.785667},{"questionId":"q6","format":"xml","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7367,"outputTokens":72,"latencyMs":2197.1337079999994},{"questionId":"q6","format":"yaml","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5025,"outputTokens":72,"latencyMs":2109.0456250000025},{"questionId":"q7","format":"json-pretty","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":6406,"outputTokens":71,"latencyMs":3442.8210830000025},{"questionId":"q7","format":"json-compact","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":3999,"outputTokens":135,"latencyMs":2297.883417000001},{"questionId":"q7","format":"toon","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2563,"outputTokens":135,"latencyMs":2089.317750000002},{"questionId":"q7","format":"csv","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2397,"outputTokens":71,"latencyMs":2221.912333},{"questionId":"q7","format":"xml","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7368,"outputTokens":135,"latencyMs":2788.8089580000014},{"questionId":"q7","format":"yaml","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5026,"outputTokens":135,"latencyMs":2084.1717910000007},{"questionId":"q8","format":"json-pretty","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":6407,"outputTokens":77,"latencyMs":3207.1332079999993},{"questionId":"q8","format":"json-compact","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4000,"outputTokens":333,"latencyMs":3978.2705830000014},{"questionId":"q8","format":"toon","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2564,"outputTokens":141,"latencyMs":2483.454833},{"questionId":"q8","format":"csv","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2398,"outputTokens":205,"latencyMs":3026.1904160000013},{"questionId":"q8","format":"xml","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7369,"outputTokens":141,"latencyMs":2303.2678330000017},{"questionId":"q8","format":"yaml","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5027,"outputTokens":205,"latencyMs":2111.9866250000014},{"questionId":"q9","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":6409,"outputTokens":391,"latencyMs":3880.3459999999977},{"questionId":"q9","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":4002,"outputTokens":1031,"latencyMs":10601.403875000004},{"questionId":"q9","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":2566,"outputTokens":135,"latencyMs":3648.207042000002},{"questionId":"q9","format":"csv","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":2400,"outputTokens":199,"latencyMs":3941.768667000004},{"questionId":"q9","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":7371,"outputTokens":199,"latencyMs":3288.114999999998},{"questionId":"q9","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5029,"outputTokens":6343,"latencyMs":44351.5315},{"questionId":"q10","format":"json-pretty","model":"gpt-5-nano","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":6406,"outputTokens":263,"latencyMs":3694.0153750000027},{"questionId":"q10","format":"json-compact","model":"gpt-5-nano","expected":"yes","actual":"true","isCorrect":true,"inputTokens":3999,"outputTokens":199,"latencyMs":2959.9164579999997},{"questionId":"q10","format":"toon","model":"gpt-5-nano","expected":"yes","actual":"true","isCorrect":true,"inputTokens":2563,"outputTokens":327,"latencyMs":4157.200499999992},{"questionId":"q10","format":"csv","model":"gpt-5-nano","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":2397,"outputTokens":199,"latencyMs":2684.010792000001},{"questionId":"q10","format":"xml","model":"gpt-5-nano","expected":"yes","actual":"true","isCorrect":true,"inputTokens":7368,"outputTokens":327,"latencyMs":6843.256250000006},{"questionId":"q10","format":"yaml","model":"gpt-5-nano","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":5026,"outputTokens":135,"latencyMs":2646.607624999997},{"questionId":"q11","format":"json-pretty","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":6406,"outputTokens":136,"latencyMs":2040.7884160000103},{"questionId":"q11","format":"json-compact","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":3999,"outputTokens":200,"latencyMs":3072.7327499999956},{"questionId":"q11","format":"toon","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2563,"outputTokens":136,"latencyMs":2512.4735840000067},{"questionId":"q11","format":"csv","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2397,"outputTokens":136,"latencyMs":2537.215207999994},{"questionId":"q11","format":"xml","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7368,"outputTokens":200,"latencyMs":3484.028041999991},{"questionId":"q11","format":"yaml","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5026,"outputTokens":136,"latencyMs":2740.545417000001},{"questionId":"q12","format":"json-pretty","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":6404,"outputTokens":519,"latencyMs":5067.75499999999},{"questionId":"q12","format":"json-compact","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":3997,"outputTokens":135,"latencyMs":2207.2960420000018},{"questionId":"q12","format":"toon","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2561,"outputTokens":71,"latencyMs":2023.4182909999945},{"questionId":"q12","format":"csv","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2395,"outputTokens":71,"latencyMs":1965.9474999999948},{"questionId":"q12","format":"xml","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7366,"outputTokens":327,"latencyMs":5445.7322910000075},{"questionId":"q12","format":"yaml","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5024,"outputTokens":135,"latencyMs":2217.8456250000017},{"questionId":"q13","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6403,"outputTokens":775,"latencyMs":6416.424459000002},{"questionId":"q13","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":3996,"outputTokens":1095,"latencyMs":10216.908708000003},{"questionId":"q13","format":"toon","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2560,"outputTokens":1415,"latencyMs":12012.967833000002},{"questionId":"q13","format":"csv","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2394,"outputTokens":1607,"latencyMs":15004.525874999992},{"questionId":"q13","format":"xml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":7365,"outputTokens":839,"latencyMs":7702.567542000004},{"questionId":"q13","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":5023,"outputTokens":903,"latencyMs":6934.4256250000035},{"questionId":"q14","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6403,"outputTokens":455,"latencyMs":6329.976332999999},{"questionId":"q14","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":3996,"outputTokens":1287,"latencyMs":11756.823124999995},{"questionId":"q14","format":"toon","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2560,"outputTokens":1351,"latencyMs":13607.471833000003},{"questionId":"q14","format":"csv","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2394,"outputTokens":1415,"latencyMs":11347.121041999999},{"questionId":"q14","format":"xml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":7365,"outputTokens":711,"latencyMs":8337.290334000005},{"questionId":"q14","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":5023,"outputTokens":647,"latencyMs":6715.685916999995},{"questionId":"q15","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6403,"outputTokens":711,"latencyMs":6485.847374999998},{"questionId":"q15","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":3996,"outputTokens":455,"latencyMs":4515.228292},{"questionId":"q15","format":"toon","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2560,"outputTokens":1415,"latencyMs":11616.520208000002},{"questionId":"q15","format":"csv","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2394,"outputTokens":1479,"latencyMs":10645.710208000004},{"questionId":"q15","format":"xml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":7365,"outputTokens":839,"latencyMs":6661.52620800001},{"questionId":"q15","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":5023,"outputTokens":583,"latencyMs":8068.320082999999},{"questionId":"q16","format":"json-pretty","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":6408,"outputTokens":2183,"latencyMs":17114.974833},{"questionId":"q16","format":"json-compact","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":4001,"outputTokens":2311,"latencyMs":19111.09525},{"questionId":"q16","format":"toon","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":2565,"outputTokens":2503,"latencyMs":18288.487999999998},{"questionId":"q16","format":"csv","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":2399,"outputTokens":2119,"latencyMs":16220.573124999995},{"questionId":"q16","format":"xml","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":7370,"outputTokens":1799,"latencyMs":12881.310708000005},{"questionId":"q16","format":"yaml","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":5028,"outputTokens":2567,"latencyMs":19749.305250000005},{"questionId":"q17","format":"json-pretty","model":"gpt-5-nano","expected":"67","actual":"67","isCorrect":true,"inputTokens":6408,"outputTokens":4679,"latencyMs":34949.00533299998},{"questionId":"q17","format":"json-compact","model":"gpt-5-nano","expected":"67","actual":"67","isCorrect":true,"inputTokens":4001,"outputTokens":2951,"latencyMs":21987.433625000005},{"questionId":"q17","format":"toon","model":"gpt-5-nano","expected":"67","actual":"66","isCorrect":false,"inputTokens":2565,"outputTokens":2567,"latencyMs":21293.442624999996},{"questionId":"q17","format":"csv","model":"gpt-5-nano","expected":"67","actual":"66","isCorrect":false,"inputTokens":2399,"outputTokens":5063,"latencyMs":44032.956208},{"questionId":"q17","format":"xml","model":"gpt-5-nano","expected":"67","actual":"68","isCorrect":false,"inputTokens":7370,"outputTokens":4231,"latencyMs":40993.50533399999},{"questionId":"q17","format":"yaml","model":"gpt-5-nano","expected":"67","actual":"67","isCorrect":true,"inputTokens":5028,"outputTokens":2119,"latencyMs":17330.996000000014},{"questionId":"q18","format":"json-pretty","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":6408,"outputTokens":1415,"latencyMs":10579.236458},{"questionId":"q18","format":"json-compact","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":4001,"outputTokens":1863,"latencyMs":19204.662540999998},{"questionId":"q18","format":"toon","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":2565,"outputTokens":1735,"latencyMs":15921.706082999997},{"questionId":"q18","format":"csv","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":2399,"outputTokens":1671,"latencyMs":14187.36045800001},{"questionId":"q18","format":"xml","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":7370,"outputTokens":2439,"latencyMs":23366.120749999987},{"questionId":"q18","format":"yaml","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":5028,"outputTokens":1415,"latencyMs":10560.425208000015},{"questionId":"q19","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":6404,"outputTokens":71,"latencyMs":1677.0029579999973},{"questionId":"q19","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":3997,"outputTokens":71,"latencyMs":2073.0098330000183},{"questionId":"q19","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2561,"outputTokens":71,"latencyMs":2217.0232499999984},{"questionId":"q19","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2395,"outputTokens":647,"latencyMs":11864.822375000018},{"questionId":"q19","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":7366,"outputTokens":135,"latencyMs":3301.152207999985},{"questionId":"q19","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":5024,"outputTokens":71,"latencyMs":3555.6330829999933},{"questionId":"q20","format":"json-pretty","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":false,"inputTokens":6405,"outputTokens":4426,"latencyMs":41580.46891700002},{"questionId":"q20","format":"json-compact","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":true,"inputTokens":3998,"outputTokens":5450,"latencyMs":46870.75433300002},{"questionId":"q20","format":"toon","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":false,"inputTokens":2562,"outputTokens":8394,"latencyMs":64883.778458999994},{"questionId":"q20","format":"csv","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":true,"inputTokens":2396,"outputTokens":5002,"latencyMs":48141.74441699998},{"questionId":"q20","format":"xml","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":false,"inputTokens":7367,"outputTokens":11978,"latencyMs":87634.05166699999},{"questionId":"q20","format":"yaml","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":false,"inputTokens":5025,"outputTokens":5258,"latencyMs":91479.50108300001},{"questionId":"q21","format":"json-pretty","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":6402,"outputTokens":2503,"latencyMs":24335.92074999999},{"questionId":"q21","format":"json-compact","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":3995,"outputTokens":839,"latencyMs":11736.013915999996},{"questionId":"q21","format":"toon","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":2559,"outputTokens":1991,"latencyMs":27924.497417000006},{"questionId":"q21","format":"csv","model":"gpt-5-nano","expected":"78","actual":"80","isCorrect":false,"inputTokens":2393,"outputTokens":2183,"latencyMs":24487.570542},{"questionId":"q21","format":"xml","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":7364,"outputTokens":1543,"latencyMs":21492.189084000012},{"questionId":"q21","format":"yaml","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":5022,"outputTokens":1031,"latencyMs":38446.63391599999},{"questionId":"q22","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":6402,"outputTokens":1159,"latencyMs":8782.016457999998},{"questionId":"q22","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3995,"outputTokens":1031,"latencyMs":8675.909083999984},{"questionId":"q22","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2559,"outputTokens":1223,"latencyMs":8873.937125000026},{"questionId":"q22","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2393,"outputTokens":2119,"latencyMs":15471.028375000024},{"questionId":"q22","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":7364,"outputTokens":1159,"latencyMs":11071.565917},{"questionId":"q22","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":5022,"outputTokens":1031,"latencyMs":8543.282542},{"questionId":"q23","format":"json-pretty","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":6410,"outputTokens":1479,"latencyMs":13175.369875000004},{"questionId":"q23","format":"json-compact","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":4003,"outputTokens":1287,"latencyMs":10265.33600000001},{"questionId":"q23","format":"toon","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2567,"outputTokens":1223,"latencyMs":9675.368832999986},{"questionId":"q23","format":"csv","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2401,"outputTokens":1607,"latencyMs":16439.982916999987},{"questionId":"q23","format":"xml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":7372,"outputTokens":1735,"latencyMs":12514.508667000016},{"questionId":"q23","format":"yaml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":5030,"outputTokens":1735,"latencyMs":13288.657208000019},{"questionId":"q24","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"10","isCorrect":false,"inputTokens":6410,"outputTokens":1223,"latencyMs":9795.180999999982},{"questionId":"q24","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4003,"outputTokens":967,"latencyMs":7360.374584000005},{"questionId":"q24","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2567,"outputTokens":967,"latencyMs":9643.991083},{"questionId":"q24","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2401,"outputTokens":1991,"latencyMs":15133.441250000003},{"questionId":"q24","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7372,"outputTokens":1607,"latencyMs":12934.877166999999},{"questionId":"q24","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5030,"outputTokens":903,"latencyMs":7409.917291999998},{"questionId":"q25","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6410,"outputTokens":2631,"latencyMs":21096.591249999998},{"questionId":"q25","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4003,"outputTokens":1095,"latencyMs":10623.016666999989},{"questionId":"q25","format":"toon","model":"gpt-5-nano","expected":"11","actual":"10","isCorrect":false,"inputTokens":2567,"outputTokens":1415,"latencyMs":11573.735833000013},{"questionId":"q25","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2401,"outputTokens":1671,"latencyMs":15761.348625000013},{"questionId":"q25","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7372,"outputTokens":2567,"latencyMs":21791.119957999996},{"questionId":"q25","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5030,"outputTokens":1223,"latencyMs":10159.468874999991},{"questionId":"q26","format":"json-pretty","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":6410,"outputTokens":1863,"latencyMs":14556.631499999989},{"questionId":"q26","format":"json-compact","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":4003,"outputTokens":2439,"latencyMs":19549.012916999985},{"questionId":"q26","format":"toon","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2567,"outputTokens":1479,"latencyMs":10861.368959000014},{"questionId":"q26","format":"csv","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2401,"outputTokens":1223,"latencyMs":12907.081083999976},{"questionId":"q26","format":"xml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":7372,"outputTokens":1479,"latencyMs":11228.751874999987},{"questionId":"q26","format":"yaml","model":"gpt-5-nano","expected":"12","actual":"10","isCorrect":false,"inputTokens":5030,"outputTokens":967,"latencyMs":12392.240124999982},{"questionId":"q27","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6410,"outputTokens":903,"latencyMs":7743.199166000006},{"questionId":"q27","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4003,"outputTokens":967,"latencyMs":7717.007332999987},{"questionId":"q27","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2567,"outputTokens":1543,"latencyMs":13466.096416999993},{"questionId":"q27","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2401,"outputTokens":1671,"latencyMs":19386.823458000028},{"questionId":"q27","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7372,"outputTokens":1351,"latencyMs":9202.514542000019},{"questionId":"q27","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5030,"outputTokens":839,"latencyMs":5914.2648340000305},{"questionId":"q28","format":"json-pretty","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":6409,"outputTokens":3271,"latencyMs":25706.06487500001},{"questionId":"q28","format":"json-compact","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":4002,"outputTokens":2695,"latencyMs":18916.239166999992},{"questionId":"q28","format":"toon","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":2566,"outputTokens":1863,"latencyMs":16402.201292000012},{"questionId":"q28","format":"csv","model":"gpt-5-nano","expected":"63","actual":"64","isCorrect":false,"inputTokens":2400,"outputTokens":3783,"latencyMs":32033.75787500001},{"questionId":"q28","format":"xml","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":7371,"outputTokens":3591,"latencyMs":34038.13491700002},{"questionId":"q28","format":"yaml","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":5029,"outputTokens":3847,"latencyMs":30728.939167000004},{"questionId":"q29","format":"json-pretty","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":6409,"outputTokens":2439,"latencyMs":20353.994166999997},{"questionId":"q29","format":"json-compact","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":4002,"outputTokens":2439,"latencyMs":19757.36262500001},{"questionId":"q29","format":"toon","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":2566,"outputTokens":2887,"latencyMs":25929.855417000013},{"questionId":"q29","format":"csv","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":2400,"outputTokens":4231,"latencyMs":31922.923416999984},{"questionId":"q29","format":"xml","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":7371,"outputTokens":3015,"latencyMs":25425.99424999999},{"questionId":"q29","format":"yaml","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":5029,"outputTokens":3207,"latencyMs":26093.998416999995},{"questionId":"q30","format":"json-pretty","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":6409,"outputTokens":2567,"latencyMs":25022.423124999972},{"questionId":"q30","format":"json-compact","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":4002,"outputTokens":2311,"latencyMs":18758.042874999985},{"questionId":"q30","format":"toon","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":2566,"outputTokens":2247,"latencyMs":18562.77574999997},{"questionId":"q30","format":"csv","model":"gpt-5-nano","expected":"39","actual":"38","isCorrect":false,"inputTokens":2400,"outputTokens":4871,"latencyMs":40234.79562499997},{"questionId":"q30","format":"xml","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":7371,"outputTokens":2375,"latencyMs":19609.509583999985},{"questionId":"q30","format":"yaml","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":5029,"outputTokens":2439,"latencyMs":19463.670209000004},{"questionId":"q31","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6410,"outputTokens":839,"latencyMs":8255.072958000004},{"questionId":"q31","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4003,"outputTokens":1223,"latencyMs":12505.736417000007},{"questionId":"q31","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2567,"outputTokens":1671,"latencyMs":13201.153250000032},{"questionId":"q31","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2401,"outputTokens":1351,"latencyMs":11218.009917000018},{"questionId":"q31","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7372,"outputTokens":1351,"latencyMs":12240.528457999986},{"questionId":"q31","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5030,"outputTokens":2183,"latencyMs":18332.564458000008},{"questionId":"q32","format":"json-pretty","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":6410,"outputTokens":711,"latencyMs":5743.938874999993},{"questionId":"q32","format":"json-compact","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":4003,"outputTokens":1735,"latencyMs":14705.464042000007},{"questionId":"q32","format":"toon","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":2567,"outputTokens":1095,"latencyMs":9447.797707999998},{"questionId":"q32","format":"csv","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":2401,"outputTokens":839,"latencyMs":6942.873625000007},{"questionId":"q32","format":"xml","model":"gpt-5-nano","expected":"8","actual":"7","isCorrect":false,"inputTokens":7372,"outputTokens":2183,"latencyMs":16337.69312499999},{"questionId":"q32","format":"yaml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":5030,"outputTokens":1223,"latencyMs":10260.930583000008},{"questionId":"q33","format":"json-pretty","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":6410,"outputTokens":967,"latencyMs":8287.556291000044},{"questionId":"q33","format":"json-compact","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":4003,"outputTokens":1991,"latencyMs":15832.841291000019},{"questionId":"q33","format":"toon","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":2567,"outputTokens":1927,"latencyMs":14752.899334000016},{"questionId":"q33","format":"csv","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":2401,"outputTokens":1607,"latencyMs":13089.083583},{"questionId":"q33","format":"xml","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":7372,"outputTokens":1351,"latencyMs":12004.315166999993},{"questionId":"q33","format":"yaml","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":5030,"outputTokens":1543,"latencyMs":14155.877500000002},{"questionId":"q34","format":"json-pretty","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":6404,"outputTokens":1415,"latencyMs":14304.43970799999},{"questionId":"q34","format":"json-compact","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":3997,"outputTokens":1159,"latencyMs":10277.40625},{"questionId":"q34","format":"toon","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2561,"outputTokens":2119,"latencyMs":18774.139125000045},{"questionId":"q34","format":"csv","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2395,"outputTokens":2503,"latencyMs":21518.568416999944},{"questionId":"q34","format":"xml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":7366,"outputTokens":1735,"latencyMs":15540.177874999994},{"questionId":"q34","format":"yaml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":5024,"outputTokens":1415,"latencyMs":13529.77120800002},{"questionId":"q35","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6404,"outputTokens":903,"latencyMs":9838.043916999944},{"questionId":"q35","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":3997,"outputTokens":775,"latencyMs":9624.56358300004},{"questionId":"q35","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2561,"outputTokens":1223,"latencyMs":11479.994999999995},{"questionId":"q35","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2395,"outputTokens":2119,"latencyMs":16858.317958},{"questionId":"q35","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7366,"outputTokens":967,"latencyMs":8445.36166600004},{"questionId":"q35","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5024,"outputTokens":839,"latencyMs":7989.79895799997},{"questionId":"q36","format":"json-pretty","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":10760,"outputTokens":137,"latencyMs":2076.940249999985},{"questionId":"q36","format":"json-compact","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":6841,"outputTokens":265,"latencyMs":3102.8519999999553},{"questionId":"q36","format":"toon","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":7278,"outputTokens":201,"latencyMs":3148.431500000006},{"questionId":"q36","format":"xml","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":12068,"outputTokens":201,"latencyMs":2814.5968749999884},{"questionId":"q36","format":"yaml","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8392,"outputTokens":137,"latencyMs":2404.5113330000313},{"questionId":"q37","format":"json-pretty","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":10760,"outputTokens":136,"latencyMs":3983.4646250000224},{"questionId":"q37","format":"json-compact","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":6841,"outputTokens":200,"latencyMs":2714.843957999954},{"questionId":"q37","format":"toon","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":7278,"outputTokens":136,"latencyMs":2891.6611250000424},{"questionId":"q37","format":"xml","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":12068,"outputTokens":200,"latencyMs":2560.106291000033},{"questionId":"q37","format":"yaml","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8392,"outputTokens":200,"latencyMs":2690.9643340000184},{"questionId":"q38","format":"json-pretty","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":10760,"outputTokens":201,"latencyMs":2398.3073330000043},{"questionId":"q38","format":"json-compact","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":6841,"outputTokens":201,"latencyMs":3097.4866250000196},{"questionId":"q38","format":"toon","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":7278,"outputTokens":201,"latencyMs":2677.376667000004},{"questionId":"q38","format":"xml","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":12068,"outputTokens":201,"latencyMs":2613.068500000052},{"questionId":"q38","format":"yaml","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8392,"outputTokens":201,"latencyMs":2712.1262500000303},{"questionId":"q39","format":"json-pretty","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":10760,"outputTokens":327,"latencyMs":3359.9519169999985},{"questionId":"q39","format":"json-compact","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":6841,"outputTokens":135,"latencyMs":2747.6355420000036},{"questionId":"q39","format":"toon","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":7278,"outputTokens":391,"latencyMs":4143.603874999972},{"questionId":"q39","format":"xml","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":12068,"outputTokens":199,"latencyMs":2685.048084000009},{"questionId":"q39","format":"yaml","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8392,"outputTokens":135,"latencyMs":2242.9143330000225},{"questionId":"q40","format":"json-pretty","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":10760,"outputTokens":394,"latencyMs":4438.404832999979},{"questionId":"q40","format":"json-compact","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":6841,"outputTokens":330,"latencyMs":5011.802582999982},{"questionId":"q40","format":"toon","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":7278,"outputTokens":202,"latencyMs":3542.613416999986},{"questionId":"q40","format":"xml","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":12068,"outputTokens":394,"latencyMs":3921.5867089999956},{"questionId":"q40","format":"yaml","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8392,"outputTokens":266,"latencyMs":5912.314874999982},{"questionId":"q41","format":"json-pretty","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":10760,"outputTokens":135,"latencyMs":2155.0715830000117},{"questionId":"q41","format":"json-compact","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":6841,"outputTokens":327,"latencyMs":3821.0887500000536},{"questionId":"q41","format":"toon","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":7278,"outputTokens":263,"latencyMs":3178.6688330000034},{"questionId":"q41","format":"xml","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":12068,"outputTokens":199,"latencyMs":2524.3564170000027},{"questionId":"q41","format":"yaml","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8392,"outputTokens":327,"latencyMs":3552.28125},{"questionId":"q42","format":"json-pretty","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":10760,"outputTokens":266,"latencyMs":3441.550917000044},{"questionId":"q42","format":"json-compact","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":6841,"outputTokens":266,"latencyMs":3261.5270829999936},{"questionId":"q42","format":"toon","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":7278,"outputTokens":202,"latencyMs":2685.566583000007},{"questionId":"q42","format":"xml","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":12068,"outputTokens":394,"latencyMs":4273.437875000003},{"questionId":"q42","format":"yaml","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8392,"outputTokens":266,"latencyMs":2843.469624999969},{"questionId":"q43","format":"json-pretty","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10760,"outputTokens":200,"latencyMs":4721.6026249999995},{"questionId":"q43","format":"json-compact","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6841,"outputTokens":328,"latencyMs":3975.6657499999856},{"questionId":"q43","format":"toon","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7278,"outputTokens":328,"latencyMs":3999.040666999994},{"questionId":"q43","format":"xml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12068,"outputTokens":264,"latencyMs":4652.639332999999},{"questionId":"q43","format":"yaml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8392,"outputTokens":328,"latencyMs":3829.222000000009},{"questionId":"q44","format":"json-pretty","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":10761,"outputTokens":140,"latencyMs":2191.818917000026},{"questionId":"q44","format":"json-compact","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":6842,"outputTokens":140,"latencyMs":2863.233583000023},{"questionId":"q44","format":"toon","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":7279,"outputTokens":140,"latencyMs":1999.2712090000277},{"questionId":"q44","format":"xml","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":12069,"outputTokens":204,"latencyMs":3021.517832999991},{"questionId":"q44","format":"yaml","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8393,"outputTokens":332,"latencyMs":5323.491916000028},{"questionId":"q45","format":"json-pretty","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":10761,"outputTokens":143,"latencyMs":3467.1724169999943},{"questionId":"q45","format":"json-compact","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":6842,"outputTokens":207,"latencyMs":3979.5487079999875},{"questionId":"q45","format":"toon","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":7279,"outputTokens":143,"latencyMs":2075.3569580000476},{"questionId":"q45","format":"xml","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":12069,"outputTokens":271,"latencyMs":3360.761583000014},{"questionId":"q45","format":"yaml","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8393,"outputTokens":79,"latencyMs":3540.7481249999837},{"questionId":"q46","format":"json-pretty","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":10761,"outputTokens":268,"latencyMs":3815.062042000005},{"questionId":"q46","format":"json-compact","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":6842,"outputTokens":396,"latencyMs":4238.7039159999695},{"questionId":"q46","format":"toon","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":7279,"outputTokens":204,"latencyMs":2662.2077500000014},{"questionId":"q46","format":"xml","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":12069,"outputTokens":140,"latencyMs":2843.7720839999965},{"questionId":"q46","format":"yaml","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8393,"outputTokens":204,"latencyMs":4767.953125},{"questionId":"q47","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":10760,"outputTokens":327,"latencyMs":3603.189249999996},{"questionId":"q47","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6841,"outputTokens":199,"latencyMs":2921.2523330000113},{"questionId":"q47","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":7278,"outputTokens":455,"latencyMs":6127.850834000041},{"questionId":"q47","format":"xml","model":"gpt-5-nano","expected":"3","actual":"10","isCorrect":false,"inputTokens":12068,"outputTokens":519,"latencyMs":5192.418166999996},{"questionId":"q47","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":8392,"outputTokens":455,"latencyMs":7608.280457999965},{"questionId":"q48","format":"json-pretty","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":10761,"outputTokens":330,"latencyMs":3521.545542000036},{"questionId":"q48","format":"json-compact","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":6842,"outputTokens":202,"latencyMs":3222.715499999991},{"questionId":"q48","format":"toon","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":7279,"outputTokens":138,"latencyMs":2931.7234590000007},{"questionId":"q48","format":"xml","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":12069,"outputTokens":202,"latencyMs":7230.87591599999},{"questionId":"q48","format":"yaml","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8393,"outputTokens":138,"latencyMs":2280.7286670000176},{"questionId":"q49","format":"json-pretty","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":10761,"outputTokens":141,"latencyMs":2470.5883330000215},{"questionId":"q49","format":"json-compact","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":6842,"outputTokens":205,"latencyMs":2900.343332999968},{"questionId":"q49","format":"toon","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":7279,"outputTokens":205,"latencyMs":2854.8742910000146},{"questionId":"q49","format":"xml","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":12069,"outputTokens":141,"latencyMs":5015.89291699999},{"questionId":"q49","format":"yaml","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8393,"outputTokens":141,"latencyMs":2197.4238340000156},{"questionId":"q50","format":"json-pretty","model":"gpt-5-nano","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":10761,"outputTokens":396,"latencyMs":8121.776167000004},{"questionId":"q50","format":"json-compact","model":"gpt-5-nano","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":6842,"outputTokens":524,"latencyMs":5608.220708999957},{"questionId":"q50","format":"toon","model":"gpt-5-nano","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":7279,"outputTokens":332,"latencyMs":4742.264792000002},{"questionId":"q50","format":"xml","model":"gpt-5-nano","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":12069,"outputTokens":332,"latencyMs":5623.460333000054},{"questionId":"q50","format":"yaml","model":"gpt-5-nano","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":8393,"outputTokens":204,"latencyMs":2709.7400409999536},{"questionId":"q51","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":10760,"outputTokens":327,"latencyMs":3973.2480000000214},{"questionId":"q51","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":6841,"outputTokens":455,"latencyMs":5220.780375000031},{"questionId":"q51","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":7278,"outputTokens":839,"latencyMs":9090.352833999961},{"questionId":"q51","format":"xml","model":"gpt-5-nano","expected":"2","actual":"6","isCorrect":false,"inputTokens":12068,"outputTokens":519,"latencyMs":5274.0245830000495},{"questionId":"q51","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":8392,"outputTokens":519,"latencyMs":5785.010666999966},{"questionId":"q52","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10757,"outputTokens":775,"latencyMs":7403.081957999966},{"questionId":"q52","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6838,"outputTokens":711,"latencyMs":6760.097416999983},{"questionId":"q52","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7275,"outputTokens":775,"latencyMs":7540.421958999999},{"questionId":"q52","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12065,"outputTokens":839,"latencyMs":7469.843333999976},{"questionId":"q52","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8389,"outputTokens":583,"latencyMs":5533.986083000025},{"questionId":"q53","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10757,"outputTokens":711,"latencyMs":8027.375500000024},{"questionId":"q53","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6838,"outputTokens":711,"latencyMs":6540.831958000024},{"questionId":"q53","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7275,"outputTokens":711,"latencyMs":6368.782166999998},{"questionId":"q53","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12065,"outputTokens":711,"latencyMs":6313.597209000029},{"questionId":"q53","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8389,"outputTokens":647,"latencyMs":6299.8524580000085},{"questionId":"q54","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10758,"outputTokens":775,"latencyMs":7374.740040999954},{"questionId":"q54","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6839,"outputTokens":711,"latencyMs":7314.57354100002},{"questionId":"q54","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7276,"outputTokens":519,"latencyMs":6731.960124999983},{"questionId":"q54","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12066,"outputTokens":647,"latencyMs":6681.448958000052},{"questionId":"q54","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8390,"outputTokens":711,"latencyMs":7164.086874999979},{"questionId":"q55","format":"json-pretty","model":"gpt-5-nano","expected":"34904.81","actual":"35004.81","isCorrect":false,"inputTokens":10758,"outputTokens":2442,"latencyMs":18872.10591599997},{"questionId":"q55","format":"json-compact","model":"gpt-5-nano","expected":"34904.81","actual":"34904.81","isCorrect":true,"inputTokens":6839,"outputTokens":4426,"latencyMs":37886.70441600005},{"questionId":"q55","format":"toon","model":"gpt-5-nano","expected":"34904.81","actual":"35004.81","isCorrect":false,"inputTokens":7276,"outputTokens":2890,"latencyMs":25755.817291999992},{"questionId":"q55","format":"xml","model":"gpt-5-nano","expected":"34904.81","actual":"35004.81","isCorrect":false,"inputTokens":12066,"outputTokens":2570,"latencyMs":21423.09054200002},{"questionId":"q55","format":"yaml","model":"gpt-5-nano","expected":"34904.81","actual":"34904.81","isCorrect":true,"inputTokens":8390,"outputTokens":2186,"latencyMs":16161.674042000028},{"questionId":"q56","format":"json-pretty","model":"gpt-5-nano","expected":"698.10","actual":"693.64","isCorrect":false,"inputTokens":10756,"outputTokens":6409,"latencyMs":48809.46754099999},{"questionId":"q56","format":"json-compact","model":"gpt-5-nano","expected":"698.10","actual":"698.10","isCorrect":true,"inputTokens":6837,"outputTokens":5961,"latencyMs":45822.211416999984},{"questionId":"q56","format":"toon","model":"gpt-5-nano","expected":"698.10","actual":"950.05","isCorrect":false,"inputTokens":7274,"outputTokens":2697,"latencyMs":21227.373750000028},{"questionId":"q56","format":"xml","model":"gpt-5-nano","expected":"698.10","actual":"698.10","isCorrect":true,"inputTokens":12064,"outputTokens":3721,"latencyMs":26520.034125000006},{"questionId":"q56","format":"yaml","model":"gpt-5-nano","expected":"698.10","actual":"693.64","isCorrect":false,"inputTokens":8388,"outputTokens":6217,"latencyMs":54472.098708000034},{"questionId":"q57","format":"json-pretty","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":10757,"outputTokens":199,"latencyMs":3240.0961670000106},{"questionId":"q57","format":"json-compact","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":6838,"outputTokens":391,"latencyMs":6134.471666999976},{"questionId":"q57","format":"toon","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":7275,"outputTokens":135,"latencyMs":2585.341582999972},{"questionId":"q57","format":"xml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":12065,"outputTokens":327,"latencyMs":3617.489916999999},{"questionId":"q57","format":"yaml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":8389,"outputTokens":199,"latencyMs":2682.431666999997},{"questionId":"q58","format":"json-pretty","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":10756,"outputTokens":842,"latencyMs":7093.496583},{"questionId":"q58","format":"json-compact","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":6837,"outputTokens":842,"latencyMs":7421.49883300002},{"questionId":"q58","format":"toon","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":7274,"outputTokens":842,"latencyMs":7471.532540999993},{"questionId":"q58","format":"xml","model":"gpt-5-nano","expected":"2152.82","actual":"1822.85","isCorrect":false,"inputTokens":12064,"outputTokens":842,"latencyMs":6895.495374999999},{"questionId":"q58","format":"yaml","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8388,"outputTokens":650,"latencyMs":6038.821250000037},{"questionId":"q59","format":"json-pretty","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":10760,"outputTokens":1415,"latencyMs":11667.665457999974},{"questionId":"q59","format":"json-compact","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":6841,"outputTokens":1863,"latencyMs":16860.552292000037},{"questionId":"q59","format":"toon","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":7278,"outputTokens":1479,"latencyMs":13681.357707999996},{"questionId":"q59","format":"xml","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":12068,"outputTokens":1671,"latencyMs":13620.787917000009},{"questionId":"q59","format":"yaml","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":8392,"outputTokens":1479,"latencyMs":12091.125792000035},{"questionId":"q60","format":"json-pretty","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":10760,"outputTokens":1415,"latencyMs":11046.142666},{"questionId":"q60","format":"json-compact","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":6841,"outputTokens":1607,"latencyMs":14428.138582999993},{"questionId":"q60","format":"toon","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":7278,"outputTokens":1735,"latencyMs":14634.961167000001},{"questionId":"q60","format":"xml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":12068,"outputTokens":1543,"latencyMs":14096.25225000002},{"questionId":"q60","format":"yaml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":8392,"outputTokens":1671,"latencyMs":18844.023499999952},{"questionId":"q61","format":"json-pretty","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":10760,"outputTokens":1223,"latencyMs":13014.50229199999},{"questionId":"q61","format":"json-compact","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":6841,"outputTokens":1415,"latencyMs":10900.717040999967},{"questionId":"q61","format":"toon","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":7278,"outputTokens":1927,"latencyMs":13485.596332999994},{"questionId":"q61","format":"xml","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":12068,"outputTokens":1415,"latencyMs":14228.706707999983},{"questionId":"q61","format":"yaml","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":8392,"outputTokens":1991,"latencyMs":15106.401208999974},{"questionId":"q62","format":"json-pretty","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":10764,"outputTokens":1543,"latencyMs":14486.385042000038},{"questionId":"q62","format":"json-compact","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":6845,"outputTokens":1031,"latencyMs":12831.842833000002},{"questionId":"q62","format":"toon","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":7282,"outputTokens":1031,"latencyMs":11625.482334},{"questionId":"q62","format":"xml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":12072,"outputTokens":967,"latencyMs":7937.718709000037},{"questionId":"q62","format":"yaml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":8396,"outputTokens":775,"latencyMs":6501.633499999996},{"questionId":"q63","format":"json-pretty","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":10764,"outputTokens":967,"latencyMs":9524.465083999967},{"questionId":"q63","format":"json-compact","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":6845,"outputTokens":1159,"latencyMs":11175.71074999997},{"questionId":"q63","format":"toon","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":7282,"outputTokens":903,"latencyMs":8199.182500000054},{"questionId":"q63","format":"xml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":12072,"outputTokens":1223,"latencyMs":13390.794166999985},{"questionId":"q63","format":"yaml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":8396,"outputTokens":967,"latencyMs":12348.245707999973},{"questionId":"q64","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10765,"outputTokens":1095,"latencyMs":10116.773791000014},{"questionId":"q64","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6846,"outputTokens":967,"latencyMs":13025.872541000019},{"questionId":"q64","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7283,"outputTokens":647,"latencyMs":9304.396834000014},{"questionId":"q64","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12073,"outputTokens":1479,"latencyMs":14533.240667000064},{"questionId":"q64","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8397,"outputTokens":711,"latencyMs":6816.599000000046},{"questionId":"q65","format":"json-pretty","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":10765,"outputTokens":1287,"latencyMs":11121.241000000038},{"questionId":"q65","format":"json-compact","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":6846,"outputTokens":967,"latencyMs":7971.9096250000875},{"questionId":"q65","format":"toon","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":7283,"outputTokens":1223,"latencyMs":13170.786708},{"questionId":"q65","format":"xml","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":12073,"outputTokens":1351,"latencyMs":11287.08370800002},{"questionId":"q65","format":"yaml","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":8397,"outputTokens":1671,"latencyMs":13525.659250000026},{"questionId":"q66","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":10764,"outputTokens":1095,"latencyMs":10712.591958999983},{"questionId":"q66","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6845,"outputTokens":967,"latencyMs":9069.787000000011},{"questionId":"q66","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":7282,"outputTokens":1031,"latencyMs":7882.628000000026},{"questionId":"q66","format":"xml","model":"gpt-5-nano","expected":"3","actual":"9","isCorrect":false,"inputTokens":12072,"outputTokens":2247,"latencyMs":18683.20766700001},{"questionId":"q66","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":8396,"outputTokens":1031,"latencyMs":8478.459499999997},{"questionId":"q67","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":10764,"outputTokens":1159,"latencyMs":9306.706791999983},{"questionId":"q67","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6845,"outputTokens":1287,"latencyMs":11594.567415999947},{"questionId":"q67","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":7282,"outputTokens":1031,"latencyMs":9524.092375000007},{"questionId":"q67","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":12072,"outputTokens":2759,"latencyMs":26369.26116600004},{"questionId":"q67","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":8396,"outputTokens":775,"latencyMs":9748.013957999996},{"questionId":"q68","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":10765,"outputTokens":1095,"latencyMs":39946.88145800005},{"questionId":"q68","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":6846,"outputTokens":903,"latencyMs":9888.969541999977},{"questionId":"q68","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":7283,"outputTokens":1095,"latencyMs":12640.796208000043},{"questionId":"q68","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":12073,"outputTokens":1351,"latencyMs":11214.770499999984},{"questionId":"q68","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":8397,"outputTokens":903,"latencyMs":9450.747916999971},{"questionId":"q69","format":"json-pretty","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":10766,"outputTokens":2503,"latencyMs":19977.371999999974},{"questionId":"q69","format":"json-compact","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":6847,"outputTokens":2375,"latencyMs":18024.229249999975},{"questionId":"q69","format":"toon","model":"gpt-5-nano","expected":"20","actual":"40","isCorrect":false,"inputTokens":7284,"outputTokens":7495,"latencyMs":58220.70745899994},{"questionId":"q69","format":"xml","model":"gpt-5-nano","expected":"20","actual":"38","isCorrect":false,"inputTokens":12074,"outputTokens":4679,"latencyMs":37425.450541},{"questionId":"q69","format":"yaml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":8398,"outputTokens":2247,"latencyMs":17404.883417000063},{"questionId":"q70","format":"json-pretty","model":"gpt-5-nano","expected":"19","actual":"19","isCorrect":true,"inputTokens":10766,"outputTokens":2311,"latencyMs":16292.80449999997},{"questionId":"q70","format":"json-compact","model":"gpt-5-nano","expected":"19","actual":"20","isCorrect":false,"inputTokens":6847,"outputTokens":2119,"latencyMs":15755.76154200011},{"questionId":"q70","format":"toon","model":"gpt-5-nano","expected":"19","actual":"19","isCorrect":true,"inputTokens":7284,"outputTokens":2823,"latencyMs":21742.35158399993},{"questionId":"q70","format":"xml","model":"gpt-5-nano","expected":"19","actual":"30","isCorrect":false,"inputTokens":12074,"outputTokens":3207,"latencyMs":24869.222708999994},{"questionId":"q70","format":"yaml","model":"gpt-5-nano","expected":"19","actual":"18","isCorrect":false,"inputTokens":8398,"outputTokens":1671,"latencyMs":12834.909708000021},{"questionId":"q71","format":"json-pretty","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3725,"outputTokens":72,"latencyMs":1824.3022920000367},{"questionId":"q71","format":"json-compact","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2398,"outputTokens":72,"latencyMs":1991.3117499999935},{"questionId":"q71","format":"toon","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1596,"outputTokens":136,"latencyMs":3694.33816699998},{"questionId":"q71","format":"csv","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1454,"outputTokens":136,"latencyMs":2339.3556249999674},{"questionId":"q71","format":"xml","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4431,"outputTokens":136,"latencyMs":2338.732917000074},{"questionId":"q71","format":"yaml","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2996,"outputTokens":136,"latencyMs":2367.6684170000954},{"questionId":"q72","format":"json-pretty","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3725,"outputTokens":202,"latencyMs":2570.8423750000075},{"questionId":"q72","format":"json-compact","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2398,"outputTokens":138,"latencyMs":6741.431541999918},{"questionId":"q72","format":"toon","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1596,"outputTokens":138,"latencyMs":2214.866916000028},{"questionId":"q72","format":"csv","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1454,"outputTokens":74,"latencyMs":2041.7232079999521},{"questionId":"q72","format":"xml","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4431,"outputTokens":138,"latencyMs":2356.0058329999447},{"questionId":"q72","format":"yaml","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2996,"outputTokens":138,"latencyMs":1795.673124999972},{"questionId":"q73","format":"json-pretty","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3726,"outputTokens":265,"latencyMs":4947.881333000027},{"questionId":"q73","format":"json-compact","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2399,"outputTokens":265,"latencyMs":2833.8702500000363},{"questionId":"q73","format":"toon","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1597,"outputTokens":201,"latencyMs":3614.8626249999506},{"questionId":"q73","format":"csv","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1455,"outputTokens":201,"latencyMs":2361.6649169999873},{"questionId":"q73","format":"xml","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4432,"outputTokens":137,"latencyMs":2093.6825420000823},{"questionId":"q73","format":"yaml","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2997,"outputTokens":137,"latencyMs":4162.406042000046},{"questionId":"q74","format":"json-pretty","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":3726,"outputTokens":135,"latencyMs":2269.4359159999294},{"questionId":"q74","format":"json-compact","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":2399,"outputTokens":199,"latencyMs":2964.297833999968},{"questionId":"q74","format":"toon","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1597,"outputTokens":391,"latencyMs":4793.913707999978},{"questionId":"q74","format":"csv","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1455,"outputTokens":71,"latencyMs":1709.6704169999575},{"questionId":"q74","format":"xml","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":4432,"outputTokens":199,"latencyMs":2772.8872919999994},{"questionId":"q74","format":"yaml","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":2997,"outputTokens":135,"latencyMs":3211.8954580000136},{"questionId":"q75","format":"json-pretty","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3725,"outputTokens":200,"latencyMs":2706.251292000059},{"questionId":"q75","format":"json-compact","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2398,"outputTokens":264,"latencyMs":4304.166708000004},{"questionId":"q75","format":"toon","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1596,"outputTokens":1672,"latencyMs":15282.85487499996},{"questionId":"q75","format":"csv","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1454,"outputTokens":136,"latencyMs":4779.248625000007},{"questionId":"q75","format":"xml","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4431,"outputTokens":200,"latencyMs":2664.8808749999152},{"questionId":"q75","format":"yaml","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2996,"outputTokens":136,"latencyMs":4078.707541999989},{"questionId":"q76","format":"json-pretty","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3725,"outputTokens":202,"latencyMs":2657.9091670000926},{"questionId":"q76","format":"json-compact","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2398,"outputTokens":202,"latencyMs":2583.576166999992},{"questionId":"q76","format":"toon","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1596,"outputTokens":522,"latencyMs":6120.337832999998},{"questionId":"q76","format":"csv","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1454,"outputTokens":138,"latencyMs":1891.5040410000365},{"questionId":"q76","format":"xml","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4431,"outputTokens":202,"latencyMs":2342.1456659999676},{"questionId":"q76","format":"yaml","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2996,"outputTokens":138,"latencyMs":2111.249959000037},{"questionId":"q77","format":"json-pretty","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3726,"outputTokens":201,"latencyMs":2276.2088330000406},{"questionId":"q77","format":"json-compact","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2399,"outputTokens":201,"latencyMs":2445.9590000000317},{"questionId":"q77","format":"toon","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1597,"outputTokens":265,"latencyMs":3605.441749999998},{"questionId":"q77","format":"csv","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1455,"outputTokens":265,"latencyMs":3432.412791000097},{"questionId":"q77","format":"xml","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4432,"outputTokens":201,"latencyMs":3005.060334000038},{"questionId":"q77","format":"yaml","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2997,"outputTokens":265,"latencyMs":4920.82916600001},{"questionId":"q78","format":"json-pretty","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":3726,"outputTokens":327,"latencyMs":9672.216999999946},{"questionId":"q78","format":"json-compact","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":2399,"outputTokens":199,"latencyMs":2258.2261660000077},{"questionId":"q78","format":"toon","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":1597,"outputTokens":135,"latencyMs":2784.1056249999674},{"questionId":"q78","format":"csv","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":1455,"outputTokens":135,"latencyMs":2266.577583999955},{"questionId":"q78","format":"xml","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":4432,"outputTokens":199,"latencyMs":3289.6401250000345},{"questionId":"q78","format":"yaml","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":2997,"outputTokens":199,"latencyMs":3341.089249999961},{"questionId":"q79","format":"json-pretty","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3725,"outputTokens":72,"latencyMs":2366.2678750000196},{"questionId":"q79","format":"json-compact","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2398,"outputTokens":200,"latencyMs":3525.4627500000643},{"questionId":"q79","format":"toon","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1596,"outputTokens":200,"latencyMs":4579.349582999945},{"questionId":"q79","format":"csv","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1454,"outputTokens":136,"latencyMs":2242.795291999937},{"questionId":"q79","format":"xml","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4431,"outputTokens":136,"latencyMs":1997.3022089999868},{"questionId":"q79","format":"yaml","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2996,"outputTokens":200,"latencyMs":3089.5134580000304},{"questionId":"q80","format":"json-pretty","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":3722,"outputTokens":263,"latencyMs":3487.994750000071},{"questionId":"q80","format":"json-compact","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":2395,"outputTokens":327,"latencyMs":3876.0249590000603},{"questionId":"q80","format":"toon","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1593,"outputTokens":327,"latencyMs":4497.503999999957},{"questionId":"q80","format":"csv","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1451,"outputTokens":327,"latencyMs":3974.2590000000782},{"questionId":"q80","format":"xml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":4428,"outputTokens":775,"latencyMs":9364.97662500001},{"questionId":"q80","format":"yaml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":2993,"outputTokens":839,"latencyMs":6536.485957999947},{"questionId":"q81","format":"json-pretty","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":3723,"outputTokens":2952,"latencyMs":25197.723415999906},{"questionId":"q81","format":"json-compact","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":2396,"outputTokens":2376,"latencyMs":19591.46325000003},{"questionId":"q81","format":"toon","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":1594,"outputTokens":3848,"latencyMs":30049.87841600005},{"questionId":"q81","format":"csv","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":1452,"outputTokens":3592,"latencyMs":32584.832832999993},{"questionId":"q81","format":"xml","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":4429,"outputTokens":2568,"latencyMs":26232.028625000035},{"questionId":"q81","format":"yaml","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":2994,"outputTokens":3784,"latencyMs":31686.203708999907},{"questionId":"q82","format":"json-pretty","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":3723,"outputTokens":2056,"latencyMs":29541.192583000055},{"questionId":"q82","format":"json-compact","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":2396,"outputTokens":2120,"latencyMs":18433.71520799992},{"questionId":"q82","format":"toon","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":1594,"outputTokens":1416,"latencyMs":16568.16829199996},{"questionId":"q82","format":"csv","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":1452,"outputTokens":2120,"latencyMs":18401.384500000044},{"questionId":"q82","format":"xml","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":4429,"outputTokens":2120,"latencyMs":71863.88837499998},{"questionId":"q82","format":"yaml","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":2994,"outputTokens":2248,"latencyMs":22854.19095900003},{"questionId":"q83","format":"json-pretty","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":3721,"outputTokens":3530,"latencyMs":29241.990583000006},{"questionId":"q83","format":"json-compact","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":2394,"outputTokens":3594,"latencyMs":41383.51104199991},{"questionId":"q83","format":"toon","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":1592,"outputTokens":3786,"latencyMs":30739.118541999953},{"questionId":"q83","format":"csv","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":1450,"outputTokens":5514,"latencyMs":47904.817541000084},{"questionId":"q83","format":"xml","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":4427,"outputTokens":2762,"latencyMs":26283.145374999964},{"questionId":"q83","format":"yaml","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":2992,"outputTokens":4042,"latencyMs":29921.262624999974},{"questionId":"q84","format":"json-pretty","model":"gpt-5-nano","expected":"0.53","actual":"0.5278333333","isCorrect":true,"inputTokens":3719,"outputTokens":2636,"latencyMs":19491.296667000046},{"questionId":"q84","format":"json-compact","model":"gpt-5-nano","expected":"0.53","actual":"0.527833","isCorrect":true,"inputTokens":2392,"outputTokens":3850,"latencyMs":27670.99208300002},{"questionId":"q84","format":"toon","model":"gpt-5-nano","expected":"0.53","actual":"0.5278333333","isCorrect":true,"inputTokens":1590,"outputTokens":5260,"latencyMs":36886.95470800006},{"questionId":"q84","format":"csv","model":"gpt-5-nano","expected":"0.53","actual":"0.5278","isCorrect":true,"inputTokens":1448,"outputTokens":3914,"latencyMs":32095.964000000036},{"questionId":"q84","format":"xml","model":"gpt-5-nano","expected":"0.53","actual":"0.5303","isCorrect":true,"inputTokens":4425,"outputTokens":2826,"latencyMs":21857.65495799994},{"questionId":"q84","format":"yaml","model":"gpt-5-nano","expected":"0.53","actual":"0.5298333333","isCorrect":true,"inputTokens":2990,"outputTokens":4492,"latencyMs":33101.25937500002},{"questionId":"q85","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3723,"outputTokens":1159,"latencyMs":9321.95083300001},{"questionId":"q85","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2396,"outputTokens":1287,"latencyMs":10380.200499999919},{"questionId":"q85","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1594,"outputTokens":1415,"latencyMs":11589.85175000003},{"questionId":"q85","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1452,"outputTokens":1671,"latencyMs":13053.38820799999},{"questionId":"q85","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4429,"outputTokens":1415,"latencyMs":13855.05920899997},{"questionId":"q85","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2994,"outputTokens":1415,"latencyMs":11732.793959000031},{"questionId":"q86","format":"json-pretty","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":3722,"outputTokens":2055,"latencyMs":15511.885458000004},{"questionId":"q86","format":"json-compact","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":2395,"outputTokens":1735,"latencyMs":12649.70991700003},{"questionId":"q86","format":"toon","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":1593,"outputTokens":2311,"latencyMs":19864.19262499991},{"questionId":"q86","format":"csv","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":1451,"outputTokens":2439,"latencyMs":21360.455709},{"questionId":"q86","format":"xml","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":4428,"outputTokens":1863,"latencyMs":14230.067458999925},{"questionId":"q86","format":"yaml","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":2993,"outputTokens":2119,"latencyMs":17830.336374999955},{"questionId":"q87","format":"json-pretty","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3729,"outputTokens":2055,"latencyMs":15676.003875000053},{"questionId":"q87","format":"json-compact","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":2402,"outputTokens":2247,"latencyMs":18072.957041000016},{"questionId":"q87","format":"toon","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1600,"outputTokens":1735,"latencyMs":17205.39591600001},{"questionId":"q87","format":"csv","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1458,"outputTokens":1671,"latencyMs":15129.71100000001},{"questionId":"q87","format":"xml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":4435,"outputTokens":1863,"latencyMs":16848.065500000026},{"questionId":"q87","format":"yaml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3000,"outputTokens":3783,"latencyMs":28171.87820799998},{"questionId":"q88","format":"json-pretty","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":3729,"outputTokens":1607,"latencyMs":12516.068333000061},{"questionId":"q88","format":"json-compact","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":2402,"outputTokens":1799,"latencyMs":14587.507709000027},{"questionId":"q88","format":"toon","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":1600,"outputTokens":1735,"latencyMs":13464.926124999998},{"questionId":"q88","format":"csv","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":1458,"outputTokens":1415,"latencyMs":10154.312957999995},{"questionId":"q88","format":"xml","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":4435,"outputTokens":1799,"latencyMs":13857.554457999999},{"questionId":"q88","format":"yaml","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":3000,"outputTokens":1991,"latencyMs":15959.20745899994},{"questionId":"q89","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3729,"outputTokens":3015,"latencyMs":21109.16279199999},{"questionId":"q89","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2402,"outputTokens":2375,"latencyMs":17738.52874999994},{"questionId":"q89","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1600,"outputTokens":2695,"latencyMs":24250.144250000012},{"questionId":"q89","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1458,"outputTokens":1863,"latencyMs":14587.208416000009},{"questionId":"q89","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4435,"outputTokens":1863,"latencyMs":15798.752917000093},{"questionId":"q89","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3000,"outputTokens":2119,"latencyMs":15580.995792000089},{"questionId":"q90","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3729,"outputTokens":1799,"latencyMs":15500.211708000046},{"questionId":"q90","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2402,"outputTokens":2247,"latencyMs":16598.138999999966},{"questionId":"q90","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1600,"outputTokens":1671,"latencyMs":15448.291624999954},{"questionId":"q90","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1458,"outputTokens":2759,"latencyMs":22686.199166000006},{"questionId":"q90","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4435,"outputTokens":2247,"latencyMs":14591.05012499995},{"questionId":"q90","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3000,"outputTokens":2119,"latencyMs":15068.102708999999},{"questionId":"q91","format":"json-pretty","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3729,"outputTokens":2055,"latencyMs":14814.978499999968},{"questionId":"q91","format":"json-compact","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":2402,"outputTokens":2631,"latencyMs":18904.37629100005},{"questionId":"q91","format":"toon","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1600,"outputTokens":1991,"latencyMs":13199.51000000001},{"questionId":"q91","format":"csv","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1458,"outputTokens":1863,"latencyMs":13311.805165999918},{"questionId":"q91","format":"xml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":4435,"outputTokens":1991,"latencyMs":14602.34366699995},{"questionId":"q91","format":"yaml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3000,"outputTokens":2183,"latencyMs":15701.061791999964},{"questionId":"q92","format":"json-pretty","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":3728,"outputTokens":1863,"latencyMs":13439.475374999922},{"questionId":"q92","format":"json-compact","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":2401,"outputTokens":1927,"latencyMs":15008.399999999907},{"questionId":"q92","format":"toon","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1599,"outputTokens":3783,"latencyMs":35872.39141700009},{"questionId":"q92","format":"csv","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1457,"outputTokens":2503,"latencyMs":20055.865292000002},{"questionId":"q92","format":"xml","model":"gpt-5-nano","expected":"32","actual":"33","isCorrect":false,"inputTokens":4434,"outputTokens":2503,"latencyMs":17435.123250000062},{"questionId":"q92","format":"yaml","model":"gpt-5-nano","expected":"32","actual":"33","isCorrect":false,"inputTokens":2999,"outputTokens":3079,"latencyMs":22982.65754200006},{"questionId":"q93","format":"json-pretty","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":3728,"outputTokens":1799,"latencyMs":13667.848667000071},{"questionId":"q93","format":"json-compact","model":"gpt-5-nano","expected":"9","actual":"10","isCorrect":false,"inputTokens":2401,"outputTokens":1927,"latencyMs":17325.981458000024},{"questionId":"q93","format":"toon","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":1599,"outputTokens":2887,"latencyMs":20446.213084000046},{"questionId":"q93","format":"csv","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":1457,"outputTokens":1735,"latencyMs":12592.209082999965},{"questionId":"q93","format":"xml","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":4434,"outputTokens":2183,"latencyMs":16554.567291000043},{"questionId":"q93","format":"yaml","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":2999,"outputTokens":1799,"latencyMs":14725.110458000097},{"questionId":"q94","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3731,"outputTokens":2439,"latencyMs":18433.54758299992},{"questionId":"q94","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2404,"outputTokens":2887,"latencyMs":19885.17279099999},{"questionId":"q94","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1602,"outputTokens":2439,"latencyMs":16427.026208999916},{"questionId":"q94","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1460,"outputTokens":2503,"latencyMs":22909.33795800002},{"questionId":"q94","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4437,"outputTokens":3207,"latencyMs":27805.63466699992},{"questionId":"q94","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3002,"outputTokens":3463,"latencyMs":33591.67854200001},{"questionId":"q95","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3731,"outputTokens":3015,"latencyMs":23251.54983299994},{"questionId":"q95","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2404,"outputTokens":2247,"latencyMs":16468.20525},{"questionId":"q95","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1602,"outputTokens":3271,"latencyMs":22610.48450000002},{"questionId":"q95","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1460,"outputTokens":2695,"latencyMs":19205.803833000013},{"questionId":"q95","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4437,"outputTokens":3015,"latencyMs":55259.601417000056},{"questionId":"q95","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3002,"outputTokens":3591,"latencyMs":32831.01295799995},{"questionId":"q96","format":"json-pretty","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":15205,"outputTokens":136,"latencyMs":4415.759916999959},{"questionId":"q96","format":"json-compact","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":11512,"outputTokens":136,"latencyMs":2476.5069170000497},{"questionId":"q96","format":"toon","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8826,"outputTokens":392,"latencyMs":6678.939291999908},{"questionId":"q96","format":"csv","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8574,"outputTokens":264,"latencyMs":3124.76004199998},{"questionId":"q96","format":"xml","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17151,"outputTokens":264,"latencyMs":3757.339666999993},{"questionId":"q96","format":"yaml","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":13187,"outputTokens":136,"latencyMs":2453.827042000019},{"questionId":"q97","format":"json-pretty","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":15207,"outputTokens":264,"latencyMs":3340.344334000023},{"questionId":"q97","format":"json-compact","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":11514,"outputTokens":136,"latencyMs":2731.200542000006},{"questionId":"q97","format":"toon","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8828,"outputTokens":456,"latencyMs":4634.077583000064},{"questionId":"q97","format":"csv","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8576,"outputTokens":264,"latencyMs":3072.655792000005},{"questionId":"q97","format":"xml","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17153,"outputTokens":328,"latencyMs":5187.99537499994},{"questionId":"q97","format":"yaml","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":13189,"outputTokens":264,"latencyMs":2972.668459000066},{"questionId":"q98","format":"json-pretty","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":15202,"outputTokens":392,"latencyMs":6565.118416999932},{"questionId":"q98","format":"json-compact","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":11509,"outputTokens":328,"latencyMs":5349.2032079999335},{"questionId":"q98","format":"toon","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8823,"outputTokens":264,"latencyMs":5895.460208999924},{"questionId":"q98","format":"csv","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8571,"outputTokens":392,"latencyMs":8778.092291999958},{"questionId":"q98","format":"xml","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17148,"outputTokens":328,"latencyMs":3495.3217089999234},{"questionId":"q98","format":"yaml","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":13184,"outputTokens":200,"latencyMs":2354.116207999992},{"questionId":"q99","format":"json-pretty","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":15208,"outputTokens":327,"latencyMs":4937.107541999896},{"questionId":"q99","format":"json-compact","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":11515,"outputTokens":263,"latencyMs":5866.194583999924},{"questionId":"q99","format":"toon","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":8829,"outputTokens":327,"latencyMs":4661.650708000059},{"questionId":"q99","format":"csv","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":8577,"outputTokens":199,"latencyMs":5637.232500000042},{"questionId":"q99","format":"xml","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":17154,"outputTokens":327,"latencyMs":4181.799791000085},{"questionId":"q99","format":"yaml","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":13190,"outputTokens":71,"latencyMs":4502.852916000062},{"questionId":"q100","format":"json-pretty","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":15202,"outputTokens":264,"latencyMs":4264.957042000024},{"questionId":"q100","format":"json-compact","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":11509,"outputTokens":264,"latencyMs":4715.0439169999445},{"questionId":"q100","format":"toon","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8823,"outputTokens":456,"latencyMs":4663.678291999968},{"questionId":"q100","format":"csv","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8571,"outputTokens":456,"latencyMs":4373.569167000009},{"questionId":"q100","format":"xml","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17148,"outputTokens":264,"latencyMs":4518.0309170000255},{"questionId":"q100","format":"yaml","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":13184,"outputTokens":200,"latencyMs":3396.0952080000425},{"questionId":"q101","format":"json-pretty","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":15207,"outputTokens":328,"latencyMs":3352.841791999992},{"questionId":"q101","format":"json-compact","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":11514,"outputTokens":200,"latencyMs":3811.8977919999743},{"questionId":"q101","format":"toon","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8828,"outputTokens":520,"latencyMs":4372.860667000059},{"questionId":"q101","format":"csv","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8576,"outputTokens":392,"latencyMs":8314.033834000002},{"questionId":"q101","format":"xml","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17153,"outputTokens":264,"latencyMs":3775.505833000061},{"questionId":"q101","format":"yaml","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":13189,"outputTokens":200,"latencyMs":3031.6716250000754},{"questionId":"q102","format":"json-pretty","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":15206,"outputTokens":263,"latencyMs":3577.9217920000665},{"questionId":"q102","format":"json-compact","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":11513,"outputTokens":391,"latencyMs":4984.194374999963},{"questionId":"q102","format":"toon","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":8827,"outputTokens":327,"latencyMs":2999.1308750000317},{"questionId":"q102","format":"csv","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":8575,"outputTokens":199,"latencyMs":2731.939916999894},{"questionId":"q102","format":"xml","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":17152,"outputTokens":263,"latencyMs":4291.760083000059},{"questionId":"q102","format":"yaml","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":13188,"outputTokens":199,"latencyMs":2982.4712919999147},{"questionId":"q103","format":"json-pretty","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":15205,"outputTokens":199,"latencyMs":3033.5258750000503},{"questionId":"q103","format":"json-compact","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":11512,"outputTokens":199,"latencyMs":2402.1904580000555},{"questionId":"q103","format":"toon","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":8826,"outputTokens":199,"latencyMs":3819.8529170000693},{"questionId":"q103","format":"csv","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":8574,"outputTokens":199,"latencyMs":5906.104833999998},{"questionId":"q103","format":"xml","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":17151,"outputTokens":327,"latencyMs":6253.140375000075},{"questionId":"q103","format":"yaml","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":13187,"outputTokens":327,"latencyMs":5041.93295799999},{"questionId":"q104","format":"json-pretty","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":15209,"outputTokens":136,"latencyMs":2659.5286250000354},{"questionId":"q104","format":"json-compact","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":11516,"outputTokens":264,"latencyMs":5142.21333300008},{"questionId":"q104","format":"toon","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8830,"outputTokens":1032,"latencyMs":10100.530999999959},{"questionId":"q104","format":"csv","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8578,"outputTokens":456,"latencyMs":4893.221375000081},{"questionId":"q104","format":"xml","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17155,"outputTokens":264,"latencyMs":4294.26020800008},{"questionId":"q104","format":"yaml","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":13191,"outputTokens":264,"latencyMs":4737.258334000013},{"questionId":"q105","format":"json-pretty","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":15205,"outputTokens":264,"latencyMs":3726.951584000024},{"questionId":"q105","format":"json-compact","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":11512,"outputTokens":264,"latencyMs":7198.318916999968},{"questionId":"q105","format":"toon","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8826,"outputTokens":584,"latencyMs":7164.169375000056},{"questionId":"q105","format":"csv","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8574,"outputTokens":264,"latencyMs":3408.346166000003},{"questionId":"q105","format":"xml","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17151,"outputTokens":264,"latencyMs":5034.2518750000745},{"questionId":"q105","format":"yaml","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":13187,"outputTokens":200,"latencyMs":2860.038958000019},{"questionId":"q106","format":"json-pretty","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":15210,"outputTokens":200,"latencyMs":6269.80237499997},{"questionId":"q106","format":"json-compact","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":11517,"outputTokens":264,"latencyMs":3151.269250000012},{"questionId":"q106","format":"toon","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8831,"outputTokens":392,"latencyMs":3999.4535840000026},{"questionId":"q106","format":"csv","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8579,"outputTokens":456,"latencyMs":5135.202249999973},{"questionId":"q106","format":"xml","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17156,"outputTokens":328,"latencyMs":3332.8847920000553},{"questionId":"q106","format":"yaml","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":13192,"outputTokens":392,"latencyMs":5556.021707999986},{"questionId":"q107","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":15201,"outputTokens":2311,"latencyMs":23159.096124999924},{"questionId":"q107","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":11508,"outputTokens":1543,"latencyMs":12534.502666999935},{"questionId":"q107","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8822,"outputTokens":199,"latencyMs":2567.922332999995},{"questionId":"q107","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8570,"outputTokens":1479,"latencyMs":14547.191708000028},{"questionId":"q107","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":17147,"outputTokens":3591,"latencyMs":33813.18545800005},{"questionId":"q107","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"155","isCorrect":false,"inputTokens":13183,"outputTokens":5447,"latencyMs":44699.56258300005},{"questionId":"q108","format":"json-pretty","model":"gpt-5-nano","expected":"15413563","actual":"15413563","isCorrect":true,"inputTokens":15204,"outputTokens":6025,"latencyMs":49491.311875000014},{"questionId":"q108","format":"json-compact","model":"gpt-5-nano","expected":"15413563","actual":"15413563","isCorrect":true,"inputTokens":11511,"outputTokens":5449,"latencyMs":44976.463749999995},{"questionId":"q108","format":"toon","model":"gpt-5-nano","expected":"15413563","actual":"15413563","isCorrect":true,"inputTokens":8825,"outputTokens":13705,"latencyMs":127624.62895799999},{"questionId":"q108","format":"csv","model":"gpt-5-nano","expected":"15413563","actual":"15412563","isCorrect":false,"inputTokens":8573,"outputTokens":5641,"latencyMs":47563.12933399994},{"questionId":"q108","format":"xml","model":"gpt-5-nano","expected":"15413563","actual":"15517376","isCorrect":false,"inputTokens":17150,"outputTokens":9545,"latencyMs":71158.95558299997},{"questionId":"q108","format":"yaml","model":"gpt-5-nano","expected":"15413563","actual":"15413563","isCorrect":true,"inputTokens":13186,"outputTokens":7049,"latencyMs":74691.67350000003},{"questionId":"q109","format":"json-pretty","model":"gpt-5-nano","expected":"2528243","actual":"1521139","isCorrect":false,"inputTokens":15204,"outputTokens":6857,"latencyMs":67510.32391699997},{"questionId":"q109","format":"json-compact","model":"gpt-5-nano","expected":"2528243","actual":"2498824","isCorrect":false,"inputTokens":11511,"outputTokens":4937,"latencyMs":37506.675708999974},{"questionId":"q109","format":"toon","model":"gpt-5-nano","expected":"2528243","actual":"2528243","isCorrect":true,"inputTokens":8825,"outputTokens":12809,"latencyMs":112831.9693750001},{"questionId":"q109","format":"csv","model":"gpt-5-nano","expected":"2528243","actual":"2529243","isCorrect":false,"inputTokens":8573,"outputTokens":12361,"latencyMs":89265.98774999997},{"questionId":"q109","format":"xml","model":"gpt-5-nano","expected":"2528243","actual":"2561476","isCorrect":false,"inputTokens":17150,"outputTokens":8393,"latencyMs":55145.15708399995},{"questionId":"q109","format":"yaml","model":"gpt-5-nano","expected":"2528243","actual":"2528243","isCorrect":true,"inputTokens":13186,"outputTokens":8393,"latencyMs":66740.92758300004},{"questionId":"q110","format":"json-pretty","model":"gpt-5-nano","expected":"154136","actual":"157962.66","isCorrect":false,"inputTokens":15203,"outputTokens":4106,"latencyMs":38557.43400000001},{"questionId":"q110","format":"json-compact","model":"gpt-5-nano","expected":"154136","actual":"154135.63","isCorrect":true,"inputTokens":11510,"outputTokens":5578,"latencyMs":67541.88345900003},{"questionId":"q110","format":"toon","model":"gpt-5-nano","expected":"154136","actual":"154135.63","isCorrect":true,"inputTokens":8824,"outputTokens":6730,"latencyMs":51204.950333000044},{"questionId":"q110","format":"csv","model":"gpt-5-nano","expected":"154136","actual":"154135.63","isCorrect":true,"inputTokens":8572,"outputTokens":5642,"latencyMs":37148.54029199993},{"questionId":"q110","format":"xml","model":"gpt-5-nano","expected":"154136","actual":"223000","isCorrect":false,"inputTokens":17149,"outputTokens":1672,"latencyMs":14603.206790999975},{"questionId":"q110","format":"yaml","model":"gpt-5-nano","expected":"154136","actual":"158305.18","isCorrect":false,"inputTokens":13185,"outputTokens":6730,"latencyMs":51263.04112499999},{"questionId":"q111","format":"json-pretty","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":15205,"outputTokens":28487,"latencyMs":190900.79233299987},{"questionId":"q111","format":"json-compact","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":11512,"outputTokens":2887,"latencyMs":18004.22108300007},{"questionId":"q111","format":"toon","model":"gpt-5-nano","expected":"41","actual":"40","isCorrect":false,"inputTokens":8826,"outputTokens":3911,"latencyMs":36760.86504200008},{"questionId":"q111","format":"csv","model":"gpt-5-nano","expected":"41","actual":"39","isCorrect":false,"inputTokens":8574,"outputTokens":4551,"latencyMs":32632.275999999954},{"questionId":"q111","format":"xml","model":"gpt-5-nano","expected":"41","actual":"43","isCorrect":false,"inputTokens":17151,"outputTokens":5959,"latencyMs":47963.752624999965},{"questionId":"q111","format":"yaml","model":"gpt-5-nano","expected":"41","actual":"39","isCorrect":false,"inputTokens":13187,"outputTokens":4615,"latencyMs":35156.557249999954},{"questionId":"q112","format":"json-pretty","model":"gpt-5-nano","expected":"53","actual":"48","isCorrect":false,"inputTokens":15205,"outputTokens":4999,"latencyMs":37420.13208300003},{"questionId":"q112","format":"json-compact","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":11512,"outputTokens":6599,"latencyMs":50662.197375000105},{"questionId":"q112","format":"toon","model":"gpt-5-nano","expected":"53","actual":"55","isCorrect":false,"inputTokens":8826,"outputTokens":4423,"latencyMs":37099.02266700007},{"questionId":"q112","format":"csv","model":"gpt-5-nano","expected":"53","actual":"56","isCorrect":false,"inputTokens":8574,"outputTokens":3719,"latencyMs":28880.48079199996},{"questionId":"q112","format":"xml","model":"gpt-5-nano","expected":"53","actual":"47","isCorrect":false,"inputTokens":17151,"outputTokens":5127,"latencyMs":38762.62162499991},{"questionId":"q112","format":"yaml","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":13187,"outputTokens":9991,"latencyMs":77426.31925000018},{"questionId":"q113","format":"json-pretty","model":"gpt-5-nano","expected":"77","actual":"77","isCorrect":true,"inputTokens":15204,"outputTokens":5447,"latencyMs":40331.298708999995},{"questionId":"q113","format":"json-compact","model":"gpt-5-nano","expected":"77","actual":"76","isCorrect":false,"inputTokens":11511,"outputTokens":2631,"latencyMs":20413.793082999997},{"questionId":"q113","format":"toon","model":"gpt-5-nano","expected":"77","actual":"77","isCorrect":true,"inputTokens":8825,"outputTokens":2311,"latencyMs":16898.795500000007},{"questionId":"q113","format":"csv","model":"gpt-5-nano","expected":"77","actual":"77","isCorrect":true,"inputTokens":8573,"outputTokens":2887,"latencyMs":21191.05854200013},{"questionId":"q113","format":"xml","model":"gpt-5-nano","expected":"77","actual":"68","isCorrect":false,"inputTokens":17150,"outputTokens":7687,"latencyMs":55218.46995800012},{"questionId":"q113","format":"yaml","model":"gpt-5-nano","expected":"77","actual":"77","isCorrect":true,"inputTokens":13186,"outputTokens":3079,"latencyMs":21970.35683299997},{"questionId":"q114","format":"json-pretty","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":15204,"outputTokens":2311,"latencyMs":16199.219916000031},{"questionId":"q114","format":"json-compact","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":11511,"outputTokens":2055,"latencyMs":14659.400208999868},{"questionId":"q114","format":"toon","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":8825,"outputTokens":4359,"latencyMs":34435.57237499999},{"questionId":"q114","format":"csv","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":8573,"outputTokens":2567,"latencyMs":18210.730375000043},{"questionId":"q114","format":"xml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":17150,"outputTokens":2055,"latencyMs":18385.114875000203},{"questionId":"q114","format":"yaml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":13186,"outputTokens":2311,"latencyMs":20261.37933399994},{"questionId":"q115","format":"json-pretty","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":15204,"outputTokens":1095,"latencyMs":8671.577082999982},{"questionId":"q115","format":"json-compact","model":"gpt-5-nano","expected":"16","actual":"15","isCorrect":false,"inputTokens":11511,"outputTokens":1031,"latencyMs":10162.498167000012},{"questionId":"q115","format":"toon","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":8825,"outputTokens":2823,"latencyMs":24796.8173750001},{"questionId":"q115","format":"csv","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":8573,"outputTokens":2055,"latencyMs":16484.410083999857},{"questionId":"q115","format":"xml","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":17150,"outputTokens":775,"latencyMs":7641.333917000098},{"questionId":"q115","format":"yaml","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":13186,"outputTokens":1927,"latencyMs":14260.779457999859},{"questionId":"q116","format":"json-pretty","model":"gpt-5-nano","expected":"49","actual":"43","isCorrect":false,"inputTokens":15204,"outputTokens":2375,"latencyMs":18652.363167000003},{"questionId":"q116","format":"json-compact","model":"gpt-5-nano","expected":"49","actual":"40","isCorrect":false,"inputTokens":11511,"outputTokens":2247,"latencyMs":15727.368292000145},{"questionId":"q116","format":"toon","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":8825,"outputTokens":3463,"latencyMs":24244.907750000013},{"questionId":"q116","format":"csv","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":8573,"outputTokens":4935,"latencyMs":32889.85933299991},{"questionId":"q116","format":"xml","model":"gpt-5-nano","expected":"49","actual":"43","isCorrect":false,"inputTokens":17150,"outputTokens":7431,"latencyMs":49845.45004200004},{"questionId":"q116","format":"yaml","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":13186,"outputTokens":3335,"latencyMs":23436.545666999882},{"questionId":"q117","format":"json-pretty","model":"gpt-5-nano","expected":"23","actual":"23","isCorrect":true,"inputTokens":15204,"outputTokens":5703,"latencyMs":37366.89908399992},{"questionId":"q117","format":"json-compact","model":"gpt-5-nano","expected":"23","actual":"23","isCorrect":true,"inputTokens":11511,"outputTokens":2631,"latencyMs":22536.738458000124},{"questionId":"q117","format":"toon","model":"gpt-5-nano","expected":"23","actual":"23","isCorrect":true,"inputTokens":8825,"outputTokens":6855,"latencyMs":58546.48245800007},{"questionId":"q117","format":"csv","model":"gpt-5-nano","expected":"23","actual":"23","isCorrect":true,"inputTokens":8573,"outputTokens":2759,"latencyMs":20400.750625000102},{"questionId":"q117","format":"xml","model":"gpt-5-nano","expected":"23","actual":"22","isCorrect":false,"inputTokens":17150,"outputTokens":8007,"latencyMs":67408.01587500004},{"questionId":"q117","format":"yaml","model":"gpt-5-nano","expected":"23","actual":"23","isCorrect":true,"inputTokens":13186,"outputTokens":8007,"latencyMs":56406.308708},{"questionId":"q118","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":15204,"outputTokens":2375,"latencyMs":16841.610582999885},{"questionId":"q118","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":11511,"outputTokens":3015,"latencyMs":22129.66941699991},{"questionId":"q118","format":"toon","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":8825,"outputTokens":5063,"latencyMs":41461.67545800004},{"questionId":"q118","format":"csv","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":8573,"outputTokens":1799,"latencyMs":13731.491125000175},{"questionId":"q118","format":"xml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":17150,"outputTokens":1671,"latencyMs":12081.078541999916},{"questionId":"q118","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":13186,"outputTokens":2311,"latencyMs":17877.154000000097},{"questionId":"q119","format":"json-pretty","model":"gpt-5-nano","expected":"57","actual":"54","isCorrect":false,"inputTokens":15211,"outputTokens":3463,"latencyMs":30047.93187500001},{"questionId":"q119","format":"json-compact","model":"gpt-5-nano","expected":"57","actual":"54","isCorrect":false,"inputTokens":11518,"outputTokens":4103,"latencyMs":45088.04829100007},{"questionId":"q119","format":"toon","model":"gpt-5-nano","expected":"57","actual":"57","isCorrect":true,"inputTokens":8832,"outputTokens":7047,"latencyMs":50745.696167000104},{"questionId":"q119","format":"csv","model":"gpt-5-nano","expected":"57","actual":"57","isCorrect":true,"inputTokens":8580,"outputTokens":5703,"latencyMs":40459.02520899987},{"questionId":"q119","format":"xml","model":"gpt-5-nano","expected":"57","actual":"84","isCorrect":false,"inputTokens":17157,"outputTokens":16455,"latencyMs":125781.46854200005},{"questionId":"q119","format":"yaml","model":"gpt-5-nano","expected":"57","actual":"57","isCorrect":true,"inputTokens":13193,"outputTokens":6663,"latencyMs":102326.54787499993},{"questionId":"q120","format":"json-pretty","model":"gpt-5-nano","expected":"43","actual":"31","isCorrect":false,"inputTokens":15211,"outputTokens":3271,"latencyMs":25686.286792000057},{"questionId":"q120","format":"json-compact","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":11518,"outputTokens":6279,"latencyMs":52420.123124999925},{"questionId":"q120","format":"toon","model":"gpt-5-nano","expected":"43","actual":"42","isCorrect":false,"inputTokens":8832,"outputTokens":8327,"latencyMs":65663.83879199997},{"questionId":"q120","format":"csv","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":8580,"outputTokens":3719,"latencyMs":26515.606124999933},{"questionId":"q120","format":"xml","model":"gpt-5-nano","expected":"43","actual":"30","isCorrect":false,"inputTokens":17157,"outputTokens":3143,"latencyMs":32663.77841699985},{"questionId":"q120","format":"yaml","model":"gpt-5-nano","expected":"43","actual":"49","isCorrect":false,"inputTokens":13193,"outputTokens":5447,"latencyMs":41647.481875000056},{"questionId":"q121","format":"json-pretty","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":15211,"outputTokens":2503,"latencyMs":19866.231833999977},{"questionId":"q121","format":"json-compact","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":11518,"outputTokens":2567,"latencyMs":20572.84479100001},{"questionId":"q121","format":"toon","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":8832,"outputTokens":7687,"latencyMs":59710.30420799996},{"questionId":"q121","format":"csv","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":8580,"outputTokens":3271,"latencyMs":25443.094375000102},{"questionId":"q121","format":"xml","model":"gpt-5-nano","expected":"25","actual":"24","isCorrect":false,"inputTokens":17157,"outputTokens":3079,"latencyMs":25214.91200000001},{"questionId":"q121","format":"yaml","model":"gpt-5-nano","expected":"25","actual":"17","isCorrect":false,"inputTokens":13193,"outputTokens":5639,"latencyMs":46368.48183299997},{"questionId":"q122","format":"json-pretty","model":"gpt-5-nano","expected":"6","actual":"8","isCorrect":false,"inputTokens":15211,"outputTokens":2439,"latencyMs":20279.95849999995},{"questionId":"q122","format":"json-compact","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":11518,"outputTokens":2823,"latencyMs":21447.17133299983},{"questionId":"q122","format":"toon","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":8832,"outputTokens":3527,"latencyMs":27965.023334000027},{"questionId":"q122","format":"csv","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":8580,"outputTokens":2567,"latencyMs":22863.875459000003},{"questionId":"q122","format":"xml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":17157,"outputTokens":3399,"latencyMs":28244.31625000015},{"questionId":"q122","format":"yaml","model":"gpt-5-nano","expected":"6","actual":"5","isCorrect":false,"inputTokens":13193,"outputTokens":2887,"latencyMs":23516.321749999886},{"questionId":"q123","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":15211,"outputTokens":4295,"latencyMs":35812.47529199999},{"questionId":"q123","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":11518,"outputTokens":2567,"latencyMs":22001.28716599988},{"questionId":"q123","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":8832,"outputTokens":3527,"latencyMs":27949.87525000004},{"questionId":"q123","format":"csv","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":8580,"outputTokens":3591,"latencyMs":30516.59579199995},{"questionId":"q123","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":17157,"outputTokens":3207,"latencyMs":25457.98716599983},{"questionId":"q123","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":13193,"outputTokens":3015,"latencyMs":23336.21820799983},{"questionId":"q124","format":"json-pretty","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":6858,"outputTokens":135,"latencyMs":2642.089500000002},{"questionId":"q124","format":"json-compact","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":4880,"outputTokens":199,"latencyMs":3509.738791999873},{"questionId":"q124","format":"toon","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5859,"outputTokens":135,"latencyMs":2451.722916000057},{"questionId":"q124","format":"xml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":7768,"outputTokens":199,"latencyMs":2887.4899579999037},{"questionId":"q124","format":"yaml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5886,"outputTokens":135,"latencyMs":2344.7639579998795},{"questionId":"q125","format":"json-pretty","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6858,"outputTokens":201,"latencyMs":3325.3229580000043},{"questionId":"q125","format":"json-compact","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4880,"outputTokens":265,"latencyMs":3112.0506659999955},{"questionId":"q125","format":"toon","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5859,"outputTokens":329,"latencyMs":3138.3786250001285},{"questionId":"q125","format":"xml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7768,"outputTokens":201,"latencyMs":2456.365125000011},{"questionId":"q125","format":"yaml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5886,"outputTokens":265,"latencyMs":3176.0160840000026},{"questionId":"q126","format":"json-pretty","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":6859,"outputTokens":455,"latencyMs":4163.004833000014},{"questionId":"q126","format":"json-compact","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":4881,"outputTokens":199,"latencyMs":3717.775041999994},{"questionId":"q126","format":"toon","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":5860,"outputTokens":199,"latencyMs":4775.366875000065},{"questionId":"q126","format":"xml","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":7769,"outputTokens":327,"latencyMs":4004.8282500000205},{"questionId":"q126","format":"yaml","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":5887,"outputTokens":199,"latencyMs":2888.953624999849},{"questionId":"q127","format":"json-pretty","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6859,"outputTokens":392,"latencyMs":4199.671832999913},{"questionId":"q127","format":"json-compact","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":4881,"outputTokens":520,"latencyMs":5690.4224580000155},{"questionId":"q127","format":"toon","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5860,"outputTokens":264,"latencyMs":3060.348708000034},{"questionId":"q127","format":"xml","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7769,"outputTokens":456,"latencyMs":4671.072417000076},{"questionId":"q127","format":"yaml","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5887,"outputTokens":200,"latencyMs":2604.2215829999186},{"questionId":"q128","format":"json-pretty","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":6858,"outputTokens":263,"latencyMs":3508.316957999952},{"questionId":"q128","format":"json-compact","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":4880,"outputTokens":263,"latencyMs":3040.1347499999683},{"questionId":"q128","format":"toon","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5859,"outputTokens":327,"latencyMs":3664.1815829998814},{"questionId":"q128","format":"xml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":7768,"outputTokens":199,"latencyMs":3045.7201249999925},{"questionId":"q128","format":"yaml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5886,"outputTokens":327,"latencyMs":4769.399749999866},{"questionId":"q129","format":"json-pretty","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6858,"outputTokens":137,"latencyMs":2659.686250000028},{"questionId":"q129","format":"json-compact","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":4880,"outputTokens":265,"latencyMs":6073.4702920001},{"questionId":"q129","format":"toon","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5859,"outputTokens":201,"latencyMs":2600.5177920002025},{"questionId":"q129","format":"xml","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7768,"outputTokens":265,"latencyMs":3274.608666999964},{"questionId":"q129","format":"yaml","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5886,"outputTokens":265,"latencyMs":5826.719209000003},{"questionId":"q130","format":"json-pretty","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":6859,"outputTokens":199,"latencyMs":3293.278500000015},{"questionId":"q130","format":"json-compact","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":4881,"outputTokens":263,"latencyMs":4044.819916999899},{"questionId":"q130","format":"toon","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":5860,"outputTokens":199,"latencyMs":4312.394667000044},{"questionId":"q130","format":"xml","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":7769,"outputTokens":391,"latencyMs":4605.550000000047},{"questionId":"q130","format":"yaml","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":5887,"outputTokens":199,"latencyMs":2351.3004580000415},{"questionId":"q131","format":"json-pretty","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":6859,"outputTokens":199,"latencyMs":2810.2617079999764},{"questionId":"q131","format":"json-compact","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":4881,"outputTokens":135,"latencyMs":2526.6270830000285},{"questionId":"q131","format":"toon","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":5860,"outputTokens":199,"latencyMs":2574.3227499998175},{"questionId":"q131","format":"xml","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":7769,"outputTokens":199,"latencyMs":3967.6630420000292},{"questionId":"q131","format":"yaml","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":5887,"outputTokens":199,"latencyMs":4692.83683299995},{"questionId":"q132","format":"json-pretty","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":6858,"outputTokens":199,"latencyMs":2618.3168750000186},{"questionId":"q132","format":"json-compact","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":4880,"outputTokens":903,"latencyMs":8403.643415999832},{"questionId":"q132","format":"toon","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5859,"outputTokens":391,"latencyMs":5556.864374999888},{"questionId":"q132","format":"xml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":7768,"outputTokens":327,"latencyMs":4956.439874999924},{"questionId":"q132","format":"yaml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5886,"outputTokens":199,"latencyMs":2702.3142909999005},{"questionId":"q133","format":"json-pretty","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6858,"outputTokens":137,"latencyMs":6340.5018329999875},{"questionId":"q133","format":"json-compact","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4880,"outputTokens":1289,"latencyMs":10921.655666999985},{"questionId":"q133","format":"toon","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5859,"outputTokens":329,"latencyMs":4083.661084000021},{"questionId":"q133","format":"xml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7768,"outputTokens":201,"latencyMs":2712.257832999807},{"questionId":"q133","format":"yaml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5886,"outputTokens":1161,"latencyMs":13487.609124999959},{"questionId":"q134","format":"json-pretty","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":6842,"outputTokens":4295,"latencyMs":31497.821125000017},{"questionId":"q134","format":"json-compact","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":4864,"outputTokens":5063,"latencyMs":39918.98462500004},{"questionId":"q134","format":"toon","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":5843,"outputTokens":263,"latencyMs":3533.640083000064},{"questionId":"q134","format":"xml","model":"gpt-5-nano","expected":"75","actual":"100","isCorrect":false,"inputTokens":7752,"outputTokens":7431,"latencyMs":61702.794250000035},{"questionId":"q134","format":"yaml","model":"gpt-5-nano","expected":"75","actual":"67","isCorrect":false,"inputTokens":5870,"outputTokens":3463,"latencyMs":26411.484875000082},{"questionId":"q135","format":"json-pretty","model":"gpt-5-nano","expected":"2453.41","actual":"2558","isCorrect":false,"inputTokens":6843,"outputTokens":10888,"latencyMs":83042.5446250001},{"questionId":"q135","format":"json-compact","model":"gpt-5-nano","expected":"2453.41","actual":"2453.4133333333","isCorrect":true,"inputTokens":4865,"outputTokens":7629,"latencyMs":55094.018666000105},{"questionId":"q135","format":"toon","model":"gpt-5-nano","expected":"2453.41","actual":"2536.72","isCorrect":false,"inputTokens":5844,"outputTokens":5770,"latencyMs":40701.20837500016},{"questionId":"q135","format":"xml","model":"gpt-5-nano","expected":"2453.41","actual":"2386.03","isCorrect":false,"inputTokens":7753,"outputTokens":4234,"latencyMs":31565.561874999898},{"questionId":"q135","format":"yaml","model":"gpt-5-nano","expected":"2453.41","actual":"2459.97 ms","isCorrect":false,"inputTokens":5871,"outputTokens":3979,"latencyMs":31334.34950000001},{"questionId":"q136","format":"json-pretty","model":"gpt-5-nano","expected":"29","actual":"26","isCorrect":false,"inputTokens":6842,"outputTokens":2695,"latencyMs":21486.44941599993},{"questionId":"q136","format":"json-compact","model":"gpt-5-nano","expected":"29","actual":"23","isCorrect":false,"inputTokens":4864,"outputTokens":2311,"latencyMs":18554.253540999955},{"questionId":"q136","format":"toon","model":"gpt-5-nano","expected":"29","actual":"23","isCorrect":false,"inputTokens":5843,"outputTokens":3591,"latencyMs":26000.71870899992},{"questionId":"q136","format":"xml","model":"gpt-5-nano","expected":"29","actual":"25","isCorrect":false,"inputTokens":7752,"outputTokens":10439,"latencyMs":83326.65795799997},{"questionId":"q136","format":"yaml","model":"gpt-5-nano","expected":"29","actual":"27","isCorrect":false,"inputTokens":5870,"outputTokens":3463,"latencyMs":27636.63966600015},{"questionId":"q137","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6842,"outputTokens":2311,"latencyMs":17983.573333000066},{"questionId":"q137","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":4864,"outputTokens":3463,"latencyMs":39156.05854200013},{"questionId":"q137","format":"toon","model":"gpt-5-nano","expected":"17","actual":"16","isCorrect":false,"inputTokens":5843,"outputTokens":13575,"latencyMs":103096.09795899992},{"questionId":"q137","format":"xml","model":"gpt-5-nano","expected":"17","actual":"14","isCorrect":false,"inputTokens":7752,"outputTokens":8135,"latencyMs":67448.66695800005},{"questionId":"q137","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"16","isCorrect":false,"inputTokens":5870,"outputTokens":12103,"latencyMs":104351.20504099992},{"questionId":"q138","format":"json-pretty","model":"gpt-5-nano","expected":"29","actual":"23","isCorrect":false,"inputTokens":6842,"outputTokens":1863,"latencyMs":15394.159792000195},{"questionId":"q138","format":"json-compact","model":"gpt-5-nano","expected":"29","actual":"28","isCorrect":false,"inputTokens":4864,"outputTokens":10887,"latencyMs":95055.6825420002},{"questionId":"q138","format":"toon","model":"gpt-5-nano","expected":"29","actual":"17","isCorrect":false,"inputTokens":5843,"outputTokens":3015,"latencyMs":22336.973833000055},{"questionId":"q138","format":"xml","model":"gpt-5-nano","expected":"29","actual":"21","isCorrect":false,"inputTokens":7752,"outputTokens":6279,"latencyMs":55273.57554200012},{"questionId":"q138","format":"yaml","model":"gpt-5-nano","expected":"29","actual":"24","isCorrect":false,"inputTokens":5870,"outputTokens":2183,"latencyMs":16989.170292000053},{"questionId":"q139","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"18","isCorrect":false,"inputTokens":6844,"outputTokens":19463,"latencyMs":160960.1156250001},{"questionId":"q139","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4866,"outputTokens":5575,"latencyMs":46369.69162499998},{"questionId":"q139","format":"toon","model":"gpt-5-nano","expected":"11","actual":"9","isCorrect":false,"inputTokens":5845,"outputTokens":4423,"latencyMs":35200.247999999905},{"questionId":"q139","format":"xml","model":"gpt-5-nano","expected":"11","actual":"19","isCorrect":false,"inputTokens":7754,"outputTokens":10695,"latencyMs":84882.94016600004},{"questionId":"q139","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"10","isCorrect":false,"inputTokens":5872,"outputTokens":4487,"latencyMs":35050.97583300015},{"questionId":"q140","format":"json-pretty","model":"gpt-5-nano","expected":"18","actual":"14","isCorrect":false,"inputTokens":6844,"outputTokens":4167,"latencyMs":33162.189375000075},{"questionId":"q140","format":"json-compact","model":"gpt-5-nano","expected":"18","actual":"40","isCorrect":false,"inputTokens":4866,"outputTokens":22215,"latencyMs":203905.71083300002},{"questionId":"q140","format":"toon","model":"gpt-5-nano","expected":"18","actual":"14","isCorrect":false,"inputTokens":5845,"outputTokens":3847,"latencyMs":30513.41970800003},{"questionId":"q140","format":"xml","model":"gpt-5-nano","expected":"18","actual":"16","isCorrect":false,"inputTokens":7754,"outputTokens":7431,"latencyMs":58706.27720899996},{"questionId":"q140","format":"yaml","model":"gpt-5-nano","expected":"18","actual":"24","isCorrect":false,"inputTokens":5872,"outputTokens":11271,"latencyMs":101555.05770900007},{"questionId":"q141","format":"json-pretty","model":"gpt-5-nano","expected":"33","actual":"33","isCorrect":true,"inputTokens":6849,"outputTokens":4295,"latencyMs":36673.49066599994},{"questionId":"q141","format":"json-compact","model":"gpt-5-nano","expected":"33","actual":"33","isCorrect":true,"inputTokens":4871,"outputTokens":5767,"latencyMs":52464.82908300008},{"questionId":"q141","format":"toon","model":"gpt-5-nano","expected":"33","actual":"33","isCorrect":true,"inputTokens":5850,"outputTokens":6087,"latencyMs":56860.8709160001},{"questionId":"q141","format":"xml","model":"gpt-5-nano","expected":"33","actual":"35","isCorrect":false,"inputTokens":7759,"outputTokens":8903,"latencyMs":74215.49379099999},{"questionId":"q141","format":"yaml","model":"gpt-5-nano","expected":"33","actual":"28","isCorrect":false,"inputTokens":5877,"outputTokens":2503,"latencyMs":22895.61141699995},{"questionId":"q142","format":"json-pretty","model":"gpt-5-nano","expected":"42","actual":"39","isCorrect":false,"inputTokens":6847,"outputTokens":2695,"latencyMs":24883.977415999863},{"questionId":"q142","format":"json-compact","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":4869,"outputTokens":3591,"latencyMs":33835.543250000104},{"questionId":"q142","format":"toon","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":5848,"outputTokens":6599,"latencyMs":62772.784166000085},{"questionId":"q142","format":"xml","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":7757,"outputTokens":3783,"latencyMs":125893.00587500003},{"questionId":"q142","format":"yaml","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":5875,"outputTokens":2951,"latencyMs":25861.298999999883},{"questionId":"q143","format":"json-pretty","model":"gpt-5-nano","expected":"25","actual":"21","isCorrect":false,"inputTokens":6843,"outputTokens":6471,"latencyMs":57175.68374999985},{"questionId":"q143","format":"json-compact","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":4865,"outputTokens":15815,"latencyMs":156164.25191699993},{"questionId":"q143","format":"toon","model":"gpt-5-nano","expected":"25","actual":"19","isCorrect":false,"inputTokens":5844,"outputTokens":4935,"latencyMs":48042.519040999934},{"questionId":"q143","format":"xml","model":"gpt-5-nano","expected":"25","actual":"20","isCorrect":false,"inputTokens":7753,"outputTokens":6023,"latencyMs":54635.70745799993},{"questionId":"q143","format":"yaml","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":5871,"outputTokens":4487,"latencyMs":42548.56787499995},{"questionId":"q144","format":"json-pretty","model":"gpt-5-nano","expected":"29","actual":"30","isCorrect":false,"inputTokens":6849,"outputTokens":5703,"latencyMs":47509.304291000124},{"questionId":"q144","format":"json-compact","model":"gpt-5-nano","expected":"29","actual":"29","isCorrect":true,"inputTokens":4871,"outputTokens":15111,"latencyMs":134634.7018749998},{"questionId":"q144","format":"toon","model":"gpt-5-nano","expected":"29","actual":"29","isCorrect":true,"inputTokens":5850,"outputTokens":2887,"latencyMs":29909.168542},{"questionId":"q144","format":"xml","model":"gpt-5-nano","expected":"29","actual":"30","isCorrect":false,"inputTokens":7759,"outputTokens":9287,"latencyMs":96027.13508400018},{"questionId":"q144","format":"yaml","model":"gpt-5-nano","expected":"29","actual":"29","isCorrect":true,"inputTokens":5877,"outputTokens":4359,"latencyMs":46109.76416699984},{"questionId":"q145","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":6849,"outputTokens":5127,"latencyMs":54735.276541},{"questionId":"q145","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":4871,"outputTokens":2631,"latencyMs":34303.451208999846},{"questionId":"q145","format":"toon","model":"gpt-5-nano","expected":"4","actual":"3","isCorrect":false,"inputTokens":5850,"outputTokens":3719,"latencyMs":43253.39220800018},{"questionId":"q145","format":"xml","model":"gpt-5-nano","expected":"4","actual":"3","isCorrect":false,"inputTokens":7759,"outputTokens":2951,"latencyMs":37003.87133300002},{"questionId":"q145","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":5877,"outputTokens":5383,"latencyMs":46064.57499999995},{"questionId":"q146","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":6851,"outputTokens":2759,"latencyMs":24990.20387499989},{"questionId":"q146","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":4873,"outputTokens":3527,"latencyMs":28991.103249999927},{"questionId":"q146","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5852,"outputTokens":6151,"latencyMs":55169.2852080001},{"questionId":"q146","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":7761,"outputTokens":4807,"latencyMs":41365.402750000125},{"questionId":"q146","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5879,"outputTokens":2503,"latencyMs":21423.665666999994},{"questionId":"q147","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":6851,"outputTokens":2311,"latencyMs":20191.79941700003},{"questionId":"q147","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":4873,"outputTokens":5127,"latencyMs":40667.05391700007},{"questionId":"q147","format":"toon","model":"gpt-5-nano","expected":"2","actual":"3","isCorrect":false,"inputTokens":5852,"outputTokens":4103,"latencyMs":35758.653625000035},{"questionId":"q147","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":7761,"outputTokens":5191,"latencyMs":42368.70191699988},{"questionId":"q147","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":5879,"outputTokens":2951,"latencyMs":26119.438083000015},{"questionId":"q148","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6851,"outputTokens":5383,"latencyMs":49430.52000000002},{"questionId":"q148","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":4873,"outputTokens":3975,"latencyMs":33286.24708299991},{"questionId":"q148","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":5852,"outputTokens":4295,"latencyMs":37179.58474999992},{"questionId":"q148","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":7761,"outputTokens":4679,"latencyMs":42021.32975000003},{"questionId":"q148","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":5879,"outputTokens":3911,"latencyMs":32745.71325000003},{"questionId":"q149","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":6849,"outputTokens":2823,"latencyMs":28571.53804200003},{"questionId":"q149","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":4871,"outputTokens":5191,"latencyMs":52367.614333000034},{"questionId":"q149","format":"toon","model":"gpt-5-nano","expected":"4","actual":"3","isCorrect":false,"inputTokens":5850,"outputTokens":3591,"latencyMs":30858.75320900022},{"questionId":"q149","format":"xml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":7759,"outputTokens":8327,"latencyMs":76327.44225000008},{"questionId":"q149","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":5877,"outputTokens":7239,"latencyMs":58981.77649999992},{"questionId":"q150","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":6849,"outputTokens":2759,"latencyMs":21856.85245799995},{"questionId":"q150","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"3","isCorrect":false,"inputTokens":4871,"outputTokens":3463,"latencyMs":27430.045124999946},{"questionId":"q150","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5850,"outputTokens":6471,"latencyMs":54548.6966250001},{"questionId":"q150","format":"xml","model":"gpt-5-nano","expected":"5","actual":"6","isCorrect":false,"inputTokens":7759,"outputTokens":10119,"latencyMs":87576.6973329999},{"questionId":"q150","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5877,"outputTokens":4295,"latencyMs":32472.19966599997},{"questionId":"q151","format":"json-pretty","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":976,"outputTokens":71,"latencyMs":2312.0224580001086},{"questionId":"q151","format":"json-compact","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":618,"outputTokens":135,"latencyMs":2231.8640000000596},{"questionId":"q151","format":"toon","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":709,"outputTokens":71,"latencyMs":2134.7074579999316},{"questionId":"q151","format":"xml","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":1060,"outputTokens":71,"latencyMs":3170.491625000024},{"questionId":"q151","format":"yaml","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":728,"outputTokens":71,"latencyMs":4572.2386660000775},{"questionId":"q152","format":"json-pretty","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":974,"outputTokens":76,"latencyMs":1885.3976249999832},{"questionId":"q152","format":"json-compact","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":616,"outputTokens":140,"latencyMs":2061.0951669998467},{"questionId":"q152","format":"toon","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":707,"outputTokens":140,"latencyMs":2638.9032089998946},{"questionId":"q152","format":"xml","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1058,"outputTokens":76,"latencyMs":2053.443042000057},{"questionId":"q152","format":"yaml","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":726,"outputTokens":140,"latencyMs":2261.1999999999534},{"questionId":"q153","format":"json-pretty","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":974,"outputTokens":72,"latencyMs":3845.7434590000194},{"questionId":"q153","format":"json-compact","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":616,"outputTokens":200,"latencyMs":2642.983917000005},{"questionId":"q153","format":"toon","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":707,"outputTokens":136,"latencyMs":2658.965042000171},{"questionId":"q153","format":"xml","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1058,"outputTokens":72,"latencyMs":2037.6524579999968},{"questionId":"q153","format":"yaml","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":726,"outputTokens":200,"latencyMs":4185.288249999983},{"questionId":"q154","format":"json-pretty","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":976,"outputTokens":71,"latencyMs":1971.2106669999193},{"questionId":"q154","format":"json-compact","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":618,"outputTokens":71,"latencyMs":2029.0975829998497},{"questionId":"q154","format":"toon","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":709,"outputTokens":135,"latencyMs":2696.0341250000056},{"questionId":"q154","format":"xml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":1060,"outputTokens":71,"latencyMs":2149.661042000167},{"questionId":"q154","format":"yaml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":728,"outputTokens":71,"latencyMs":1868.1827089998405},{"questionId":"q155","format":"json-pretty","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":974,"outputTokens":136,"latencyMs":2445.285457999911},{"questionId":"q155","format":"json-compact","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":616,"outputTokens":136,"latencyMs":2233.945000000065},{"questionId":"q155","format":"toon","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":707,"outputTokens":200,"latencyMs":2702.534250000026},{"questionId":"q155","format":"xml","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1058,"outputTokens":136,"latencyMs":2075.463834000053},{"questionId":"q155","format":"yaml","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":726,"outputTokens":72,"latencyMs":5153.698083000025},{"questionId":"q156","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":976,"outputTokens":71,"latencyMs":1629.6089579998516},{"questionId":"q156","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":618,"outputTokens":135,"latencyMs":4391.98208300001},{"questionId":"q156","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":709,"outputTokens":135,"latencyMs":2099.9590000000317},{"questionId":"q156","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1060,"outputTokens":71,"latencyMs":1909.576124999905},{"questionId":"q156","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":728,"outputTokens":135,"latencyMs":2411.902791999979},{"questionId":"q157","format":"json-pretty","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":976,"outputTokens":136,"latencyMs":2482.494249999989},{"questionId":"q157","format":"json-compact","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":618,"outputTokens":72,"latencyMs":1850.1961659998633},{"questionId":"q157","format":"toon","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":709,"outputTokens":200,"latencyMs":2955.7467910000123},{"questionId":"q157","format":"xml","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1060,"outputTokens":200,"latencyMs":2802.7335420001764},{"questionId":"q157","format":"yaml","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":728,"outputTokens":136,"latencyMs":2228.2546250000596},{"questionId":"q158","format":"json-pretty","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":974,"outputTokens":71,"latencyMs":1945.2352499999106},{"questionId":"q158","format":"json-compact","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":616,"outputTokens":71,"latencyMs":2144.7766249999404},{"questionId":"q158","format":"toon","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":707,"outputTokens":199,"latencyMs":2729.1954579998273},{"questionId":"q158","format":"xml","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":1058,"outputTokens":135,"latencyMs":2579.960625000065},{"questionId":"q158","format":"yaml","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":726,"outputTokens":71,"latencyMs":2418.635832999833},{"questionId":"q159","format":"json-pretty","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":975,"outputTokens":136,"latencyMs":3062.0771659999155},{"questionId":"q159","format":"json-compact","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":617,"outputTokens":136,"latencyMs":4291.742499999935},{"questionId":"q159","format":"toon","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":708,"outputTokens":328,"latencyMs":3913.950291000074},{"questionId":"q159","format":"xml","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1059,"outputTokens":136,"latencyMs":2528.3267919998616},{"questionId":"q159","format":"yaml","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":727,"outputTokens":136,"latencyMs":2194.895041999873},{"questionId":"q160","format":"json-pretty","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":976,"outputTokens":139,"latencyMs":3661.4919579999987},{"questionId":"q160","format":"json-compact","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":618,"outputTokens":203,"latencyMs":3087.264417000115},{"questionId":"q160","format":"toon","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":709,"outputTokens":75,"latencyMs":2045.724208000116},{"questionId":"q160","format":"xml","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1060,"outputTokens":75,"latencyMs":2745.066542000044},{"questionId":"q160","format":"yaml","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":728,"outputTokens":843,"latencyMs":8787.615250000032},{"questionId":"q161","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":976,"outputTokens":71,"latencyMs":2095.7312079998665},{"questionId":"q161","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":618,"outputTokens":135,"latencyMs":3192.4400830001105},{"questionId":"q161","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":709,"outputTokens":135,"latencyMs":2619.6019999999553},{"questionId":"q161","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1060,"outputTokens":71,"latencyMs":2165.53766599996},{"questionId":"q161","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":728,"outputTokens":135,"latencyMs":2236.4154999998864},{"questionId":"q162","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":976,"outputTokens":7,"latencyMs":1959.035666000098},{"questionId":"q162","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":618,"outputTokens":135,"latencyMs":3408.7168330000713},{"questionId":"q162","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":709,"outputTokens":71,"latencyMs":7638.709082999965},{"questionId":"q162","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1060,"outputTokens":71,"latencyMs":2790.4492919999175},{"questionId":"q162","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":728,"outputTokens":71,"latencyMs":2064.096915999893},{"questionId":"q163","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":975,"outputTokens":135,"latencyMs":2720.4852919999976},{"questionId":"q163","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":617,"outputTokens":135,"latencyMs":3240.151833000127},{"questionId":"q163","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":708,"outputTokens":199,"latencyMs":4423.658624999924},{"questionId":"q163","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1059,"outputTokens":71,"latencyMs":2253.4459999999963},{"questionId":"q163","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":727,"outputTokens":135,"latencyMs":4213.069040999981},{"questionId":"q164","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":975,"outputTokens":199,"latencyMs":3178.958667000057},{"questionId":"q164","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":617,"outputTokens":775,"latencyMs":8963.453499999829},{"questionId":"q164","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":708,"outputTokens":327,"latencyMs":4819.837041999912},{"questionId":"q164","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1059,"outputTokens":135,"latencyMs":2564.8647080000956},{"questionId":"q164","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":727,"outputTokens":199,"latencyMs":2959.5384170000907},{"questionId":"q165","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":975,"outputTokens":71,"latencyMs":3220.785208000103},{"questionId":"q165","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":617,"outputTokens":199,"latencyMs":2784.003792000003},{"questionId":"q165","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":708,"outputTokens":71,"latencyMs":3810.677124999929},{"questionId":"q165","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1059,"outputTokens":135,"latencyMs":1838.4450000000652},{"questionId":"q165","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":727,"outputTokens":135,"latencyMs":3023.823166999966},{"questionId":"q166","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":979,"outputTokens":71,"latencyMs":4515.81495800009},{"questionId":"q166","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":621,"outputTokens":327,"latencyMs":4036.937458000146},{"questionId":"q166","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":712,"outputTokens":327,"latencyMs":3925.6301249999087},{"questionId":"q166","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1063,"outputTokens":263,"latencyMs":4828.367165999953},{"questionId":"q166","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":731,"outputTokens":135,"latencyMs":2436.752584000118},{"questionId":"q167","format":"json-pretty","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":975,"outputTokens":263,"latencyMs":3149.5658330000006},{"questionId":"q167","format":"json-compact","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":617,"outputTokens":199,"latencyMs":2797.849542000098},{"questionId":"q167","format":"toon","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":708,"outputTokens":263,"latencyMs":3504.1114999998827},{"questionId":"q167","format":"xml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1059,"outputTokens":135,"latencyMs":3626.619750000071},{"questionId":"q167","format":"yaml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":727,"outputTokens":263,"latencyMs":8001.90012500016},{"questionId":"q168","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":977,"outputTokens":71,"latencyMs":1770.4174170000479},{"questionId":"q168","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":619,"outputTokens":647,"latencyMs":6697.601999999955},{"questionId":"q168","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":710,"outputTokens":327,"latencyMs":3620.5571250000503},{"questionId":"q168","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":1061,"outputTokens":135,"latencyMs":2246.236124999821},{"questionId":"q168","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":729,"outputTokens":135,"latencyMs":2095.6032910000067},{"questionId":"q169","format":"json-pretty","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":979,"outputTokens":263,"latencyMs":5218.508042000001},{"questionId":"q169","format":"json-compact","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":621,"outputTokens":391,"latencyMs":4260.963791999966},{"questionId":"q169","format":"toon","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":712,"outputTokens":263,"latencyMs":3208.7124580000527},{"questionId":"q169","format":"xml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":1063,"outputTokens":263,"latencyMs":3158.103209000081},{"questionId":"q169","format":"yaml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":731,"outputTokens":135,"latencyMs":2637.7113749999553},{"questionId":"q170","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":978,"outputTokens":199,"latencyMs":4182.437665999867},{"questionId":"q170","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":620,"outputTokens":263,"latencyMs":3155.0406659999862},{"questionId":"q170","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":711,"outputTokens":391,"latencyMs":5566.256125000073},{"questionId":"q170","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":1062,"outputTokens":327,"latencyMs":3721.184208000079},{"questionId":"q170","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":730,"outputTokens":199,"latencyMs":4757.437958000228},{"questionId":"q171","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":980,"outputTokens":199,"latencyMs":2493.2133329999633},{"questionId":"q171","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":622,"outputTokens":199,"latencyMs":3028.5618330000434},{"questionId":"q171","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":713,"outputTokens":327,"latencyMs":3776.8497919999063},{"questionId":"q171","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1064,"outputTokens":327,"latencyMs":5966.682250000071},{"questionId":"q171","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":732,"outputTokens":135,"latencyMs":2331.9263340001926},{"questionId":"q172","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":980,"outputTokens":135,"latencyMs":2097.668916999828},{"questionId":"q172","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":622,"outputTokens":327,"latencyMs":3725.9484159999993},{"questionId":"q172","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":713,"outputTokens":135,"latencyMs":2032.4292919998989},{"questionId":"q172","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1064,"outputTokens":199,"latencyMs":2836.448790999828},{"questionId":"q172","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":732,"outputTokens":199,"latencyMs":3434.1892089999747},{"questionId":"q173","format":"json-pretty","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":981,"outputTokens":135,"latencyMs":2831.8669160001446},{"questionId":"q173","format":"json-compact","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":623,"outputTokens":391,"latencyMs":6664.1556669999845},{"questionId":"q173","format":"toon","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":714,"outputTokens":263,"latencyMs":3239.020249999827},{"questionId":"q173","format":"xml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1065,"outputTokens":455,"latencyMs":5118.517375000054},{"questionId":"q173","format":"yaml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":733,"outputTokens":327,"latencyMs":5797.415291999932},{"questionId":"q174","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":978,"outputTokens":71,"latencyMs":1952.9836659999564},{"questionId":"q174","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":620,"outputTokens":391,"latencyMs":4394.613874999806},{"questionId":"q174","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":711,"outputTokens":199,"latencyMs":2786.7329579999205},{"questionId":"q174","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1062,"outputTokens":135,"latencyMs":2533.21154199983},{"questionId":"q174","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":730,"outputTokens":71,"latencyMs":1956.7482919998001},{"questionId":"q175","format":"json-pretty","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":982,"outputTokens":327,"latencyMs":3973.723250000039},{"questionId":"q175","format":"json-compact","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":624,"outputTokens":391,"latencyMs":4001.693250000011},{"questionId":"q175","format":"toon","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":715,"outputTokens":391,"latencyMs":4934.054459000006},{"questionId":"q175","format":"xml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1066,"outputTokens":327,"latencyMs":3737.762875000015},{"questionId":"q175","format":"yaml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":734,"outputTokens":263,"latencyMs":3000.23054099991},{"questionId":"q176","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":976,"outputTokens":135,"latencyMs":3078.639083000133},{"questionId":"q176","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":618,"outputTokens":263,"latencyMs":3533.0008750001434},{"questionId":"q176","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":709,"outputTokens":263,"latencyMs":3243.636959000025},{"questionId":"q176","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1060,"outputTokens":263,"latencyMs":6352.7193330000155},{"questionId":"q176","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":728,"outputTokens":135,"latencyMs":4659.21166699985},{"questionId":"q177","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":984,"outputTokens":199,"latencyMs":3261.5107500001322},{"questionId":"q177","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":626,"outputTokens":199,"latencyMs":3467.698916000081},{"questionId":"q177","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":717,"outputTokens":135,"latencyMs":2226.124458000064},{"questionId":"q177","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1068,"outputTokens":263,"latencyMs":3677.065958000021},{"questionId":"q177","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":736,"outputTokens":135,"latencyMs":2577.7424580000807},{"questionId":"q178","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":979,"outputTokens":199,"latencyMs":13625.448916999856},{"questionId":"q178","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":621,"outputTokens":135,"latencyMs":2291.9423330000136},{"questionId":"q178","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":712,"outputTokens":199,"latencyMs":2938.0128330001608},{"questionId":"q178","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1063,"outputTokens":519,"latencyMs":5407.197042000014},{"questionId":"q178","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":731,"outputTokens":135,"latencyMs":2133.6571669999976},{"questionId":"q179","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":978,"outputTokens":199,"latencyMs":2824.474042000016},{"questionId":"q179","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":620,"outputTokens":199,"latencyMs":2885.682041000109},{"questionId":"q179","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":711,"outputTokens":391,"latencyMs":5116.009333000053},{"questionId":"q179","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1062,"outputTokens":263,"latencyMs":3732.189458999783},{"questionId":"q179","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":730,"outputTokens":199,"latencyMs":3656.646790999919},{"questionId":"q180","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":6404,"outputTokens":71,"latencyMs":2266.199166000122},{"questionId":"q180","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":3997,"outputTokens":199,"latencyMs":2985.217999999877},{"questionId":"q180","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2561,"outputTokens":71,"latencyMs":4531.09912499995},{"questionId":"q180","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2395,"outputTokens":327,"latencyMs":7071.067792000016},{"questionId":"q180","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":7366,"outputTokens":135,"latencyMs":2746.68429200002},{"questionId":"q180","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":5024,"outputTokens":71,"latencyMs":2643.376791999908},{"questionId":"q181","format":"json-pretty","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id, name, email, department, salary, yearsExperience, active","isCorrect":true,"inputTokens":6409,"outputTokens":340,"latencyMs":3842.7833749998827},{"questionId":"q181","format":"json-compact","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4002,"outputTokens":402,"latencyMs":4222.851749999914},{"questionId":"q181","format":"toon","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2566,"outputTokens":210,"latencyMs":2795.296166999964},{"questionId":"q181","format":"csv","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2400,"outputTokens":82,"latencyMs":2186.6787089998834},{"questionId":"q181","format":"xml","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id, name, email, department, salary, yearsExperience, active","isCorrect":true,"inputTokens":7371,"outputTokens":276,"latencyMs":3283.7072499999776},{"questionId":"q181","format":"yaml","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id, name, email, department, salary, yearsExperience, active","isCorrect":true,"inputTokens":5029,"outputTokens":340,"latencyMs":3938.074459000025},{"questionId":"q182","format":"json-pretty","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":6407,"outputTokens":263,"latencyMs":3906.2781659998},{"questionId":"q182","format":"json-compact","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":4000,"outputTokens":327,"latencyMs":4669.161083000014},{"questionId":"q182","format":"toon","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":2564,"outputTokens":71,"latencyMs":3148.013666999992},{"questionId":"q182","format":"csv","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":2398,"outputTokens":71,"latencyMs":2121.9527920000255},{"questionId":"q182","format":"xml","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":7369,"outputTokens":263,"latencyMs":4802.542292000027},{"questionId":"q182","format":"yaml","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":5027,"outputTokens":327,"latencyMs":3741.182999999961},{"questionId":"q183","format":"json-pretty","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":6408,"outputTokens":71,"latencyMs":2065.5990830000956},{"questionId":"q183","format":"json-compact","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4001,"outputTokens":71,"latencyMs":1998.4624169999734},{"questionId":"q183","format":"toon","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2565,"outputTokens":71,"latencyMs":1797.8245419999585},{"questionId":"q183","format":"csv","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2399,"outputTokens":199,"latencyMs":2815.201624999987},{"questionId":"q183","format":"xml","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7370,"outputTokens":135,"latencyMs":4208.477582999971},{"questionId":"q183","format":"yaml","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5028,"outputTokens":71,"latencyMs":2256.8574999999255},{"questionId":"q184","format":"json-pretty","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":6408,"outputTokens":204,"latencyMs":3212.476624999894},{"questionId":"q184","format":"json-compact","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4001,"outputTokens":140,"latencyMs":2664.868083999958},{"questionId":"q184","format":"toon","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2565,"outputTokens":140,"latencyMs":5919.902500000084},{"questionId":"q184","format":"csv","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2399,"outputTokens":76,"latencyMs":1970.7766670000274},{"questionId":"q184","format":"xml","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7370,"outputTokens":76,"latencyMs":1976.160666000098},{"questionId":"q184","format":"yaml","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5028,"outputTokens":140,"latencyMs":2378.865332999965},{"questionId":"q185","format":"json-pretty","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":6405,"outputTokens":135,"latencyMs":2776.5269589999225},{"questionId":"q185","format":"json-compact","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":3998,"outputTokens":135,"latencyMs":2518.771375000011},{"questionId":"q185","format":"toon","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":2562,"outputTokens":199,"latencyMs":3686.2868329999037},{"questionId":"q185","format":"csv","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":2396,"outputTokens":967,"latencyMs":9257.057583000045},{"questionId":"q185","format":"xml","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":7367,"outputTokens":135,"latencyMs":4153.484124999959},{"questionId":"q185","format":"yaml","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":5025,"outputTokens":135,"latencyMs":3150.407000000123},{"questionId":"q186","format":"json-pretty","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":10757,"outputTokens":199,"latencyMs":2903.4743750002235},{"questionId":"q186","format":"json-compact","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":6838,"outputTokens":583,"latencyMs":5690.455374999903},{"questionId":"q186","format":"toon","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":7275,"outputTokens":71,"latencyMs":7394.790583000053},{"questionId":"q186","format":"xml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":12065,"outputTokens":199,"latencyMs":11446.779499999946},{"questionId":"q186","format":"yaml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":8389,"outputTokens":199,"latencyMs":2758.905333999777},{"questionId":"q187","format":"json-pretty","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":10764,"outputTokens":599,"latencyMs":5310.473666999955},{"questionId":"q187","format":"json-compact","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":6845,"outputTokens":789,"latencyMs":6727.976667000214},{"questionId":"q187","format":"toon","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":7282,"outputTokens":471,"latencyMs":5490.880708000157},{"questionId":"q187","format":"xml","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":12072,"outputTokens":343,"latencyMs":4612.854832999874},{"questionId":"q187","format":"yaml","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":8396,"outputTokens":791,"latencyMs":7194.165915999794},{"questionId":"q188","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":10760,"outputTokens":327,"latencyMs":3178.9103329998907},{"questionId":"q188","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":6841,"outputTokens":327,"latencyMs":3662.467750000069},{"questionId":"q188","format":"toon","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":7278,"outputTokens":455,"latencyMs":4551.9334169998765},{"questionId":"q188","format":"xml","model":"gpt-5-nano","expected":"4","actual":"10","isCorrect":false,"inputTokens":12068,"outputTokens":519,"latencyMs":5273.39154200023},{"questionId":"q188","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":8392,"outputTokens":327,"latencyMs":4245.433791999938},{"questionId":"q189","format":"json-pretty","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":10765,"outputTokens":268,"latencyMs":4681.627957999939},{"questionId":"q189","format":"json-compact","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":6846,"outputTokens":332,"latencyMs":3301.5220000001136},{"questionId":"q189","format":"toon","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":7283,"outputTokens":140,"latencyMs":4202.679125000024},{"questionId":"q189","format":"xml","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":12073,"outputTokens":204,"latencyMs":2711.6467079999857},{"questionId":"q189","format":"yaml","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku, name, quantity, price","isCorrect":true,"inputTokens":8397,"outputTokens":205,"latencyMs":2543.4837499998976},{"questionId":"q190","format":"json-pretty","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10761,"outputTokens":72,"latencyMs":2149.682041999884},{"questionId":"q190","format":"json-compact","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6842,"outputTokens":200,"latencyMs":2211.3391670000274},{"questionId":"q190","format":"toon","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7279,"outputTokens":136,"latencyMs":2225.490125000011},{"questionId":"q190","format":"xml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12069,"outputTokens":200,"latencyMs":3754.3431250001304},{"questionId":"q190","format":"yaml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8393,"outputTokens":136,"latencyMs":7293.941040999955},{"questionId":"q191","format":"json-pretty","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":10766,"outputTokens":333,"latencyMs":4800.030750000151},{"questionId":"q191","format":"json-compact","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":6847,"outputTokens":331,"latencyMs":5688.367417000001},{"questionId":"q191","format":"toon","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":7284,"outputTokens":205,"latencyMs":3750.244292000076},{"questionId":"q191","format":"xml","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":12074,"outputTokens":205,"latencyMs":3307.1429999999236},{"questionId":"q191","format":"yaml","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id, name, email, phone","isCorrect":true,"inputTokens":8398,"outputTokens":141,"latencyMs":4616.1370419999585},{"questionId":"q192","format":"json-pretty","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":3721,"outputTokens":327,"latencyMs":4998.584792000009},{"questionId":"q192","format":"json-compact","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":2394,"outputTokens":839,"latencyMs":9367.604667000007},{"questionId":"q192","format":"toon","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1592,"outputTokens":391,"latencyMs":4071.9847500000615},{"questionId":"q192","format":"csv","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1450,"outputTokens":455,"latencyMs":6016.693167000078},{"questionId":"q192","format":"xml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":4427,"outputTokens":839,"latencyMs":8399.051541999914},{"questionId":"q192","format":"yaml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":2992,"outputTokens":199,"latencyMs":2688.4976670001633},{"questionId":"q193","format":"json-pretty","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date, views, clicks, conversions, revenue, bounceRate","isCorrect":true,"inputTokens":3725,"outputTokens":274,"latencyMs":4121.197374999989},{"questionId":"q193","format":"json-compact","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2398,"outputTokens":276,"latencyMs":3618.0884169999044},{"questionId":"q193","format":"toon","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1596,"outputTokens":148,"latencyMs":5133.002667000052},{"questionId":"q193","format":"csv","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1454,"outputTokens":148,"latencyMs":2504.0228329999372},{"questionId":"q193","format":"xml","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date, views, clicks, conversions, revenue, bounceRate","isCorrect":true,"inputTokens":4431,"outputTokens":274,"latencyMs":3449.389417000115},{"questionId":"q193","format":"yaml","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2996,"outputTokens":532,"latencyMs":6014.739541999996},{"questionId":"q194","format":"json-pretty","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3724,"outputTokens":136,"latencyMs":3948.2680420000106},{"questionId":"q194","format":"json-compact","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2397,"outputTokens":264,"latencyMs":2758.7947919999715},{"questionId":"q194","format":"toon","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1595,"outputTokens":200,"latencyMs":2665.807042000117},{"questionId":"q194","format":"csv","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1453,"outputTokens":200,"latencyMs":2793.7808749999385},{"questionId":"q194","format":"xml","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":4430,"outputTokens":200,"latencyMs":2658.43429200002},{"questionId":"q194","format":"yaml","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2995,"outputTokens":200,"latencyMs":3203.553791999817},{"questionId":"q195","format":"json-pretty","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3725,"outputTokens":76,"latencyMs":2192.951583000133},{"questionId":"q195","format":"json-compact","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2398,"outputTokens":204,"latencyMs":2600.9522079997696},{"questionId":"q195","format":"toon","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1596,"outputTokens":76,"latencyMs":4160.560415999964},{"questionId":"q195","format":"csv","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1454,"outputTokens":140,"latencyMs":2145.1014999998733},{"questionId":"q195","format":"xml","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4431,"outputTokens":12,"latencyMs":2984.335833000019},{"questionId":"q195","format":"yaml","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2996,"outputTokens":76,"latencyMs":2231.3227499998175},{"questionId":"q196","format":"json-pretty","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":3721,"outputTokens":135,"latencyMs":3135.1317079998553},{"questionId":"q196","format":"json-compact","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":2394,"outputTokens":71,"latencyMs":2038.079417000059},{"questionId":"q196","format":"toon","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":1592,"outputTokens":199,"latencyMs":2405.8865420001093},{"questionId":"q196","format":"csv","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":1450,"outputTokens":71,"latencyMs":2052.968958000187},{"questionId":"q196","format":"xml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":4427,"outputTokens":135,"latencyMs":2958.8703749999404},{"questionId":"q196","format":"yaml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":2992,"outputTokens":135,"latencyMs":2353.1580829999875},{"questionId":"q197","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":15201,"outputTokens":1287,"latencyMs":11019.641290999949},{"questionId":"q197","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":11508,"outputTokens":1287,"latencyMs":10685.38570899982},{"questionId":"q197","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8822,"outputTokens":199,"latencyMs":4465.280124999816},{"questionId":"q197","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8570,"outputTokens":1351,"latencyMs":14746.95849999995},{"questionId":"q197","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":17147,"outputTokens":2375,"latencyMs":21686.653166999808},{"questionId":"q197","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":13183,"outputTokens":3463,"latencyMs":28692.80233399989},{"questionId":"q198","format":"json-pretty","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":15206,"outputTokens":287,"latencyMs":3100.767833000049},{"questionId":"q198","format":"json-compact","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":11513,"outputTokens":351,"latencyMs":3610.665959000122},{"questionId":"q198","format":"toon","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8827,"outputTokens":159,"latencyMs":2709.3721249999944},{"questionId":"q198","format":"csv","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8575,"outputTokens":223,"latencyMs":3709.3974999999627},{"questionId":"q198","format":"xml","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":17152,"outputTokens":415,"latencyMs":4834.6889170000795},{"questionId":"q198","format":"yaml","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id, name, repo, description, createdAt, updatedAt, pushedAt, stars, watchers, forks, defaultBranch","isCorrect":false,"inputTokens":13188,"outputTokens":351,"latencyMs":4287.428334000055},{"questionId":"q199","format":"json-pretty","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":15206,"outputTokens":265,"latencyMs":3169.219291999936},{"questionId":"q199","format":"json-compact","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":11513,"outputTokens":393,"latencyMs":4319.117707999889},{"questionId":"q199","format":"toon","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":8827,"outputTokens":329,"latencyMs":3787.7225410002284},{"questionId":"q199","format":"csv","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":8575,"outputTokens":201,"latencyMs":3099.9900420000777},{"questionId":"q199","format":"xml","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":17152,"outputTokens":329,"latencyMs":3604.399583000224},{"questionId":"q199","format":"yaml","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":13188,"outputTokens":393,"latencyMs":4602.328416999895},{"questionId":"q200","format":"json-pretty","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":15205,"outputTokens":265,"latencyMs":5139.138416999951},{"questionId":"q200","format":"json-compact","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":11512,"outputTokens":329,"latencyMs":4392.627249999903},{"questionId":"q200","format":"toon","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8826,"outputTokens":265,"latencyMs":3076.617999999784},{"questionId":"q200","format":"csv","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8574,"outputTokens":393,"latencyMs":4265.7371660000645},{"questionId":"q200","format":"xml","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17151,"outputTokens":265,"latencyMs":7332.103000000119},{"questionId":"q200","format":"yaml","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":13187,"outputTokens":265,"latencyMs":3677.992083000019},{"questionId":"q201","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":15202,"outputTokens":135,"latencyMs":3846.9981249999255},{"questionId":"q201","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":11509,"outputTokens":327,"latencyMs":3941.1369579997845},{"questionId":"q201","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":8823,"outputTokens":199,"latencyMs":2715.6938750003465},{"questionId":"q201","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":8571,"outputTokens":199,"latencyMs":3929.0370829999447},{"questionId":"q201","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":17148,"outputTokens":135,"latencyMs":2409.6185420001857},{"questionId":"q201","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":13184,"outputTokens":263,"latencyMs":2841.174083000049},{"questionId":"q202","format":"json-pretty","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":6842,"outputTokens":4039,"latencyMs":30658.40908400016},{"questionId":"q202","format":"json-compact","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":4864,"outputTokens":8007,"latencyMs":70745.20808399981},{"questionId":"q202","format":"toon","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":5843,"outputTokens":199,"latencyMs":2693.741791999899},{"questionId":"q202","format":"xml","model":"gpt-5-nano","expected":"75","actual":"100","isCorrect":false,"inputTokens":7752,"outputTokens":5831,"latencyMs":45934.20145899989},{"questionId":"q202","format":"yaml","model":"gpt-5-nano","expected":"75","actual":"49","isCorrect":false,"inputTokens":5870,"outputTokens":6919,"latencyMs":57812.3292080001},{"questionId":"q203","format":"json-pretty","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error","isCorrect":true,"inputTokens":6851,"outputTokens":470,"latencyMs":4051.8215000000782},{"questionId":"q203","format":"json-compact","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":4873,"outputTokens":658,"latencyMs":7203.393791999668},{"questionId":"q203","format":"toon","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error, message, stack, retryable","isCorrect":false,"inputTokens":5852,"outputTokens":1245,"latencyMs":11520.45708299987},{"questionId":"q203","format":"xml","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error, message, stack, retryable","isCorrect":true,"inputTokens":7761,"outputTokens":669,"latencyMs":7284.115042000078},{"questionId":"q203","format":"yaml","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error, message, stack, retryable","isCorrect":true,"inputTokens":5879,"outputTokens":1117,"latencyMs":9623.01495799981},{"questionId":"q204","format":"json-pretty","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":6846,"outputTokens":135,"latencyMs":2629.631916000042},{"questionId":"q204","format":"json-compact","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":4868,"outputTokens":263,"latencyMs":3570.4422909999266},{"questionId":"q204","format":"toon","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5847,"outputTokens":199,"latencyMs":2593.2370839999057},{"questionId":"q204","format":"xml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":7756,"outputTokens":199,"latencyMs":8825.160583000164},{"questionId":"q204","format":"yaml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5874,"outputTokens":135,"latencyMs":2405.8004589998163}] \ No newline at end of file +[{"questionId":"q1","format":"json-pretty","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":6453,"outputTokens":72,"latencyMs":2972.5673339999994},{"questionId":"q1","format":"json-compact","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":4046,"outputTokens":136,"latencyMs":3134.6921249999996},{"questionId":"q1","format":"toon","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2610,"outputTokens":136,"latencyMs":3895.96425},{"questionId":"q1","format":"csv","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2444,"outputTokens":136,"latencyMs":3544.769292},{"questionId":"q1","format":"xml","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7415,"outputTokens":264,"latencyMs":7031.435208000001},{"questionId":"q1","format":"yaml","model":"gpt-5-nano","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5073,"outputTokens":136,"latencyMs":3521.167166},{"questionId":"q2","format":"json-pretty","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":6453,"outputTokens":135,"latencyMs":3347.5196659999997},{"questionId":"q2","format":"json-compact","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":4046,"outputTokens":263,"latencyMs":5199.591958},{"questionId":"q2","format":"toon","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2610,"outputTokens":199,"latencyMs":3885.8805},{"questionId":"q2","format":"csv","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2444,"outputTokens":135,"latencyMs":3269.682833},{"questionId":"q2","format":"xml","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7415,"outputTokens":199,"latencyMs":6971.545625000001},{"questionId":"q2","format":"yaml","model":"gpt-5-nano","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5073,"outputTokens":135,"latencyMs":5154.9494159999995},{"questionId":"q3","format":"json-pretty","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":6455,"outputTokens":204,"latencyMs":3733.00875},{"questionId":"q3","format":"json-compact","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4048,"outputTokens":268,"latencyMs":3525.1048329999994},{"questionId":"q3","format":"toon","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2612,"outputTokens":268,"latencyMs":8655.205000000002},{"questionId":"q3","format":"csv","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2446,"outputTokens":204,"latencyMs":6536.7831670000005},{"questionId":"q3","format":"xml","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7417,"outputTokens":268,"latencyMs":5954.473},{"questionId":"q3","format":"yaml","model":"gpt-5-nano","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5075,"outputTokens":268,"latencyMs":3160.4033339999996},{"questionId":"q4","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":6455,"outputTokens":199,"latencyMs":3313.826750000001},{"questionId":"q4","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4048,"outputTokens":199,"latencyMs":3131.2316250000003},{"questionId":"q4","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2612,"outputTokens":199,"latencyMs":5442.334000000001},{"questionId":"q4","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2446,"outputTokens":199,"latencyMs":3414.6267500000013},{"questionId":"q4","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":7417,"outputTokens":327,"latencyMs":8402.244708},{"questionId":"q4","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":5075,"outputTokens":199,"latencyMs":3475.437167},{"questionId":"q5","format":"json-pretty","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":6451,"outputTokens":263,"latencyMs":4907.695667},{"questionId":"q5","format":"json-compact","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":4044,"outputTokens":327,"latencyMs":4696.913291999999},{"questionId":"q5","format":"toon","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":2608,"outputTokens":391,"latencyMs":3964.1052500000005},{"questionId":"q5","format":"csv","model":"gpt-5-nano","expected":"no","actual":"0","isCorrect":true,"inputTokens":2442,"outputTokens":391,"latencyMs":4546.052125},{"questionId":"q5","format":"xml","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":7413,"outputTokens":263,"latencyMs":12447.142749999999},{"questionId":"q5","format":"yaml","model":"gpt-5-nano","expected":"no","actual":"false","isCorrect":true,"inputTokens":5071,"outputTokens":327,"latencyMs":5445.827582999998},{"questionId":"q6","format":"json-pretty","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":6452,"outputTokens":200,"latencyMs":2964.3259579999994},{"questionId":"q6","format":"json-compact","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":4045,"outputTokens":136,"latencyMs":2476.4536669999998},{"questionId":"q6","format":"toon","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2609,"outputTokens":264,"latencyMs":10081.144249999998},{"questionId":"q6","format":"csv","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2443,"outputTokens":200,"latencyMs":3566.9233750000003},{"questionId":"q6","format":"xml","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7414,"outputTokens":200,"latencyMs":3328.505791999998},{"questionId":"q6","format":"yaml","model":"gpt-5-nano","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5072,"outputTokens":136,"latencyMs":2627.9952919999996},{"questionId":"q7","format":"json-pretty","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":6453,"outputTokens":199,"latencyMs":6494.151333},{"questionId":"q7","format":"json-compact","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":4046,"outputTokens":327,"latencyMs":7962.695958},{"questionId":"q7","format":"toon","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2610,"outputTokens":135,"latencyMs":3294.491083000001},{"questionId":"q7","format":"csv","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2444,"outputTokens":71,"latencyMs":2967.147291999998},{"questionId":"q7","format":"xml","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7415,"outputTokens":199,"latencyMs":5902.4375},{"questionId":"q7","format":"yaml","model":"gpt-5-nano","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5073,"outputTokens":135,"latencyMs":2749.074292000001},{"questionId":"q8","format":"json-pretty","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":6454,"outputTokens":397,"latencyMs":6029.30125},{"questionId":"q8","format":"json-compact","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4047,"outputTokens":525,"latencyMs":9383.724667000002},{"questionId":"q8","format":"toon","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2611,"outputTokens":205,"latencyMs":6669.472833},{"questionId":"q8","format":"csv","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2445,"outputTokens":269,"latencyMs":4525.544707999998},{"questionId":"q8","format":"xml","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7416,"outputTokens":205,"latencyMs":3729.2334999999985},{"questionId":"q8","format":"yaml","model":"gpt-5-nano","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5074,"outputTokens":205,"latencyMs":5299.043415999997},{"questionId":"q9","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":6456,"outputTokens":199,"latencyMs":8089.861666000001},{"questionId":"q9","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":4049,"outputTokens":135,"latencyMs":3481.7531249999993},{"questionId":"q9","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":2613,"outputTokens":519,"latencyMs":6562.223791000004},{"questionId":"q9","format":"csv","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":2447,"outputTokens":199,"latencyMs":6497.464292000004},{"questionId":"q9","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":7418,"outputTokens":263,"latencyMs":4308.770542000006},{"questionId":"q9","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5076,"outputTokens":455,"latencyMs":7165.029666999995},{"questionId":"q10","format":"json-pretty","model":"gpt-5-nano","expected":"yes","actual":"true","isCorrect":true,"inputTokens":6453,"outputTokens":199,"latencyMs":7847.269749999992},{"questionId":"q10","format":"json-compact","model":"gpt-5-nano","expected":"yes","actual":"true","isCorrect":true,"inputTokens":4046,"outputTokens":263,"latencyMs":5162.878791000003},{"questionId":"q10","format":"toon","model":"gpt-5-nano","expected":"yes","actual":"false","isCorrect":false,"inputTokens":2610,"outputTokens":3015,"latencyMs":44119.86958299999},{"questionId":"q10","format":"csv","model":"gpt-5-nano","expected":"yes","actual":"1","isCorrect":true,"inputTokens":2444,"outputTokens":711,"latencyMs":10732.124499999998},{"questionId":"q10","format":"xml","model":"gpt-5-nano","expected":"yes","actual":"true","isCorrect":true,"inputTokens":7415,"outputTokens":455,"latencyMs":7670.041000000005},{"questionId":"q10","format":"yaml","model":"gpt-5-nano","expected":"yes","actual":"false","isCorrect":false,"inputTokens":5073,"outputTokens":1671,"latencyMs":32276.611915999994},{"questionId":"q11","format":"json-pretty","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":6453,"outputTokens":200,"latencyMs":5021.2276249999995},{"questionId":"q11","format":"json-compact","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":4046,"outputTokens":136,"latencyMs":4285.859167000002},{"questionId":"q11","format":"toon","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2610,"outputTokens":136,"latencyMs":2937.344834000003},{"questionId":"q11","format":"csv","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2444,"outputTokens":136,"latencyMs":2420.909792000006},{"questionId":"q11","format":"xml","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7415,"outputTokens":136,"latencyMs":2657.114041000008},{"questionId":"q11","format":"yaml","model":"gpt-5-nano","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5073,"outputTokens":136,"latencyMs":4141.019750000007},{"questionId":"q12","format":"json-pretty","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":6451,"outputTokens":327,"latencyMs":8326.443249999997},{"questionId":"q12","format":"json-compact","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":4044,"outputTokens":71,"latencyMs":1961.1611669999984},{"questionId":"q12","format":"toon","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2608,"outputTokens":455,"latencyMs":9416.313375000012},{"questionId":"q12","format":"csv","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2442,"outputTokens":199,"latencyMs":3679.7195409999986},{"questionId":"q12","format":"xml","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7413,"outputTokens":391,"latencyMs":4654.398666000008},{"questionId":"q12","format":"yaml","model":"gpt-5-nano","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5071,"outputTokens":263,"latencyMs":5227.314999999988},{"questionId":"q13","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6450,"outputTokens":583,"latencyMs":11891.486790999988},{"questionId":"q13","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":4043,"outputTokens":903,"latencyMs":11808.044291999991},{"questionId":"q13","format":"toon","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2607,"outputTokens":1031,"latencyMs":13869.627583000009},{"questionId":"q13","format":"csv","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2441,"outputTokens":1095,"latencyMs":11982.031124999994},{"questionId":"q13","format":"xml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":7412,"outputTokens":1159,"latencyMs":12268.117834000004},{"questionId":"q13","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":5070,"outputTokens":903,"latencyMs":14245.349500000011},{"questionId":"q14","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6450,"outputTokens":583,"latencyMs":10854.242750000005},{"questionId":"q14","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":4043,"outputTokens":1543,"latencyMs":14848.513416999995},{"questionId":"q14","format":"toon","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2607,"outputTokens":1351,"latencyMs":18436.125499999995},{"questionId":"q14","format":"csv","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2441,"outputTokens":583,"latencyMs":7873.872290999992},{"questionId":"q14","format":"xml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":7412,"outputTokens":1159,"latencyMs":16891.326750000007},{"questionId":"q14","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":5070,"outputTokens":775,"latencyMs":11653.730375},{"questionId":"q15","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6450,"outputTokens":711,"latencyMs":10938.143958},{"questionId":"q15","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":4043,"outputTokens":839,"latencyMs":9845.249500000005},{"questionId":"q15","format":"toon","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2607,"outputTokens":1351,"latencyMs":17360.869999999995},{"questionId":"q15","format":"csv","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":2441,"outputTokens":583,"latencyMs":6245.138082999998},{"questionId":"q15","format":"xml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":7412,"outputTokens":583,"latencyMs":6926.366333000013},{"questionId":"q15","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":5070,"outputTokens":1159,"latencyMs":10682.860499999995},{"questionId":"q16","format":"json-pretty","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":6455,"outputTokens":2887,"latencyMs":26044.378417},{"questionId":"q16","format":"json-compact","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":4048,"outputTokens":2311,"latencyMs":20886.991708999994},{"questionId":"q16","format":"toon","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":2612,"outputTokens":3207,"latencyMs":38211.708791},{"questionId":"q16","format":"csv","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":2446,"outputTokens":3591,"latencyMs":34722.982124999995},{"questionId":"q16","format":"xml","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":7417,"outputTokens":2759,"latencyMs":27677.160040999996},{"questionId":"q16","format":"yaml","model":"gpt-5-nano","expected":"91","actual":"91","isCorrect":true,"inputTokens":5075,"outputTokens":2183,"latencyMs":21999.34112499999},{"questionId":"q17","format":"json-pretty","model":"gpt-5-nano","expected":"67","actual":"68","isCorrect":false,"inputTokens":6455,"outputTokens":2375,"latencyMs":24885.276625},{"questionId":"q17","format":"json-compact","model":"gpt-5-nano","expected":"67","actual":"67","isCorrect":true,"inputTokens":4048,"outputTokens":2567,"latencyMs":23865.78125},{"questionId":"q17","format":"toon","model":"gpt-5-nano","expected":"67","actual":"67","isCorrect":true,"inputTokens":2612,"outputTokens":3271,"latencyMs":33953.05562499999},{"questionId":"q17","format":"csv","model":"gpt-5-nano","expected":"67","actual":"66","isCorrect":false,"inputTokens":2446,"outputTokens":2695,"latencyMs":28272.904916999993},{"questionId":"q17","format":"xml","model":"gpt-5-nano","expected":"67","actual":"66","isCorrect":false,"inputTokens":7417,"outputTokens":2183,"latencyMs":20128.4455},{"questionId":"q17","format":"yaml","model":"gpt-5-nano","expected":"67","actual":"67","isCorrect":true,"inputTokens":5075,"outputTokens":2887,"latencyMs":27213.535542000012},{"questionId":"q18","format":"json-pretty","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":6455,"outputTokens":2119,"latencyMs":23143.38708300001},{"questionId":"q18","format":"json-compact","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":4048,"outputTokens":1735,"latencyMs":19879.81758399999},{"questionId":"q18","format":"toon","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":2612,"outputTokens":1159,"latencyMs":11657.12725000002},{"questionId":"q18","format":"csv","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":2446,"outputTokens":1607,"latencyMs":17132.650707999986},{"questionId":"q18","format":"xml","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":7417,"outputTokens":2119,"latencyMs":27134.673207999993},{"questionId":"q18","format":"yaml","model":"gpt-5-nano","expected":"41","actual":"42","isCorrect":false,"inputTokens":5075,"outputTokens":1671,"latencyMs":17106.441208000004},{"questionId":"q19","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":6451,"outputTokens":135,"latencyMs":2293.6627090000256},{"questionId":"q19","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":4044,"outputTokens":135,"latencyMs":2780.268083999981},{"questionId":"q19","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2608,"outputTokens":199,"latencyMs":4030.7550000000047},{"questionId":"q19","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2442,"outputTokens":327,"latencyMs":4502.81479199999},{"questionId":"q19","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":7413,"outputTokens":199,"latencyMs":5037.844874999981},{"questionId":"q19","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":5071,"outputTokens":199,"latencyMs":2760.7317500000063},{"questionId":"q20","format":"json-pretty","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":true,"inputTokens":6452,"outputTokens":4554,"latencyMs":44195.818792000005},{"questionId":"q20","format":"json-compact","model":"gpt-5-nano","expected":"96503","actual":"96493.32","isCorrect":false,"inputTokens":4045,"outputTokens":4362,"latencyMs":49358.45320799999},{"questionId":"q20","format":"toon","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":true,"inputTokens":2609,"outputTokens":7882,"latencyMs":80184.430708},{"questionId":"q20","format":"csv","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":true,"inputTokens":2443,"outputTokens":8778,"latencyMs":95858.013875},{"questionId":"q20","format":"xml","model":"gpt-5-nano","expected":"96503","actual":"97054.06","isCorrect":false,"inputTokens":7414,"outputTokens":4426,"latencyMs":53834.34679099999},{"questionId":"q20","format":"yaml","model":"gpt-5-nano","expected":"96503","actual":"96503.32","isCorrect":true,"inputTokens":5072,"outputTokens":6666,"latencyMs":84141.234291},{"questionId":"q21","format":"json-pretty","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":6449,"outputTokens":1415,"latencyMs":16387.246041000006},{"questionId":"q21","format":"json-compact","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":4042,"outputTokens":1671,"latencyMs":25604.649542},{"questionId":"q21","format":"toon","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":2606,"outputTokens":1735,"latencyMs":26253.37858400002},{"questionId":"q21","format":"csv","model":"gpt-5-nano","expected":"78","actual":"80","isCorrect":false,"inputTokens":2440,"outputTokens":2439,"latencyMs":42881.88175},{"questionId":"q21","format":"xml","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":7411,"outputTokens":1543,"latencyMs":18824.236875000002},{"questionId":"q21","format":"yaml","model":"gpt-5-nano","expected":"78","actual":"78","isCorrect":true,"inputTokens":5069,"outputTokens":1351,"latencyMs":19731.303249999997},{"questionId":"q22","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":6449,"outputTokens":1031,"latencyMs":13784.215957999986},{"questionId":"q22","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4042,"outputTokens":1223,"latencyMs":13702.792790999985},{"questionId":"q22","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2606,"outputTokens":1223,"latencyMs":14749.500458000024},{"questionId":"q22","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2440,"outputTokens":1927,"latencyMs":21055.402042},{"questionId":"q22","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":7411,"outputTokens":1031,"latencyMs":16298.56808300002},{"questionId":"q22","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":5069,"outputTokens":1031,"latencyMs":15148.553374999989},{"questionId":"q23","format":"json-pretty","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":6457,"outputTokens":1223,"latencyMs":23748.731792000006},{"questionId":"q23","format":"json-compact","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":4050,"outputTokens":1095,"latencyMs":13191.228082999995},{"questionId":"q23","format":"toon","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2614,"outputTokens":1799,"latencyMs":19708.061292},{"questionId":"q23","format":"csv","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2448,"outputTokens":1863,"latencyMs":17711.145375000022},{"questionId":"q23","format":"xml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":7419,"outputTokens":1287,"latencyMs":14610.536499999987},{"questionId":"q23","format":"yaml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":5077,"outputTokens":1927,"latencyMs":21769.996958999982},{"questionId":"q24","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6457,"outputTokens":1159,"latencyMs":14361.063832999993},{"questionId":"q24","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4050,"outputTokens":903,"latencyMs":10174.464332999982},{"questionId":"q24","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2614,"outputTokens":1415,"latencyMs":14791.934709000023},{"questionId":"q24","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2448,"outputTokens":903,"latencyMs":10505.627374999982},{"questionId":"q24","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7419,"outputTokens":1415,"latencyMs":15141.258583000017},{"questionId":"q24","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5077,"outputTokens":2503,"latencyMs":25832.709875},{"questionId":"q25","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6457,"outputTokens":1415,"latencyMs":17542.110833000013},{"questionId":"q25","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4050,"outputTokens":1415,"latencyMs":15454.643125000002},{"questionId":"q25","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2614,"outputTokens":1671,"latencyMs":18372.684709000023},{"questionId":"q25","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2448,"outputTokens":1607,"latencyMs":18959.7825},{"questionId":"q25","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7419,"outputTokens":1351,"latencyMs":17531.771584000002},{"questionId":"q25","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5077,"outputTokens":1863,"latencyMs":20752.042625000002},{"questionId":"q26","format":"json-pretty","model":"gpt-5-nano","expected":"12","actual":"11","isCorrect":false,"inputTokens":6457,"outputTokens":903,"latencyMs":11434.620958999993},{"questionId":"q26","format":"json-compact","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":4050,"outputTokens":1799,"latencyMs":26667.313249999977},{"questionId":"q26","format":"toon","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2614,"outputTokens":1543,"latencyMs":19438.584040999995},{"questionId":"q26","format":"csv","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2448,"outputTokens":1863,"latencyMs":24016.536166999984},{"questionId":"q26","format":"xml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":7419,"outputTokens":1031,"latencyMs":12378.093457999988},{"questionId":"q26","format":"yaml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":5077,"outputTokens":1735,"latencyMs":18425.30349999998},{"questionId":"q27","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6457,"outputTokens":1031,"latencyMs":13011.211625000025},{"questionId":"q27","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"10","isCorrect":false,"inputTokens":4050,"outputTokens":1031,"latencyMs":12447.250166999991},{"questionId":"q27","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2614,"outputTokens":1799,"latencyMs":21169.804915999994},{"questionId":"q27","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2448,"outputTokens":1095,"latencyMs":13325.901125000004},{"questionId":"q27","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7419,"outputTokens":1799,"latencyMs":20041.018916},{"questionId":"q27","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5077,"outputTokens":839,"latencyMs":9830.15854200002},{"questionId":"q28","format":"json-pretty","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":6456,"outputTokens":2695,"latencyMs":34929.70891699998},{"questionId":"q28","format":"json-compact","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":4049,"outputTokens":2695,"latencyMs":32068.38629199998},{"questionId":"q28","format":"toon","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":2613,"outputTokens":3207,"latencyMs":35007.753916999965},{"questionId":"q28","format":"csv","model":"gpt-5-nano","expected":"63","actual":"64","isCorrect":false,"inputTokens":2447,"outputTokens":4295,"latencyMs":44454.070875000034},{"questionId":"q28","format":"xml","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":7418,"outputTokens":4359,"latencyMs":53078.10720800003},{"questionId":"q28","format":"yaml","model":"gpt-5-nano","expected":"63","actual":"63","isCorrect":true,"inputTokens":5076,"outputTokens":5383,"latencyMs":62424.056374999986},{"questionId":"q29","format":"json-pretty","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":6456,"outputTokens":3527,"latencyMs":38803.08712500002},{"questionId":"q29","format":"json-compact","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":4049,"outputTokens":2631,"latencyMs":36088.47983299999},{"questionId":"q29","format":"toon","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":2613,"outputTokens":3463,"latencyMs":37541.076541999995},{"questionId":"q29","format":"csv","model":"gpt-5-nano","expected":"53","actual":"54","isCorrect":false,"inputTokens":2447,"outputTokens":4423,"latencyMs":47284.71529199998},{"questionId":"q29","format":"xml","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":7418,"outputTokens":2375,"latencyMs":26526.219709000026},{"questionId":"q29","format":"yaml","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":5076,"outputTokens":2183,"latencyMs":22403.027584000025},{"questionId":"q30","format":"json-pretty","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":6456,"outputTokens":1607,"latencyMs":17674.940834000008},{"questionId":"q30","format":"json-compact","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":4049,"outputTokens":3591,"latencyMs":36116.21895800001},{"questionId":"q30","format":"toon","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":2613,"outputTokens":3271,"latencyMs":28577.131834},{"questionId":"q30","format":"csv","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":2447,"outputTokens":3527,"latencyMs":35360.875459},{"questionId":"q30","format":"xml","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":7418,"outputTokens":2247,"latencyMs":22521.478082999995},{"questionId":"q30","format":"yaml","model":"gpt-5-nano","expected":"39","actual":"39","isCorrect":true,"inputTokens":5076,"outputTokens":2503,"latencyMs":25049.46987499995},{"questionId":"q31","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6457,"outputTokens":2375,"latencyMs":23942.841625},{"questionId":"q31","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4050,"outputTokens":1223,"latencyMs":32165.900916000013},{"questionId":"q31","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2614,"outputTokens":1351,"latencyMs":14162.266958999971},{"questionId":"q31","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2448,"outputTokens":1479,"latencyMs":17867.209082999965},{"questionId":"q31","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7419,"outputTokens":1671,"latencyMs":16419.887457999983},{"questionId":"q31","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5077,"outputTokens":1735,"latencyMs":18486.571375},{"questionId":"q32","format":"json-pretty","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":6457,"outputTokens":1351,"latencyMs":13326.963042000018},{"questionId":"q32","format":"json-compact","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":4050,"outputTokens":1287,"latencyMs":12924.029000000039},{"questionId":"q32","format":"toon","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":2614,"outputTokens":1671,"latencyMs":68753.054917},{"questionId":"q32","format":"csv","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":2448,"outputTokens":1735,"latencyMs":20531.763833000034},{"questionId":"q32","format":"xml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":7419,"outputTokens":1479,"latencyMs":25654.030582999985},{"questionId":"q32","format":"yaml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":5077,"outputTokens":1159,"latencyMs":14334.90933299996},{"questionId":"q33","format":"json-pretty","model":"gpt-5-nano","expected":"15","actual":"16","isCorrect":false,"inputTokens":6457,"outputTokens":1991,"latencyMs":22980.818790999998},{"questionId":"q33","format":"json-compact","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":4050,"outputTokens":1223,"latencyMs":14265.010167},{"questionId":"q33","format":"toon","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":2614,"outputTokens":1223,"latencyMs":16021.799958000018},{"questionId":"q33","format":"csv","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":2448,"outputTokens":1095,"latencyMs":12047.490749999997},{"questionId":"q33","format":"xml","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":7419,"outputTokens":1735,"latencyMs":20477.510917000007},{"questionId":"q33","format":"yaml","model":"gpt-5-nano","expected":"15","actual":"15","isCorrect":true,"inputTokens":5077,"outputTokens":1159,"latencyMs":12986.920958000002},{"questionId":"q34","format":"json-pretty","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":6451,"outputTokens":647,"latencyMs":7786.604583000008},{"questionId":"q34","format":"json-compact","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":4044,"outputTokens":1159,"latencyMs":12812.286625000008},{"questionId":"q34","format":"toon","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2608,"outputTokens":1287,"latencyMs":13650.962291000003},{"questionId":"q34","format":"csv","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":2442,"outputTokens":1543,"latencyMs":22916.750333000033},{"questionId":"q34","format":"xml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":7413,"outputTokens":1735,"latencyMs":27190.99129199999},{"questionId":"q34","format":"yaml","model":"gpt-5-nano","expected":"12","actual":"12","isCorrect":true,"inputTokens":5071,"outputTokens":1223,"latencyMs":18113.423624999996},{"questionId":"q35","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6451,"outputTokens":839,"latencyMs":14319.936749999993},{"questionId":"q35","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":4044,"outputTokens":1159,"latencyMs":19572.69550000003},{"questionId":"q35","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2608,"outputTokens":2183,"latencyMs":27670.17745899997},{"questionId":"q35","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":2442,"outputTokens":2055,"latencyMs":30155.347083},{"questionId":"q35","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":7413,"outputTokens":1735,"latencyMs":17707.77020899998},{"questionId":"q35","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5071,"outputTokens":1671,"latencyMs":17215.395207999973},{"questionId":"q36","format":"json-pretty","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":10807,"outputTokens":329,"latencyMs":7442.371583},{"questionId":"q36","format":"json-compact","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":6888,"outputTokens":329,"latencyMs":6396.415417000011},{"questionId":"q36","format":"toon","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":7325,"outputTokens":393,"latencyMs":4602.9237079999875},{"questionId":"q36","format":"xml","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":12115,"outputTokens":265,"latencyMs":4785.739166999992},{"questionId":"q36","format":"yaml","model":"gpt-5-nano","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8439,"outputTokens":137,"latencyMs":7061.959375000035},{"questionId":"q37","format":"json-pretty","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":10807,"outputTokens":328,"latencyMs":6893.168333999987},{"questionId":"q37","format":"json-compact","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":6888,"outputTokens":392,"latencyMs":5599.888457999972},{"questionId":"q37","format":"toon","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":7325,"outputTokens":136,"latencyMs":3737.4124590000138},{"questionId":"q37","format":"xml","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":12115,"outputTokens":200,"latencyMs":6961.4233330000425},{"questionId":"q37","format":"yaml","model":"gpt-5-nano","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8439,"outputTokens":264,"latencyMs":5140.496583},{"questionId":"q38","format":"json-pretty","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":10807,"outputTokens":585,"latencyMs":7864.396957999968},{"questionId":"q38","format":"json-compact","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":6888,"outputTokens":521,"latencyMs":6421.59816600004},{"questionId":"q38","format":"toon","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":7325,"outputTokens":265,"latencyMs":3733.0714159999625},{"questionId":"q38","format":"xml","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":12115,"outputTokens":393,"latencyMs":9006.905709000013},{"questionId":"q38","format":"yaml","model":"gpt-5-nano","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8439,"outputTokens":329,"latencyMs":6850.975207999989},{"questionId":"q39","format":"json-pretty","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":10807,"outputTokens":391,"latencyMs":5922.136790999968},{"questionId":"q39","format":"json-compact","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":6888,"outputTokens":391,"latencyMs":5514.317334000021},{"questionId":"q39","format":"toon","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":7325,"outputTokens":199,"latencyMs":5819.365749999997},{"questionId":"q39","format":"xml","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":12115,"outputTokens":327,"latencyMs":4577.510000000009},{"questionId":"q39","format":"yaml","model":"gpt-5-nano","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8439,"outputTokens":199,"latencyMs":3300.337999999989},{"questionId":"q40","format":"json-pretty","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":10807,"outputTokens":522,"latencyMs":10160.32945900003},{"questionId":"q40","format":"json-compact","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":6888,"outputTokens":586,"latencyMs":8726.273291999998},{"questionId":"q40","format":"toon","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":7325,"outputTokens":458,"latencyMs":7168.197749999992},{"questionId":"q40","format":"xml","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":12115,"outputTokens":330,"latencyMs":9622.921792000008},{"questionId":"q40","format":"yaml","model":"gpt-5-nano","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8439,"outputTokens":458,"latencyMs":12277.717749999953},{"questionId":"q41","format":"json-pretty","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":10807,"outputTokens":647,"latencyMs":10452.618916000007},{"questionId":"q41","format":"json-compact","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":6888,"outputTokens":327,"latencyMs":7993.501333000022},{"questionId":"q41","format":"toon","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":7325,"outputTokens":391,"latencyMs":6246.408166999987},{"questionId":"q41","format":"xml","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":12115,"outputTokens":327,"latencyMs":6159.920333000016},{"questionId":"q41","format":"yaml","model":"gpt-5-nano","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8439,"outputTokens":135,"latencyMs":4936.892832999991},{"questionId":"q42","format":"json-pretty","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":10807,"outputTokens":330,"latencyMs":6513.42883400002},{"questionId":"q42","format":"json-compact","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":6888,"outputTokens":266,"latencyMs":5169.701290999947},{"questionId":"q42","format":"toon","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":7325,"outputTokens":394,"latencyMs":6359.4133749999455},{"questionId":"q42","format":"xml","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":12115,"outputTokens":458,"latencyMs":7373.879582999973},{"questionId":"q42","format":"yaml","model":"gpt-5-nano","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8439,"outputTokens":586,"latencyMs":9219.977708999999},{"questionId":"q43","format":"json-pretty","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10807,"outputTokens":456,"latencyMs":6681.0621670000255},{"questionId":"q43","format":"json-compact","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6888,"outputTokens":456,"latencyMs":6576.267416999966},{"questionId":"q43","format":"toon","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7325,"outputTokens":328,"latencyMs":5022.24679200002},{"questionId":"q43","format":"xml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12115,"outputTokens":200,"latencyMs":3512.9928749999963},{"questionId":"q43","format":"yaml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8439,"outputTokens":264,"latencyMs":5002.190166999993},{"questionId":"q44","format":"json-pretty","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":10808,"outputTokens":332,"latencyMs":4617.067666999996},{"questionId":"q44","format":"json-compact","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":6889,"outputTokens":780,"latencyMs":10020.49199999997},{"questionId":"q44","format":"toon","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":7326,"outputTokens":268,"latencyMs":7557.820707999985},{"questionId":"q44","format":"xml","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":12116,"outputTokens":588,"latencyMs":8139.475207999989},{"questionId":"q44","format":"yaml","model":"gpt-5-nano","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8440,"outputTokens":268,"latencyMs":4885.101999999955},{"questionId":"q45","format":"json-pretty","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":10808,"outputTokens":399,"latencyMs":5571.73229200003},{"questionId":"q45","format":"json-compact","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":6889,"outputTokens":655,"latencyMs":10517.11179200001},{"questionId":"q45","format":"toon","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":7326,"outputTokens":271,"latencyMs":4059.1732920000213},{"questionId":"q45","format":"xml","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":12116,"outputTokens":143,"latencyMs":3110.5762920000125},{"questionId":"q45","format":"yaml","model":"gpt-5-nano","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8440,"outputTokens":207,"latencyMs":3715.018374999985},{"questionId":"q46","format":"json-pretty","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":10808,"outputTokens":844,"latencyMs":10064.53433299996},{"questionId":"q46","format":"json-compact","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":6889,"outputTokens":716,"latencyMs":11513.566083000042},{"questionId":"q46","format":"toon","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":7326,"outputTokens":332,"latencyMs":4870.685041000019},{"questionId":"q46","format":"xml","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":12116,"outputTokens":396,"latencyMs":5502.409417000017},{"questionId":"q46","format":"yaml","model":"gpt-5-nano","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8440,"outputTokens":204,"latencyMs":3546.790500000003},{"questionId":"q47","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":10807,"outputTokens":711,"latencyMs":8666.248917000019},{"questionId":"q47","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6888,"outputTokens":391,"latencyMs":5154.15979200002},{"questionId":"q47","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":7325,"outputTokens":455,"latencyMs":6469.648125000007},{"questionId":"q47","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":12115,"outputTokens":967,"latencyMs":12026.984790999966},{"questionId":"q47","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":8439,"outputTokens":327,"latencyMs":4457.898167000036},{"questionId":"q48","format":"json-pretty","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":10808,"outputTokens":266,"latencyMs":4677.685125000018},{"questionId":"q48","format":"json-compact","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":6889,"outputTokens":458,"latencyMs":7593.830499999982},{"questionId":"q48","format":"toon","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":7326,"outputTokens":330,"latencyMs":7164.58362499997},{"questionId":"q48","format":"xml","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":12116,"outputTokens":202,"latencyMs":4569.4382500000065},{"questionId":"q48","format":"yaml","model":"gpt-5-nano","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8440,"outputTokens":266,"latencyMs":8655.245208000008},{"questionId":"q49","format":"json-pretty","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":10808,"outputTokens":333,"latencyMs":9397.804249999986},{"questionId":"q49","format":"json-compact","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":6889,"outputTokens":525,"latencyMs":7369.637999999977},{"questionId":"q49","format":"toon","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":7326,"outputTokens":269,"latencyMs":7476.084625000018},{"questionId":"q49","format":"xml","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":12116,"outputTokens":205,"latencyMs":4457.102749999962},{"questionId":"q49","format":"yaml","model":"gpt-5-nano","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8440,"outputTokens":205,"latencyMs":3285.5180420000106},{"questionId":"q50","format":"json-pretty","model":"gpt-5-nano","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":10808,"outputTokens":332,"latencyMs":5102.7447909999755},{"questionId":"q50","format":"json-compact","model":"gpt-5-nano","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":6889,"outputTokens":652,"latencyMs":8489.679457999999},{"questionId":"q50","format":"toon","model":"gpt-5-nano","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":7326,"outputTokens":204,"latencyMs":7314.751374999993},{"questionId":"q50","format":"xml","model":"gpt-5-nano","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":12116,"outputTokens":332,"latencyMs":5297.356582999986},{"questionId":"q50","format":"yaml","model":"gpt-5-nano","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":8440,"outputTokens":460,"latencyMs":5892.525124999986},{"questionId":"q51","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":10807,"outputTokens":519,"latencyMs":7676.625582999957},{"questionId":"q51","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":6888,"outputTokens":711,"latencyMs":10736.315040999965},{"questionId":"q51","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":7325,"outputTokens":327,"latencyMs":7610.965416999999},{"questionId":"q51","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":12115,"outputTokens":647,"latencyMs":9436.054707999981},{"questionId":"q51","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":8439,"outputTokens":583,"latencyMs":7257.893417000014},{"questionId":"q52","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10804,"outputTokens":1031,"latencyMs":12227.468916999991},{"questionId":"q52","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6885,"outputTokens":903,"latencyMs":10091.694916000008},{"questionId":"q52","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7322,"outputTokens":839,"latencyMs":10802.154916999978},{"questionId":"q52","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12112,"outputTokens":1287,"latencyMs":15510.80912499997},{"questionId":"q52","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8436,"outputTokens":775,"latencyMs":11378.500208000012},{"questionId":"q53","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10804,"outputTokens":775,"latencyMs":8308.13866700005},{"questionId":"q53","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6885,"outputTokens":775,"latencyMs":9544.55587500002},{"questionId":"q53","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7322,"outputTokens":775,"latencyMs":17279.684707999986},{"questionId":"q53","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12112,"outputTokens":967,"latencyMs":9357.227749999962},{"questionId":"q53","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8436,"outputTokens":647,"latencyMs":7357.085124999983},{"questionId":"q54","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10805,"outputTokens":903,"latencyMs":9983.735792000021},{"questionId":"q54","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6886,"outputTokens":775,"latencyMs":8389.489208000014},{"questionId":"q54","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7323,"outputTokens":711,"latencyMs":8864.96325000003},{"questionId":"q54","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12113,"outputTokens":903,"latencyMs":11110.859708000033},{"questionId":"q54","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8437,"outputTokens":647,"latencyMs":9254.883999999962},{"questionId":"q55","format":"json-pretty","model":"gpt-5-nano","expected":"34904.81","actual":"34682.16","isCorrect":false,"inputTokens":10805,"outputTokens":3210,"latencyMs":35287.34470799996},{"questionId":"q55","format":"json-compact","model":"gpt-5-nano","expected":"34904.81","actual":"35004.81","isCorrect":false,"inputTokens":6886,"outputTokens":3082,"latencyMs":53411.28241600003},{"questionId":"q55","format":"toon","model":"gpt-5-nano","expected":"34904.81","actual":"34904.81","isCorrect":true,"inputTokens":7323,"outputTokens":5386,"latencyMs":65610.21349999995},{"questionId":"q55","format":"xml","model":"gpt-5-nano","expected":"34904.81","actual":"35004.81","isCorrect":false,"inputTokens":12113,"outputTokens":3082,"latencyMs":30146.02779099997},{"questionId":"q55","format":"yaml","model":"gpt-5-nano","expected":"34904.81","actual":"34904.81","isCorrect":true,"inputTokens":8437,"outputTokens":5194,"latencyMs":47443.21795800002},{"questionId":"q56","format":"json-pretty","model":"gpt-5-nano","expected":"698.10","actual":"698.10","isCorrect":true,"inputTokens":10803,"outputTokens":7433,"latencyMs":99668.25054200005},{"questionId":"q56","format":"json-compact","model":"gpt-5-nano","expected":"698.10","actual":"698.10","isCorrect":true,"inputTokens":6884,"outputTokens":2825,"latencyMs":29726.150667000038},{"questionId":"q56","format":"toon","model":"gpt-5-nano","expected":"698.10","actual":"698.0962","isCorrect":true,"inputTokens":7321,"outputTokens":3402,"latencyMs":29333.87345799996},{"questionId":"q56","format":"xml","model":"gpt-5-nano","expected":"698.10","actual":"698.10","isCorrect":true,"inputTokens":12111,"outputTokens":8713,"latencyMs":88680.83875000005},{"questionId":"q56","format":"yaml","model":"gpt-5-nano","expected":"698.10","actual":"700.10","isCorrect":false,"inputTokens":8435,"outputTokens":2185,"latencyMs":22504.17745899997},{"questionId":"q57","format":"json-pretty","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":10804,"outputTokens":263,"latencyMs":3635.718082999985},{"questionId":"q57","format":"json-compact","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":6885,"outputTokens":455,"latencyMs":6344.6660000000265},{"questionId":"q57","format":"toon","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":7322,"outputTokens":135,"latencyMs":2214.3594589999993},{"questionId":"q57","format":"xml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":12112,"outputTokens":199,"latencyMs":3028.1589169999934},{"questionId":"q57","format":"yaml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":8436,"outputTokens":263,"latencyMs":3726.436791999964},{"questionId":"q58","format":"json-pretty","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":10803,"outputTokens":1034,"latencyMs":17297.540125000058},{"questionId":"q58","format":"json-compact","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":6884,"outputTokens":1226,"latencyMs":15772.636416999972},{"questionId":"q58","format":"toon","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":7321,"outputTokens":714,"latencyMs":7838.149208000046},{"questionId":"q58","format":"xml","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":12111,"outputTokens":1418,"latencyMs":18521.650375000027},{"questionId":"q58","format":"yaml","model":"gpt-5-nano","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8435,"outputTokens":1546,"latencyMs":16359.941374999937},{"questionId":"q59","format":"json-pretty","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":10807,"outputTokens":1543,"latencyMs":27746.01329100004},{"questionId":"q59","format":"json-compact","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":6888,"outputTokens":1799,"latencyMs":21588.628625000012},{"questionId":"q59","format":"toon","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":7325,"outputTokens":1607,"latencyMs":22641.205915999948},{"questionId":"q59","format":"xml","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":12115,"outputTokens":2119,"latencyMs":21735.99774999998},{"questionId":"q59","format":"yaml","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":8439,"outputTokens":1415,"latencyMs":15331.604541999986},{"questionId":"q60","format":"json-pretty","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":10807,"outputTokens":1671,"latencyMs":23176.00108299998},{"questionId":"q60","format":"json-compact","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":6888,"outputTokens":1031,"latencyMs":11122.897291000001},{"questionId":"q60","format":"toon","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":7325,"outputTokens":2119,"latencyMs":44203.42183300003},{"questionId":"q60","format":"xml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":12115,"outputTokens":1415,"latencyMs":21619.647499999963},{"questionId":"q60","format":"yaml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":8439,"outputTokens":2055,"latencyMs":20646.457915999927},{"questionId":"q61","format":"json-pretty","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":10807,"outputTokens":1799,"latencyMs":22455.639375000028},{"questionId":"q61","format":"json-compact","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":6888,"outputTokens":1223,"latencyMs":12465.433750000084},{"questionId":"q61","format":"toon","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":7325,"outputTokens":1543,"latencyMs":17901.21987499995},{"questionId":"q61","format":"xml","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":12115,"outputTokens":1351,"latencyMs":21725.661124999984},{"questionId":"q61","format":"yaml","model":"gpt-5-nano","expected":"28","actual":"28","isCorrect":true,"inputTokens":8439,"outputTokens":1479,"latencyMs":14143.484124999959},{"questionId":"q62","format":"json-pretty","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":10811,"outputTokens":1095,"latencyMs":11719.594000000041},{"questionId":"q62","format":"json-compact","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":6892,"outputTokens":1543,"latencyMs":16026.440790999914},{"questionId":"q62","format":"toon","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":7329,"outputTokens":839,"latencyMs":8525.078959000064},{"questionId":"q62","format":"xml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":12119,"outputTokens":1031,"latencyMs":11283.568582999986},{"questionId":"q62","format":"yaml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":8443,"outputTokens":1159,"latencyMs":13248.422166000004},{"questionId":"q63","format":"json-pretty","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":10811,"outputTokens":903,"latencyMs":10624.811125000007},{"questionId":"q63","format":"json-compact","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":6892,"outputTokens":1287,"latencyMs":15027.778207999887},{"questionId":"q63","format":"toon","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":7329,"outputTokens":967,"latencyMs":10102.057166000013},{"questionId":"q63","format":"xml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":12119,"outputTokens":1223,"latencyMs":14080.474375000107},{"questionId":"q63","format":"yaml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":8443,"outputTokens":839,"latencyMs":10806.409125000006},{"questionId":"q64","format":"json-pretty","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":10812,"outputTokens":647,"latencyMs":8619.796208000043},{"questionId":"q64","format":"json-compact","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":6893,"outputTokens":1095,"latencyMs":11266.89533300011},{"questionId":"q64","format":"toon","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":7330,"outputTokens":903,"latencyMs":10153.941749999998},{"questionId":"q64","format":"xml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":12120,"outputTokens":903,"latencyMs":10022.941333999974},{"questionId":"q64","format":"yaml","model":"gpt-5-nano","expected":"10","actual":"10","isCorrect":true,"inputTokens":8444,"outputTokens":1031,"latencyMs":11607.239833},{"questionId":"q65","format":"json-pretty","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":10812,"outputTokens":839,"latencyMs":9680.83516700007},{"questionId":"q65","format":"json-compact","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":6893,"outputTokens":1351,"latencyMs":13172.463165999972},{"questionId":"q65","format":"toon","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":7330,"outputTokens":967,"latencyMs":13761.158374999999},{"questionId":"q65","format":"xml","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":12120,"outputTokens":775,"latencyMs":8579.024916999973},{"questionId":"q65","format":"yaml","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":8444,"outputTokens":967,"latencyMs":11227.277834000066},{"questionId":"q66","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":10811,"outputTokens":1095,"latencyMs":11719.79470800003},{"questionId":"q66","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6892,"outputTokens":1287,"latencyMs":12974.757499999949},{"questionId":"q66","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":7329,"outputTokens":1159,"latencyMs":12100.158374999999},{"questionId":"q66","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":12119,"outputTokens":1607,"latencyMs":18089.76549999998},{"questionId":"q66","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":8443,"outputTokens":1351,"latencyMs":14901.317249999964},{"questionId":"q67","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":10811,"outputTokens":967,"latencyMs":13135.226917000022},{"questionId":"q67","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6892,"outputTokens":1479,"latencyMs":17517.77812500007},{"questionId":"q67","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":7329,"outputTokens":1159,"latencyMs":14243.380082999938},{"questionId":"q67","format":"xml","model":"gpt-5-nano","expected":"3","actual":"8","isCorrect":false,"inputTokens":12119,"outputTokens":4551,"latencyMs":60141.679957999964},{"questionId":"q67","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":8443,"outputTokens":1223,"latencyMs":14046.266375000007},{"questionId":"q68","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":10812,"outputTokens":1095,"latencyMs":13080.296417000005},{"questionId":"q68","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":6893,"outputTokens":1095,"latencyMs":12948.556707999902},{"questionId":"q68","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":7330,"outputTokens":967,"latencyMs":11987.064000000013},{"questionId":"q68","format":"xml","model":"gpt-5-nano","expected":"5","actual":"4","isCorrect":false,"inputTokens":12120,"outputTokens":2375,"latencyMs":25660.169375000056},{"questionId":"q68","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":8444,"outputTokens":1095,"latencyMs":13157.490292000002},{"questionId":"q69","format":"json-pretty","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":10813,"outputTokens":1991,"latencyMs":22131.704499999993},{"questionId":"q69","format":"json-compact","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":6894,"outputTokens":2503,"latencyMs":35345.66633400007},{"questionId":"q69","format":"toon","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":7331,"outputTokens":1799,"latencyMs":20451.80954200006},{"questionId":"q69","format":"xml","model":"gpt-5-nano","expected":"20","actual":"23","isCorrect":false,"inputTokens":12121,"outputTokens":4423,"latencyMs":45258.41941600002},{"questionId":"q69","format":"yaml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":8445,"outputTokens":3335,"latencyMs":34735.526083000004},{"questionId":"q70","format":"json-pretty","model":"gpt-5-nano","expected":"19","actual":"19","isCorrect":true,"inputTokens":10813,"outputTokens":2887,"latencyMs":41997.15891700005},{"questionId":"q70","format":"json-compact","model":"gpt-5-nano","expected":"19","actual":"19","isCorrect":true,"inputTokens":6894,"outputTokens":2503,"latencyMs":39163.075124999974},{"questionId":"q70","format":"toon","model":"gpt-5-nano","expected":"19","actual":"19","isCorrect":true,"inputTokens":7331,"outputTokens":2503,"latencyMs":27668.932207999984},{"questionId":"q70","format":"xml","model":"gpt-5-nano","expected":"19","actual":"30","isCorrect":false,"inputTokens":12121,"outputTokens":7303,"latencyMs":115098.00795900007},{"questionId":"q70","format":"yaml","model":"gpt-5-nano","expected":"19","actual":"18","isCorrect":false,"inputTokens":8445,"outputTokens":2247,"latencyMs":46982.19175},{"questionId":"q71","format":"json-pretty","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3772,"outputTokens":72,"latencyMs":2913.5479589999886},{"questionId":"q71","format":"json-compact","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2445,"outputTokens":136,"latencyMs":2803.939374999958},{"questionId":"q71","format":"toon","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1643,"outputTokens":200,"latencyMs":6143.997375000035},{"questionId":"q71","format":"csv","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1501,"outputTokens":264,"latencyMs":3737.9859579999465},{"questionId":"q71","format":"xml","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4478,"outputTokens":136,"latencyMs":6451.102792000049},{"questionId":"q71","format":"yaml","model":"gpt-5-nano","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3043,"outputTokens":136,"latencyMs":3775.4380420000525},{"questionId":"q72","format":"json-pretty","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3772,"outputTokens":202,"latencyMs":5033.039417000022},{"questionId":"q72","format":"json-compact","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2445,"outputTokens":266,"latencyMs":4111.494624999934},{"questionId":"q72","format":"toon","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1643,"outputTokens":394,"latencyMs":8638.389167000074},{"questionId":"q72","format":"csv","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1501,"outputTokens":458,"latencyMs":8524.106499999994},{"questionId":"q72","format":"xml","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4478,"outputTokens":330,"latencyMs":3970.999124999973},{"questionId":"q72","format":"yaml","model":"gpt-5-nano","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3043,"outputTokens":394,"latencyMs":4908.646542000002},{"questionId":"q73","format":"json-pretty","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3773,"outputTokens":393,"latencyMs":5176.399416},{"questionId":"q73","format":"json-compact","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2446,"outputTokens":393,"latencyMs":4801.460207999917},{"questionId":"q73","format":"toon","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1644,"outputTokens":393,"latencyMs":6057.693375000032},{"questionId":"q73","format":"csv","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1502,"outputTokens":265,"latencyMs":3750.650166999898},{"questionId":"q73","format":"xml","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4479,"outputTokens":265,"latencyMs":3854.7792090000585},{"questionId":"q73","format":"yaml","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3044,"outputTokens":393,"latencyMs":6347.295958000002},{"questionId":"q74","format":"json-pretty","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":3773,"outputTokens":199,"latencyMs":2818.6481669999193},{"questionId":"q74","format":"json-compact","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":2446,"outputTokens":583,"latencyMs":7016.024041999946},{"questionId":"q74","format":"toon","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1644,"outputTokens":327,"latencyMs":11679.409291999997},{"questionId":"q74","format":"csv","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1502,"outputTokens":263,"latencyMs":8099.556624999968},{"questionId":"q74","format":"xml","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":4479,"outputTokens":263,"latencyMs":5838.912250000052},{"questionId":"q74","format":"yaml","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":3044,"outputTokens":199,"latencyMs":4037.007084000041},{"questionId":"q75","format":"json-pretty","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3772,"outputTokens":200,"latencyMs":3196.2014169999165},{"questionId":"q75","format":"json-compact","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2445,"outputTokens":456,"latencyMs":5860.588916999986},{"questionId":"q75","format":"toon","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1643,"outputTokens":136,"latencyMs":7298.248332999996},{"questionId":"q75","format":"csv","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1501,"outputTokens":136,"latencyMs":2674.1493330000667},{"questionId":"q75","format":"xml","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4478,"outputTokens":136,"latencyMs":4064.031667000032},{"questionId":"q75","format":"yaml","model":"gpt-5-nano","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3043,"outputTokens":264,"latencyMs":5344.256874999963},{"questionId":"q76","format":"json-pretty","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3772,"outputTokens":522,"latencyMs":7330.072791999904},{"questionId":"q76","format":"json-compact","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2445,"outputTokens":650,"latencyMs":6851.327042000019},{"questionId":"q76","format":"toon","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1643,"outputTokens":522,"latencyMs":5857.433624999947},{"questionId":"q76","format":"csv","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1501,"outputTokens":522,"latencyMs":6343.291249999893},{"questionId":"q76","format":"xml","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4478,"outputTokens":266,"latencyMs":4330.397083999938},{"questionId":"q76","format":"yaml","model":"gpt-5-nano","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3043,"outputTokens":394,"latencyMs":5960.957958000014},{"questionId":"q77","format":"json-pretty","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3773,"outputTokens":329,"latencyMs":5319.284916999983},{"questionId":"q77","format":"json-compact","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2446,"outputTokens":329,"latencyMs":4361.255540999933},{"questionId":"q77","format":"toon","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1644,"outputTokens":585,"latencyMs":7900.699583000038},{"questionId":"q77","format":"csv","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1502,"outputTokens":457,"latencyMs":5575.410874999943},{"questionId":"q77","format":"xml","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4479,"outputTokens":521,"latencyMs":8356.306833000039},{"questionId":"q77","format":"yaml","model":"gpt-5-nano","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3044,"outputTokens":329,"latencyMs":7722.601000000024},{"questionId":"q78","format":"json-pretty","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":3773,"outputTokens":263,"latencyMs":5472.443542000023},{"questionId":"q78","format":"json-compact","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":2446,"outputTokens":135,"latencyMs":3127.7964580000844},{"questionId":"q78","format":"toon","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":1644,"outputTokens":263,"latencyMs":3561.4773750000168},{"questionId":"q78","format":"csv","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":1502,"outputTokens":199,"latencyMs":2641.831709000049},{"questionId":"q78","format":"xml","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":4479,"outputTokens":263,"latencyMs":3984.2812919999706},{"questionId":"q78","format":"yaml","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":3044,"outputTokens":327,"latencyMs":4146.360374999931},{"questionId":"q79","format":"json-pretty","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3772,"outputTokens":200,"latencyMs":3001.3016669999342},{"questionId":"q79","format":"json-compact","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2445,"outputTokens":200,"latencyMs":3991.4621250000782},{"questionId":"q79","format":"toon","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1643,"outputTokens":456,"latencyMs":7423.164000000106},{"questionId":"q79","format":"csv","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1501,"outputTokens":328,"latencyMs":3855.023082999978},{"questionId":"q79","format":"xml","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4478,"outputTokens":264,"latencyMs":4289.763542000088},{"questionId":"q79","format":"yaml","model":"gpt-5-nano","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3043,"outputTokens":200,"latencyMs":3733.6695410000393},{"questionId":"q80","format":"json-pretty","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":3769,"outputTokens":903,"latencyMs":9863.215167000075},{"questionId":"q80","format":"json-compact","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":2442,"outputTokens":775,"latencyMs":9231.59695799998},{"questionId":"q80","format":"toon","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1640,"outputTokens":391,"latencyMs":5704.973082999932},{"questionId":"q80","format":"csv","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1498,"outputTokens":583,"latencyMs":8583.113332999987},{"questionId":"q80","format":"xml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":4475,"outputTokens":327,"latencyMs":5083.1711249999935},{"questionId":"q80","format":"yaml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":3040,"outputTokens":455,"latencyMs":6700.828666999936},{"questionId":"q81","format":"json-pretty","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":3770,"outputTokens":1608,"latencyMs":18562.350708000013},{"questionId":"q81","format":"json-compact","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":2443,"outputTokens":2312,"latencyMs":21658.89724999992},{"questionId":"q81","format":"toon","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":1641,"outputTokens":2760,"latencyMs":26115.369374999893},{"questionId":"q81","format":"csv","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":1499,"outputTokens":2760,"latencyMs":27018.220417000004},{"questionId":"q81","format":"xml","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":4476,"outputTokens":3208,"latencyMs":35106.79574999993},{"questionId":"q81","format":"yaml","model":"gpt-5-nano","expected":"328320","actual":"328320","isCorrect":true,"inputTokens":3041,"outputTokens":3144,"latencyMs":28490.40104200004},{"questionId":"q82","format":"json-pretty","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":3770,"outputTokens":2056,"latencyMs":22616.934583000024},{"questionId":"q82","format":"json-compact","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":2443,"outputTokens":2376,"latencyMs":29998.364584000083},{"questionId":"q82","format":"toon","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":1641,"outputTokens":3080,"latencyMs":32440.0187919999},{"questionId":"q82","format":"csv","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":1499,"outputTokens":1864,"latencyMs":32674.871374999988},{"questionId":"q82","format":"xml","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":4476,"outputTokens":1992,"latencyMs":18328.068833000027},{"questionId":"q82","format":"yaml","model":"gpt-5-nano","expected":"1791","actual":"1791","isCorrect":true,"inputTokens":3041,"outputTokens":2440,"latencyMs":26599.709708000068},{"questionId":"q83","format":"json-pretty","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":3768,"outputTokens":3594,"latencyMs":31722.614457999938},{"questionId":"q83","format":"json-compact","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":2441,"outputTokens":6346,"latencyMs":61432.27987500001},{"questionId":"q83","format":"toon","model":"gpt-5-nano","expected":"311695.88","actual":"310695.88","isCorrect":false,"inputTokens":1639,"outputTokens":6922,"latencyMs":67581.54674999998},{"questionId":"q83","format":"csv","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":1497,"outputTokens":3658,"latencyMs":34084.16679099994},{"questionId":"q83","format":"xml","model":"gpt-5-nano","expected":"311695.88","actual":"310795.88","isCorrect":false,"inputTokens":4474,"outputTokens":7178,"latencyMs":72630.23875000002},{"questionId":"q83","format":"yaml","model":"gpt-5-nano","expected":"311695.88","actual":"311695.88","isCorrect":true,"inputTokens":3039,"outputTokens":5898,"latencyMs":57679.497999999905},{"questionId":"q84","format":"json-pretty","model":"gpt-5-nano","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3766,"outputTokens":7369,"latencyMs":67930.0475000001},{"questionId":"q84","format":"json-compact","model":"gpt-5-nano","expected":"0.53","actual":"0.5278333333333333","isCorrect":true,"inputTokens":2439,"outputTokens":4942,"latencyMs":46617.24916699994},{"questionId":"q84","format":"toon","model":"gpt-5-nano","expected":"0.53","actual":"0.527833","isCorrect":true,"inputTokens":1637,"outputTokens":4874,"latencyMs":46005.71258299996},{"questionId":"q84","format":"csv","model":"gpt-5-nano","expected":"0.53","actual":"0.5278333333","isCorrect":true,"inputTokens":1495,"outputTokens":3532,"latencyMs":33943.250624999986},{"questionId":"q84","format":"xml","model":"gpt-5-nano","expected":"0.53","actual":"0.5298333333","isCorrect":true,"inputTokens":4472,"outputTokens":4044,"latencyMs":37510.19779100001},{"questionId":"q84","format":"yaml","model":"gpt-5-nano","expected":"0.53","actual":"0.5278333333","isCorrect":true,"inputTokens":3037,"outputTokens":4556,"latencyMs":55096.31458399992},{"questionId":"q85","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3770,"outputTokens":1351,"latencyMs":13053.103250000044},{"questionId":"q85","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2443,"outputTokens":1031,"latencyMs":11122.031167000066},{"questionId":"q85","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1641,"outputTokens":1863,"latencyMs":19323.529084000038},{"questionId":"q85","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1499,"outputTokens":1479,"latencyMs":18111.95750000002},{"questionId":"q85","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4476,"outputTokens":1671,"latencyMs":26929.74866599997},{"questionId":"q85","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3041,"outputTokens":1479,"latencyMs":14530.826667000074},{"questionId":"q86","format":"json-pretty","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":3769,"outputTokens":1543,"latencyMs":13622.04912500002},{"questionId":"q86","format":"json-compact","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":2442,"outputTokens":2055,"latencyMs":23578.416874999995},{"questionId":"q86","format":"toon","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":1640,"outputTokens":3143,"latencyMs":42258.583417000016},{"questionId":"q86","format":"csv","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":1498,"outputTokens":2631,"latencyMs":24667.923749999958},{"questionId":"q86","format":"xml","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":4475,"outputTokens":2247,"latencyMs":20338.54649999994},{"questionId":"q86","format":"yaml","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":3040,"outputTokens":1863,"latencyMs":19668.138083999977},{"questionId":"q87","format":"json-pretty","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3776,"outputTokens":1735,"latencyMs":17175.41733299999},{"questionId":"q87","format":"json-compact","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":2449,"outputTokens":2119,"latencyMs":21621.97758299997},{"questionId":"q87","format":"toon","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1647,"outputTokens":2183,"latencyMs":20844.98749999993},{"questionId":"q87","format":"csv","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1505,"outputTokens":1671,"latencyMs":17007.475209000055},{"questionId":"q87","format":"xml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":4482,"outputTokens":2247,"latencyMs":20809.911709000007},{"questionId":"q87","format":"yaml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3047,"outputTokens":2439,"latencyMs":23743.970708000008},{"questionId":"q88","format":"json-pretty","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":3776,"outputTokens":1479,"latencyMs":13780.006082999986},{"questionId":"q88","format":"json-compact","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":2449,"outputTokens":2567,"latencyMs":28457.375708000036},{"questionId":"q88","format":"toon","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":1647,"outputTokens":1927,"latencyMs":19491.54999999993},{"questionId":"q88","format":"csv","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":1505,"outputTokens":1223,"latencyMs":18888.93787500006},{"questionId":"q88","format":"xml","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":4482,"outputTokens":2055,"latencyMs":19879.96829199989},{"questionId":"q88","format":"yaml","model":"gpt-5-nano","expected":"14","actual":"14","isCorrect":true,"inputTokens":3047,"outputTokens":2439,"latencyMs":37028.00633300003},{"questionId":"q89","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3776,"outputTokens":1799,"latencyMs":20709.313749999972},{"questionId":"q89","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2449,"outputTokens":3143,"latencyMs":28282.831917000003},{"questionId":"q89","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1647,"outputTokens":2055,"latencyMs":19895.249166999944},{"questionId":"q89","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1505,"outputTokens":1927,"latencyMs":24627.73229199997},{"questionId":"q89","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4482,"outputTokens":2631,"latencyMs":26081.29570799996},{"questionId":"q89","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3047,"outputTokens":1991,"latencyMs":19726.860791999963},{"questionId":"q90","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3776,"outputTokens":2567,"latencyMs":24450.25691700005},{"questionId":"q90","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2449,"outputTokens":3079,"latencyMs":30192.66949999996},{"questionId":"q90","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1647,"outputTokens":2695,"latencyMs":30800.806582999998},{"questionId":"q90","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1505,"outputTokens":1735,"latencyMs":17525.293582999962},{"questionId":"q90","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4482,"outputTokens":2503,"latencyMs":24294.877791999956},{"questionId":"q90","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3047,"outputTokens":2887,"latencyMs":27950.61812499992},{"questionId":"q91","format":"json-pretty","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3776,"outputTokens":2695,"latencyMs":41832.30825},{"questionId":"q91","format":"json-compact","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":2449,"outputTokens":2311,"latencyMs":25780.131374999997},{"questionId":"q91","format":"toon","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1647,"outputTokens":2247,"latencyMs":25229.332374999998},{"questionId":"q91","format":"csv","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":1505,"outputTokens":2311,"latencyMs":22338.296957999934},{"questionId":"q91","format":"xml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":4482,"outputTokens":2887,"latencyMs":38730.74291699997},{"questionId":"q91","format":"yaml","model":"gpt-5-nano","expected":"20","actual":"20","isCorrect":true,"inputTokens":3047,"outputTokens":3655,"latencyMs":35328.02654200001},{"questionId":"q92","format":"json-pretty","model":"gpt-5-nano","expected":"32","actual":"33","isCorrect":false,"inputTokens":3775,"outputTokens":2247,"latencyMs":25864.640083999955},{"questionId":"q92","format":"json-compact","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":2448,"outputTokens":2503,"latencyMs":25477.055166000035},{"questionId":"q92","format":"toon","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1646,"outputTokens":3399,"latencyMs":31212.052334000007},{"questionId":"q92","format":"csv","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":1504,"outputTokens":1607,"latencyMs":16476.131416000077},{"questionId":"q92","format":"xml","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":4481,"outputTokens":2695,"latencyMs":40359.28500000003},{"questionId":"q92","format":"yaml","model":"gpt-5-nano","expected":"32","actual":"32","isCorrect":true,"inputTokens":3046,"outputTokens":2439,"latencyMs":30665.69058299996},{"questionId":"q93","format":"json-pretty","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":3775,"outputTokens":1927,"latencyMs":21325.32583300001},{"questionId":"q93","format":"json-compact","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":2448,"outputTokens":2183,"latencyMs":22164.555750000058},{"questionId":"q93","format":"toon","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":1646,"outputTokens":2119,"latencyMs":21662.80249999999},{"questionId":"q93","format":"csv","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":1504,"outputTokens":1927,"latencyMs":20278.839750000043},{"questionId":"q93","format":"xml","model":"gpt-5-nano","expected":"9","actual":"9","isCorrect":true,"inputTokens":4481,"outputTokens":1479,"latencyMs":16767.767083000042},{"questionId":"q93","format":"yaml","model":"gpt-5-nano","expected":"9","actual":"2025-01-17,2025-01-20,2025-01-27,2025-01-28,2025-01-30,2025-02-06,2025-02-10,2025-02-11,2025-02-12","isCorrect":false,"inputTokens":3046,"outputTokens":2180,"latencyMs":27798.925208},{"questionId":"q94","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3778,"outputTokens":2439,"latencyMs":34332.84691700002},{"questionId":"q94","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2451,"outputTokens":2183,"latencyMs":28876.632458999986},{"questionId":"q94","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1649,"outputTokens":3399,"latencyMs":29958.79008299997},{"questionId":"q94","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1507,"outputTokens":3463,"latencyMs":36254.60154099995},{"questionId":"q94","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4484,"outputTokens":3399,"latencyMs":33635.2489169999},{"questionId":"q94","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3049,"outputTokens":2951,"latencyMs":26639.627958000056},{"questionId":"q95","format":"json-pretty","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3778,"outputTokens":2375,"latencyMs":24043.562916999916},{"questionId":"q95","format":"json-compact","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":2451,"outputTokens":3399,"latencyMs":36727.34020800004},{"questionId":"q95","format":"toon","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1649,"outputTokens":2695,"latencyMs":28240.418083999888},{"questionId":"q95","format":"csv","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":1507,"outputTokens":3143,"latencyMs":32131.14024999994},{"questionId":"q95","format":"xml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":4484,"outputTokens":3079,"latencyMs":30882.657292000018},{"questionId":"q95","format":"yaml","model":"gpt-5-nano","expected":"22","actual":"22","isCorrect":true,"inputTokens":3049,"outputTokens":3143,"latencyMs":34199.13054200006},{"questionId":"q96","format":"json-pretty","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":15252,"outputTokens":136,"latencyMs":3174.137834000052},{"questionId":"q96","format":"json-compact","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":11559,"outputTokens":264,"latencyMs":4427.513500000001},{"questionId":"q96","format":"toon","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8873,"outputTokens":328,"latencyMs":4197.75774999999},{"questionId":"q96","format":"csv","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8621,"outputTokens":264,"latencyMs":4365.791124999989},{"questionId":"q96","format":"xml","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17198,"outputTokens":328,"latencyMs":8591.86379199999},{"questionId":"q96","format":"yaml","model":"gpt-5-nano","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":13234,"outputTokens":136,"latencyMs":3006.3902920000255},{"questionId":"q97","format":"json-pretty","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":15254,"outputTokens":328,"latencyMs":5031.756458000047},{"questionId":"q97","format":"json-compact","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":11561,"outputTokens":328,"latencyMs":4921.948124999995},{"questionId":"q97","format":"toon","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8875,"outputTokens":328,"latencyMs":13561.781500000041},{"questionId":"q97","format":"csv","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8623,"outputTokens":328,"latencyMs":11962.30929200002},{"questionId":"q97","format":"xml","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17200,"outputTokens":328,"latencyMs":8242.271916999947},{"questionId":"q97","format":"yaml","model":"gpt-5-nano","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":13236,"outputTokens":264,"latencyMs":7252.942959000007},{"questionId":"q98","format":"json-pretty","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":15249,"outputTokens":264,"latencyMs":4860.508666999987},{"questionId":"q98","format":"json-compact","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":11556,"outputTokens":392,"latencyMs":5948.768499999889},{"questionId":"q98","format":"toon","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8870,"outputTokens":264,"latencyMs":3931.105542000034},{"questionId":"q98","format":"csv","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8618,"outputTokens":648,"latencyMs":11943.225541999913},{"questionId":"q98","format":"xml","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17195,"outputTokens":328,"latencyMs":5136.993999999948},{"questionId":"q98","format":"yaml","model":"gpt-5-nano","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":13231,"outputTokens":264,"latencyMs":3974.5073329999577},{"questionId":"q99","format":"json-pretty","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":15255,"outputTokens":583,"latencyMs":10160.71020900004},{"questionId":"q99","format":"json-compact","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":11562,"outputTokens":519,"latencyMs":10971.125875000027},{"questionId":"q99","format":"toon","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":8876,"outputTokens":327,"latencyMs":4399.097415999975},{"questionId":"q99","format":"csv","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":8624,"outputTokens":327,"latencyMs":5930.175540999975},{"questionId":"q99","format":"xml","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":17201,"outputTokens":391,"latencyMs":7765.284375000047},{"questionId":"q99","format":"yaml","model":"gpt-5-nano","expected":"master","actual":"master","isCorrect":true,"inputTokens":13237,"outputTokens":391,"latencyMs":4973.2529159999685},{"questionId":"q100","format":"json-pretty","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":15249,"outputTokens":136,"latencyMs":3776.490165999974},{"questionId":"q100","format":"json-compact","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":11556,"outputTokens":328,"latencyMs":7673.985999999917},{"questionId":"q100","format":"toon","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8870,"outputTokens":392,"latencyMs":8448.222042000038},{"questionId":"q100","format":"csv","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8618,"outputTokens":328,"latencyMs":5834.679583000019},{"questionId":"q100","format":"xml","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17195,"outputTokens":328,"latencyMs":4700.3877090000315},{"questionId":"q100","format":"yaml","model":"gpt-5-nano","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":13231,"outputTokens":328,"latencyMs":4259.143250000081},{"questionId":"q101","format":"json-pretty","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":15254,"outputTokens":264,"latencyMs":4328.258417000063},{"questionId":"q101","format":"json-compact","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":11561,"outputTokens":264,"latencyMs":3439.5950410000514},{"questionId":"q101","format":"toon","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8875,"outputTokens":584,"latencyMs":6392.925374999992},{"questionId":"q101","format":"csv","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8623,"outputTokens":264,"latencyMs":4369.833875000011},{"questionId":"q101","format":"xml","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17200,"outputTokens":264,"latencyMs":4026.3382080000592},{"questionId":"q101","format":"yaml","model":"gpt-5-nano","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":13236,"outputTokens":328,"latencyMs":3988.631625000038},{"questionId":"q102","format":"json-pretty","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":15253,"outputTokens":263,"latencyMs":6256.574583000038},{"questionId":"q102","format":"json-compact","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":11560,"outputTokens":263,"latencyMs":4265.1440410000505},{"questionId":"q102","format":"toon","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":8874,"outputTokens":455,"latencyMs":10965.912249999936},{"questionId":"q102","format":"csv","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":8622,"outputTokens":391,"latencyMs":7398.446083000046},{"questionId":"q102","format":"xml","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":17199,"outputTokens":391,"latencyMs":5909.86704199994},{"questionId":"q102","format":"yaml","model":"gpt-5-nano","expected":"678","actual":"678","isCorrect":true,"inputTokens":13235,"outputTokens":199,"latencyMs":3187.290791999898},{"questionId":"q103","format":"json-pretty","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":15252,"outputTokens":519,"latencyMs":6490.3748749999795},{"questionId":"q103","format":"json-compact","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":11559,"outputTokens":583,"latencyMs":7666.581874999916},{"questionId":"q103","format":"toon","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":8873,"outputTokens":327,"latencyMs":6757.961624999996},{"questionId":"q103","format":"csv","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":8621,"outputTokens":263,"latencyMs":4993.553249999997},{"questionId":"q103","format":"xml","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":17198,"outputTokens":327,"latencyMs":8451.123458999908},{"questionId":"q103","format":"yaml","model":"gpt-5-nano","expected":"main","actual":"main","isCorrect":true,"inputTokens":13234,"outputTokens":199,"latencyMs":3843.894541000016},{"questionId":"q104","format":"json-pretty","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":15256,"outputTokens":392,"latencyMs":11129.69512499997},{"questionId":"q104","format":"json-compact","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":11563,"outputTokens":392,"latencyMs":8905.517333000083},{"questionId":"q104","format":"toon","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8877,"outputTokens":392,"latencyMs":6319.255083000055},{"questionId":"q104","format":"csv","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8625,"outputTokens":328,"latencyMs":7528.265875000041},{"questionId":"q104","format":"xml","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17202,"outputTokens":392,"latencyMs":13579.722625000053},{"questionId":"q104","format":"yaml","model":"gpt-5-nano","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":13238,"outputTokens":328,"latencyMs":10575.226957999985},{"questionId":"q105","format":"json-pretty","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":15252,"outputTokens":200,"latencyMs":5185.377082999912},{"questionId":"q105","format":"json-compact","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":11559,"outputTokens":264,"latencyMs":4393.0949580000015},{"questionId":"q105","format":"toon","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8873,"outputTokens":456,"latencyMs":6270.737916999962},{"questionId":"q105","format":"csv","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8621,"outputTokens":328,"latencyMs":4558.527000000002},{"questionId":"q105","format":"xml","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17198,"outputTokens":328,"latencyMs":5035.306250000023},{"questionId":"q105","format":"yaml","model":"gpt-5-nano","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":13234,"outputTokens":328,"latencyMs":6407.646999999997},{"questionId":"q106","format":"json-pretty","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":15257,"outputTokens":136,"latencyMs":3376.4645419999724},{"questionId":"q106","format":"json-compact","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":11564,"outputTokens":392,"latencyMs":4399.214457999915},{"questionId":"q106","format":"toon","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8878,"outputTokens":456,"latencyMs":5250.13595799997},{"questionId":"q106","format":"csv","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8626,"outputTokens":584,"latencyMs":10384.269833000028},{"questionId":"q106","format":"xml","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17203,"outputTokens":328,"latencyMs":4593.475250000018},{"questionId":"q106","format":"yaml","model":"gpt-5-nano","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":13239,"outputTokens":328,"latencyMs":9014.461250000051},{"questionId":"q107","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":15248,"outputTokens":2375,"latencyMs":24542.61075000011},{"questionId":"q107","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":11555,"outputTokens":711,"latencyMs":7960.394625000074},{"questionId":"q107","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8869,"outputTokens":199,"latencyMs":2844.534417000017},{"questionId":"q107","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8617,"outputTokens":4167,"latencyMs":85494.36879199988},{"questionId":"q107","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":17194,"outputTokens":2503,"latencyMs":24175.646917000064},{"questionId":"q107","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":13230,"outputTokens":3783,"latencyMs":37382.685666000005},{"questionId":"q108","format":"json-pretty","model":"gpt-5-nano","expected":"15413563","actual":"12284443","isCorrect":false,"inputTokens":15251,"outputTokens":4745,"latencyMs":52110.05108299991},{"questionId":"q108","format":"json-compact","model":"gpt-5-nano","expected":"15413563","actual":"15527138","isCorrect":false,"inputTokens":11558,"outputTokens":9417,"latencyMs":101241.22870800004},{"questionId":"q108","format":"toon","model":"gpt-5-nano","expected":"15413563","actual":"15413563","isCorrect":true,"inputTokens":8872,"outputTokens":5257,"latencyMs":54524.812208000105},{"questionId":"q108","format":"csv","model":"gpt-5-nano","expected":"15413563","actual":"15413563","isCorrect":true,"inputTokens":8620,"outputTokens":5321,"latencyMs":62112.67933299986},{"questionId":"q108","format":"xml","model":"gpt-5-nano","expected":"15413563","actual":"18856320","isCorrect":false,"inputTokens":17197,"outputTokens":11785,"latencyMs":138915.09487500007},{"questionId":"q108","format":"yaml","model":"gpt-5-nano","expected":"15413563","actual":"18617253","isCorrect":false,"inputTokens":13233,"outputTokens":9097,"latencyMs":90010.27312499995},{"questionId":"q109","format":"json-pretty","model":"gpt-5-nano","expected":"2528243","actual":"2528243","isCorrect":true,"inputTokens":15251,"outputTokens":7561,"latencyMs":100665.4815},{"questionId":"q109","format":"json-compact","model":"gpt-5-nano","expected":"2528243","actual":"2528243","isCorrect":true,"inputTokens":11558,"outputTokens":5385,"latencyMs":63163.41504199989},{"questionId":"q109","format":"toon","model":"gpt-5-nano","expected":"2528243","actual":"2528243","isCorrect":true,"inputTokens":8872,"outputTokens":13577,"latencyMs":136019.44545800006},{"questionId":"q109","format":"csv","model":"gpt-5-nano","expected":"2528243","actual":"2831139","isCorrect":false,"inputTokens":8620,"outputTokens":8137,"latencyMs":83168.071},{"questionId":"q109","format":"xml","model":"gpt-5-nano","expected":"2528243","actual":"3500000","isCorrect":false,"inputTokens":17197,"outputTokens":12745,"latencyMs":298602.1220839999},{"questionId":"q109","format":"yaml","model":"gpt-5-nano","expected":"2528243","actual":"11131566","isCorrect":false,"inputTokens":13233,"outputTokens":21577,"latencyMs":197418.5528330002},{"questionId":"q110","format":"json-pretty","model":"gpt-5-nano","expected":"154136","actual":"154135.63","isCorrect":false,"inputTokens":15250,"outputTokens":10762,"latencyMs":100897.18295899988},{"questionId":"q110","format":"json-compact","model":"gpt-5-nano","expected":"154136","actual":"154135.63","isCorrect":false,"inputTokens":11557,"outputTokens":6602,"latencyMs":69045.86083299993},{"questionId":"q110","format":"toon","model":"gpt-5-nano","expected":"154136","actual":"154125.63","isCorrect":false,"inputTokens":8871,"outputTokens":5642,"latencyMs":53949.60254199989},{"questionId":"q110","format":"csv","model":"gpt-5-nano","expected":"154136","actual":"154135.63","isCorrect":false,"inputTokens":8619,"outputTokens":4874,"latencyMs":52207.93591700005},{"questionId":"q110","format":"xml","model":"gpt-5-nano","expected":"154136","actual":"157742.648351648","isCorrect":false,"inputTokens":17196,"outputTokens":4940,"latencyMs":71313.78808299988},{"questionId":"q110","format":"yaml","model":"gpt-5-nano","expected":"154136","actual":"154136","isCorrect":true,"inputTokens":13232,"outputTokens":5704,"latencyMs":55499.88716699998},{"questionId":"q111","format":"json-pretty","model":"gpt-5-nano","expected":"41","actual":"27","isCorrect":false,"inputTokens":15252,"outputTokens":2183,"latencyMs":21596.972249999875},{"questionId":"q111","format":"json-compact","model":"gpt-5-nano","expected":"41","actual":"41","isCorrect":true,"inputTokens":11559,"outputTokens":2631,"latencyMs":30117.775832999963},{"questionId":"q111","format":"toon","model":"gpt-5-nano","expected":"41","actual":"40","isCorrect":false,"inputTokens":8873,"outputTokens":3015,"latencyMs":37560.24325000006},{"questionId":"q111","format":"csv","model":"gpt-5-nano","expected":"41","actual":"40","isCorrect":false,"inputTokens":8621,"outputTokens":3399,"latencyMs":39291.82483300008},{"questionId":"q111","format":"xml","model":"gpt-5-nano","expected":"41","actual":"27","isCorrect":false,"inputTokens":17198,"outputTokens":9607,"latencyMs":96459.7463750001},{"questionId":"q111","format":"yaml","model":"gpt-5-nano","expected":"41","actual":"39","isCorrect":false,"inputTokens":13234,"outputTokens":4743,"latencyMs":42681.62850000011},{"questionId":"q112","format":"json-pretty","model":"gpt-5-nano","expected":"53","actual":"61","isCorrect":false,"inputTokens":15252,"outputTokens":5831,"latencyMs":55109.62650000001},{"questionId":"q112","format":"json-compact","model":"gpt-5-nano","expected":"53","actual":"54","isCorrect":false,"inputTokens":11559,"outputTokens":7175,"latencyMs":94090.2629170001},{"questionId":"q112","format":"toon","model":"gpt-5-nano","expected":"53","actual":"60","isCorrect":false,"inputTokens":8873,"outputTokens":2439,"latencyMs":22994.73654199997},{"questionId":"q112","format":"csv","model":"gpt-5-nano","expected":"53","actual":"59","isCorrect":false,"inputTokens":8621,"outputTokens":7367,"latencyMs":62292.922792},{"questionId":"q112","format":"xml","model":"gpt-5-nano","expected":"53","actual":"60","isCorrect":false,"inputTokens":17198,"outputTokens":7687,"latencyMs":68955.88562500011},{"questionId":"q112","format":"yaml","model":"gpt-5-nano","expected":"53","actual":"53","isCorrect":true,"inputTokens":13234,"outputTokens":2759,"latencyMs":25175.004457999952},{"questionId":"q113","format":"json-pretty","model":"gpt-5-nano","expected":"77","actual":"77","isCorrect":true,"inputTokens":15251,"outputTokens":2567,"latencyMs":31751.425124999834},{"questionId":"q113","format":"json-compact","model":"gpt-5-nano","expected":"77","actual":"64","isCorrect":false,"inputTokens":11558,"outputTokens":2695,"latencyMs":24752.101749999914},{"questionId":"q113","format":"toon","model":"gpt-5-nano","expected":"77","actual":"77","isCorrect":true,"inputTokens":8872,"outputTokens":4039,"latencyMs":58847.84733400005},{"questionId":"q113","format":"csv","model":"gpt-5-nano","expected":"77","actual":"77","isCorrect":true,"inputTokens":8620,"outputTokens":3015,"latencyMs":28625.518792000134},{"questionId":"q113","format":"xml","model":"gpt-5-nano","expected":"77","actual":"73","isCorrect":false,"inputTokens":17197,"outputTokens":4999,"latencyMs":43475.059833999956},{"questionId":"q113","format":"yaml","model":"gpt-5-nano","expected":"77","actual":"74","isCorrect":false,"inputTokens":13233,"outputTokens":15879,"latencyMs":133066.7952080001},{"questionId":"q114","format":"json-pretty","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":15251,"outputTokens":1607,"latencyMs":17589.96158400015},{"questionId":"q114","format":"json-compact","model":"gpt-5-nano","expected":"37","actual":"36","isCorrect":false,"inputTokens":11558,"outputTokens":10759,"latencyMs":91386.12729199999},{"questionId":"q114","format":"toon","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":8872,"outputTokens":2951,"latencyMs":28628.738499999978},{"questionId":"q114","format":"csv","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":8620,"outputTokens":2055,"latencyMs":16901.366458999924},{"questionId":"q114","format":"xml","model":"gpt-5-nano","expected":"37","actual":"27","isCorrect":false,"inputTokens":17197,"outputTokens":1863,"latencyMs":16063.757082999917},{"questionId":"q114","format":"yaml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":13233,"outputTokens":3719,"latencyMs":40554.34516599984},{"questionId":"q115","format":"json-pretty","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":15251,"outputTokens":967,"latencyMs":15059.88116599992},{"questionId":"q115","format":"json-compact","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":11558,"outputTokens":1095,"latencyMs":10164.44991599978},{"questionId":"q115","format":"toon","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":8872,"outputTokens":3719,"latencyMs":44044.071624999866},{"questionId":"q115","format":"csv","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":8620,"outputTokens":2183,"latencyMs":19417.32862499985},{"questionId":"q115","format":"xml","model":"gpt-5-nano","expected":"16","actual":"15","isCorrect":false,"inputTokens":17197,"outputTokens":1607,"latencyMs":15047.773334000027},{"questionId":"q115","format":"yaml","model":"gpt-5-nano","expected":"16","actual":"16","isCorrect":true,"inputTokens":13233,"outputTokens":1543,"latencyMs":36046.026458000066},{"questionId":"q116","format":"json-pretty","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":15251,"outputTokens":3271,"latencyMs":76326.07350000017},{"questionId":"q116","format":"json-compact","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":11558,"outputTokens":3655,"latencyMs":32130.930374999996},{"questionId":"q116","format":"toon","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":8872,"outputTokens":12615,"latencyMs":155529.18491700012},{"questionId":"q116","format":"csv","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":8620,"outputTokens":6407,"latencyMs":55902.06070800009},{"questionId":"q116","format":"xml","model":"gpt-5-nano","expected":"49","actual":"106","isCorrect":false,"inputTokens":17197,"outputTokens":7495,"latencyMs":64000.08562499983},{"questionId":"q116","format":"yaml","model":"gpt-5-nano","expected":"49","actual":"49","isCorrect":true,"inputTokens":13233,"outputTokens":3591,"latencyMs":31902.165125000058},{"questionId":"q117","format":"json-pretty","model":"gpt-5-nano","expected":"23","actual":"29","isCorrect":false,"inputTokens":15251,"outputTokens":7751,"latencyMs":65168.02249999996},{"questionId":"q117","format":"json-compact","model":"gpt-5-nano","expected":"23","actual":"21","isCorrect":false,"inputTokens":11558,"outputTokens":3207,"latencyMs":28594.051167000085},{"questionId":"q117","format":"toon","model":"gpt-5-nano","expected":"23","actual":"23","isCorrect":true,"inputTokens":8872,"outputTokens":10503,"latencyMs":98151.87975000008},{"questionId":"q117","format":"csv","model":"gpt-5-nano","expected":"23","actual":"21","isCorrect":false,"inputTokens":8620,"outputTokens":2183,"latencyMs":20484.786165999947},{"questionId":"q117","format":"xml","model":"gpt-5-nano","expected":"23","actual":"21","isCorrect":false,"inputTokens":17197,"outputTokens":4615,"latencyMs":43518.77370800008},{"questionId":"q117","format":"yaml","model":"gpt-5-nano","expected":"23","actual":"23","isCorrect":true,"inputTokens":13233,"outputTokens":3975,"latencyMs":35627.89633300016},{"questionId":"q118","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":15251,"outputTokens":3015,"latencyMs":26725.319833999965},{"questionId":"q118","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":11558,"outputTokens":1799,"latencyMs":18761.737124999985},{"questionId":"q118","format":"toon","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":8872,"outputTokens":2759,"latencyMs":24030.234415999847},{"questionId":"q118","format":"csv","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":8620,"outputTokens":2311,"latencyMs":20654.35191700002},{"questionId":"q118","format":"xml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":17197,"outputTokens":1799,"latencyMs":17041.010666999966},{"questionId":"q118","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"5","isCorrect":false,"inputTokens":13233,"outputTokens":3463,"latencyMs":33392.745709000155},{"questionId":"q119","format":"json-pretty","model":"gpt-5-nano","expected":"57","actual":"56","isCorrect":false,"inputTokens":15258,"outputTokens":4103,"latencyMs":38199.90779100009},{"questionId":"q119","format":"json-compact","model":"gpt-5-nano","expected":"57","actual":"73","isCorrect":false,"inputTokens":11565,"outputTokens":12743,"latencyMs":131865.93516699993},{"questionId":"q119","format":"toon","model":"gpt-5-nano","expected":"57","actual":"57","isCorrect":true,"inputTokens":8879,"outputTokens":4167,"latencyMs":39757.292166},{"questionId":"q119","format":"csv","model":"gpt-5-nano","expected":"57","actual":"57","isCorrect":true,"inputTokens":8627,"outputTokens":7239,"latencyMs":81548.7545830002},{"questionId":"q119","format":"xml","model":"gpt-5-nano","expected":"57","actual":"60","isCorrect":false,"inputTokens":17204,"outputTokens":15303,"latencyMs":134679.453584},{"questionId":"q119","format":"yaml","model":"gpt-5-nano","expected":"57","actual":"57","isCorrect":true,"inputTokens":13240,"outputTokens":4167,"latencyMs":38586.18929200014},{"questionId":"q120","format":"json-pretty","model":"gpt-5-nano","expected":"43","actual":"57","isCorrect":false,"inputTokens":15258,"outputTokens":6407,"latencyMs":74895.58804200008},{"questionId":"q120","format":"json-compact","model":"gpt-5-nano","expected":"43","actual":"32","isCorrect":false,"inputTokens":11565,"outputTokens":3015,"latencyMs":35560.51104200003},{"questionId":"q120","format":"toon","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":8879,"outputTokens":5191,"latencyMs":56691.20683400007},{"questionId":"q120","format":"csv","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":8627,"outputTokens":4231,"latencyMs":38942.962958000135},{"questionId":"q120","format":"xml","model":"gpt-5-nano","expected":"43","actual":"39","isCorrect":false,"inputTokens":17204,"outputTokens":11847,"latencyMs":143298.53979200008},{"questionId":"q120","format":"yaml","model":"gpt-5-nano","expected":"43","actual":"43","isCorrect":true,"inputTokens":13240,"outputTokens":5767,"latencyMs":58260.77429199987},{"questionId":"q121","format":"json-pretty","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":15258,"outputTokens":2951,"latencyMs":30454.88316700002},{"questionId":"q121","format":"json-compact","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":11565,"outputTokens":4423,"latencyMs":44076.22399999993},{"questionId":"q121","format":"toon","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":8879,"outputTokens":4231,"latencyMs":37241.785167000024},{"questionId":"q121","format":"csv","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":8627,"outputTokens":5895,"latencyMs":90620.1480419999},{"questionId":"q121","format":"xml","model":"gpt-5-nano","expected":"25","actual":"18","isCorrect":false,"inputTokens":17204,"outputTokens":3975,"latencyMs":40301.10383399995},{"questionId":"q121","format":"yaml","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":13240,"outputTokens":3911,"latencyMs":53672.49758299999},{"questionId":"q122","format":"json-pretty","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":15258,"outputTokens":3463,"latencyMs":35417.804375000065},{"questionId":"q122","format":"json-compact","model":"gpt-5-nano","expected":"6","actual":"5","isCorrect":false,"inputTokens":11565,"outputTokens":4487,"latencyMs":46868.0803749999},{"questionId":"q122","format":"toon","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":8879,"outputTokens":4551,"latencyMs":45573.12279199995},{"questionId":"q122","format":"csv","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":8627,"outputTokens":2887,"latencyMs":28422.857124999864},{"questionId":"q122","format":"xml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":17204,"outputTokens":3143,"latencyMs":29955.825083999895},{"questionId":"q122","format":"yaml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":13240,"outputTokens":5575,"latencyMs":48339.364959000144},{"questionId":"q123","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":15258,"outputTokens":3143,"latencyMs":25295.857915999833},{"questionId":"q123","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":11565,"outputTokens":3079,"latencyMs":29557.22145800013},{"questionId":"q123","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":8879,"outputTokens":2503,"latencyMs":22121.652333999984},{"questionId":"q123","format":"csv","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":8627,"outputTokens":2695,"latencyMs":31378.58437499986},{"questionId":"q123","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":17204,"outputTokens":1799,"latencyMs":20389.638000000035},{"questionId":"q123","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":13240,"outputTokens":3079,"latencyMs":28901.529666000046},{"questionId":"q124","format":"json-pretty","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":6905,"outputTokens":263,"latencyMs":3115.7104579999577},{"questionId":"q124","format":"json-compact","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":4927,"outputTokens":327,"latencyMs":7230.5603749998845},{"questionId":"q124","format":"toon","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5906,"outputTokens":263,"latencyMs":6441.958833999932},{"questionId":"q124","format":"xml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":7815,"outputTokens":199,"latencyMs":3481.11870799982},{"questionId":"q124","format":"yaml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5933,"outputTokens":135,"latencyMs":2440.1648330001626},{"questionId":"q125","format":"json-pretty","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6905,"outputTokens":841,"latencyMs":9103.615125000011},{"questionId":"q125","format":"json-compact","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4927,"outputTokens":1097,"latencyMs":10853.658833999885},{"questionId":"q125","format":"toon","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5906,"outputTokens":393,"latencyMs":4811.6409579999745},{"questionId":"q125","format":"xml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7815,"outputTokens":265,"latencyMs":3462.576958999969},{"questionId":"q125","format":"yaml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5933,"outputTokens":201,"latencyMs":4551.6412080000155},{"questionId":"q126","format":"json-pretty","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":6906,"outputTokens":199,"latencyMs":5946.616957999999},{"questionId":"q126","format":"json-compact","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":4928,"outputTokens":391,"latencyMs":4862.764540999895},{"questionId":"q126","format":"toon","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":5907,"outputTokens":199,"latencyMs":4436.1078329999},{"questionId":"q126","format":"xml","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":7816,"outputTokens":263,"latencyMs":3710.672332999995},{"questionId":"q126","format":"yaml","model":"gpt-5-nano","expected":"424","actual":"424","isCorrect":true,"inputTokens":5934,"outputTokens":263,"latencyMs":3584.6445420000236},{"questionId":"q127","format":"json-pretty","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6906,"outputTokens":264,"latencyMs":3332.7081249998882},{"questionId":"q127","format":"json-compact","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":4928,"outputTokens":328,"latencyMs":6029.872375000035},{"questionId":"q127","format":"toon","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5907,"outputTokens":264,"latencyMs":6360.82320899982},{"questionId":"q127","format":"xml","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7816,"outputTokens":328,"latencyMs":4304.171290999977},{"questionId":"q127","format":"yaml","model":"gpt-5-nano","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":5934,"outputTokens":328,"latencyMs":6850.6273750001565},{"questionId":"q128","format":"json-pretty","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":6905,"outputTokens":1095,"latencyMs":25444.60245799995},{"questionId":"q128","format":"json-compact","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":4927,"outputTokens":839,"latencyMs":11782.655000000028},{"questionId":"q128","format":"toon","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5906,"outputTokens":263,"latencyMs":5465.98116700002},{"questionId":"q128","format":"xml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":7815,"outputTokens":263,"latencyMs":3371.2434590000194},{"questionId":"q128","format":"yaml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5933,"outputTokens":263,"latencyMs":3541.5000830001663},{"questionId":"q129","format":"json-pretty","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6905,"outputTokens":905,"latencyMs":9463.865125000011},{"questionId":"q129","format":"json-compact","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":4927,"outputTokens":649,"latencyMs":6622.134208000032},{"questionId":"q129","format":"toon","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5906,"outputTokens":329,"latencyMs":4798.912999999942},{"questionId":"q129","format":"xml","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7815,"outputTokens":265,"latencyMs":5167.431041999953},{"questionId":"q129","format":"yaml","model":"gpt-5-nano","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":5933,"outputTokens":393,"latencyMs":4693.827333000023},{"questionId":"q130","format":"json-pretty","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":6906,"outputTokens":199,"latencyMs":2821.266375000123},{"questionId":"q130","format":"json-compact","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":4928,"outputTokens":839,"latencyMs":8631.246000000043},{"questionId":"q130","format":"toon","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":5907,"outputTokens":327,"latencyMs":4855.562292000046},{"questionId":"q130","format":"xml","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":7816,"outputTokens":519,"latencyMs":7240.806624999968},{"questionId":"q130","format":"yaml","model":"gpt-5-nano","expected":"435","actual":"435","isCorrect":true,"inputTokens":5934,"outputTokens":1031,"latencyMs":10435.050374999875},{"questionId":"q131","format":"json-pretty","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":6906,"outputTokens":199,"latencyMs":3034.7289579999633},{"questionId":"q131","format":"json-compact","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":4928,"outputTokens":327,"latencyMs":3241.3320420000236},{"questionId":"q131","format":"toon","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":5907,"outputTokens":391,"latencyMs":5222.304125000024},{"questionId":"q131","format":"xml","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":7816,"outputTokens":263,"latencyMs":3285.6503329998814},{"questionId":"q131","format":"yaml","model":"gpt-5-nano","expected":"408","actual":"408","isCorrect":true,"inputTokens":5934,"outputTokens":135,"latencyMs":3403.779457999859},{"questionId":"q132","format":"json-pretty","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":6905,"outputTokens":967,"latencyMs":12630.437167000026},{"questionId":"q132","format":"json-compact","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":4927,"outputTokens":1095,"latencyMs":12474.425874999957},{"questionId":"q132","format":"toon","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5906,"outputTokens":1863,"latencyMs":15583.305916999932},{"questionId":"q132","format":"xml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":7815,"outputTokens":455,"latencyMs":6376.325249999994},{"questionId":"q132","format":"yaml","model":"gpt-5-nano","expected":"error","actual":"error","isCorrect":true,"inputTokens":5933,"outputTokens":327,"latencyMs":5189.892333999975},{"questionId":"q133","format":"json-pretty","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6905,"outputTokens":457,"latencyMs":4789.633792000124},{"questionId":"q133","format":"json-compact","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4927,"outputTokens":969,"latencyMs":9603.09604199999},{"questionId":"q133","format":"toon","model":"gpt-5-nano","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":5906,"outputTokens":265,"latencyMs":3472.784415999893},{"questionId":"q133","format":"xml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/payments","isCorrect":false,"inputTokens":7815,"outputTokens":1098,"latencyMs":14847.909750000108},{"questionId":"q133","format":"yaml","model":"gpt-5-nano","expected":"/api/users","actual":"/api/payments","isCorrect":false,"inputTokens":5933,"outputTokens":1354,"latencyMs":14238.748833000194},{"questionId":"q134","format":"json-pretty","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":6889,"outputTokens":3719,"latencyMs":29588.24908400001},{"questionId":"q134","format":"json-compact","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":4911,"outputTokens":5319,"latencyMs":40931.71183300018},{"questionId":"q134","format":"toon","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":5890,"outputTokens":391,"latencyMs":5362.043415999971},{"questionId":"q134","format":"xml","model":"gpt-5-nano","expected":"75","actual":"100","isCorrect":false,"inputTokens":7799,"outputTokens":2247,"latencyMs":34099.03204199998},{"questionId":"q134","format":"yaml","model":"gpt-5-nano","expected":"75","actual":"100","isCorrect":false,"inputTokens":5917,"outputTokens":20167,"latencyMs":191462.27824999997},{"questionId":"q135","format":"json-pretty","model":"gpt-5-nano","expected":"2453.41","actual":"2413.3866666667","isCorrect":false,"inputTokens":6890,"outputTokens":10189,"latencyMs":114932.37154199998},{"questionId":"q135","format":"json-compact","model":"gpt-5-nano","expected":"2453.41","actual":"2344","isCorrect":false,"inputTokens":4912,"outputTokens":12488,"latencyMs":178401.16920799995},{"questionId":"q135","format":"toon","model":"gpt-5-nano","expected":"2453.41","actual":"2399.5942028985507","isCorrect":false,"inputTokens":5891,"outputTokens":12494,"latencyMs":106734.32024999987},{"questionId":"q135","format":"xml","model":"gpt-5-nano","expected":"2453.41","actual":"2453.4133333333","isCorrect":true,"inputTokens":7800,"outputTokens":9613,"latencyMs":79749.86854199995},{"questionId":"q135","format":"yaml","model":"gpt-5-nano","expected":"2453.41","actual":"2513","isCorrect":false,"inputTokens":5918,"outputTokens":9352,"latencyMs":79398.59566699993},{"questionId":"q136","format":"json-pretty","model":"gpt-5-nano","expected":"29","actual":"29","isCorrect":true,"inputTokens":6889,"outputTokens":4615,"latencyMs":42187.74462500005},{"questionId":"q136","format":"json-compact","model":"gpt-5-nano","expected":"29","actual":"29","isCorrect":true,"inputTokens":4911,"outputTokens":3527,"latencyMs":32269.509624999948},{"questionId":"q136","format":"toon","model":"gpt-5-nano","expected":"29","actual":"23","isCorrect":false,"inputTokens":5890,"outputTokens":9671,"latencyMs":76910.37262500008},{"questionId":"q136","format":"xml","model":"gpt-5-nano","expected":"29","actual":"28","isCorrect":false,"inputTokens":7799,"outputTokens":10887,"latencyMs":88385.73287499999},{"questionId":"q136","format":"yaml","model":"gpt-5-nano","expected":"29","actual":"33","isCorrect":false,"inputTokens":5917,"outputTokens":13319,"latencyMs":135069.12662500003},{"questionId":"q137","format":"json-pretty","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":6889,"outputTokens":2503,"latencyMs":24155.693625000073},{"questionId":"q137","format":"json-compact","model":"gpt-5-nano","expected":"17","actual":"17","isCorrect":true,"inputTokens":4911,"outputTokens":8519,"latencyMs":80310.74116600002},{"questionId":"q137","format":"toon","model":"gpt-5-nano","expected":"17","actual":"13","isCorrect":false,"inputTokens":5890,"outputTokens":6855,"latencyMs":60660.83295800001},{"questionId":"q137","format":"xml","model":"gpt-5-nano","expected":"17","actual":"13","isCorrect":false,"inputTokens":7799,"outputTokens":3847,"latencyMs":60666.268124999944},{"questionId":"q137","format":"yaml","model":"gpt-5-nano","expected":"17","actual":"14","isCorrect":false,"inputTokens":5917,"outputTokens":7303,"latencyMs":57974.093916999875},{"questionId":"q138","format":"json-pretty","model":"gpt-5-nano","expected":"29","actual":"23","isCorrect":false,"inputTokens":6889,"outputTokens":3463,"latencyMs":38306.33962500002},{"questionId":"q138","format":"json-compact","model":"gpt-5-nano","expected":"29","actual":"29","isCorrect":true,"inputTokens":4911,"outputTokens":3591,"latencyMs":29538.051624999847},{"questionId":"q138","format":"toon","model":"gpt-5-nano","expected":"29","actual":"38","isCorrect":false,"inputTokens":5890,"outputTokens":15815,"latencyMs":129636.9376660001},{"questionId":"q138","format":"xml","model":"gpt-5-nano","expected":"29","actual":"22","isCorrect":false,"inputTokens":7799,"outputTokens":10503,"latencyMs":76536.67662499985},{"questionId":"q138","format":"yaml","model":"gpt-5-nano","expected":"29","actual":"50","isCorrect":false,"inputTokens":5917,"outputTokens":13319,"latencyMs":151121.41308299988},{"questionId":"q139","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":6891,"outputTokens":4103,"latencyMs":56042.69449999998},{"questionId":"q139","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"10","isCorrect":false,"inputTokens":4913,"outputTokens":4743,"latencyMs":37126.876166999806},{"questionId":"q139","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5892,"outputTokens":5639,"latencyMs":40804.70775000006},{"questionId":"q139","format":"xml","model":"gpt-5-nano","expected":"11","actual":"6","isCorrect":false,"inputTokens":7801,"outputTokens":9799,"latencyMs":92226.40016699978},{"questionId":"q139","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":5919,"outputTokens":12039,"latencyMs":93280.16320900014},{"questionId":"q140","format":"json-pretty","model":"gpt-5-nano","expected":"18","actual":"18","isCorrect":true,"inputTokens":6891,"outputTokens":4295,"latencyMs":41815.3666660001},{"questionId":"q140","format":"json-compact","model":"gpt-5-nano","expected":"18","actual":"18","isCorrect":true,"inputTokens":4913,"outputTokens":3335,"latencyMs":29692.3865410001},{"questionId":"q140","format":"toon","model":"gpt-5-nano","expected":"18","actual":"17","isCorrect":false,"inputTokens":5892,"outputTokens":6599,"latencyMs":97809.67475},{"questionId":"q140","format":"xml","model":"gpt-5-nano","expected":"18","actual":"15","isCorrect":false,"inputTokens":7801,"outputTokens":12039,"latencyMs":123416.02841699985},{"questionId":"q140","format":"yaml","model":"gpt-5-nano","expected":"18","actual":"17","isCorrect":false,"inputTokens":5919,"outputTokens":10567,"latencyMs":91551.0178749999},{"questionId":"q141","format":"json-pretty","model":"gpt-5-nano","expected":"33","actual":"33","isCorrect":true,"inputTokens":6896,"outputTokens":3271,"latencyMs":30817.282166999998},{"questionId":"q141","format":"json-compact","model":"gpt-5-nano","expected":"33","actual":"33","isCorrect":true,"inputTokens":4918,"outputTokens":5703,"latencyMs":63405.342667000135},{"questionId":"q141","format":"toon","model":"gpt-5-nano","expected":"33","actual":"34","isCorrect":false,"inputTokens":5897,"outputTokens":14151,"latencyMs":112034.90975000011},{"questionId":"q141","format":"xml","model":"gpt-5-nano","expected":"33","actual":"37","isCorrect":false,"inputTokens":7806,"outputTokens":15175,"latencyMs":145415.13762499997},{"questionId":"q141","format":"yaml","model":"gpt-5-nano","expected":"33","actual":"63","isCorrect":false,"inputTokens":5924,"outputTokens":18759,"latencyMs":151461.663834},{"questionId":"q142","format":"json-pretty","model":"gpt-5-nano","expected":"42","actual":"43","isCorrect":false,"inputTokens":6894,"outputTokens":5959,"latencyMs":47470.96787500009},{"questionId":"q142","format":"json-compact","model":"gpt-5-nano","expected":"42","actual":"42","isCorrect":true,"inputTokens":4916,"outputTokens":5383,"latencyMs":53329.43183299992},{"questionId":"q142","format":"toon","model":"gpt-5-nano","expected":"42","actual":"43","isCorrect":false,"inputTokens":5895,"outputTokens":10375,"latencyMs":85387.93562499993},{"questionId":"q142","format":"xml","model":"gpt-5-nano","expected":"42","actual":"60","isCorrect":false,"inputTokens":7804,"outputTokens":12167,"latencyMs":93500.84408300021},{"questionId":"q142","format":"yaml","model":"gpt-5-nano","expected":"42","actual":"80","isCorrect":false,"inputTokens":5922,"outputTokens":9671,"latencyMs":379540.6658340001},{"questionId":"q143","format":"json-pretty","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":6890,"outputTokens":4743,"latencyMs":38821.55270899995},{"questionId":"q143","format":"json-compact","model":"gpt-5-nano","expected":"25","actual":"25","isCorrect":true,"inputTokens":4912,"outputTokens":6727,"latencyMs":57143.98108300008},{"questionId":"q143","format":"toon","model":"gpt-5-nano","expected":"25","actual":"27","isCorrect":false,"inputTokens":5891,"outputTokens":14407,"latencyMs":121313.45120800007},{"questionId":"q143","format":"xml","model":"gpt-5-nano","expected":"25","actual":"19","isCorrect":false,"inputTokens":7800,"outputTokens":7559,"latencyMs":69642.35850000009},{"questionId":"q143","format":"yaml","model":"gpt-5-nano","expected":"25","actual":"17","isCorrect":false,"inputTokens":5918,"outputTokens":16135,"latencyMs":161343.49933400005},{"questionId":"q144","format":"json-pretty","model":"gpt-5-nano","expected":"29","actual":"34","isCorrect":false,"inputTokens":6896,"outputTokens":20167,"latencyMs":230741.27374999993},{"questionId":"q144","format":"json-compact","model":"gpt-5-nano","expected":"29","actual":"29","isCorrect":true,"inputTokens":4918,"outputTokens":7495,"latencyMs":63636.350584000116},{"questionId":"q144","format":"toon","model":"gpt-5-nano","expected":"29","actual":"36","isCorrect":false,"inputTokens":5897,"outputTokens":16263,"latencyMs":130179.59895799984},{"questionId":"q144","format":"xml","model":"gpt-5-nano","expected":"29","actual":"37","isCorrect":false,"inputTokens":7806,"outputTokens":11591,"latencyMs":113961.67562500015},{"questionId":"q144","format":"yaml","model":"gpt-5-nano","expected":"29","actual":"37","isCorrect":false,"inputTokens":5924,"outputTokens":16519,"latencyMs":140941.021834},{"questionId":"q145","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":6896,"outputTokens":5447,"latencyMs":46888.73112499993},{"questionId":"q145","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":4918,"outputTokens":7111,"latencyMs":74291.24454099988},{"questionId":"q145","format":"toon","model":"gpt-5-nano","expected":"4","actual":"3","isCorrect":false,"inputTokens":5897,"outputTokens":4423,"latencyMs":38978.35370799992},{"questionId":"q145","format":"xml","model":"gpt-5-nano","expected":"4","actual":"3","isCorrect":false,"inputTokens":7806,"outputTokens":4039,"latencyMs":36546.21775000007},{"questionId":"q145","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":5924,"outputTokens":4167,"latencyMs":39038.52904099994},{"questionId":"q146","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"4","isCorrect":false,"inputTokens":6898,"outputTokens":3015,"latencyMs":26637.197584000183},{"questionId":"q146","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":4920,"outputTokens":2887,"latencyMs":26040.22404200025},{"questionId":"q146","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5899,"outputTokens":3463,"latencyMs":29262.439125000034},{"questionId":"q146","format":"xml","model":"gpt-5-nano","expected":"5","actual":"6","isCorrect":false,"inputTokens":7808,"outputTokens":6215,"latencyMs":58986.4326249999},{"questionId":"q146","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5926,"outputTokens":4743,"latencyMs":38633.85925000021},{"questionId":"q147","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":6898,"outputTokens":3079,"latencyMs":27888.549333999865},{"questionId":"q147","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":4920,"outputTokens":4679,"latencyMs":43776.82637499971},{"questionId":"q147","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":5899,"outputTokens":5447,"latencyMs":50479.701083000284},{"questionId":"q147","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":7808,"outputTokens":4999,"latencyMs":58853.683209000155},{"questionId":"q147","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":5926,"outputTokens":3655,"latencyMs":31553.769249999896},{"questionId":"q148","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":6898,"outputTokens":2759,"latencyMs":47123.68912500003},{"questionId":"q148","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":4920,"outputTokens":4359,"latencyMs":41352.168209000025},{"questionId":"q148","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":5899,"outputTokens":6343,"latencyMs":55856.09641700005},{"questionId":"q148","format":"xml","model":"gpt-5-nano","expected":"3","actual":"2","isCorrect":false,"inputTokens":7808,"outputTokens":4103,"latencyMs":35685.38595899986},{"questionId":"q148","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":5926,"outputTokens":6343,"latencyMs":51239.12116699992},{"questionId":"q149","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":6896,"outputTokens":6855,"latencyMs":90414.68912500003},{"questionId":"q149","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":4918,"outputTokens":2695,"latencyMs":27534.36641700007},{"questionId":"q149","format":"toon","model":"gpt-5-nano","expected":"4","actual":"3","isCorrect":false,"inputTokens":5897,"outputTokens":4487,"latencyMs":42209.12216599984},{"questionId":"q149","format":"xml","model":"gpt-5-nano","expected":"4","actual":"3","isCorrect":false,"inputTokens":7806,"outputTokens":6279,"latencyMs":75733.36095800018},{"questionId":"q149","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":5924,"outputTokens":7687,"latencyMs":63058.29333399981},{"questionId":"q150","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":6896,"outputTokens":4103,"latencyMs":40312.71462500002},{"questionId":"q150","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":4918,"outputTokens":4167,"latencyMs":35873.915792000014},{"questionId":"q150","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":5897,"outputTokens":5703,"latencyMs":57978.6901250002},{"questionId":"q150","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":7806,"outputTokens":6023,"latencyMs":54796.04729200015},{"questionId":"q150","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"4","isCorrect":false,"inputTokens":5924,"outputTokens":5127,"latencyMs":43498.875916999765},{"questionId":"q151","format":"json-pretty","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":1023,"outputTokens":583,"latencyMs":6742.807415999938},{"questionId":"q151","format":"json-compact","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":665,"outputTokens":455,"latencyMs":8014.11641700007},{"questionId":"q151","format":"toon","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":756,"outputTokens":135,"latencyMs":3002.658749999944},{"questionId":"q151","format":"xml","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":1107,"outputTokens":135,"latencyMs":2293.2929159998894},{"questionId":"q151","format":"yaml","model":"gpt-5-nano","expected":"development","actual":"development","isCorrect":true,"inputTokens":775,"outputTokens":135,"latencyMs":3505.548708999995},{"questionId":"q152","format":"json-pretty","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1021,"outputTokens":268,"latencyMs":3357.363291999791},{"questionId":"q152","format":"json-compact","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":663,"outputTokens":396,"latencyMs":4544.950250000227},{"questionId":"q152","format":"toon","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":754,"outputTokens":204,"latencyMs":3910.6737079997547},{"questionId":"q152","format":"xml","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1105,"outputTokens":140,"latencyMs":2807.8532920000143},{"questionId":"q152","format":"yaml","model":"gpt-5-nano","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":773,"outputTokens":204,"latencyMs":6755.754457999952},{"questionId":"q153","format":"json-pretty","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1021,"outputTokens":136,"latencyMs":2664.359124999959},{"questionId":"q153","format":"json-compact","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":663,"outputTokens":136,"latencyMs":2253.774665999692},{"questionId":"q153","format":"toon","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":754,"outputTokens":200,"latencyMs":5005.998707999941},{"questionId":"q153","format":"xml","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1105,"outputTokens":72,"latencyMs":2703.680333000142},{"questionId":"q153","format":"yaml","model":"gpt-5-nano","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":773,"outputTokens":72,"latencyMs":1897.0315409996547},{"questionId":"q154","format":"json-pretty","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":1023,"outputTokens":135,"latencyMs":3792.394541000016},{"questionId":"q154","format":"json-compact","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":665,"outputTokens":135,"latencyMs":4632.95924999984},{"questionId":"q154","format":"toon","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":756,"outputTokens":135,"latencyMs":2394.134250000119},{"questionId":"q154","format":"xml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":1107,"outputTokens":135,"latencyMs":2430.5971249998547},{"questionId":"q154","format":"yaml","model":"gpt-5-nano","expected":"37","actual":"37","isCorrect":true,"inputTokens":775,"outputTokens":135,"latencyMs":2441.4810000001453},{"questionId":"q155","format":"json-pretty","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1021,"outputTokens":72,"latencyMs":1814.3805419998243},{"questionId":"q155","format":"json-compact","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":663,"outputTokens":200,"latencyMs":3557.1527090002783},{"questionId":"q155","format":"toon","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":754,"outputTokens":136,"latencyMs":3990.5303329997696},{"questionId":"q155","format":"xml","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1105,"outputTokens":136,"latencyMs":3151.5690419999883},{"questionId":"q155","format":"yaml","model":"gpt-5-nano","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":773,"outputTokens":136,"latencyMs":2329.0516249998473},{"questionId":"q156","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1023,"outputTokens":135,"latencyMs":1982.217999999877},{"questionId":"q156","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":665,"outputTokens":199,"latencyMs":2489.357166999951},{"questionId":"q156","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":756,"outputTokens":199,"latencyMs":3184.5403330000117},{"questionId":"q156","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1107,"outputTokens":199,"latencyMs":3035.3530419999734},{"questionId":"q156","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":775,"outputTokens":199,"latencyMs":2576.0412079999223},{"questionId":"q157","format":"json-pretty","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1023,"outputTokens":136,"latencyMs":15388.45091599971},{"questionId":"q157","format":"json-compact","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":665,"outputTokens":200,"latencyMs":3027.245583000127},{"questionId":"q157","format":"toon","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":756,"outputTokens":200,"latencyMs":2682.324666999746},{"questionId":"q157","format":"xml","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1107,"outputTokens":72,"latencyMs":2551.2237090002745},{"questionId":"q157","format":"yaml","model":"gpt-5-nano","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":775,"outputTokens":136,"latencyMs":4109.2170409997925},{"questionId":"q158","format":"json-pretty","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":1021,"outputTokens":199,"latencyMs":3716.7424159999937},{"questionId":"q158","format":"json-compact","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":663,"outputTokens":455,"latencyMs":6808.871625000145},{"questionId":"q158","format":"toon","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":754,"outputTokens":327,"latencyMs":4373.244665999897},{"questionId":"q158","format":"xml","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":1105,"outputTokens":135,"latencyMs":2383.4876660001464},{"questionId":"q158","format":"yaml","model":"gpt-5-nano","expected":"real","actual":"real","isCorrect":true,"inputTokens":773,"outputTokens":263,"latencyMs":3284.6608330002055},{"questionId":"q159","format":"json-pretty","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1022,"outputTokens":136,"latencyMs":2568.9396250001155},{"questionId":"q159","format":"json-compact","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":664,"outputTokens":136,"latencyMs":2548.5491659999825},{"questionId":"q159","format":"toon","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":755,"outputTokens":200,"latencyMs":2528.12133300025},{"questionId":"q159","format":"xml","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1106,"outputTokens":200,"latencyMs":3261.567334000021},{"questionId":"q159","format":"yaml","model":"gpt-5-nano","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":774,"outputTokens":264,"latencyMs":5840.089374999981},{"questionId":"q160","format":"json-pretty","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1023,"outputTokens":459,"latencyMs":4751.093458000105},{"questionId":"q160","format":"json-compact","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":665,"outputTokens":715,"latencyMs":9806.253459000029},{"questionId":"q160","format":"toon","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":756,"outputTokens":267,"latencyMs":3160.0135419997387},{"questionId":"q160","format":"xml","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1107,"outputTokens":459,"latencyMs":4843.594291999936},{"questionId":"q160","format":"yaml","model":"gpt-5-nano","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":775,"outputTokens":395,"latencyMs":5162.69116699975},{"questionId":"q161","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1023,"outputTokens":135,"latencyMs":3906.6883749999106},{"questionId":"q161","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":665,"outputTokens":263,"latencyMs":3832.8434589998797},{"questionId":"q161","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":756,"outputTokens":263,"latencyMs":4753.713458000217},{"questionId":"q161","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1107,"outputTokens":199,"latencyMs":6292.803332999814},{"questionId":"q161","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":775,"outputTokens":199,"latencyMs":6083.018833000213},{"questionId":"q162","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1023,"outputTokens":135,"latencyMs":2117.091250000056},{"questionId":"q162","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":665,"outputTokens":199,"latencyMs":5661.312124999706},{"questionId":"q162","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":756,"outputTokens":263,"latencyMs":5603.599000000395},{"questionId":"q162","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1107,"outputTokens":135,"latencyMs":4710.21570800012},{"questionId":"q162","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":775,"outputTokens":199,"latencyMs":7471.390374999959},{"questionId":"q163","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1022,"outputTokens":199,"latencyMs":4701.419417000376},{"questionId":"q163","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":664,"outputTokens":199,"latencyMs":3847.8547080000862},{"questionId":"q163","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":755,"outputTokens":263,"latencyMs":3375.736333000008},{"questionId":"q163","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1106,"outputTokens":263,"latencyMs":4812.272791000083},{"questionId":"q163","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":774,"outputTokens":199,"latencyMs":2934.4373750002123},{"questionId":"q164","format":"json-pretty","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1022,"outputTokens":135,"latencyMs":2458.2781249997206},{"questionId":"q164","format":"json-compact","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":664,"outputTokens":263,"latencyMs":3522.378250000067},{"questionId":"q164","format":"toon","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":755,"outputTokens":327,"latencyMs":4799.6738340002485},{"questionId":"q164","format":"xml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":1106,"outputTokens":199,"latencyMs":3384.5427500000224},{"questionId":"q164","format":"yaml","model":"gpt-5-nano","expected":"2","actual":"2","isCorrect":true,"inputTokens":774,"outputTokens":263,"latencyMs":5075.06341599999},{"questionId":"q165","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1022,"outputTokens":199,"latencyMs":2597.252208999824},{"questionId":"q165","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":664,"outputTokens":199,"latencyMs":2931.3202499998733},{"questionId":"q165","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":755,"outputTokens":71,"latencyMs":2898.455083000008},{"questionId":"q165","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1106,"outputTokens":263,"latencyMs":3072.9117919998243},{"questionId":"q165","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":774,"outputTokens":71,"latencyMs":2456.2880830001086},{"questionId":"q166","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1026,"outputTokens":199,"latencyMs":2803.085833000019},{"questionId":"q166","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":668,"outputTokens":839,"latencyMs":7020.810707999859},{"questionId":"q166","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":759,"outputTokens":199,"latencyMs":2661.654792000074},{"questionId":"q166","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1110,"outputTokens":327,"latencyMs":5022.77420799993},{"questionId":"q166","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":778,"outputTokens":263,"latencyMs":3315.3438749997877},{"questionId":"q167","format":"json-pretty","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1022,"outputTokens":263,"latencyMs":4004.776000000071},{"questionId":"q167","format":"json-compact","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":664,"outputTokens":327,"latencyMs":4605.751166999806},{"questionId":"q167","format":"toon","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":755,"outputTokens":327,"latencyMs":5653.84929200029},{"questionId":"q167","format":"xml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1106,"outputTokens":263,"latencyMs":3888.8481250000186},{"questionId":"q167","format":"yaml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":774,"outputTokens":263,"latencyMs":4843.462165999692},{"questionId":"q168","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":1024,"outputTokens":199,"latencyMs":2777.7275000000373},{"questionId":"q168","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":666,"outputTokens":711,"latencyMs":10276.33304100018},{"questionId":"q168","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":757,"outputTokens":199,"latencyMs":4521.871375000104},{"questionId":"q168","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":1108,"outputTokens":199,"latencyMs":5794.047832999844},{"questionId":"q168","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":776,"outputTokens":135,"latencyMs":3790.2669999999925},{"questionId":"q169","format":"json-pretty","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":1026,"outputTokens":391,"latencyMs":5940.847500000149},{"questionId":"q169","format":"json-compact","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":668,"outputTokens":455,"latencyMs":4762.850875000004},{"questionId":"q169","format":"toon","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":759,"outputTokens":455,"latencyMs":4719.768250000197},{"questionId":"q169","format":"xml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":1110,"outputTokens":327,"latencyMs":6340.48066599993},{"questionId":"q169","format":"yaml","model":"gpt-5-nano","expected":"8","actual":"8","isCorrect":true,"inputTokens":778,"outputTokens":263,"latencyMs":3185.5706250001676},{"questionId":"q170","format":"json-pretty","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":1025,"outputTokens":455,"latencyMs":4706.524750000332},{"questionId":"q170","format":"json-compact","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":667,"outputTokens":327,"latencyMs":5000.733999999706},{"questionId":"q170","format":"toon","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":758,"outputTokens":455,"latencyMs":6487.487999999896},{"questionId":"q170","format":"xml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":1109,"outputTokens":455,"latencyMs":8960.450375000015},{"questionId":"q170","format":"yaml","model":"gpt-5-nano","expected":"5","actual":"5","isCorrect":true,"inputTokens":777,"outputTokens":391,"latencyMs":5475.017541000154},{"questionId":"q171","format":"json-pretty","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1027,"outputTokens":263,"latencyMs":3693.9329169997945},{"questionId":"q171","format":"json-compact","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":669,"outputTokens":263,"latencyMs":3648.1655830000527},{"questionId":"q171","format":"toon","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":760,"outputTokens":967,"latencyMs":8853.470167000312},{"questionId":"q171","format":"xml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":1111,"outputTokens":263,"latencyMs":3237.0124579998665},{"questionId":"q171","format":"yaml","model":"gpt-5-nano","expected":"3","actual":"3","isCorrect":true,"inputTokens":779,"outputTokens":263,"latencyMs":4246.907124999911},{"questionId":"q172","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1027,"outputTokens":199,"latencyMs":3634.9288749997504},{"questionId":"q172","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":669,"outputTokens":327,"latencyMs":20629.421875},{"questionId":"q172","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":760,"outputTokens":199,"latencyMs":3223.972542000003},{"questionId":"q172","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1111,"outputTokens":327,"latencyMs":13383.079999999609},{"questionId":"q172","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":779,"outputTokens":263,"latencyMs":4296.551624999847},{"questionId":"q173","format":"json-pretty","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1028,"outputTokens":199,"latencyMs":2751.2610419997945},{"questionId":"q173","format":"json-compact","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":670,"outputTokens":327,"latencyMs":4416.645750000142},{"questionId":"q173","format":"toon","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":761,"outputTokens":519,"latencyMs":7847.565500000026},{"questionId":"q173","format":"xml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1112,"outputTokens":263,"latencyMs":6915.099750000052},{"questionId":"q173","format":"yaml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":780,"outputTokens":391,"latencyMs":6582.503958000336},{"questionId":"q174","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1025,"outputTokens":135,"latencyMs":2869.2113329996355},{"questionId":"q174","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":667,"outputTokens":263,"latencyMs":3683.578083000146},{"questionId":"q174","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":758,"outputTokens":327,"latencyMs":11444.25062499987},{"questionId":"q174","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1109,"outputTokens":327,"latencyMs":11302.633040999994},{"questionId":"q174","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":777,"outputTokens":135,"latencyMs":3082.4122499995865},{"questionId":"q175","format":"json-pretty","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1029,"outputTokens":327,"latencyMs":4507.2793749999255},{"questionId":"q175","format":"json-compact","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":671,"outputTokens":263,"latencyMs":4098.119249999989},{"questionId":"q175","format":"toon","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":762,"outputTokens":327,"latencyMs":10054.765292000026},{"questionId":"q175","format":"xml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":1113,"outputTokens":391,"latencyMs":6727.330041999929},{"questionId":"q175","format":"yaml","model":"gpt-5-nano","expected":"0","actual":"0","isCorrect":true,"inputTokens":781,"outputTokens":391,"latencyMs":7630.485625000205},{"questionId":"q176","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1023,"outputTokens":135,"latencyMs":2451.910416999832},{"questionId":"q176","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":665,"outputTokens":263,"latencyMs":3689.5402919999324},{"questionId":"q176","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":756,"outputTokens":199,"latencyMs":2873.5955409999005},{"questionId":"q176","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1107,"outputTokens":199,"latencyMs":3277.6729580000974},{"questionId":"q176","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":775,"outputTokens":263,"latencyMs":4645.139667000156},{"questionId":"q177","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1031,"outputTokens":327,"latencyMs":3554.097957999911},{"questionId":"q177","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":673,"outputTokens":263,"latencyMs":3744.1457079998218},{"questionId":"q177","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":764,"outputTokens":263,"latencyMs":6140.8108749999665},{"questionId":"q177","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1115,"outputTokens":327,"latencyMs":7183.078166999854},{"questionId":"q177","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":783,"outputTokens":263,"latencyMs":5394.631290999707},{"questionId":"q178","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1026,"outputTokens":135,"latencyMs":2505.3744160002097},{"questionId":"q178","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":668,"outputTokens":775,"latencyMs":11512.826999999583},{"questionId":"q178","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":759,"outputTokens":583,"latencyMs":9364.836542000063},{"questionId":"q178","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1110,"outputTokens":199,"latencyMs":4985.15625},{"questionId":"q178","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":778,"outputTokens":199,"latencyMs":3276.027791999746},{"questionId":"q179","format":"json-pretty","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1025,"outputTokens":199,"latencyMs":4783.0132499998435},{"questionId":"q179","format":"json-compact","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":667,"outputTokens":839,"latencyMs":9481.153750000056},{"questionId":"q179","format":"toon","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":758,"outputTokens":199,"latencyMs":4011.8218750003725},{"questionId":"q179","format":"xml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":1109,"outputTokens":327,"latencyMs":4096.573667000048},{"questionId":"q179","format":"yaml","model":"gpt-5-nano","expected":"1","actual":"1","isCorrect":true,"inputTokens":777,"outputTokens":455,"latencyMs":5798.487083999906},{"questionId":"q180","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":6451,"outputTokens":135,"latencyMs":5243.732333999593},{"questionId":"q180","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":4044,"outputTokens":135,"latencyMs":2556.379874999635},{"questionId":"q180","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2608,"outputTokens":135,"latencyMs":1891.7810420002788},{"questionId":"q180","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":2442,"outputTokens":263,"latencyMs":5116.138582999818},{"questionId":"q180","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":7413,"outputTokens":135,"latencyMs":5086.265749999788},{"questionId":"q180","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":5071,"outputTokens":135,"latencyMs":2536.9129999997094},{"questionId":"q181","format":"json-pretty","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":6456,"outputTokens":338,"latencyMs":7833.342083000112},{"questionId":"q181","format":"json-compact","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4049,"outputTokens":594,"latencyMs":5790.048458999954},{"questionId":"q181","format":"toon","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2613,"outputTokens":146,"latencyMs":3634.676667000167},{"questionId":"q181","format":"csv","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2447,"outputTokens":210,"latencyMs":3299.1852080002427},{"questionId":"q181","format":"xml","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":7418,"outputTokens":466,"latencyMs":5790.619957999792},{"questionId":"q181","format":"yaml","model":"gpt-5-nano","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":5076,"outputTokens":466,"latencyMs":5859.513791000005},{"questionId":"q182","format":"json-pretty","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":6454,"outputTokens":775,"latencyMs":11483.762374999933},{"questionId":"q182","format":"json-compact","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":4047,"outputTokens":519,"latencyMs":8336.065458999947},{"questionId":"q182","format":"toon","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":2611,"outputTokens":327,"latencyMs":4580.0390419997275},{"questionId":"q182","format":"csv","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":2445,"outputTokens":135,"latencyMs":4742.354375000112},{"questionId":"q182","format":"xml","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":7416,"outputTokens":263,"latencyMs":4211.900249999948},{"questionId":"q182","format":"yaml","model":"gpt-5-nano","expected":"email","actual":"email","isCorrect":true,"inputTokens":5074,"outputTokens":327,"latencyMs":4381.881250000093},{"questionId":"q183","format":"json-pretty","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":6455,"outputTokens":327,"latencyMs":4497.625540999696},{"questionId":"q183","format":"json-compact","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4048,"outputTokens":199,"latencyMs":3268.267374999821},{"questionId":"q183","format":"toon","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2612,"outputTokens":263,"latencyMs":5449.473415999673},{"questionId":"q183","format":"csv","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2446,"outputTokens":263,"latencyMs":4097.444000000134},{"questionId":"q183","format":"xml","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7417,"outputTokens":263,"latencyMs":4184.440708999988},{"questionId":"q183","format":"yaml","model":"gpt-5-nano","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5075,"outputTokens":135,"latencyMs":2946.645917000249},{"questionId":"q184","format":"json-pretty","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":6455,"outputTokens":204,"latencyMs":9614.618167000357},{"questionId":"q184","format":"json-compact","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4048,"outputTokens":140,"latencyMs":2873.793583000079},{"questionId":"q184","format":"toon","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2612,"outputTokens":140,"latencyMs":2877.5963750001974},{"questionId":"q184","format":"csv","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2446,"outputTokens":204,"latencyMs":3551.0457079997286},{"questionId":"q184","format":"xml","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7417,"outputTokens":140,"latencyMs":2625.599458000157},{"questionId":"q184","format":"yaml","model":"gpt-5-nano","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5075,"outputTokens":204,"latencyMs":2858.9412090000696},{"questionId":"q185","format":"json-pretty","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":6452,"outputTokens":199,"latencyMs":3881.6359999999404},{"questionId":"q185","format":"json-compact","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":4045,"outputTokens":263,"latencyMs":20388.577374999877},{"questionId":"q185","format":"toon","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":2609,"outputTokens":199,"latencyMs":3270.6690000002272},{"questionId":"q185","format":"csv","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":2443,"outputTokens":3591,"latencyMs":29106.583208000287},{"questionId":"q185","format":"xml","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":7414,"outputTokens":391,"latencyMs":4287.198749999981},{"questionId":"q185","format":"yaml","model":"gpt-5-nano","expected":"7","actual":"7","isCorrect":true,"inputTokens":5072,"outputTokens":327,"latencyMs":3268.473958000075},{"questionId":"q186","format":"json-pretty","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":10804,"outputTokens":583,"latencyMs":5497.04287499981},{"questionId":"q186","format":"json-compact","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":6885,"outputTokens":519,"latencyMs":8948.515999999829},{"questionId":"q186","format":"toon","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":7322,"outputTokens":199,"latencyMs":4607.618165999651},{"questionId":"q186","format":"xml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":12112,"outputTokens":263,"latencyMs":3848.347834000364},{"questionId":"q186","format":"yaml","model":"gpt-5-nano","expected":"50","actual":"50","isCorrect":true,"inputTokens":8436,"outputTokens":263,"latencyMs":3796.977042000275},{"questionId":"q187","format":"json-pretty","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orders","isCorrect":false,"inputTokens":10811,"outputTokens":455,"latencyMs":5749.156665999908},{"questionId":"q187","format":"json-compact","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":6892,"outputTokens":725,"latencyMs":8672.660540999845},{"questionId":"q187","format":"toon","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":7329,"outputTokens":533,"latencyMs":5036.442500000354},{"questionId":"q187","format":"xml","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":12119,"outputTokens":405,"latencyMs":4477.925249999855},{"questionId":"q187","format":"yaml","model":"gpt-5-nano","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":8443,"outputTokens":533,"latencyMs":5181.2483749999665},{"questionId":"q188","format":"json-pretty","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":10807,"outputTokens":263,"latencyMs":4242.422333000228},{"questionId":"q188","format":"json-compact","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":6888,"outputTokens":519,"latencyMs":5718.889750000089},{"questionId":"q188","format":"toon","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":7325,"outputTokens":327,"latencyMs":5340.791166000068},{"questionId":"q188","format":"xml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":12115,"outputTokens":967,"latencyMs":9309.043292000424},{"questionId":"q188","format":"yaml","model":"gpt-5-nano","expected":"4","actual":"4","isCorrect":true,"inputTokens":8439,"outputTokens":583,"latencyMs":6073.824750000145},{"questionId":"q189","format":"json-pretty","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":10812,"outputTokens":140,"latencyMs":4690.233665999956},{"questionId":"q189","format":"json-compact","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":6893,"outputTokens":268,"latencyMs":5466.577833000105},{"questionId":"q189","format":"toon","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":7330,"outputTokens":204,"latencyMs":2964.15870800009},{"questionId":"q189","format":"xml","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":12120,"outputTokens":332,"latencyMs":4515.02179200016},{"questionId":"q189","format":"yaml","model":"gpt-5-nano","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":8444,"outputTokens":268,"latencyMs":8201.164624999743},{"questionId":"q190","format":"json-pretty","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10808,"outputTokens":200,"latencyMs":4878.779084000271},{"questionId":"q190","format":"json-compact","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6889,"outputTokens":200,"latencyMs":3476.9356669997796},{"questionId":"q190","format":"toon","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7326,"outputTokens":200,"latencyMs":2631.7864590003155},{"questionId":"q190","format":"xml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12116,"outputTokens":200,"latencyMs":3225.6047499999404},{"questionId":"q190","format":"yaml","model":"gpt-5-nano","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8440,"outputTokens":264,"latencyMs":6446.187125000171},{"questionId":"q191","format":"json-pretty","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":10813,"outputTokens":395,"latencyMs":4756.288000000175},{"questionId":"q191","format":"json-compact","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":6894,"outputTokens":267,"latencyMs":4393.102415999863},{"questionId":"q191","format":"toon","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":7331,"outputTokens":267,"latencyMs":5854.2481249999255},{"questionId":"q191","format":"xml","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":12121,"outputTokens":267,"latencyMs":6437.172999999952},{"questionId":"q191","format":"yaml","model":"gpt-5-nano","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":8445,"outputTokens":203,"latencyMs":3253.0284589999355},{"questionId":"q192","format":"json-pretty","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":3768,"outputTokens":839,"latencyMs":7548.902958999854},{"questionId":"q192","format":"json-compact","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":2441,"outputTokens":647,"latencyMs":8269.41454199981},{"questionId":"q192","format":"toon","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1639,"outputTokens":263,"latencyMs":8128.112708000001},{"questionId":"q192","format":"csv","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":1497,"outputTokens":967,"latencyMs":14334.201832999941},{"questionId":"q192","format":"xml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":4474,"outputTokens":903,"latencyMs":11843.660625000019},{"questionId":"q192","format":"yaml","model":"gpt-5-nano","expected":"60","actual":"60","isCorrect":true,"inputTokens":3039,"outputTokens":839,"latencyMs":12383.660041000228},{"questionId":"q193","format":"json-pretty","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3772,"outputTokens":212,"latencyMs":3716.6943749999627},{"questionId":"q193","format":"json-compact","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2445,"outputTokens":276,"latencyMs":4321.190417000093},{"questionId":"q193","format":"toon","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1643,"outputTokens":276,"latencyMs":3628.611583000049},{"questionId":"q193","format":"csv","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1501,"outputTokens":276,"latencyMs":3329.8520830003545},{"questionId":"q193","format":"xml","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4478,"outputTokens":276,"latencyMs":8496.843958000187},{"questionId":"q193","format":"yaml","model":"gpt-5-nano","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3043,"outputTokens":340,"latencyMs":4394.631957999896},{"questionId":"q194","format":"json-pretty","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3771,"outputTokens":392,"latencyMs":7423.73466600012},{"questionId":"q194","format":"json-compact","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2444,"outputTokens":584,"latencyMs":5982.649750000332},{"questionId":"q194","format":"toon","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1642,"outputTokens":200,"latencyMs":4239.526542000007},{"questionId":"q194","format":"csv","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1500,"outputTokens":200,"latencyMs":5031.122084000148},{"questionId":"q194","format":"xml","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":4477,"outputTokens":264,"latencyMs":7930.7731669996865},{"questionId":"q194","format":"yaml","model":"gpt-5-nano","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3042,"outputTokens":264,"latencyMs":3106.251792000141},{"questionId":"q195","format":"json-pretty","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3772,"outputTokens":524,"latencyMs":8153.412459000014},{"questionId":"q195","format":"json-compact","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2445,"outputTokens":588,"latencyMs":7503.793999999762},{"questionId":"q195","format":"toon","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1643,"outputTokens":204,"latencyMs":3050.2509169997647},{"questionId":"q195","format":"csv","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1501,"outputTokens":140,"latencyMs":2617.243791999761},{"questionId":"q195","format":"xml","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4478,"outputTokens":204,"latencyMs":3606.4029169999994},{"questionId":"q195","format":"yaml","model":"gpt-5-nano","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3043,"outputTokens":140,"latencyMs":2635.89858300006},{"questionId":"q196","format":"json-pretty","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":3768,"outputTokens":199,"latencyMs":2703.3255830002017},{"questionId":"q196","format":"json-compact","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":2441,"outputTokens":327,"latencyMs":8638.098624999635},{"questionId":"q196","format":"toon","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":1639,"outputTokens":263,"latencyMs":5401.915333000012},{"questionId":"q196","format":"csv","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":1497,"outputTokens":327,"latencyMs":5221.692542000208},{"questionId":"q196","format":"xml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":4474,"outputTokens":263,"latencyMs":3620.0856249998324},{"questionId":"q196","format":"yaml","model":"gpt-5-nano","expected":"6","actual":"6","isCorrect":true,"inputTokens":3039,"outputTokens":263,"latencyMs":3699.8902920000255},{"questionId":"q197","format":"json-pretty","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":15248,"outputTokens":1415,"latencyMs":16647.309458999895},{"questionId":"q197","format":"json-compact","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":11555,"outputTokens":1415,"latencyMs":28783.25216699997},{"questionId":"q197","format":"toon","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8869,"outputTokens":199,"latencyMs":2771.9329590001144},{"questionId":"q197","format":"csv","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":8617,"outputTokens":1543,"latencyMs":13179.85262500029},{"questionId":"q197","format":"xml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":17194,"outputTokens":2439,"latencyMs":22739.320625000168},{"questionId":"q197","format":"yaml","model":"gpt-5-nano","expected":"100","actual":"100","isCorrect":true,"inputTokens":13230,"outputTokens":6023,"latencyMs":51610.507999999914},{"questionId":"q198","format":"json-pretty","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":15253,"outputTokens":415,"latencyMs":4503.3462910000235},{"questionId":"q198","format":"json-compact","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":11560,"outputTokens":543,"latencyMs":5954.608874999918},{"questionId":"q198","format":"toon","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8874,"outputTokens":351,"latencyMs":5119.701166999992},{"questionId":"q198","format":"csv","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8622,"outputTokens":223,"latencyMs":3383.965749999974},{"questionId":"q198","format":"xml","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":17199,"outputTokens":351,"latencyMs":8255.69604199985},{"questionId":"q198","format":"yaml","model":"gpt-5-nano","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":13235,"outputTokens":415,"latencyMs":5688.796708000358},{"questionId":"q199","format":"json-pretty","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":15253,"outputTokens":713,"latencyMs":6935.332459000405},{"questionId":"q199","format":"json-compact","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":11560,"outputTokens":777,"latencyMs":7743.049292000011},{"questionId":"q199","format":"toon","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":8874,"outputTokens":329,"latencyMs":7626.734916999936},{"questionId":"q199","format":"csv","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":8622,"outputTokens":201,"latencyMs":3741.813084000256},{"questionId":"q199","format":"xml","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":17199,"outputTokens":329,"latencyMs":4252.712875000201},{"questionId":"q199","format":"yaml","model":"gpt-5-nano","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":13235,"outputTokens":329,"latencyMs":4048.160000000149},{"questionId":"q200","format":"json-pretty","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":15252,"outputTokens":265,"latencyMs":4140.997250000015},{"questionId":"q200","format":"json-compact","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":11559,"outputTokens":457,"latencyMs":4518.3047080002725},{"questionId":"q200","format":"toon","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8873,"outputTokens":393,"latencyMs":4574.140791999642},{"questionId":"q200","format":"csv","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8621,"outputTokens":585,"latencyMs":5762.444084000308},{"questionId":"q200","format":"xml","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17198,"outputTokens":393,"latencyMs":4484.691707999911},{"questionId":"q200","format":"yaml","model":"gpt-5-nano","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":13234,"outputTokens":265,"latencyMs":4706.706375000067},{"questionId":"q201","format":"json-pretty","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":15249,"outputTokens":327,"latencyMs":3816.0244579999708},{"questionId":"q201","format":"json-compact","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":11556,"outputTokens":455,"latencyMs":6546.3870419999585},{"questionId":"q201","format":"toon","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":8870,"outputTokens":263,"latencyMs":4477.5936670000665},{"questionId":"q201","format":"csv","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":8618,"outputTokens":455,"latencyMs":5239.246875000186},{"questionId":"q201","format":"xml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":17195,"outputTokens":647,"latencyMs":7984.748166000005},{"questionId":"q201","format":"yaml","model":"gpt-5-nano","expected":"11","actual":"11","isCorrect":true,"inputTokens":13231,"outputTokens":263,"latencyMs":8407.612875000108},{"questionId":"q202","format":"json-pretty","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":6889,"outputTokens":2951,"latencyMs":27031.601250000298},{"questionId":"q202","format":"json-compact","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":4911,"outputTokens":10631,"latencyMs":81286.11437499989},{"questionId":"q202","format":"toon","model":"gpt-5-nano","expected":"75","actual":"75","isCorrect":true,"inputTokens":5890,"outputTokens":135,"latencyMs":3970.110458999872},{"questionId":"q202","format":"xml","model":"gpt-5-nano","expected":"75","actual":"58","isCorrect":false,"inputTokens":7799,"outputTokens":4871,"latencyMs":149805.12412499962},{"questionId":"q202","format":"yaml","model":"gpt-5-nano","expected":"75","actual":"100","isCorrect":false,"inputTokens":5917,"outputTokens":13703,"latencyMs":155739.1889589997},{"questionId":"q203","format":"json-pretty","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":6898,"outputTokens":466,"latencyMs":9108.469208000228},{"questionId":"q203","format":"json-compact","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":4920,"outputTokens":1106,"latencyMs":10545.836332999635},{"questionId":"q203","format":"toon","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":5899,"outputTokens":594,"latencyMs":8183.054624999873},{"questionId":"q203","format":"xml","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":7808,"outputTokens":1426,"latencyMs":11678.348749999888},{"questionId":"q203","format":"yaml","model":"gpt-5-nano","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":5926,"outputTokens":850,"latencyMs":8783.712416999973},{"questionId":"q204","format":"json-pretty","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":6893,"outputTokens":455,"latencyMs":4951.810749999713},{"questionId":"q204","format":"json-compact","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":4915,"outputTokens":583,"latencyMs":8194.983541999944},{"questionId":"q204","format":"toon","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5894,"outputTokens":263,"latencyMs":8032.524958000053},{"questionId":"q204","format":"xml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":7803,"outputTokens":263,"latencyMs":5646.691916999873},{"questionId":"q204","format":"yaml","model":"gpt-5-nano","expected":"info","actual":"info","isCorrect":true,"inputTokens":5921,"outputTokens":199,"latencyMs":3208.1868750001304},{"questionId":"q205","format":"json-pretty","model":"gpt-5-nano","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1378,"outputTokens":455,"latencyMs":5234.0922079999},{"questionId":"q205","format":"json-compact","model":"gpt-5-nano","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":891,"outputTokens":1159,"latencyMs":13605.088709000032},{"questionId":"q205","format":"toon","model":"gpt-5-nano","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":639,"outputTokens":711,"latencyMs":6515.72516599996},{"questionId":"q205","format":"csv","model":"gpt-5-nano","expected":"YES","actual":"NO","isCorrect":false,"inputTokens":587,"outputTokens":1479,"latencyMs":14114.901416999754},{"questionId":"q205","format":"xml","model":"gpt-5-nano","expected":"YES","actual":"NO","isCorrect":false,"inputTokens":1561,"outputTokens":327,"latencyMs":5021.389500000048},{"questionId":"q205","format":"yaml","model":"gpt-5-nano","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1097,"outputTokens":455,"latencyMs":5526.448458000086},{"questionId":"q206","format":"json-pretty","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1192,"outputTokens":647,"latencyMs":6677.833459000103},{"questionId":"q206","format":"json-compact","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":777,"outputTokens":455,"latencyMs":5277.759208000265},{"questionId":"q206","format":"toon","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":569,"outputTokens":1543,"latencyMs":16482.88245799998},{"questionId":"q206","format":"csv","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":520,"outputTokens":647,"latencyMs":8937.574082999956},{"questionId":"q206","format":"xml","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1345,"outputTokens":391,"latencyMs":5002.396875000093},{"questionId":"q206","format":"yaml","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":953,"outputTokens":711,"latencyMs":6825.064792000223},{"questionId":"q207","format":"json-pretty","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1572,"outputTokens":1351,"latencyMs":11435.325457999948},{"questionId":"q207","format":"json-compact","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1013,"outputTokens":1671,"latencyMs":17690.187415999826},{"questionId":"q207","format":"toon","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":716,"outputTokens":3399,"latencyMs":31119.575375000015},{"questionId":"q207","format":"csv","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":661,"outputTokens":1031,"latencyMs":13081.246334000025},{"questionId":"q207","format":"xml","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1786,"outputTokens":455,"latencyMs":8677.37220800016},{"questionId":"q207","format":"yaml","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1251,"outputTokens":903,"latencyMs":8989.367707999889},{"questionId":"q208","format":"json-pretty","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1370,"outputTokens":775,"latencyMs":7549.4047079999},{"questionId":"q208","format":"json-compact","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":886,"outputTokens":1095,"latencyMs":9804.55941700004},{"questionId":"q208","format":"toon","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1114,"outputTokens":967,"latencyMs":8704.191707999911},{"questionId":"q208","format":"csv","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":584,"outputTokens":519,"latencyMs":8088.646707999986},{"questionId":"q208","format":"xml","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1552,"outputTokens":583,"latencyMs":8100.714499999769},{"questionId":"q208","format":"yaml","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1090,"outputTokens":967,"latencyMs":8288.674542000052},{"questionId":"q209","format":"json-pretty","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1326,"outputTokens":647,"latencyMs":6657.2364159999415},{"questionId":"q209","format":"json-compact","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":851,"outputTokens":1159,"latencyMs":13106.269707999658},{"questionId":"q209","format":"toon","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1078,"outputTokens":1351,"latencyMs":22118.10566599993},{"questionId":"q209","format":"csv","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":424,"outputTokens":1031,"latencyMs":10323.110000000335},{"questionId":"q209","format":"xml","model":"gpt-5-nano","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1505,"outputTokens":455,"latencyMs":5428.757249999791},{"questionId":"q209","format":"yaml","model":"gpt-5-nano","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1054,"outputTokens":1671,"latencyMs":15239.667082999833}] \ No newline at end of file diff --git a/benchmarks/results/accuracy/models/grok-4-fast-non-reasoning b/benchmarks/results/accuracy/models/grok-4-fast-non-reasoning index 6b27ca5..4e0ae2d 100644 --- a/benchmarks/results/accuracy/models/grok-4-fast-non-reasoning +++ b/benchmarks/results/accuracy/models/grok-4-fast-non-reasoning @@ -1 +1 @@ -[{"questionId":"q1","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":6514,"outputTokens":2,"latencyMs":761.5262920000023},{"questionId":"q1","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":4105,"outputTokens":2,"latencyMs":500.59695899999497},{"questionId":"q1","format":"toon","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2684,"outputTokens":2,"latencyMs":515.6211670000048},{"questionId":"q1","format":"csv","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2519,"outputTokens":2,"latencyMs":476.0241249999963},{"questionId":"q1","format":"xml","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7451,"outputTokens":2,"latencyMs":778.9134579999954},{"questionId":"q1","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5147,"outputTokens":2,"latencyMs":701.711166000001},{"questionId":"q2","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":6514,"outputTokens":1,"latencyMs":711.5992909999986},{"questionId":"q2","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":4105,"outputTokens":1,"latencyMs":595.440042000002},{"questionId":"q2","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2684,"outputTokens":1,"latencyMs":472.6217500000057},{"questionId":"q2","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2519,"outputTokens":1,"latencyMs":473.1861250000002},{"questionId":"q2","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7451,"outputTokens":1,"latencyMs":388.14058300000033},{"questionId":"q2","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5147,"outputTokens":1,"latencyMs":401.74412499999744},{"questionId":"q3","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":6517,"outputTokens":7,"latencyMs":775.6467080000002},{"questionId":"q3","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4108,"outputTokens":7,"latencyMs":387.1924579999977},{"questionId":"q3","format":"toon","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2687,"outputTokens":7,"latencyMs":372.3358339999977},{"questionId":"q3","format":"csv","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2522,"outputTokens":7,"latencyMs":468.0272499999992},{"questionId":"q3","format":"xml","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7454,"outputTokens":7,"latencyMs":490.00308299999597},{"questionId":"q3","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5150,"outputTokens":7,"latencyMs":375.0840840000019},{"questionId":"q4","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":6517,"outputTokens":1,"latencyMs":419.2505420000016},{"questionId":"q4","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":4108,"outputTokens":1,"latencyMs":347.2774999999965},{"questionId":"q4","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":2687,"outputTokens":1,"latencyMs":725.2204169999968},{"questionId":"q4","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":2522,"outputTokens":1,"latencyMs":408.4358329999959},{"questionId":"q4","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":7454,"outputTokens":1,"latencyMs":432.94354100000055},{"questionId":"q4","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":5150,"outputTokens":1,"latencyMs":324.5074590000004},{"questionId":"q5","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"no","actual":"No","isCorrect":true,"inputTokens":6512,"outputTokens":1,"latencyMs":466.5721249999988},{"questionId":"q5","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"no","actual":"No","isCorrect":true,"inputTokens":4103,"outputTokens":1,"latencyMs":340.88862500000687},{"questionId":"q5","format":"toon","model":"grok-4-fast-non-reasoning","expected":"no","actual":"No","isCorrect":true,"inputTokens":2682,"outputTokens":1,"latencyMs":354.3313750000016},{"questionId":"q5","format":"csv","model":"grok-4-fast-non-reasoning","expected":"no","actual":"No","isCorrect":true,"inputTokens":2517,"outputTokens":1,"latencyMs":367.6174170000013},{"questionId":"q5","format":"xml","model":"grok-4-fast-non-reasoning","expected":"no","actual":"No","isCorrect":true,"inputTokens":7449,"outputTokens":1,"latencyMs":485.8099999999977},{"questionId":"q5","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"no","actual":"No\n\n## Explanation\n\nThe reasoning involves parsing the YAML structure to locate the specific employee record. The data is organized under the \"employees\" key as a list of objects, each containing fields like id, name, email, department, salary, yearsExperience, and active. To answer the query, identify the entry where the name field equals \"Tracy Gleason,\" which corresponds to id: 9. In that record, the active field is set to false, indicating the employee is not active. This direct examination of the relevant YAML entry confirms the status.","isCorrect":true,"inputTokens":5145,"outputTokens":112,"latencyMs":975.8160419999986},{"questionId":"q6","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":6513,"outputTokens":2,"latencyMs":465.20220800000243},{"questionId":"q6","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":4104,"outputTokens":2,"latencyMs":369.96195799999987},{"questionId":"q6","format":"toon","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2683,"outputTokens":2,"latencyMs":325.0040829999998},{"questionId":"q6","format":"csv","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2518,"outputTokens":2,"latencyMs":355.7384999999995},{"questionId":"q6","format":"xml","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7450,"outputTokens":2,"latencyMs":407.23608299999614},{"questionId":"q6","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5146,"outputTokens":2,"latencyMs":363.005167000003},{"questionId":"q7","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":6513,"outputTokens":1,"latencyMs":472.03954100000556},{"questionId":"q7","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":4104,"outputTokens":1,"latencyMs":442.02470800000447},{"questionId":"q7","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2683,"outputTokens":1,"latencyMs":355.5347499999989},{"questionId":"q7","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2518,"outputTokens":1,"latencyMs":328.25824999999895},{"questionId":"q7","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7450,"outputTokens":1,"latencyMs":372.78304200000275},{"questionId":"q7","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5146,"outputTokens":1,"latencyMs":396.5722090000054},{"questionId":"q8","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":6516,"outputTokens":6,"latencyMs":401.1356250000026},{"questionId":"q8","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4107,"outputTokens":6,"latencyMs":454.8651249999966},{"questionId":"q8","format":"toon","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2686,"outputTokens":6,"latencyMs":325.653459000001},{"questionId":"q8","format":"csv","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2521,"outputTokens":6,"latencyMs":343.06095799999457},{"questionId":"q8","format":"xml","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7453,"outputTokens":6,"latencyMs":421.2090840000019},{"questionId":"q8","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5149,"outputTokens":6,"latencyMs":468.93291700000555},{"questionId":"q9","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":6517,"outputTokens":1,"latencyMs":528.4802909999999},{"questionId":"q9","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":4108,"outputTokens":1,"latencyMs":362.1042499999967},{"questionId":"q9","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":2687,"outputTokens":1,"latencyMs":468.5052080000023},{"questionId":"q9","format":"csv","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":2522,"outputTokens":1,"latencyMs":428.89395800000057},{"questionId":"q9","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":7454,"outputTokens":1,"latencyMs":606.0681249999907},{"questionId":"q9","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":5150,"outputTokens":1,"latencyMs":919.7102500000037},{"questionId":"q10","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":6514,"outputTokens":1,"latencyMs":500.4227499999979},{"questionId":"q10","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":4105,"outputTokens":1,"latencyMs":442.85050000000047},{"questionId":"q10","format":"toon","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":2684,"outputTokens":1,"latencyMs":384.0657910000009},{"questionId":"q10","format":"csv","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":2519,"outputTokens":1,"latencyMs":444.0281249999971},{"questionId":"q10","format":"xml","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":7451,"outputTokens":1,"latencyMs":778.1915000000008},{"questionId":"q10","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":5147,"outputTokens":1,"latencyMs":655.6432500000083},{"questionId":"q11","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":6515,"outputTokens":2,"latencyMs":382.5429589999985},{"questionId":"q11","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":4106,"outputTokens":2,"latencyMs":440.61229200000525},{"questionId":"q11","format":"toon","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2685,"outputTokens":2,"latencyMs":490.61737500000163},{"questionId":"q11","format":"csv","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2520,"outputTokens":2,"latencyMs":311.47699999999895},{"questionId":"q11","format":"xml","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7452,"outputTokens":2,"latencyMs":421.51524999999674},{"questionId":"q11","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5148,"outputTokens":2,"latencyMs":372.16904200000863},{"questionId":"q12","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":6513,"outputTokens":1,"latencyMs":394.88895800001046},{"questionId":"q12","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":4104,"outputTokens":1,"latencyMs":398.95141700000386},{"questionId":"q12","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2683,"outputTokens":1,"latencyMs":357.5086250000022},{"questionId":"q12","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2518,"outputTokens":1,"latencyMs":335.627250000005},{"questionId":"q12","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7450,"outputTokens":1,"latencyMs":431.76062499999534},{"questionId":"q12","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5146,"outputTokens":1,"latencyMs":403.6072090000089},{"questionId":"q13","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":6512,"outputTokens":1,"latencyMs":404.34020799999416},{"questionId":"q13","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":4103,"outputTokens":1,"latencyMs":445.0642499999958},{"questionId":"q13","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2682,"outputTokens":1,"latencyMs":337.8610420000041},{"questionId":"q13","format":"csv","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2517,"outputTokens":1,"latencyMs":395.6776250000112},{"questionId":"q13","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":7449,"outputTokens":1,"latencyMs":476.2642909999995},{"questionId":"q13","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":5145,"outputTokens":1,"latencyMs":366.7690830000065},{"questionId":"q14","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":6512,"outputTokens":1,"latencyMs":465.09870899999805},{"questionId":"q14","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":4103,"outputTokens":1,"latencyMs":359.2127920000057},{"questionId":"q14","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2682,"outputTokens":1,"latencyMs":334.1157499999972},{"questionId":"q14","format":"csv","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":2517,"outputTokens":1,"latencyMs":528.625749999992},{"questionId":"q14","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":7449,"outputTokens":1,"latencyMs":353.411500000002},{"questionId":"q14","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":5145,"outputTokens":1,"latencyMs":370.83195800000976},{"questionId":"q15","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":6512,"outputTokens":1,"latencyMs":526.565957999992},{"questionId":"q15","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":4103,"outputTokens":1,"latencyMs":985.0588749999879},{"questionId":"q15","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2682,"outputTokens":1,"latencyMs":334.4548749999958},{"questionId":"q15","format":"csv","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":2517,"outputTokens":1,"latencyMs":383.49925000000803},{"questionId":"q15","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":7449,"outputTokens":1,"latencyMs":628.7822909999959},{"questionId":"q15","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":5145,"outputTokens":1,"latencyMs":398.5317919999943},{"questionId":"q16","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"91","actual":"84","isCorrect":false,"inputTokens":6517,"outputTokens":1,"latencyMs":389.80308300000615},{"questionId":"q16","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"91","actual":"95","isCorrect":false,"inputTokens":4108,"outputTokens":1,"latencyMs":408.20829200001026},{"questionId":"q16","format":"toon","model":"grok-4-fast-non-reasoning","expected":"91","actual":"85","isCorrect":false,"inputTokens":2687,"outputTokens":1,"latencyMs":426.5163329999923},{"questionId":"q16","format":"csv","model":"grok-4-fast-non-reasoning","expected":"91","actual":"78","isCorrect":false,"inputTokens":2522,"outputTokens":1,"latencyMs":340.91258299999754},{"questionId":"q16","format":"xml","model":"grok-4-fast-non-reasoning","expected":"91","actual":"83","isCorrect":false,"inputTokens":7454,"outputTokens":1,"latencyMs":438.6693749999977},{"questionId":"q16","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"91","actual":"89","isCorrect":false,"inputTokens":5150,"outputTokens":1,"latencyMs":386.27850000000035},{"questionId":"q17","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"67","actual":"67","isCorrect":true,"inputTokens":6517,"outputTokens":1,"latencyMs":346.692292000007},{"questionId":"q17","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"67","actual":"67","isCorrect":true,"inputTokens":4108,"outputTokens":1,"latencyMs":387.7595000000001},{"questionId":"q17","format":"toon","model":"grok-4-fast-non-reasoning","expected":"67","actual":"65","isCorrect":false,"inputTokens":2687,"outputTokens":1,"latencyMs":349.2559170000022},{"questionId":"q17","format":"csv","model":"grok-4-fast-non-reasoning","expected":"67","actual":"58","isCorrect":false,"inputTokens":2522,"outputTokens":1,"latencyMs":362.42529100000684},{"questionId":"q17","format":"xml","model":"grok-4-fast-non-reasoning","expected":"67","actual":"58","isCorrect":false,"inputTokens":7454,"outputTokens":1,"latencyMs":487.1877500000119},{"questionId":"q17","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"67","actual":"63","isCorrect":false,"inputTokens":5150,"outputTokens":1,"latencyMs":545.6530420000199},{"questionId":"q18","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"41","actual":"47","isCorrect":false,"inputTokens":6517,"outputTokens":1,"latencyMs":515.1853750000009},{"questionId":"q18","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"41","actual":"44","isCorrect":false,"inputTokens":4108,"outputTokens":1,"latencyMs":665.0683749999735},{"questionId":"q18","format":"toon","model":"grok-4-fast-non-reasoning","expected":"41","actual":"43","isCorrect":false,"inputTokens":2687,"outputTokens":1,"latencyMs":481.14145900000585},{"questionId":"q18","format":"csv","model":"grok-4-fast-non-reasoning","expected":"41","actual":"41","isCorrect":true,"inputTokens":2522,"outputTokens":1,"latencyMs":403.5092080000031},{"questionId":"q18","format":"xml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"41","isCorrect":true,"inputTokens":7454,"outputTokens":1,"latencyMs":505.2139160000079},{"questionId":"q18","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"44","isCorrect":false,"inputTokens":5150,"outputTokens":1,"latencyMs":425.5977079999866},{"questionId":"q19","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":6513,"outputTokens":1,"latencyMs":539.9807079999882},{"questionId":"q19","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":4104,"outputTokens":1,"latencyMs":553.1321670000034},{"questionId":"q19","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2683,"outputTokens":1,"latencyMs":390.7068750000035},{"questionId":"q19","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2518,"outputTokens":1,"latencyMs":352.33854199998314},{"questionId":"q19","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":7450,"outputTokens":1,"latencyMs":425.1197500000126},{"questionId":"q19","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":5146,"outputTokens":1,"latencyMs":361.4464999999909},{"questionId":"q20","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"100528","isCorrect":false,"inputTokens":6514,"outputTokens":2,"latencyMs":509.49049999998533},{"questionId":"q20","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"{\"averageSalary\":98125.09}","isCorrect":false,"inputTokens":4105,"outputTokens":9,"latencyMs":383.7856250000186},{"questionId":"q20","format":"toon","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"100683","isCorrect":false,"inputTokens":2684,"outputTokens":2,"latencyMs":344.03820799998357},{"questionId":"q20","format":"csv","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"99075","isCorrect":false,"inputTokens":2519,"outputTokens":2,"latencyMs":352.11241699999664},{"questionId":"q20","format":"xml","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"100218","isCorrect":false,"inputTokens":7451,"outputTokens":2,"latencyMs":481.4297080000106},{"questionId":"q20","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"97079","isCorrect":false,"inputTokens":5147,"outputTokens":2,"latencyMs":402.71816700001364},{"questionId":"q21","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"78","actual":"82","isCorrect":false,"inputTokens":6511,"outputTokens":1,"latencyMs":379.5406249999942},{"questionId":"q21","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"78","actual":"82","isCorrect":false,"inputTokens":4102,"outputTokens":1,"latencyMs":369.2469169999822},{"questionId":"q21","format":"toon","model":"grok-4-fast-non-reasoning","expected":"78","actual":"83","isCorrect":false,"inputTokens":2681,"outputTokens":1,"latencyMs":337.66729100001976},{"questionId":"q21","format":"csv","model":"grok-4-fast-non-reasoning","expected":"78","actual":"84","isCorrect":false,"inputTokens":2516,"outputTokens":1,"latencyMs":328.7916660000046},{"questionId":"q21","format":"xml","model":"grok-4-fast-non-reasoning","expected":"78","actual":"84","isCorrect":false,"inputTokens":7448,"outputTokens":1,"latencyMs":441.60566600001766},{"questionId":"q21","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"78","actual":"85","isCorrect":false,"inputTokens":5144,"outputTokens":1,"latencyMs":369.4047079999873},{"questionId":"q22","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"18","isCorrect":false,"inputTokens":6511,"outputTokens":1,"latencyMs":445.49179199998616},{"questionId":"q22","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":4102,"outputTokens":1,"latencyMs":352.1461249999993},{"questionId":"q22","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":2681,"outputTokens":1,"latencyMs":345.1410420000029},{"questionId":"q22","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"27","isCorrect":false,"inputTokens":2516,"outputTokens":1,"latencyMs":961.7747089999903},{"questionId":"q22","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"23","isCorrect":false,"inputTokens":7448,"outputTokens":1,"latencyMs":464.33974999998463},{"questionId":"q22","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"16","isCorrect":false,"inputTokens":5144,"outputTokens":1,"latencyMs":353.5139999999956},{"questionId":"q23","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"12","actual":"12","isCorrect":true,"inputTokens":6519,"outputTokens":1,"latencyMs":413.5964169999934},{"questionId":"q23","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"12","actual":"12","isCorrect":true,"inputTokens":4110,"outputTokens":1,"latencyMs":411.6941250000091},{"questionId":"q23","format":"toon","model":"grok-4-fast-non-reasoning","expected":"12","actual":"18","isCorrect":false,"inputTokens":2689,"outputTokens":1,"latencyMs":397.0174999999872},{"questionId":"q23","format":"csv","model":"grok-4-fast-non-reasoning","expected":"12","actual":"25","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":336.8227080000215},{"questionId":"q23","format":"xml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"17","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":309.1291659999988},{"questionId":"q23","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"14","isCorrect":false,"inputTokens":5152,"outputTokens":1,"latencyMs":471.53241600000183},{"questionId":"q24","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"9","isCorrect":false,"inputTokens":6519,"outputTokens":1,"latencyMs":341.24662499999977},{"questionId":"q24","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":4110,"outputTokens":1,"latencyMs":502.4761250000156},{"questionId":"q24","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":2689,"outputTokens":1,"latencyMs":351.05816700001014},{"questionId":"q24","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"21","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":355.1737500000163},{"questionId":"q24","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":511.81508299999405},{"questionId":"q24","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"13","isCorrect":false,"inputTokens":5152,"outputTokens":1,"latencyMs":357.3190829999803},{"questionId":"q25","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":6519,"outputTokens":1,"latencyMs":537.1332079999847},{"questionId":"q25","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"8","isCorrect":false,"inputTokens":4110,"outputTokens":1,"latencyMs":347.58433300000615},{"questionId":"q25","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"17","isCorrect":false,"inputTokens":2689,"outputTokens":1,"latencyMs":384.00466700000106},{"questionId":"q25","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"17","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":478.09291700000176},{"questionId":"q25","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"8","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":429.55516699998407},{"questionId":"q25","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":5152,"outputTokens":1,"latencyMs":289.97816600001534},{"questionId":"q26","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"12","actual":"7","isCorrect":false,"inputTokens":6519,"outputTokens":1,"latencyMs":438.1736250000249},{"questionId":"q26","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"12","actual":"7","isCorrect":false,"inputTokens":4110,"outputTokens":1,"latencyMs":457.51041699998314},{"questionId":"q26","format":"toon","model":"grok-4-fast-non-reasoning","expected":"12","actual":"18","isCorrect":false,"inputTokens":2689,"outputTokens":1,"latencyMs":412.01324999998906},{"questionId":"q26","format":"csv","model":"grok-4-fast-non-reasoning","expected":"12","actual":"18","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":404.7447500000126},{"questionId":"q26","format":"xml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"6","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":574.9759580000245},{"questionId":"q26","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"8","isCorrect":false,"inputTokens":5152,"outputTokens":1,"latencyMs":429.6673750000191},{"questionId":"q27","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"16","isCorrect":false,"inputTokens":6519,"outputTokens":1,"latencyMs":515.0940410000039},{"questionId":"q27","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":4110,"outputTokens":1,"latencyMs":498.8452920000127},{"questionId":"q27","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"19","isCorrect":false,"inputTokens":2689,"outputTokens":1,"latencyMs":386.86858400001074},{"questionId":"q27","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"19","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":418.4909579999803},{"questionId":"q27","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":535.0260000000126},{"questionId":"q27","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"14","isCorrect":false,"inputTokens":5152,"outputTokens":1,"latencyMs":330.5213749999821},{"questionId":"q28","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"63","actual":"78","isCorrect":false,"inputTokens":6518,"outputTokens":1,"latencyMs":474.9635410000046},{"questionId":"q28","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"63","actual":"76","isCorrect":false,"inputTokens":4109,"outputTokens":1,"latencyMs":421.59404199998244},{"questionId":"q28","format":"toon","model":"grok-4-fast-non-reasoning","expected":"63","actual":"79","isCorrect":false,"inputTokens":2688,"outputTokens":1,"latencyMs":415.17358299999614},{"questionId":"q28","format":"csv","model":"grok-4-fast-non-reasoning","expected":"63","actual":"79","isCorrect":false,"inputTokens":2523,"outputTokens":1,"latencyMs":322.5785409999953},{"questionId":"q28","format":"xml","model":"grok-4-fast-non-reasoning","expected":"63","actual":"79","isCorrect":false,"inputTokens":7455,"outputTokens":1,"latencyMs":453.7339590000047},{"questionId":"q28","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"63","actual":"79","isCorrect":false,"inputTokens":5151,"outputTokens":1,"latencyMs":398.2559170000022},{"questionId":"q29","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"53","actual":"52","isCorrect":false,"inputTokens":6518,"outputTokens":1,"latencyMs":442.93283400000655},{"questionId":"q29","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"53","actual":"50","isCorrect":false,"inputTokens":4109,"outputTokens":1,"latencyMs":343.1851670000178},{"questionId":"q29","format":"toon","model":"grok-4-fast-non-reasoning","expected":"53","actual":"58","isCorrect":false,"inputTokens":2688,"outputTokens":1,"latencyMs":359.6128749999916},{"questionId":"q29","format":"csv","model":"grok-4-fast-non-reasoning","expected":"53","actual":"63","isCorrect":false,"inputTokens":2523,"outputTokens":1,"latencyMs":572.2567500000005},{"questionId":"q29","format":"xml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"57","isCorrect":false,"inputTokens":7455,"outputTokens":1,"latencyMs":363.640167000005},{"questionId":"q29","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"59","isCorrect":false,"inputTokens":5151,"outputTokens":1,"latencyMs":355.43204200000037},{"questionId":"q30","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"39","actual":"32","isCorrect":false,"inputTokens":6518,"outputTokens":1,"latencyMs":405.43745900000795},{"questionId":"q30","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"39","actual":"30","isCorrect":false,"inputTokens":4109,"outputTokens":1,"latencyMs":342.7166670000006},{"questionId":"q30","format":"toon","model":"grok-4-fast-non-reasoning","expected":"39","actual":"31","isCorrect":false,"inputTokens":2688,"outputTokens":1,"latencyMs":328.27799999999115},{"questionId":"q30","format":"csv","model":"grok-4-fast-non-reasoning","expected":"39","actual":"38","isCorrect":false,"inputTokens":2523,"outputTokens":1,"latencyMs":356.0969160000095},{"questionId":"q30","format":"xml","model":"grok-4-fast-non-reasoning","expected":"39","actual":"28","isCorrect":false,"inputTokens":7455,"outputTokens":1,"latencyMs":527.7865420000162},{"questionId":"q30","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"39","actual":"28","isCorrect":false,"inputTokens":5151,"outputTokens":1,"latencyMs":365.9529160000093},{"questionId":"q31","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"16","isCorrect":false,"inputTokens":6519,"outputTokens":1,"latencyMs":427.5779170000169},{"questionId":"q31","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":4110,"outputTokens":1,"latencyMs":367.35283399999025},{"questionId":"q31","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"27","isCorrect":false,"inputTokens":2689,"outputTokens":1,"latencyMs":362.78220899999724},{"questionId":"q31","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"24","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":439.2202919999836},{"questionId":"q31","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"16","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":336.6583329999994},{"questionId":"q31","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"13","isCorrect":false,"inputTokens":5152,"outputTokens":1,"latencyMs":344.7207500000077},{"questionId":"q32","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"8","actual":"14","isCorrect":false,"inputTokens":6519,"outputTokens":1,"latencyMs":362.5337500000023},{"questionId":"q32","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":4110,"outputTokens":1,"latencyMs":422.4656660000037},{"questionId":"q32","format":"toon","model":"grok-4-fast-non-reasoning","expected":"8","actual":"19","isCorrect":false,"inputTokens":2689,"outputTokens":1,"latencyMs":354.0910000000149},{"questionId":"q32","format":"csv","model":"grok-4-fast-non-reasoning","expected":"8","actual":"21","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":3352.1979170000122},{"questionId":"q32","format":"xml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"18","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":401.66245899998466},{"questionId":"q32","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"15","isCorrect":false,"inputTokens":5152,"outputTokens":1,"latencyMs":357.48762500000885},{"questionId":"q33","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"15","actual":"14","isCorrect":false,"inputTokens":6519,"outputTokens":1,"latencyMs":464.58083300001454},{"questionId":"q33","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"15","actual":"12","isCorrect":false,"inputTokens":4110,"outputTokens":1,"latencyMs":339.05816700001014},{"questionId":"q33","format":"toon","model":"grok-4-fast-non-reasoning","expected":"15","actual":"15","isCorrect":true,"inputTokens":2689,"outputTokens":1,"latencyMs":322.0691250000091},{"questionId":"q33","format":"csv","model":"grok-4-fast-non-reasoning","expected":"15","actual":"17","isCorrect":false,"inputTokens":2524,"outputTokens":1,"latencyMs":326.74733400001423},{"questionId":"q33","format":"xml","model":"grok-4-fast-non-reasoning","expected":"15","actual":"13","isCorrect":false,"inputTokens":7456,"outputTokens":1,"latencyMs":415.8575830000045},{"questionId":"q33","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"15","actual":"15","isCorrect":true,"inputTokens":5152,"outputTokens":1,"latencyMs":373.81012499998906},{"questionId":"q34","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":6513,"outputTokens":1,"latencyMs":346.1486670000013},{"questionId":"q34","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":4104,"outputTokens":1,"latencyMs":435.0612919999985},{"questionId":"q34","format":"toon","model":"grok-4-fast-non-reasoning","expected":"12","actual":"20","isCorrect":false,"inputTokens":2683,"outputTokens":1,"latencyMs":5182.362292000034},{"questionId":"q34","format":"csv","model":"grok-4-fast-non-reasoning","expected":"12","actual":"20","isCorrect":false,"inputTokens":2518,"outputTokens":1,"latencyMs":473.3698329999461},{"questionId":"q34","format":"xml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":7450,"outputTokens":1,"latencyMs":601.1534170000232},{"questionId":"q34","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":5146,"outputTokens":1,"latencyMs":459.3080830000108},{"questionId":"q35","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"19","isCorrect":false,"inputTokens":6513,"outputTokens":1,"latencyMs":561.8072500000126},{"questionId":"q35","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"14","isCorrect":false,"inputTokens":4104,"outputTokens":1,"latencyMs":440.85999999998603},{"questionId":"q35","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"19","isCorrect":false,"inputTokens":2683,"outputTokens":1,"latencyMs":386.76491700002225},{"questionId":"q35","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"20","isCorrect":false,"inputTokens":2518,"outputTokens":1,"latencyMs":405.76008400000865},{"questionId":"q35","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"21","isCorrect":false,"inputTokens":7450,"outputTokens":1,"latencyMs":5173.215124999988},{"questionId":"q35","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"19","isCorrect":false,"inputTokens":5146,"outputTokens":1,"latencyMs":426.1099169999943},{"questionId":"q36","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":10956,"outputTokens":3,"latencyMs":566.8539999999921},{"questionId":"q36","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":6935,"outputTokens":3,"latencyMs":414.9709579999908},{"questionId":"q36","format":"toon","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":7358,"outputTokens":3,"latencyMs":523.9318339999882},{"questionId":"q36","format":"xml","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":12282,"outputTokens":3,"latencyMs":558.1745830000145},{"questionId":"q36","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8521,"outputTokens":3,"latencyMs":512.4016249999986},{"questionId":"q37","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":10956,"outputTokens":2,"latencyMs":663.1533329999656},{"questionId":"q37","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":6935,"outputTokens":2,"latencyMs":623.3991669999668},{"questionId":"q37","format":"toon","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":7358,"outputTokens":2,"latencyMs":464.65949999995064},{"questionId":"q37","format":"xml","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":12282,"outputTokens":2,"latencyMs":550.274167000025},{"questionId":"q37","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8521,"outputTokens":2,"latencyMs":539.2086249999702},{"questionId":"q38","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":10956,"outputTokens":3,"latencyMs":640.108666999964},{"questionId":"q38","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":6935,"outputTokens":3,"latencyMs":460.2317499999772},{"questionId":"q38","format":"toon","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":7358,"outputTokens":3,"latencyMs":589.3290419999976},{"questionId":"q38","format":"xml","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":12282,"outputTokens":3,"latencyMs":697.7994999999646},{"questionId":"q38","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8521,"outputTokens":3,"latencyMs":514.6527089999872},{"questionId":"q39","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":10956,"outputTokens":1,"latencyMs":575.9620829999913},{"questionId":"q39","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":6935,"outputTokens":1,"latencyMs":397.76016599999275},{"questionId":"q39","format":"toon","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":7358,"outputTokens":1,"latencyMs":553.4510000000009},{"questionId":"q39","format":"xml","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":12282,"outputTokens":1,"latencyMs":590.0400000000373},{"questionId":"q39","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8521,"outputTokens":1,"latencyMs":531.2707499999669},{"questionId":"q40","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":10956,"outputTokens":4,"latencyMs":540.1289589999942},{"questionId":"q40","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":6935,"outputTokens":4,"latencyMs":477.3901659999974},{"questionId":"q40","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":7358,"outputTokens":4,"latencyMs":455.8447499999893},{"questionId":"q40","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":12282,"outputTokens":4,"latencyMs":412.3774169999524},{"questionId":"q40","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8521,"outputTokens":4,"latencyMs":482.77370800002245},{"questionId":"q41","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":10956,"outputTokens":1,"latencyMs":602.7812920000288},{"questionId":"q41","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":6935,"outputTokens":1,"latencyMs":428.48158400005195},{"questionId":"q41","format":"toon","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":7358,"outputTokens":1,"latencyMs":407.5885000000126},{"questionId":"q41","format":"xml","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":12282,"outputTokens":1,"latencyMs":537.2538749999949},{"questionId":"q41","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8521,"outputTokens":1,"latencyMs":344.9529999999795},{"questionId":"q42","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":10956,"outputTokens":4,"latencyMs":549.2679170000483},{"questionId":"q42","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":6935,"outputTokens":4,"latencyMs":405.8097500000149},{"questionId":"q42","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":7358,"outputTokens":4,"latencyMs":480.70799999998417},{"questionId":"q42","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":12282,"outputTokens":4,"latencyMs":561.4101249999949},{"questionId":"q42","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8521,"outputTokens":4,"latencyMs":539.778667000006},{"questionId":"q43","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10956,"outputTokens":2,"latencyMs":758.497583999997},{"questionId":"q43","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6935,"outputTokens":2,"latencyMs":520.9442500000005},{"questionId":"q43","format":"toon","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7358,"outputTokens":2,"latencyMs":449.63329099997645},{"questionId":"q43","format":"xml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12282,"outputTokens":2,"latencyMs":738.6963749999995},{"questionId":"q43","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8521,"outputTokens":2,"latencyMs":622.8419170000125},{"questionId":"q44","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":10957,"outputTokens":6,"latencyMs":757.1417499999516},{"questionId":"q44","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":6936,"outputTokens":6,"latencyMs":902.6479169999948},{"questionId":"q44","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":7359,"outputTokens":6,"latencyMs":625.1648339999956},{"questionId":"q44","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":12283,"outputTokens":6,"latencyMs":842.1151660000323},{"questionId":"q44","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8522,"outputTokens":6,"latencyMs":820.8025420000195},{"questionId":"q45","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":10957,"outputTokens":9,"latencyMs":899.1557079999475},{"questionId":"q45","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":6936,"outputTokens":9,"latencyMs":520.8798330000136},{"questionId":"q45","format":"toon","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":7359,"outputTokens":9,"latencyMs":565.7481670000125},{"questionId":"q45","format":"xml","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":12283,"outputTokens":9,"latencyMs":464.50062500004424},{"questionId":"q45","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8522,"outputTokens":9,"latencyMs":459.5018329999875},{"questionId":"q46","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":10957,"outputTokens":6,"latencyMs":452.3087500000256},{"questionId":"q46","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":6936,"outputTokens":6,"latencyMs":510.81529200001387},{"questionId":"q46","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":7359,"outputTokens":6,"latencyMs":437.3572909999639},{"questionId":"q46","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":12283,"outputTokens":6,"latencyMs":634.0092500000028},{"questionId":"q46","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8522,"outputTokens":6,"latencyMs":628.0129170000437},{"questionId":"q47","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":10956,"outputTokens":1,"latencyMs":552.6011249999865},{"questionId":"q47","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":6935,"outputTokens":1,"latencyMs":358.6222499999567},{"questionId":"q47","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":7358,"outputTokens":1,"latencyMs":371.4452910000109},{"questionId":"q47","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":12282,"outputTokens":1,"latencyMs":569.9203330000164},{"questionId":"q47","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":8521,"outputTokens":1,"latencyMs":636.978833000001},{"questionId":"q48","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":10957,"outputTokens":4,"latencyMs":665.875624999986},{"questionId":"q48","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":6936,"outputTokens":4,"latencyMs":487.7277499999618},{"questionId":"q48","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":7359,"outputTokens":4,"latencyMs":426.6503340000054},{"questionId":"q48","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":12283,"outputTokens":4,"latencyMs":678.2607500000158},{"questionId":"q48","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8522,"outputTokens":4,"latencyMs":480.61691599996993},{"questionId":"q49","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":10957,"outputTokens":7,"latencyMs":464.1542499999632},{"questionId":"q49","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":6936,"outputTokens":7,"latencyMs":443.76504199998453},{"questionId":"q49","format":"toon","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":7359,"outputTokens":7,"latencyMs":438.383124999993},{"questionId":"q49","format":"xml","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":12283,"outputTokens":7,"latencyMs":470.35570899996674},{"questionId":"q49","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8522,"outputTokens":7,"latencyMs":530.829832999967},{"questionId":"q50","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":10957,"outputTokens":6,"latencyMs":475.8803750000079},{"questionId":"q50","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":6936,"outputTokens":6,"latencyMs":441.16533300001174},{"questionId":"q50","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":7359,"outputTokens":6,"latencyMs":467.2861660000053},{"questionId":"q50","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":12283,"outputTokens":6,"latencyMs":400.08862500003306},{"questionId":"q50","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2025-09-20","actual":"2025-09-20","isCorrect":true,"inputTokens":8522,"outputTokens":6,"latencyMs":510.7727500000037},{"questionId":"q51","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":10956,"outputTokens":1,"latencyMs":410.404709000024},{"questionId":"q51","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"6","isCorrect":false,"inputTokens":6935,"outputTokens":1,"latencyMs":376.0299160000286},{"questionId":"q51","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":7358,"outputTokens":1,"latencyMs":342.0409160000272},{"questionId":"q51","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"6","isCorrect":false,"inputTokens":12282,"outputTokens":1,"latencyMs":569.6756670000032},{"questionId":"q51","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"6","isCorrect":false,"inputTokens":8521,"outputTokens":1,"latencyMs":493.2434169999906},{"questionId":"q52","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"12","isCorrect":false,"inputTokens":10953,"outputTokens":1,"latencyMs":659.4615829999675},{"questionId":"q52","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":6932,"outputTokens":1,"latencyMs":446.51141700003063},{"questionId":"q52","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":7355,"outputTokens":1,"latencyMs":338.02987499994924},{"questionId":"q52","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"12","isCorrect":false,"inputTokens":12279,"outputTokens":1,"latencyMs":352.6787079999922},{"questionId":"q52","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":8518,"outputTokens":1,"latencyMs":391.76895800000057},{"questionId":"q53","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":10953,"outputTokens":1,"latencyMs":632.9206249999697},{"questionId":"q53","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":6932,"outputTokens":1,"latencyMs":389.9041660000221},{"questionId":"q53","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":7355,"outputTokens":1,"latencyMs":630.5223750000005},{"questionId":"q53","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":12279,"outputTokens":1,"latencyMs":480.4078749999753},{"questionId":"q53","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":8518,"outputTokens":1,"latencyMs":488.68216600001324},{"questionId":"q54","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":10954,"outputTokens":1,"latencyMs":665.2814580000122},{"questionId":"q54","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"8","isCorrect":false,"inputTokens":6933,"outputTokens":1,"latencyMs":595.6088749999763},{"questionId":"q54","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":7356,"outputTokens":1,"latencyMs":574.077249999973},{"questionId":"q54","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":12280,"outputTokens":1,"latencyMs":812.8848330000183},{"questionId":"q54","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":8519,"outputTokens":1,"latencyMs":554.0167910000309},{"questionId":"q55","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"33855.03","isCorrect":false,"inputTokens":10954,"outputTokens":4,"latencyMs":691.8770420000073},{"questionId":"q55","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"27875.82","isCorrect":false,"inputTokens":6933,"outputTokens":4,"latencyMs":622.2279170000111},{"questionId":"q55","format":"toon","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"27670.94","isCorrect":false,"inputTokens":7356,"outputTokens":4,"latencyMs":555.1483330000192},{"questionId":"q55","format":"xml","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"24971.85","isCorrect":false,"inputTokens":12280,"outputTokens":4,"latencyMs":620.6162089999998},{"questionId":"q55","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"23931.39","isCorrect":false,"inputTokens":8519,"outputTokens":4,"latencyMs":552.059917000006},{"questionId":"q56","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"729.95","isCorrect":false,"inputTokens":10952,"outputTokens":3,"latencyMs":463.0561250000028},{"questionId":"q56","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"749.76","isCorrect":false,"inputTokens":6931,"outputTokens":3,"latencyMs":398.0659999999916},{"questionId":"q56","format":"toon","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"839.12","isCorrect":false,"inputTokens":7354,"outputTokens":3,"latencyMs":389.1647499999963},{"questionId":"q56","format":"xml","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"917.11","isCorrect":false,"inputTokens":12278,"outputTokens":3,"latencyMs":483.8302079999703},{"questionId":"q56","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"739.07","isCorrect":false,"inputTokens":8517,"outputTokens":3,"latencyMs":391.6107089999714},{"questionId":"q57","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":10953,"outputTokens":1,"latencyMs":460.79416699998546},{"questionId":"q57","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":6932,"outputTokens":1,"latencyMs":396.0734999999986},{"questionId":"q57","format":"toon","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":7355,"outputTokens":1,"latencyMs":419.44516699999804},{"questionId":"q57","format":"xml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":12279,"outputTokens":1,"latencyMs":419.5071660000249},{"questionId":"q57","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":8518,"outputTokens":1,"latencyMs":657.1186250000028},{"questionId":"q58","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":10952,"outputTokens":4,"latencyMs":534.1697500000009},{"questionId":"q58","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":6931,"outputTokens":4,"latencyMs":470.96020799997495},{"questionId":"q58","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":7354,"outputTokens":4,"latencyMs":427.40983299998334},{"questionId":"q58","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":12278,"outputTokens":4,"latencyMs":794.905832999968},{"questionId":"q58","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8517,"outputTokens":4,"latencyMs":360.6685830000206},{"questionId":"q59","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"43","actual":"33","isCorrect":false,"inputTokens":10956,"outputTokens":1,"latencyMs":619.0210419999785},{"questionId":"q59","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"43","actual":"35","isCorrect":false,"inputTokens":6935,"outputTokens":1,"latencyMs":430.27820800000336},{"questionId":"q59","format":"toon","model":"grok-4-fast-non-reasoning","expected":"43","actual":"38","isCorrect":false,"inputTokens":7358,"outputTokens":1,"latencyMs":434.2036659999867},{"questionId":"q59","format":"xml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"29","isCorrect":false,"inputTokens":12282,"outputTokens":1,"latencyMs":662.0124999999534},{"questionId":"q59","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"30","isCorrect":false,"inputTokens":8521,"outputTokens":1,"latencyMs":495.84416700003203},{"questionId":"q60","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"37","actual":"25","isCorrect":false,"inputTokens":10956,"outputTokens":1,"latencyMs":465.0403749999823},{"questionId":"q60","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"37","actual":"27","isCorrect":false,"inputTokens":6935,"outputTokens":1,"latencyMs":397.51029199996265},{"questionId":"q60","format":"toon","model":"grok-4-fast-non-reasoning","expected":"37","actual":"29","isCorrect":false,"inputTokens":7358,"outputTokens":1,"latencyMs":455.74883300001966},{"questionId":"q60","format":"xml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"25","isCorrect":false,"inputTokens":12282,"outputTokens":1,"latencyMs":568.1555420000223},{"questionId":"q60","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"25","isCorrect":false,"inputTokens":8521,"outputTokens":1,"latencyMs":470.5608749999665},{"questionId":"q61","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"28","actual":"20","isCorrect":false,"inputTokens":10956,"outputTokens":1,"latencyMs":385.2380830000038},{"questionId":"q61","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"28","actual":"16","isCorrect":false,"inputTokens":6935,"outputTokens":1,"latencyMs":874.853375000006},{"questionId":"q61","format":"toon","model":"grok-4-fast-non-reasoning","expected":"28","actual":"20","isCorrect":false,"inputTokens":7358,"outputTokens":1,"latencyMs":450.92887499998324},{"questionId":"q61","format":"xml","model":"grok-4-fast-non-reasoning","expected":"28","actual":"15","isCorrect":false,"inputTokens":12282,"outputTokens":1,"latencyMs":529.7754159999895},{"questionId":"q61","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"28","actual":"18","isCorrect":false,"inputTokens":8521,"outputTokens":1,"latencyMs":382.37229099997785},{"questionId":"q62","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"8","actual":"7","isCorrect":false,"inputTokens":10960,"outputTokens":1,"latencyMs":551.9976659999811},{"questionId":"q62","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"8","actual":"8","isCorrect":true,"inputTokens":6939,"outputTokens":1,"latencyMs":314.4155000000028},{"questionId":"q62","format":"toon","model":"grok-4-fast-non-reasoning","expected":"8","actual":"8","isCorrect":true,"inputTokens":7362,"outputTokens":1,"latencyMs":335.5158340000198},{"questionId":"q62","format":"xml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":12286,"outputTokens":1,"latencyMs":396.2446250000503},{"questionId":"q62","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"6","isCorrect":false,"inputTokens":8525,"outputTokens":1,"latencyMs":436.60504200001014},{"questionId":"q63","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6","actual":"8","isCorrect":false,"inputTokens":10960,"outputTokens":1,"latencyMs":356.6454590000212},{"questionId":"q63","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6","actual":"7","isCorrect":false,"inputTokens":6939,"outputTokens":1,"latencyMs":435.14762499998324},{"questionId":"q63","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":7362,"outputTokens":1,"latencyMs":452.6919999999809},{"questionId":"q63","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"10","isCorrect":false,"inputTokens":12286,"outputTokens":1,"latencyMs":467.1829999999609},{"questionId":"q63","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"8","isCorrect":false,"inputTokens":8525,"outputTokens":1,"latencyMs":381.6025839999784},{"questionId":"q64","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"9","isCorrect":false,"inputTokens":10961,"outputTokens":1,"latencyMs":592.5714999999618},{"questionId":"q64","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":6940,"outputTokens":1,"latencyMs":473.8131660000072},{"questionId":"q64","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"11","isCorrect":false,"inputTokens":7363,"outputTokens":1,"latencyMs":609.5063330000266},{"questionId":"q64","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"9","isCorrect":false,"inputTokens":12287,"outputTokens":1,"latencyMs":572.6724580000155},{"questionId":"q64","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"7","isCorrect":false,"inputTokens":8526,"outputTokens":1,"latencyMs":756.585625000007},{"questionId":"q65","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"9","actual":"6","isCorrect":false,"inputTokens":10961,"outputTokens":1,"latencyMs":539.3232500000158},{"questionId":"q65","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"9","actual":"6","isCorrect":false,"inputTokens":6940,"outputTokens":1,"latencyMs":495.69016599998577},{"questionId":"q65","format":"toon","model":"grok-4-fast-non-reasoning","expected":"9","actual":"6","isCorrect":false,"inputTokens":7363,"outputTokens":1,"latencyMs":541.1602499999572},{"questionId":"q65","format":"xml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"7","isCorrect":false,"inputTokens":12287,"outputTokens":1,"latencyMs":452.36516600003233},{"questionId":"q65","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"5","isCorrect":false,"inputTokens":8526,"outputTokens":1,"latencyMs":600.1375000000116},{"questionId":"q66","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":10960,"outputTokens":1,"latencyMs":398.22683300002245},{"questionId":"q66","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"0","isCorrect":false,"inputTokens":6939,"outputTokens":1,"latencyMs":355.4437909999979},{"questionId":"q66","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":7362,"outputTokens":1,"latencyMs":421.15708400000585},{"questionId":"q66","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":12286,"outputTokens":1,"latencyMs":580.3251659999951},{"questionId":"q66","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"0","isCorrect":false,"inputTokens":8525,"outputTokens":1,"latencyMs":404.0891249999986},{"questionId":"q67","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"5","isCorrect":false,"inputTokens":10960,"outputTokens":1,"latencyMs":635.1647499999963},{"questionId":"q67","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":6939,"outputTokens":1,"latencyMs":467.4698750000098},{"questionId":"q67","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":7362,"outputTokens":1,"latencyMs":442.25512499996694},{"questionId":"q67","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"6","isCorrect":false,"inputTokens":12286,"outputTokens":1,"latencyMs":548.8180839999695},{"questionId":"q67","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":8525,"outputTokens":1,"latencyMs":463.51537500001723},{"questionId":"q68","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":10961,"outputTokens":1,"latencyMs":458.2318749999977},{"questionId":"q68","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":6940,"outputTokens":1,"latencyMs":396.87979199999245},{"questionId":"q68","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"3","isCorrect":false,"inputTokens":7363,"outputTokens":1,"latencyMs":413.3233749999781},{"questionId":"q68","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"7","isCorrect":false,"inputTokens":12287,"outputTokens":1,"latencyMs":378.3361249999725},{"questionId":"q68","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"3","isCorrect":false,"inputTokens":8526,"outputTokens":1,"latencyMs":612.5968749999884},{"questionId":"q69","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"20","actual":"12","isCorrect":false,"inputTokens":10962,"outputTokens":1,"latencyMs":604.1281670000171},{"questionId":"q69","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"20","actual":"7","isCorrect":false,"inputTokens":6941,"outputTokens":1,"latencyMs":347.44645799999125},{"questionId":"q69","format":"toon","model":"grok-4-fast-non-reasoning","expected":"20","actual":"7","isCorrect":false,"inputTokens":7364,"outputTokens":1,"latencyMs":426.0782500000205},{"questionId":"q69","format":"xml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"14","isCorrect":false,"inputTokens":12288,"outputTokens":1,"latencyMs":522.594625000027},{"questionId":"q69","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"7","isCorrect":false,"inputTokens":8527,"outputTokens":1,"latencyMs":575.0527080000029},{"questionId":"q70","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"19","actual":"9","isCorrect":false,"inputTokens":10962,"outputTokens":1,"latencyMs":409.7545829999726},{"questionId":"q70","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"19","actual":"8","isCorrect":false,"inputTokens":6941,"outputTokens":1,"latencyMs":407.54341600002954},{"questionId":"q70","format":"toon","model":"grok-4-fast-non-reasoning","expected":"19","actual":"10","isCorrect":false,"inputTokens":7364,"outputTokens":1,"latencyMs":356.71583299996564},{"questionId":"q70","format":"xml","model":"grok-4-fast-non-reasoning","expected":"19","actual":"12","isCorrect":false,"inputTokens":12288,"outputTokens":1,"latencyMs":385.07058400003007},{"questionId":"q70","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"19","actual":"8","isCorrect":false,"inputTokens":8527,"outputTokens":1,"latencyMs":548.9263749999809},{"questionId":"q71","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3826,"outputTokens":2,"latencyMs":329.69054199999664},{"questionId":"q71","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2497,"outputTokens":2,"latencyMs":346.65991699998267},{"questionId":"q71","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1756,"outputTokens":2,"latencyMs":361.6430839999812},{"questionId":"q71","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1615,"outputTokens":2,"latencyMs":340.7061250000261},{"questionId":"q71","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4472,"outputTokens":2,"latencyMs":375.5275000000256},{"questionId":"q71","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3156,"outputTokens":2,"latencyMs":381.57520899997326},{"questionId":"q72","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3826,"outputTokens":4,"latencyMs":334.3855420000036},{"questionId":"q72","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2497,"outputTokens":4,"latencyMs":431.2075000000186},{"questionId":"q72","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1756,"outputTokens":4,"latencyMs":417.43779199995333},{"questionId":"q72","format":"csv","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1615,"outputTokens":4,"latencyMs":361.8493749999907},{"questionId":"q72","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4472,"outputTokens":4,"latencyMs":368.65762499999255},{"questionId":"q72","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3156,"outputTokens":4,"latencyMs":392.60095799999544},{"questionId":"q73","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3827,"outputTokens":3,"latencyMs":377.71758300001966},{"questionId":"q73","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2498,"outputTokens":3,"latencyMs":371.80779099999927},{"questionId":"q73","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1757,"outputTokens":3,"latencyMs":345.0765829999582},{"questionId":"q73","format":"csv","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1616,"outputTokens":3,"latencyMs":535.0971670000581},{"questionId":"q73","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4473,"outputTokens":3,"latencyMs":469.2517920000246},{"questionId":"q73","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3157,"outputTokens":3,"latencyMs":496.89879199990537},{"questionId":"q74","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":3827,"outputTokens":1,"latencyMs":486.6630000000587},{"questionId":"q74","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":2498,"outputTokens":1,"latencyMs":469.28129199997056},{"questionId":"q74","format":"toon","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":1757,"outputTokens":1,"latencyMs":459.7913750000298},{"questionId":"q74","format":"csv","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":1616,"outputTokens":1,"latencyMs":472.8293330000015},{"questionId":"q74","format":"xml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":4473,"outputTokens":1,"latencyMs":553.0435829999624},{"questionId":"q74","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":3157,"outputTokens":1,"latencyMs":483.9387919999426},{"questionId":"q75","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3826,"outputTokens":2,"latencyMs":449.8296669999836},{"questionId":"q75","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2497,"outputTokens":2,"latencyMs":305.42183300002944},{"questionId":"q75","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1756,"outputTokens":2,"latencyMs":317.5880410000682},{"questionId":"q75","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1615,"outputTokens":2,"latencyMs":275.8347500000382},{"questionId":"q75","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4472,"outputTokens":2,"latencyMs":371.0335840000771},{"questionId":"q75","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3156,"outputTokens":2,"latencyMs":309.938790999935},{"questionId":"q76","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3826,"outputTokens":4,"latencyMs":507.03887499996927},{"questionId":"q76","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2497,"outputTokens":4,"latencyMs":391.26070800004527},{"questionId":"q76","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1756,"outputTokens":4,"latencyMs":354.82066600001417},{"questionId":"q76","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1615,"outputTokens":4,"latencyMs":388.5782500000205},{"questionId":"q76","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4472,"outputTokens":4,"latencyMs":410.16641700000037},{"questionId":"q76","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3156,"outputTokens":4,"latencyMs":325.99412500008475},{"questionId":"q77","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3827,"outputTokens":3,"latencyMs":489.9527920000255},{"questionId":"q77","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2498,"outputTokens":3,"latencyMs":335.36324999993667},{"questionId":"q77","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1757,"outputTokens":3,"latencyMs":306.72416699992027},{"questionId":"q77","format":"csv","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1616,"outputTokens":3,"latencyMs":328.4217919999501},{"questionId":"q77","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4473,"outputTokens":3,"latencyMs":334.49016699998174},{"questionId":"q77","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3157,"outputTokens":3,"latencyMs":374.46616700000595},{"questionId":"q78","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":3827,"outputTokens":1,"latencyMs":433.6642499999143},{"questionId":"q78","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":2498,"outputTokens":1,"latencyMs":305.99808299995493},{"questionId":"q78","format":"toon","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":1757,"outputTokens":1,"latencyMs":348.12816700001713},{"questionId":"q78","format":"csv","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":1616,"outputTokens":1,"latencyMs":318.9747079999652},{"questionId":"q78","format":"xml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":4473,"outputTokens":1,"latencyMs":466.9804170000134},{"questionId":"q78","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":3157,"outputTokens":1,"latencyMs":324.3272089998936},{"questionId":"q79","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3826,"outputTokens":2,"latencyMs":465.95758299995214},{"questionId":"q79","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2497,"outputTokens":2,"latencyMs":329.2257090000203},{"questionId":"q79","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1756,"outputTokens":2,"latencyMs":592.95550000004},{"questionId":"q79","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1615,"outputTokens":2,"latencyMs":338.29350000002887},{"questionId":"q79","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4472,"outputTokens":2,"latencyMs":340.1784580000676},{"questionId":"q79","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3156,"outputTokens":2,"latencyMs":387.2401250000112},{"questionId":"q80","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"60","actual":"59","isCorrect":false,"inputTokens":3823,"outputTokens":1,"latencyMs":373.2088750000112},{"questionId":"q80","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"60","actual":"58","isCorrect":false,"inputTokens":2494,"outputTokens":1,"latencyMs":355.5565829999978},{"questionId":"q80","format":"toon","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":1753,"outputTokens":1,"latencyMs":306.8279999999795},{"questionId":"q80","format":"csv","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":1612,"outputTokens":1,"latencyMs":425.7756669999799},{"questionId":"q80","format":"xml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"59","isCorrect":false,"inputTokens":4469,"outputTokens":1,"latencyMs":409.0429160000058},{"questionId":"q80","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"62","isCorrect":false,"inputTokens":3153,"outputTokens":1,"latencyMs":403.6800000000512},{"questionId":"q81","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"298902","isCorrect":false,"inputTokens":3824,"outputTokens":2,"latencyMs":396.22729199996684},{"questionId":"q81","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"338,607","isCorrect":false,"inputTokens":2495,"outputTokens":3,"latencyMs":390.9813749999739},{"questionId":"q81","format":"toon","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"317687","isCorrect":false,"inputTokens":1754,"outputTokens":2,"latencyMs":341.0073749999283},{"questionId":"q81","format":"csv","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"385,820","isCorrect":false,"inputTokens":1613,"outputTokens":3,"latencyMs":315.14962500007823},{"questionId":"q81","format":"xml","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"352779","isCorrect":false,"inputTokens":4470,"outputTokens":2,"latencyMs":484.66212499991525},{"questionId":"q81","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"258292","isCorrect":false,"inputTokens":3154,"outputTokens":2,"latencyMs":414.0405000000028},{"questionId":"q82","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1277","isCorrect":false,"inputTokens":3824,"outputTokens":2,"latencyMs":403.11999999999534},{"questionId":"q82","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1559","isCorrect":false,"inputTokens":2495,"outputTokens":2,"latencyMs":474.6341670000693},{"questionId":"q82","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1299","isCorrect":false,"inputTokens":1754,"outputTokens":2,"latencyMs":444.76437500002794},{"questionId":"q82","format":"csv","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1452","isCorrect":false,"inputTokens":1613,"outputTokens":2,"latencyMs":423.95737499999814},{"questionId":"q82","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1022","isCorrect":false,"inputTokens":4470,"outputTokens":2,"latencyMs":491.33337500004563},{"questionId":"q82","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1150","isCorrect":false,"inputTokens":3154,"outputTokens":2,"latencyMs":442.90133300004527},{"questionId":"q83","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"374352.03","isCorrect":false,"inputTokens":3822,"outputTokens":4,"latencyMs":400.19537500001024},{"questionId":"q83","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"{\"totalRevenue\":302475.91}","isCorrect":false,"inputTokens":2493,"outputTokens":9,"latencyMs":448.9623329999158},{"questionId":"q83","format":"toon","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"321046.69","isCorrect":false,"inputTokens":1752,"outputTokens":4,"latencyMs":420.30504199990537},{"questionId":"q83","format":"csv","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"281805.25","isCorrect":false,"inputTokens":1611,"outputTokens":4,"latencyMs":335.90145800006576},{"questionId":"q83","format":"xml","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"286287.61","isCorrect":false,"inputTokens":4468,"outputTokens":4,"latencyMs":387.11108300008345},{"questionId":"q83","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"290456.64","isCorrect":false,"inputTokens":3152,"outputTokens":4,"latencyMs":391.97983399999794},{"questionId":"q84","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3820,"outputTokens":3,"latencyMs":290.4728330000071},{"questionId":"q84","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.52","isCorrect":false,"inputTokens":2491,"outputTokens":3,"latencyMs":359.40116700006183},{"questionId":"q84","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1750,"outputTokens":3,"latencyMs":350.5647919999901},{"questionId":"q84","format":"csv","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1609,"outputTokens":3,"latencyMs":499.4252909999341},{"questionId":"q84","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4466,"outputTokens":3,"latencyMs":468.5248329999158},{"questionId":"q84","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3150,"outputTokens":3,"latencyMs":472.97824999992736},{"questionId":"q85","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"18","isCorrect":false,"inputTokens":3824,"outputTokens":1,"latencyMs":315.1796669999603},{"questionId":"q85","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"14","isCorrect":false,"inputTokens":2495,"outputTokens":1,"latencyMs":335.01720799994655},{"questionId":"q85","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"13","isCorrect":false,"inputTokens":1754,"outputTokens":1,"latencyMs":322.39274999999907},{"questionId":"q85","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"15","isCorrect":false,"inputTokens":1613,"outputTokens":1,"latencyMs":402.57325000001583},{"questionId":"q85","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"17","isCorrect":false,"inputTokens":4470,"outputTokens":1,"latencyMs":337.38025000004563},{"questionId":"q85","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"17","isCorrect":false,"inputTokens":3154,"outputTokens":1,"latencyMs":316.03812499996275},{"questionId":"q86","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"42","actual":"38","isCorrect":false,"inputTokens":3823,"outputTokens":1,"latencyMs":368.8665000000037},{"questionId":"q86","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"42","actual":"36","isCorrect":false,"inputTokens":2494,"outputTokens":1,"latencyMs":326.2820410000859},{"questionId":"q86","format":"toon","model":"grok-4-fast-non-reasoning","expected":"42","actual":"53","isCorrect":false,"inputTokens":1753,"outputTokens":1,"latencyMs":336.5452500000829},{"questionId":"q86","format":"csv","model":"grok-4-fast-non-reasoning","expected":"42","actual":"56","isCorrect":false,"inputTokens":1612,"outputTokens":1,"latencyMs":314.9193750000559},{"questionId":"q86","format":"xml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"58","isCorrect":false,"inputTokens":4469,"outputTokens":1,"latencyMs":435.84954100009054},{"questionId":"q86","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"27","isCorrect":false,"inputTokens":3153,"outputTokens":1,"latencyMs":359.82695899996907},{"questionId":"q87","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"20","actual":"9","isCorrect":false,"inputTokens":3830,"outputTokens":1,"latencyMs":488.08299999998417},{"questionId":"q87","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"20","actual":"10","isCorrect":false,"inputTokens":2501,"outputTokens":1,"latencyMs":314.9802909999853},{"questionId":"q87","format":"toon","model":"grok-4-fast-non-reasoning","expected":"20","actual":"16","isCorrect":false,"inputTokens":1760,"outputTokens":1,"latencyMs":335.9362919999985},{"questionId":"q87","format":"csv","model":"grok-4-fast-non-reasoning","expected":"20","actual":"15","isCorrect":false,"inputTokens":1619,"outputTokens":1,"latencyMs":314.2439589999849},{"questionId":"q87","format":"xml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"12","isCorrect":false,"inputTokens":4476,"outputTokens":1,"latencyMs":360.1144590000622},{"questionId":"q87","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"12","isCorrect":false,"inputTokens":3160,"outputTokens":1,"latencyMs":356.3955830000341},{"questionId":"q88","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"14","actual":"8","isCorrect":false,"inputTokens":3830,"outputTokens":1,"latencyMs":411.14433300006203},{"questionId":"q88","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"14","actual":"8","isCorrect":false,"inputTokens":2501,"outputTokens":1,"latencyMs":318.47408300009556},{"questionId":"q88","format":"toon","model":"grok-4-fast-non-reasoning","expected":"14","actual":"8","isCorrect":false,"inputTokens":1760,"outputTokens":1,"latencyMs":300.919249999919},{"questionId":"q88","format":"csv","model":"grok-4-fast-non-reasoning","expected":"14","actual":"9","isCorrect":false,"inputTokens":1619,"outputTokens":1,"latencyMs":323.51470800000243},{"questionId":"q88","format":"xml","model":"grok-4-fast-non-reasoning","expected":"14","actual":"9","isCorrect":false,"inputTokens":4476,"outputTokens":1,"latencyMs":432.13320800010115},{"questionId":"q88","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"14","actual":"5","isCorrect":false,"inputTokens":3160,"outputTokens":1,"latencyMs":326.80045799992513},{"questionId":"q89","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"6","isCorrect":false,"inputTokens":3830,"outputTokens":1,"latencyMs":488.59304099995643},{"questionId":"q89","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"8","isCorrect":false,"inputTokens":2501,"outputTokens":1,"latencyMs":342.5887080000248},{"questionId":"q89","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"9","isCorrect":false,"inputTokens":1760,"outputTokens":1,"latencyMs":515.8002500000875},{"questionId":"q89","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"10","isCorrect":false,"inputTokens":1619,"outputTokens":1,"latencyMs":339.3334999999497},{"questionId":"q89","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"8","isCorrect":false,"inputTokens":4476,"outputTokens":1,"latencyMs":363.9139580000192},{"questionId":"q89","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"8","isCorrect":false,"inputTokens":3160,"outputTokens":1,"latencyMs":380.3202090000268},{"questionId":"q90","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"8","isCorrect":false,"inputTokens":3830,"outputTokens":1,"latencyMs":343.7908339999849},{"questionId":"q90","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"5","isCorrect":false,"inputTokens":2501,"outputTokens":1,"latencyMs":417.3149999999441},{"questionId":"q90","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"14","isCorrect":false,"inputTokens":1760,"outputTokens":1,"latencyMs":379.86295800004154},{"questionId":"q90","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"9","isCorrect":false,"inputTokens":1619,"outputTokens":1,"latencyMs":411.82212499994785},{"questionId":"q90","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"6","isCorrect":false,"inputTokens":4476,"outputTokens":1,"latencyMs":450.1183749999618},{"questionId":"q90","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"9","isCorrect":false,"inputTokens":3160,"outputTokens":1,"latencyMs":466.2122920000693},{"questionId":"q91","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"20","actual":"9","isCorrect":false,"inputTokens":3830,"outputTokens":1,"latencyMs":393.5609580000164},{"questionId":"q91","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"20","actual":"8","isCorrect":false,"inputTokens":2501,"outputTokens":1,"latencyMs":416.19550000003073},{"questionId":"q91","format":"toon","model":"grok-4-fast-non-reasoning","expected":"20","actual":"11","isCorrect":false,"inputTokens":1760,"outputTokens":1,"latencyMs":389.74620799999684},{"questionId":"q91","format":"csv","model":"grok-4-fast-non-reasoning","expected":"20","actual":"6","isCorrect":false,"inputTokens":1619,"outputTokens":1,"latencyMs":396.1777909999946},{"questionId":"q91","format":"xml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"7","isCorrect":false,"inputTokens":4476,"outputTokens":1,"latencyMs":404.6151250000112},{"questionId":"q91","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"5","isCorrect":false,"inputTokens":3160,"outputTokens":1,"latencyMs":408.80200000002515},{"questionId":"q92","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"32","actual":"13","isCorrect":false,"inputTokens":3829,"outputTokens":1,"latencyMs":410.44745800003875},{"questionId":"q92","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"32","actual":"12","isCorrect":false,"inputTokens":2500,"outputTokens":1,"latencyMs":592.8343340000138},{"questionId":"q92","format":"toon","model":"grok-4-fast-non-reasoning","expected":"32","actual":"13","isCorrect":false,"inputTokens":1759,"outputTokens":1,"latencyMs":367.4747920000227},{"questionId":"q92","format":"csv","model":"grok-4-fast-non-reasoning","expected":"32","actual":"17","isCorrect":false,"inputTokens":1618,"outputTokens":1,"latencyMs":341.0896669999929},{"questionId":"q92","format":"xml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"13","isCorrect":false,"inputTokens":4475,"outputTokens":1,"latencyMs":361.27491699997336},{"questionId":"q92","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"12","isCorrect":false,"inputTokens":3159,"outputTokens":1,"latencyMs":325.8553339999635},{"questionId":"q93","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"9","actual":"8","isCorrect":false,"inputTokens":3829,"outputTokens":1,"latencyMs":337.24675000004936},{"questionId":"q93","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"9","actual":"5","isCorrect":false,"inputTokens":2500,"outputTokens":1,"latencyMs":375.48058299999684},{"questionId":"q93","format":"toon","model":"grok-4-fast-non-reasoning","expected":"9","actual":"9","isCorrect":true,"inputTokens":1759,"outputTokens":1,"latencyMs":348.889542000019},{"questionId":"q93","format":"csv","model":"grok-4-fast-non-reasoning","expected":"9","actual":"15","isCorrect":false,"inputTokens":1618,"outputTokens":1,"latencyMs":415.4302499999758},{"questionId":"q93","format":"xml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"5","isCorrect":false,"inputTokens":4475,"outputTokens":1,"latencyMs":360.1652499999618},{"questionId":"q93","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"5","isCorrect":false,"inputTokens":3159,"outputTokens":1,"latencyMs":343.6735000000335},{"questionId":"q94","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"26","isCorrect":false,"inputTokens":3832,"outputTokens":1,"latencyMs":350.66125000000466},{"questionId":"q94","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"24","isCorrect":false,"inputTokens":2503,"outputTokens":1,"latencyMs":340.76887500006706},{"questionId":"q94","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"26","isCorrect":false,"inputTokens":1762,"outputTokens":1,"latencyMs":324.38254200003576},{"questionId":"q94","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":1621,"outputTokens":1,"latencyMs":340.17829100007657},{"questionId":"q94","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"30","isCorrect":false,"inputTokens":4478,"outputTokens":1,"latencyMs":328.704207999981},{"questionId":"q94","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":3162,"outputTokens":1,"latencyMs":316.7704580000136},{"questionId":"q95","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"25","isCorrect":false,"inputTokens":3832,"outputTokens":1,"latencyMs":319.32687500002794},{"questionId":"q95","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":2503,"outputTokens":1,"latencyMs":313.49662500002887},{"questionId":"q95","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"27","isCorrect":false,"inputTokens":1762,"outputTokens":1,"latencyMs":318.79358399997},{"questionId":"q95","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"25","isCorrect":false,"inputTokens":1621,"outputTokens":1,"latencyMs":335.59158399992157},{"questionId":"q95","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"25","isCorrect":false,"inputTokens":4478,"outputTokens":1,"latencyMs":354.7229590000352},{"questionId":"q95","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"21","isCorrect":false,"inputTokens":3162,"outputTokens":1,"latencyMs":395.67291700001806},{"questionId":"q96","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":15243,"outputTokens":2,"latencyMs":633.4637090000324},{"questionId":"q96","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":11547,"outputTokens":2,"latencyMs":545.5627910000039},{"questionId":"q96","format":"toon","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8870,"outputTokens":2,"latencyMs":527.8811659999192},{"questionId":"q96","format":"csv","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8620,"outputTokens":2,"latencyMs":665.69316600007},{"questionId":"q96","format":"xml","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17155,"outputTokens":2,"latencyMs":728.3304999999236},{"questionId":"q96","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":13241,"outputTokens":2,"latencyMs":652.9882920000236},{"questionId":"q97","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":15244,"outputTokens":2,"latencyMs":686.7121670000488},{"questionId":"q97","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":11548,"outputTokens":2,"latencyMs":2075.6001669999678},{"questionId":"q97","format":"toon","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8871,"outputTokens":2,"latencyMs":628.827874999959},{"questionId":"q97","format":"csv","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8621,"outputTokens":2,"latencyMs":506.2241670000367},{"questionId":"q97","format":"xml","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17156,"outputTokens":2,"latencyMs":872.7743750000373},{"questionId":"q97","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":13242,"outputTokens":2,"latencyMs":675.5760409999639},{"questionId":"q98","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":15240,"outputTokens":2,"latencyMs":742.5285840000724},{"questionId":"q98","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":11544,"outputTokens":2,"latencyMs":368.8186670000432},{"questionId":"q98","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8867,"outputTokens":2,"latencyMs":538.2703749999637},{"questionId":"q98","format":"csv","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8617,"outputTokens":2,"latencyMs":628.0049169999547},{"questionId":"q98","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17152,"outputTokens":2,"latencyMs":860.1127499999711},{"questionId":"q98","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":13238,"outputTokens":2,"latencyMs":667.9358750000829},{"questionId":"q99","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":15245,"outputTokens":1,"latencyMs":550.7519999999786},{"questionId":"q99","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":11549,"outputTokens":1,"latencyMs":735.2369169999147},{"questionId":"q99","format":"toon","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":8872,"outputTokens":1,"latencyMs":480.1288329999661},{"questionId":"q99","format":"csv","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":8622,"outputTokens":1,"latencyMs":611.9510419999715},{"questionId":"q99","format":"xml","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":17157,"outputTokens":1,"latencyMs":895.7609999999404},{"questionId":"q99","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":13243,"outputTokens":1,"latencyMs":696.1840829999419},{"questionId":"q100","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":15240,"outputTokens":2,"latencyMs":794.0231669999193},{"questionId":"q100","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":11544,"outputTokens":2,"latencyMs":724.3973750000587},{"questionId":"q100","format":"toon","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8867,"outputTokens":2,"latencyMs":518.8924579999875},{"questionId":"q100","format":"csv","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8617,"outputTokens":2,"latencyMs":534.8386249999749},{"questionId":"q100","format":"xml","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17152,"outputTokens":2,"latencyMs":818.5332919999491},{"questionId":"q100","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":13238,"outputTokens":2,"latencyMs":555.0767089999281},{"questionId":"q101","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":15244,"outputTokens":2,"latencyMs":2691.729290999938},{"questionId":"q101","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":11548,"outputTokens":2,"latencyMs":2325.1379590000724},{"questionId":"q101","format":"toon","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8871,"outputTokens":2,"latencyMs":345.6618330000201},{"questionId":"q101","format":"csv","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8621,"outputTokens":2,"latencyMs":668.4125830000266},{"questionId":"q101","format":"xml","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17156,"outputTokens":2,"latencyMs":943.1000420000637},{"questionId":"q101","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":13242,"outputTokens":2,"latencyMs":565.3708750000224},{"questionId":"q102","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":15244,"outputTokens":1,"latencyMs":608.4350410000188},{"questionId":"q102","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":11548,"outputTokens":1,"latencyMs":566.187750000041},{"questionId":"q102","format":"toon","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":8871,"outputTokens":1,"latencyMs":615.8182909999741},{"questionId":"q102","format":"csv","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":8621,"outputTokens":1,"latencyMs":331.2258750000037},{"questionId":"q102","format":"xml","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":17156,"outputTokens":1,"latencyMs":808.9863749999786},{"questionId":"q102","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":13242,"outputTokens":1,"latencyMs":578.7096250000177},{"questionId":"q103","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":15243,"outputTokens":1,"latencyMs":498.01545900001656},{"questionId":"q103","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":11547,"outputTokens":1,"latencyMs":760.3997080000117},{"questionId":"q103","format":"toon","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":8870,"outputTokens":1,"latencyMs":597.3555000000633},{"questionId":"q103","format":"csv","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":8620,"outputTokens":1,"latencyMs":428.3412500000559},{"questionId":"q103","format":"xml","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":17155,"outputTokens":1,"latencyMs":696.7395000000251},{"questionId":"q103","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":13241,"outputTokens":1,"latencyMs":641.2926670000888},{"questionId":"q104","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":15246,"outputTokens":2,"latencyMs":802.5000839999411},{"questionId":"q104","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":11550,"outputTokens":2,"latencyMs":341.1172909999732},{"questionId":"q104","format":"toon","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8873,"outputTokens":2,"latencyMs":350.4842499999795},{"questionId":"q104","format":"csv","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8623,"outputTokens":2,"latencyMs":496.36562500009313},{"questionId":"q104","format":"xml","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17158,"outputTokens":2,"latencyMs":804.3026250000112},{"questionId":"q104","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":13244,"outputTokens":2,"latencyMs":713.7414159999462},{"questionId":"q105","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":15243,"outputTokens":2,"latencyMs":804.12591599999},{"questionId":"q105","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":11547,"outputTokens":2,"latencyMs":542.6296659999061},{"questionId":"q105","format":"toon","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8870,"outputTokens":2,"latencyMs":604.3150419999147},{"questionId":"q105","format":"csv","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8620,"outputTokens":2,"latencyMs":546.6159579999512},{"questionId":"q105","format":"xml","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17155,"outputTokens":2,"latencyMs":580.3338750000112},{"questionId":"q105","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":13241,"outputTokens":2,"latencyMs":627.8868750000838},{"questionId":"q106","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":15247,"outputTokens":2,"latencyMs":340.95262500003446},{"questionId":"q106","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":11551,"outputTokens":2,"latencyMs":632.842709000106},{"questionId":"q106","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8874,"outputTokens":2,"latencyMs":455.9128330000676},{"questionId":"q106","format":"csv","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8624,"outputTokens":2,"latencyMs":510.64554199995473},{"questionId":"q106","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17159,"outputTokens":2,"latencyMs":710.8952500000596},{"questionId":"q106","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":13245,"outputTokens":2,"latencyMs":570.6464580000611},{"questionId":"q107","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":15239,"outputTokens":1,"latencyMs":778.3499580000062},{"questionId":"q107","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":11543,"outputTokens":1,"latencyMs":716.1798330000602},{"questionId":"q107","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8866,"outputTokens":1,"latencyMs":672.0294170000125},{"questionId":"q107","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8616,"outputTokens":1,"latencyMs":628.1176660000347},{"questionId":"q107","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"70","isCorrect":false,"inputTokens":17151,"outputTokens":1,"latencyMs":878.0192500000121},{"questionId":"q107","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":13237,"outputTokens":1,"latencyMs":455.5393749999348},{"questionId":"q108","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"11,516,777","isCorrect":false,"inputTokens":15242,"outputTokens":5,"latencyMs":532.8352090000408},{"questionId":"q108","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"15930023","isCorrect":false,"inputTokens":11546,"outputTokens":3,"latencyMs":772.8401669999585},{"questionId":"q108","format":"toon","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"10000000","isCorrect":false,"inputTokens":8869,"outputTokens":3,"latencyMs":471.41133299993817},{"questionId":"q108","format":"csv","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"12759625","isCorrect":false,"inputTokens":8619,"outputTokens":3,"latencyMs":560.2020839999896},{"questionId":"q108","format":"xml","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"11947297","isCorrect":false,"inputTokens":17154,"outputTokens":3,"latencyMs":866.3124169999501},{"questionId":"q108","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"7,804,745","isCorrect":false,"inputTokens":13240,"outputTokens":5,"latencyMs":627.4507079999894},{"questionId":"q109","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1861310","isCorrect":false,"inputTokens":15242,"outputTokens":3,"latencyMs":633.7115830000257},{"questionId":"q109","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1376255","isCorrect":false,"inputTokens":11546,"outputTokens":3,"latencyMs":572.0788750000065},{"questionId":"q109","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"905874","isCorrect":false,"inputTokens":8869,"outputTokens":2,"latencyMs":625.1871249999385},{"questionId":"q109","format":"csv","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1094841","isCorrect":false,"inputTokens":8619,"outputTokens":3,"latencyMs":667.4393329999875},{"questionId":"q109","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1413102","isCorrect":false,"inputTokens":17154,"outputTokens":3,"latencyMs":768.5377500000177},{"questionId":"q109","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1266780","isCorrect":false,"inputTokens":13240,"outputTokens":3,"latencyMs":356.4237499999581},{"questionId":"q110","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"148950.58","isCorrect":false,"inputTokens":15241,"outputTokens":4,"latencyMs":449.2469999999739},{"questionId":"q110","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"{\"averageStars\":136899.08}","isCorrect":false,"inputTokens":11545,"outputTokens":9,"latencyMs":691.4008339999709},{"questionId":"q110","format":"toon","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"140801","isCorrect":false,"inputTokens":8868,"outputTokens":2,"latencyMs":711.2465000000084},{"questionId":"q110","format":"csv","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"134431","isCorrect":false,"inputTokens":8618,"outputTokens":2,"latencyMs":640.9051250000484},{"questionId":"q110","format":"xml","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"126649","isCorrect":false,"inputTokens":17153,"outputTokens":2,"latencyMs":476.703375000041},{"questionId":"q110","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"126389","isCorrect":false,"inputTokens":13239,"outputTokens":2,"latencyMs":681.1675840000389},{"questionId":"q111","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"41","actual":"39","isCorrect":false,"inputTokens":15243,"outputTokens":1,"latencyMs":716.4929579999298},{"questionId":"q111","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"41","actual":"39","isCorrect":false,"inputTokens":11547,"outputTokens":1,"latencyMs":698.8853339999914},{"questionId":"q111","format":"toon","model":"grok-4-fast-non-reasoning","expected":"41","actual":"25","isCorrect":false,"inputTokens":8870,"outputTokens":1,"latencyMs":522.2306660000468},{"questionId":"q111","format":"csv","model":"grok-4-fast-non-reasoning","expected":"41","actual":"22","isCorrect":false,"inputTokens":8620,"outputTokens":1,"latencyMs":468.0676659999881},{"questionId":"q111","format":"xml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"25","isCorrect":false,"inputTokens":17155,"outputTokens":1,"latencyMs":704.2454579999903},{"questionId":"q111","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"25","isCorrect":false,"inputTokens":13241,"outputTokens":1,"latencyMs":744.2714999999152},{"questionId":"q112","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"53","actual":"39","isCorrect":false,"inputTokens":15243,"outputTokens":1,"latencyMs":655.9435419999063},{"questionId":"q112","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"53","actual":"23","isCorrect":false,"inputTokens":11547,"outputTokens":1,"latencyMs":629.3833749999758},{"questionId":"q112","format":"toon","model":"grok-4-fast-non-reasoning","expected":"53","actual":"27","isCorrect":false,"inputTokens":8870,"outputTokens":1,"latencyMs":520.7202910000924},{"questionId":"q112","format":"csv","model":"grok-4-fast-non-reasoning","expected":"53","actual":"35","isCorrect":false,"inputTokens":8620,"outputTokens":1,"latencyMs":432.4779170000693},{"questionId":"q112","format":"xml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"32","isCorrect":false,"inputTokens":17155,"outputTokens":1,"latencyMs":846.315082999994},{"questionId":"q112","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"34","isCorrect":false,"inputTokens":13241,"outputTokens":1,"latencyMs":391.02062500000466},{"questionId":"q113","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"77","actual":"59","isCorrect":false,"inputTokens":15242,"outputTokens":1,"latencyMs":490.0183750001015},{"questionId":"q113","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"77","actual":"68","isCorrect":false,"inputTokens":11546,"outputTokens":1,"latencyMs":403.00137499999255},{"questionId":"q113","format":"toon","model":"grok-4-fast-non-reasoning","expected":"77","actual":"59","isCorrect":false,"inputTokens":8869,"outputTokens":1,"latencyMs":358.87187499995343},{"questionId":"q113","format":"csv","model":"grok-4-fast-non-reasoning","expected":"77","actual":"66","isCorrect":false,"inputTokens":8619,"outputTokens":1,"latencyMs":502.90829199994914},{"questionId":"q113","format":"xml","model":"grok-4-fast-non-reasoning","expected":"77","actual":"47","isCorrect":false,"inputTokens":17154,"outputTokens":1,"latencyMs":970.4605000000447},{"questionId":"q113","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"77","actual":"61","isCorrect":false,"inputTokens":13240,"outputTokens":1,"latencyMs":688.6402080000844},{"questionId":"q114","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"37","actual":"35","isCorrect":false,"inputTokens":15242,"outputTokens":1,"latencyMs":903.6797080000397},{"questionId":"q114","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"37","actual":"35","isCorrect":false,"inputTokens":11546,"outputTokens":1,"latencyMs":794.250207999954},{"questionId":"q114","format":"toon","model":"grok-4-fast-non-reasoning","expected":"37","actual":"36","isCorrect":false,"inputTokens":8869,"outputTokens":1,"latencyMs":390.6796669999603},{"questionId":"q114","format":"csv","model":"grok-4-fast-non-reasoning","expected":"37","actual":"29","isCorrect":false,"inputTokens":8619,"outputTokens":1,"latencyMs":540.5698340000818},{"questionId":"q114","format":"xml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"25","isCorrect":false,"inputTokens":17154,"outputTokens":1,"latencyMs":737.7287500000093},{"questionId":"q114","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"35","isCorrect":false,"inputTokens":13240,"outputTokens":1,"latencyMs":426.6078330000164},{"questionId":"q115","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"16","actual":"16","isCorrect":true,"inputTokens":15242,"outputTokens":1,"latencyMs":680.2153750000289},{"questionId":"q115","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"16","actual":"16","isCorrect":true,"inputTokens":11546,"outputTokens":1,"latencyMs":545.5063340000343},{"questionId":"q115","format":"toon","model":"grok-4-fast-non-reasoning","expected":"16","actual":"25","isCorrect":false,"inputTokens":8869,"outputTokens":1,"latencyMs":678.1834579999559},{"questionId":"q115","format":"csv","model":"grok-4-fast-non-reasoning","expected":"16","actual":"25","isCorrect":false,"inputTokens":8619,"outputTokens":1,"latencyMs":441.6597920000786},{"questionId":"q115","format":"xml","model":"grok-4-fast-non-reasoning","expected":"16","actual":"20","isCorrect":false,"inputTokens":17154,"outputTokens":1,"latencyMs":956.2092079999857},{"questionId":"q115","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"16","actual":"16","isCorrect":true,"inputTokens":13240,"outputTokens":1,"latencyMs":699.404084000038},{"questionId":"q116","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"49","actual":"71","isCorrect":false,"inputTokens":15242,"outputTokens":1,"latencyMs":770.1492500000168},{"questionId":"q116","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"49","actual":"60","isCorrect":false,"inputTokens":11546,"outputTokens":1,"latencyMs":573.6035000000848},{"questionId":"q116","format":"toon","model":"grok-4-fast-non-reasoning","expected":"49","actual":"73","isCorrect":false,"inputTokens":8869,"outputTokens":1,"latencyMs":570.5019590000156},{"questionId":"q116","format":"csv","model":"grok-4-fast-non-reasoning","expected":"49","actual":"68","isCorrect":false,"inputTokens":8619,"outputTokens":1,"latencyMs":463.01512500003446},{"questionId":"q116","format":"xml","model":"grok-4-fast-non-reasoning","expected":"49","actual":"37","isCorrect":false,"inputTokens":17154,"outputTokens":1,"latencyMs":931.2610420000274},{"questionId":"q116","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"49","actual":"53","isCorrect":false,"inputTokens":13240,"outputTokens":1,"latencyMs":596.768875000067},{"questionId":"q117","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"23","actual":"28","isCorrect":false,"inputTokens":15242,"outputTokens":1,"latencyMs":619.4133330000332},{"questionId":"q117","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"23","actual":"42","isCorrect":false,"inputTokens":11546,"outputTokens":1,"latencyMs":466.2274169999873},{"questionId":"q117","format":"toon","model":"grok-4-fast-non-reasoning","expected":"23","actual":"60","isCorrect":false,"inputTokens":8869,"outputTokens":1,"latencyMs":700.3706249999814},{"questionId":"q117","format":"csv","model":"grok-4-fast-non-reasoning","expected":"23","actual":"54","isCorrect":false,"inputTokens":8619,"outputTokens":1,"latencyMs":356.86966700002085},{"questionId":"q117","format":"xml","model":"grok-4-fast-non-reasoning","expected":"23","actual":"26","isCorrect":false,"inputTokens":17154,"outputTokens":1,"latencyMs":411.84579199994914},{"questionId":"q117","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"23","actual":"17","isCorrect":false,"inputTokens":13240,"outputTokens":1,"latencyMs":448.826999999932},{"questionId":"q118","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"3","isCorrect":false,"inputTokens":15242,"outputTokens":1,"latencyMs":393.56099999998696},{"questionId":"q118","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"2","isCorrect":false,"inputTokens":11546,"outputTokens":1,"latencyMs":562.6259589999681},{"questionId":"q118","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"2","isCorrect":false,"inputTokens":8869,"outputTokens":1,"latencyMs":613.5504169999622},{"questionId":"q118","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4","actual":"1","isCorrect":false,"inputTokens":8619,"outputTokens":1,"latencyMs":351.3422080000164},{"questionId":"q118","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"2","isCorrect":false,"inputTokens":17154,"outputTokens":1,"latencyMs":849.3879170001019},{"questionId":"q118","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"1","isCorrect":false,"inputTokens":13240,"outputTokens":1,"latencyMs":493.12075000000186},{"questionId":"q119","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"57","actual":"32","isCorrect":false,"inputTokens":15249,"outputTokens":1,"latencyMs":808.0619589999551},{"questionId":"q119","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"57","actual":"31","isCorrect":false,"inputTokens":11553,"outputTokens":1,"latencyMs":605.4609580000397},{"questionId":"q119","format":"toon","model":"grok-4-fast-non-reasoning","expected":"57","actual":"20","isCorrect":false,"inputTokens":8876,"outputTokens":1,"latencyMs":432.17312499997206},{"questionId":"q119","format":"csv","model":"grok-4-fast-non-reasoning","expected":"57","actual":"25","isCorrect":false,"inputTokens":8626,"outputTokens":1,"latencyMs":590.7101250000997},{"questionId":"q119","format":"xml","model":"grok-4-fast-non-reasoning","expected":"57","actual":"30","isCorrect":false,"inputTokens":17161,"outputTokens":1,"latencyMs":392.77558400004636},{"questionId":"q119","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"57","actual":"18","isCorrect":false,"inputTokens":13247,"outputTokens":1,"latencyMs":675.5407080000732},{"questionId":"q120","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"43","actual":"23","isCorrect":false,"inputTokens":15249,"outputTokens":1,"latencyMs":393.07983299996704},{"questionId":"q120","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"43","actual":"22","isCorrect":false,"inputTokens":11553,"outputTokens":1,"latencyMs":683.4502920000814},{"questionId":"q120","format":"toon","model":"grok-4-fast-non-reasoning","expected":"43","actual":"11","isCorrect":false,"inputTokens":8876,"outputTokens":1,"latencyMs":381.63970800000243},{"questionId":"q120","format":"csv","model":"grok-4-fast-non-reasoning","expected":"43","actual":"25","isCorrect":false,"inputTokens":8626,"outputTokens":1,"latencyMs":405.6352499999339},{"questionId":"q120","format":"xml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"28","isCorrect":false,"inputTokens":17161,"outputTokens":1,"latencyMs":381.923625000054},{"questionId":"q120","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"22","isCorrect":false,"inputTokens":13247,"outputTokens":1,"latencyMs":572.6517920000479},{"questionId":"q121","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"25","actual":"18","isCorrect":false,"inputTokens":15249,"outputTokens":1,"latencyMs":477.9099579999456},{"questionId":"q121","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"25","actual":"9","isCorrect":false,"inputTokens":11553,"outputTokens":1,"latencyMs":662.4345409999369},{"questionId":"q121","format":"toon","model":"grok-4-fast-non-reasoning","expected":"25","actual":"8","isCorrect":false,"inputTokens":8876,"outputTokens":1,"latencyMs":533.1207090000389},{"questionId":"q121","format":"csv","model":"grok-4-fast-non-reasoning","expected":"25","actual":"6","isCorrect":false,"inputTokens":8626,"outputTokens":1,"latencyMs":461.1072919999715},{"questionId":"q121","format":"xml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"15","isCorrect":false,"inputTokens":17161,"outputTokens":1,"latencyMs":403.9090829999186},{"questionId":"q121","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"9","isCorrect":false,"inputTokens":13247,"outputTokens":1,"latencyMs":411.6952090000268},{"questionId":"q122","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6","actual":"5","isCorrect":false,"inputTokens":15249,"outputTokens":1,"latencyMs":374.8737080000574},{"questionId":"q122","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6","actual":"3","isCorrect":false,"inputTokens":11553,"outputTokens":1,"latencyMs":423.4539169999771},{"questionId":"q122","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":8876,"outputTokens":1,"latencyMs":615.6106669999426},{"questionId":"q122","format":"csv","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":8626,"outputTokens":1,"latencyMs":403.406500000041},{"questionId":"q122","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":17161,"outputTokens":1,"latencyMs":780.1899579999736},{"questionId":"q122","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"2","isCorrect":false,"inputTokens":13247,"outputTokens":1,"latencyMs":394.3780000000261},{"questionId":"q123","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"11","isCorrect":false,"inputTokens":15249,"outputTokens":1,"latencyMs":779.1762080000481},{"questionId":"q123","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"3","isCorrect":false,"inputTokens":11553,"outputTokens":1,"latencyMs":322.2744999999413},{"questionId":"q123","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":8876,"outputTokens":1,"latencyMs":820.2505830000155},{"questionId":"q123","format":"csv","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":8626,"outputTokens":1,"latencyMs":738.11020799994},{"questionId":"q123","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"4","isCorrect":false,"inputTokens":17161,"outputTokens":1,"latencyMs":640.8369999999413},{"questionId":"q123","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"3","isCorrect":false,"inputTokens":13247,"outputTokens":1,"latencyMs":493.1192909999518},{"questionId":"q124","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7010,"outputTokens":1,"latencyMs":628.3985000000102},{"questionId":"q124","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":4922,"outputTokens":1,"latencyMs":544.5020409999415},{"questionId":"q124","format":"toon","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6011,"outputTokens":1,"latencyMs":547.3039170000702},{"questionId":"q124","format":"xml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7920,"outputTokens":1,"latencyMs":664.1819159999723},{"questionId":"q124","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6037,"outputTokens":1,"latencyMs":488.5942090000026},{"questionId":"q125","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7010,"outputTokens":2,"latencyMs":542.185999999987},{"questionId":"q125","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4922,"outputTokens":2,"latencyMs":504.43029099993873},{"questionId":"q125","format":"toon","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6011,"outputTokens":2,"latencyMs":391.89174999995157},{"questionId":"q125","format":"xml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7920,"outputTokens":2,"latencyMs":517.6699580000713},{"questionId":"q125","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6037,"outputTokens":2,"latencyMs":365.8610000000335},{"questionId":"q126","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":7011,"outputTokens":1,"latencyMs":454.36612499994226},{"questionId":"q126","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":4923,"outputTokens":1,"latencyMs":456.68258300004527},{"questionId":"q126","format":"toon","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":6012,"outputTokens":1,"latencyMs":361.28120900003705},{"questionId":"q126","format":"xml","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":7921,"outputTokens":1,"latencyMs":414.62033299996983},{"questionId":"q126","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":6038,"outputTokens":1,"latencyMs":351.6857079999754},{"questionId":"q127","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7011,"outputTokens":2,"latencyMs":376.3115000000689},{"questionId":"q127","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":4923,"outputTokens":2,"latencyMs":399.4570839999942},{"questionId":"q127","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6012,"outputTokens":2,"latencyMs":413.27624999999534},{"questionId":"q127","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7921,"outputTokens":2,"latencyMs":655.8041659999872},{"questionId":"q127","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6038,"outputTokens":2,"latencyMs":562.9127499999013},{"questionId":"q128","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":7010,"outputTokens":1,"latencyMs":411.6166669999948},{"questionId":"q128","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":4922,"outputTokens":1,"latencyMs":520.3250410000328},{"questionId":"q128","format":"toon","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6011,"outputTokens":1,"latencyMs":630.7051249999786},{"questionId":"q128","format":"xml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":7920,"outputTokens":1,"latencyMs":407.46475000004284},{"questionId":"q128","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6037,"outputTokens":1,"latencyMs":382.6142080000136},{"questionId":"q129","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7010,"outputTokens":2,"latencyMs":465.02712499990594},{"questionId":"q129","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":4922,"outputTokens":2,"latencyMs":335.43270899995696},{"questionId":"q129","format":"toon","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6011,"outputTokens":2,"latencyMs":400.3914169999771},{"questionId":"q129","format":"xml","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7920,"outputTokens":2,"latencyMs":431.80858300009277},{"questionId":"q129","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6037,"outputTokens":2,"latencyMs":383.0126670000609},{"questionId":"q130","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":7011,"outputTokens":1,"latencyMs":393.6459999999497},{"questionId":"q130","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":4923,"outputTokens":1,"latencyMs":334.10466699989047},{"questionId":"q130","format":"toon","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":6012,"outputTokens":1,"latencyMs":591.420792000019},{"questionId":"q130","format":"xml","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":7921,"outputTokens":1,"latencyMs":345.5883749999339},{"questionId":"q130","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":6038,"outputTokens":1,"latencyMs":387.33283400000073},{"questionId":"q131","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":7011,"outputTokens":1,"latencyMs":380.9183750000084},{"questionId":"q131","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":4923,"outputTokens":1,"latencyMs":374.1097090000985},{"questionId":"q131","format":"toon","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":6012,"outputTokens":1,"latencyMs":379.5562079999363},{"questionId":"q131","format":"xml","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":7921,"outputTokens":1,"latencyMs":600.8733340000035},{"questionId":"q131","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":6038,"outputTokens":1,"latencyMs":518.0287910000188},{"questionId":"q132","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7010,"outputTokens":1,"latencyMs":381.24399999994785},{"questionId":"q132","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":4922,"outputTokens":1,"latencyMs":377.2337080000434},{"questionId":"q132","format":"toon","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6011,"outputTokens":1,"latencyMs":394.7509999999311},{"questionId":"q132","format":"xml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7920,"outputTokens":1,"latencyMs":634.0575410000747},{"questionId":"q132","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6037,"outputTokens":1,"latencyMs":424.10337499994785},{"questionId":"q133","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7010,"outputTokens":2,"latencyMs":354.91070800006855},{"questionId":"q133","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4922,"outputTokens":2,"latencyMs":375.06462499999907},{"questionId":"q133","format":"toon","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6011,"outputTokens":2,"latencyMs":616.2190839999821},{"questionId":"q133","format":"xml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7920,"outputTokens":2,"latencyMs":613.9362500000279},{"questionId":"q133","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6037,"outputTokens":2,"latencyMs":446.4282499999972},{"questionId":"q134","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"75","actual":"70","isCorrect":false,"inputTokens":6994,"outputTokens":1,"latencyMs":492.752833000035},{"questionId":"q134","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"75","actual":"70","isCorrect":false,"inputTokens":4906,"outputTokens":1,"latencyMs":630.5677920000162},{"questionId":"q134","format":"toon","model":"grok-4-fast-non-reasoning","expected":"75","actual":"75","isCorrect":true,"inputTokens":5995,"outputTokens":1,"latencyMs":448.093499999959},{"questionId":"q134","format":"xml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"60","isCorrect":false,"inputTokens":7904,"outputTokens":1,"latencyMs":602.6525409999304},{"questionId":"q134","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"58","isCorrect":false,"inputTokens":6021,"outputTokens":1,"latencyMs":588.9808330000378},{"questionId":"q135","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2384","isCorrect":false,"inputTokens":6995,"outputTokens":2,"latencyMs":548.9166670000413},{"questionId":"q135","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2315","isCorrect":false,"inputTokens":4907,"outputTokens":2,"latencyMs":472.4483750000363},{"questionId":"q135","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"1779","isCorrect":false,"inputTokens":5996,"outputTokens":2,"latencyMs":347.96562500006985},{"questionId":"q135","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2198","isCorrect":false,"inputTokens":7905,"outputTokens":2,"latencyMs":428.2542080000276},{"questionId":"q135","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"1686","isCorrect":false,"inputTokens":6022,"outputTokens":2,"latencyMs":378.906917000073},{"questionId":"q136","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"29","actual":"25","isCorrect":false,"inputTokens":6994,"outputTokens":1,"latencyMs":461.1518749999814},{"questionId":"q136","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"29","actual":"24","isCorrect":false,"inputTokens":4906,"outputTokens":1,"latencyMs":392.55816600006074},{"questionId":"q136","format":"toon","model":"grok-4-fast-non-reasoning","expected":"29","actual":"27","isCorrect":false,"inputTokens":5995,"outputTokens":1,"latencyMs":379.6563750000205},{"questionId":"q136","format":"xml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"28","isCorrect":false,"inputTokens":7904,"outputTokens":1,"latencyMs":504.56883300002664},{"questionId":"q136","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"31","isCorrect":false,"inputTokens":6021,"outputTokens":1,"latencyMs":625.3374590000603},{"questionId":"q137","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"16","isCorrect":false,"inputTokens":6994,"outputTokens":1,"latencyMs":498.84766700002365},{"questionId":"q137","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"14","isCorrect":false,"inputTokens":4906,"outputTokens":1,"latencyMs":398.41633300005924},{"questionId":"q137","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"15","isCorrect":false,"inputTokens":5995,"outputTokens":1,"latencyMs":394.8762080000015},{"questionId":"q137","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"16","isCorrect":false,"inputTokens":7904,"outputTokens":1,"latencyMs":474.63279200007673},{"questionId":"q137","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"16","isCorrect":false,"inputTokens":6021,"outputTokens":1,"latencyMs":354.0374159999192},{"questionId":"q138","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"29","actual":"24","isCorrect":false,"inputTokens":6994,"outputTokens":1,"latencyMs":348.90487500000745},{"questionId":"q138","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"29","actual":"23","isCorrect":false,"inputTokens":4906,"outputTokens":1,"latencyMs":597.8686250000028},{"questionId":"q138","format":"toon","model":"grok-4-fast-non-reasoning","expected":"29","actual":"25","isCorrect":false,"inputTokens":5995,"outputTokens":1,"latencyMs":1130.912291000015},{"questionId":"q138","format":"xml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"20","isCorrect":false,"inputTokens":7904,"outputTokens":1,"latencyMs":392.9054999999935},{"questionId":"q138","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"24","isCorrect":false,"inputTokens":6021,"outputTokens":1,"latencyMs":485.8165830000071},{"questionId":"q139","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":6996,"outputTokens":1,"latencyMs":360.9347500000149},{"questionId":"q139","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":4908,"outputTokens":1,"latencyMs":369.8820419999538},{"questionId":"q139","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":5997,"outputTokens":1,"latencyMs":330.68237499997485},{"questionId":"q139","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":7906,"outputTokens":1,"latencyMs":464.4594999999972},{"questionId":"q139","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"16","isCorrect":false,"inputTokens":6023,"outputTokens":1,"latencyMs":329.4231250000885},{"questionId":"q140","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"18","actual":"15","isCorrect":false,"inputTokens":6996,"outputTokens":1,"latencyMs":860.1005410000216},{"questionId":"q140","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"18","actual":"14","isCorrect":false,"inputTokens":4908,"outputTokens":1,"latencyMs":435.3988750000717},{"questionId":"q140","format":"toon","model":"grok-4-fast-non-reasoning","expected":"18","actual":"16","isCorrect":false,"inputTokens":5997,"outputTokens":1,"latencyMs":384.7552499999292},{"questionId":"q140","format":"xml","model":"grok-4-fast-non-reasoning","expected":"18","actual":"15","isCorrect":false,"inputTokens":7906,"outputTokens":1,"latencyMs":561.0219999999972},{"questionId":"q140","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"18","actual":"15","isCorrect":false,"inputTokens":6023,"outputTokens":1,"latencyMs":404.68016699992586},{"questionId":"q141","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"33","actual":"27","isCorrect":false,"inputTokens":7000,"outputTokens":1,"latencyMs":471.05758300004527},{"questionId":"q141","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"33","actual":"29","isCorrect":false,"inputTokens":4912,"outputTokens":1,"latencyMs":362.88241700001527},{"questionId":"q141","format":"toon","model":"grok-4-fast-non-reasoning","expected":"33","actual":"39","isCorrect":false,"inputTokens":6001,"outputTokens":1,"latencyMs":362.282583000022},{"questionId":"q141","format":"xml","model":"grok-4-fast-non-reasoning","expected":"33","actual":"28","isCorrect":false,"inputTokens":7910,"outputTokens":1,"latencyMs":334.56137500004843},{"questionId":"q141","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"33","actual":"32","isCorrect":false,"inputTokens":6027,"outputTokens":1,"latencyMs":362.18050000001676},{"questionId":"q142","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"42","actual":"25","isCorrect":false,"inputTokens":6999,"outputTokens":1,"latencyMs":390.17516600003},{"questionId":"q142","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"42","actual":"18","isCorrect":false,"inputTokens":4911,"outputTokens":1,"latencyMs":349.779291999992},{"questionId":"q142","format":"toon","model":"grok-4-fast-non-reasoning","expected":"42","actual":"25","isCorrect":false,"inputTokens":6000,"outputTokens":1,"latencyMs":365.62591699999757},{"questionId":"q142","format":"xml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"23","isCorrect":false,"inputTokens":7909,"outputTokens":1,"latencyMs":409.50941699999385},{"questionId":"q142","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"25","isCorrect":false,"inputTokens":6026,"outputTokens":1,"latencyMs":352.8945000000531},{"questionId":"q143","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"25","actual":"23","isCorrect":false,"inputTokens":6995,"outputTokens":1,"latencyMs":417.6676669999724},{"questionId":"q143","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"25","actual":"27","isCorrect":false,"inputTokens":4907,"outputTokens":1,"latencyMs":511.1363329999149},{"questionId":"q143","format":"toon","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":5996,"outputTokens":1,"latencyMs":446.0274170000339},{"questionId":"q143","format":"xml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":7905,"outputTokens":1,"latencyMs":500.85004099993967},{"questionId":"q143","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"26","isCorrect":false,"inputTokens":6022,"outputTokens":1,"latencyMs":486.2238330000546},{"questionId":"q144","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"29","actual":"20","isCorrect":false,"inputTokens":7001,"outputTokens":1,"latencyMs":459.67733299999963},{"questionId":"q144","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"29","actual":"23","isCorrect":false,"inputTokens":4913,"outputTokens":1,"latencyMs":447.0490840000566},{"questionId":"q144","format":"toon","model":"grok-4-fast-non-reasoning","expected":"29","actual":"25","isCorrect":false,"inputTokens":6002,"outputTokens":1,"latencyMs":443.00808299996424},{"questionId":"q144","format":"xml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"25","isCorrect":false,"inputTokens":7911,"outputTokens":1,"latencyMs":450.24762499995995},{"questionId":"q144","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"29","isCorrect":true,"inputTokens":6028,"outputTokens":1,"latencyMs":446.1748750000261},{"questionId":"q145","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"3","isCorrect":false,"inputTokens":7001,"outputTokens":1,"latencyMs":477.9445419999538},{"questionId":"q145","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"3","isCorrect":false,"inputTokens":4913,"outputTokens":1,"latencyMs":369.05883300001733},{"questionId":"q145","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"3","isCorrect":false,"inputTokens":6002,"outputTokens":1,"latencyMs":352.532332999981},{"questionId":"q145","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"3","isCorrect":false,"inputTokens":7911,"outputTokens":1,"latencyMs":402.71041599998716},{"questionId":"q145","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"2","isCorrect":false,"inputTokens":6028,"outputTokens":1,"latencyMs":343.9370839999756},{"questionId":"q146","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":7003,"outputTokens":1,"latencyMs":448.52666600001976},{"questionId":"q146","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":4915,"outputTokens":1,"latencyMs":307.28262499999255},{"questionId":"q146","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":6004,"outputTokens":1,"latencyMs":381.33808399992995},{"questionId":"q146","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":7913,"outputTokens":1,"latencyMs":428.02612499997485},{"questionId":"q146","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":6030,"outputTokens":1,"latencyMs":386.1064169999445},{"questionId":"q147","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"4","isCorrect":false,"inputTokens":7003,"outputTokens":1,"latencyMs":499.48870899993926},{"questionId":"q147","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"6","isCorrect":false,"inputTokens":4915,"outputTokens":1,"latencyMs":600.1385419999715},{"questionId":"q147","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"7","isCorrect":false,"inputTokens":6004,"outputTokens":1,"latencyMs":373.59070800000336},{"questionId":"q147","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"5","isCorrect":false,"inputTokens":7913,"outputTokens":1,"latencyMs":358.8839160000207},{"questionId":"q147","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"4","isCorrect":false,"inputTokens":6030,"outputTokens":1,"latencyMs":354.7287920000963},{"questionId":"q148","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":7003,"outputTokens":1,"latencyMs":451.58479100000113},{"questionId":"q148","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":4915,"outputTokens":1,"latencyMs":327.36525000003166},{"questionId":"q148","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":6004,"outputTokens":1,"latencyMs":370.73895899998024},{"questionId":"q148","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":7913,"outputTokens":1,"latencyMs":404.0799999999581},{"questionId":"q148","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":6030,"outputTokens":1,"latencyMs":429.8884170000674},{"questionId":"q149","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"6","isCorrect":false,"inputTokens":7001,"outputTokens":1,"latencyMs":377.94491600000765},{"questionId":"q149","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"8","isCorrect":false,"inputTokens":4913,"outputTokens":1,"latencyMs":391.6597499999916},{"questionId":"q149","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"7","isCorrect":false,"inputTokens":6002,"outputTokens":1,"latencyMs":368.6678339999635},{"questionId":"q149","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"8","isCorrect":false,"inputTokens":7911,"outputTokens":1,"latencyMs":362.266958000022},{"questionId":"q149","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"6","isCorrect":false,"inputTokens":6028,"outputTokens":1,"latencyMs":392.51866699999664},{"questionId":"q150","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"3","isCorrect":false,"inputTokens":7001,"outputTokens":1,"latencyMs":375.9136250000447},{"questionId":"q150","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":4913,"outputTokens":1,"latencyMs":410.7285840000259},{"questionId":"q150","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":6002,"outputTokens":1,"latencyMs":378.04991600010544},{"questionId":"q150","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"4","isCorrect":false,"inputTokens":7911,"outputTokens":1,"latencyMs":330.7129170000553},{"questionId":"q150","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":6028,"outputTokens":1,"latencyMs":473.70275000005495},{"questionId":"q151","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":1127,"outputTokens":1,"latencyMs":422.4119579999242},{"questionId":"q151","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":749,"outputTokens":1,"latencyMs":364.578332999954},{"questionId":"q151","format":"toon","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":861,"outputTokens":1,"latencyMs":358.74412500008475},{"questionId":"q151","format":"xml","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":1199,"outputTokens":1,"latencyMs":380.1164169999538},{"questionId":"q151","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":879,"outputTokens":1,"latencyMs":452.7212920000311},{"questionId":"q152","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1125,"outputTokens":5,"latencyMs":317.5407500000438},{"questionId":"q152","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":747,"outputTokens":5,"latencyMs":382.5822080000071},{"questionId":"q152","format":"toon","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":859,"outputTokens":5,"latencyMs":339.4631250000093},{"questionId":"q152","format":"xml","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1197,"outputTokens":5,"latencyMs":363.8777499999851},{"questionId":"q152","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":877,"outputTokens":5,"latencyMs":317.45404099998996},{"questionId":"q153","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1125,"outputTokens":2,"latencyMs":318.85229199996684},{"questionId":"q153","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":747,"outputTokens":2,"latencyMs":346.7132499999134},{"questionId":"q153","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":859,"outputTokens":2,"latencyMs":405.8940409999341},{"questionId":"q153","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1197,"outputTokens":2,"latencyMs":546.3707089999225},{"questionId":"q153","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":877,"outputTokens":2,"latencyMs":392.1274170000106},{"questionId":"q154","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":1127,"outputTokens":1,"latencyMs":409.99079099996015},{"questionId":"q154","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":749,"outputTokens":1,"latencyMs":575.8202499998733},{"questionId":"q154","format":"toon","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":861,"outputTokens":1,"latencyMs":513.9864590000361},{"questionId":"q154","format":"xml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":1199,"outputTokens":1,"latencyMs":407.92458300013095},{"questionId":"q154","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":879,"outputTokens":1,"latencyMs":462.17279199999757},{"questionId":"q155","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1125,"outputTokens":2,"latencyMs":363.9769170000218},{"questionId":"q155","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":747,"outputTokens":2,"latencyMs":375.0419999998994},{"questionId":"q155","format":"toon","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":859,"outputTokens":2,"latencyMs":297.3989999999758},{"questionId":"q155","format":"xml","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1197,"outputTokens":2,"latencyMs":319.11029199999757},{"questionId":"q155","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":877,"outputTokens":2,"latencyMs":326.12866599997506},{"questionId":"q156","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1127,"outputTokens":1,"latencyMs":584.0956250000745},{"questionId":"q156","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":749,"outputTokens":1,"latencyMs":366.14229099988006},{"questionId":"q156","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":861,"outputTokens":1,"latencyMs":350.41783299995586},{"questionId":"q156","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1199,"outputTokens":1,"latencyMs":416.20175000000745},{"questionId":"q156","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":879,"outputTokens":1,"latencyMs":348.1779169999063},{"questionId":"q157","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1127,"outputTokens":2,"latencyMs":325.81333300005645},{"questionId":"q157","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":749,"outputTokens":2,"latencyMs":364.9545409998391},{"questionId":"q157","format":"toon","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":861,"outputTokens":2,"latencyMs":768.2352499999106},{"questionId":"q157","format":"xml","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1199,"outputTokens":2,"latencyMs":349.34429200016893},{"questionId":"q157","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":879,"outputTokens":2,"latencyMs":304.9921669999603},{"questionId":"q158","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":1125,"outputTokens":1,"latencyMs":352.1822500000708},{"questionId":"q158","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":747,"outputTokens":1,"latencyMs":335.11220800015144},{"questionId":"q158","format":"toon","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":859,"outputTokens":1,"latencyMs":318.23408299987204},{"questionId":"q158","format":"xml","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":1197,"outputTokens":1,"latencyMs":290.9962920001708},{"questionId":"q158","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":877,"outputTokens":1,"latencyMs":355.078125},{"questionId":"q159","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1126,"outputTokens":2,"latencyMs":359.9554169999901},{"questionId":"q159","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":748,"outputTokens":2,"latencyMs":388.3081249999814},{"questionId":"q159","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":860,"outputTokens":2,"latencyMs":306.580375000136},{"questionId":"q159","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1198,"outputTokens":2,"latencyMs":367.22133399988525},{"questionId":"q159","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":878,"outputTokens":2,"latencyMs":318.70112500013784},{"questionId":"q160","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1127,"outputTokens":5,"latencyMs":357.5999169999268},{"questionId":"q160","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":749,"outputTokens":5,"latencyMs":434.7398749999702},{"questionId":"q160","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":861,"outputTokens":5,"latencyMs":320.847708000103},{"questionId":"q160","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1199,"outputTokens":5,"latencyMs":444.52779099997133},{"questionId":"q160","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":879,"outputTokens":5,"latencyMs":346.64495799993165},{"questionId":"q161","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1127,"outputTokens":1,"latencyMs":441.18654099991545},{"questionId":"q161","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":749,"outputTokens":1,"latencyMs":316.80200000014156},{"questionId":"q161","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":861,"outputTokens":1,"latencyMs":344.1644170000218},{"questionId":"q161","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1199,"outputTokens":1,"latencyMs":359.67345799994655},{"questionId":"q161","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":879,"outputTokens":1,"latencyMs":314.257708999794},{"questionId":"q162","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1127,"outputTokens":1,"latencyMs":341.3015829999931},{"questionId":"q162","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":749,"outputTokens":1,"latencyMs":402.5352079998702},{"questionId":"q162","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":861,"outputTokens":1,"latencyMs":330.62899999995716},{"questionId":"q162","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1199,"outputTokens":1,"latencyMs":321.2940420000814},{"questionId":"q162","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":879,"outputTokens":1,"latencyMs":341.8873340000864},{"questionId":"q163","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1126,"outputTokens":1,"latencyMs":358.171624999959},{"questionId":"q163","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":748,"outputTokens":1,"latencyMs":325.0455830001738},{"questionId":"q163","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":860,"outputTokens":1,"latencyMs":434.60591699997894},{"questionId":"q163","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1198,"outputTokens":1,"latencyMs":369.04483399982564},{"questionId":"q163","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":878,"outputTokens":1,"latencyMs":389.55891599995084},{"questionId":"q164","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1126,"outputTokens":1,"latencyMs":440.0353750002105},{"questionId":"q164","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":748,"outputTokens":1,"latencyMs":438.29116599983536},{"questionId":"q164","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":860,"outputTokens":1,"latencyMs":453.78141699987464},{"questionId":"q164","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1198,"outputTokens":1,"latencyMs":464.6766250000801},{"questionId":"q164","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":878,"outputTokens":1,"latencyMs":462.33987500006333},{"questionId":"q165","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1126,"outputTokens":1,"latencyMs":448.2880409997888},{"questionId":"q165","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":748,"outputTokens":1,"latencyMs":426.0887919999659},{"questionId":"q165","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":860,"outputTokens":1,"latencyMs":323.5241249999963},{"questionId":"q165","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1198,"outputTokens":1,"latencyMs":326.37737499992363},{"questionId":"q165","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":878,"outputTokens":1,"latencyMs":315.89241699990816},{"questionId":"q166","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1130,"outputTokens":1,"latencyMs":332.59687500004657},{"questionId":"q166","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":752,"outputTokens":1,"latencyMs":328.14174999995157},{"questionId":"q166","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":864,"outputTokens":1,"latencyMs":478.95383300003596},{"questionId":"q166","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1202,"outputTokens":1,"latencyMs":344.2700000000186},{"questionId":"q166","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":882,"outputTokens":1,"latencyMs":278.3616669999901},{"questionId":"q167","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1126,"outputTokens":1,"latencyMs":303.0478749999311},{"questionId":"q167","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0","actual":"2","isCorrect":false,"inputTokens":748,"outputTokens":1,"latencyMs":358.3665420000907},{"questionId":"q167","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0","actual":"2","isCorrect":false,"inputTokens":860,"outputTokens":1,"latencyMs":367.6374580000993},{"questionId":"q167","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1198,"outputTokens":1,"latencyMs":357.58566700015217},{"questionId":"q167","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"2","isCorrect":false,"inputTokens":878,"outputTokens":1,"latencyMs":347.233709000051},{"questionId":"q168","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":1128,"outputTokens":1,"latencyMs":311.53012500004843},{"questionId":"q168","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":750,"outputTokens":1,"latencyMs":402.30229199980386},{"questionId":"q168","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":862,"outputTokens":1,"latencyMs":314.2168750001583},{"questionId":"q168","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":1200,"outputTokens":1,"latencyMs":337.4619160001166},{"questionId":"q168","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":880,"outputTokens":1,"latencyMs":310.02666600001976},{"questionId":"q169","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"8","actual":"11","isCorrect":false,"inputTokens":1130,"outputTokens":1,"latencyMs":349.0553749999963},{"questionId":"q169","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":752,"outputTokens":1,"latencyMs":323.92966700019315},{"questionId":"q169","format":"toon","model":"grok-4-fast-non-reasoning","expected":"8","actual":"8","isCorrect":true,"inputTokens":864,"outputTokens":1,"latencyMs":420.6140000000596},{"questionId":"q169","format":"xml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":1202,"outputTokens":1,"latencyMs":339.4013749998994},{"questionId":"q169","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"10","isCorrect":false,"inputTokens":882,"outputTokens":1,"latencyMs":350.1278329999186},{"questionId":"q170","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":1129,"outputTokens":1,"latencyMs":337.0966250000056},{"questionId":"q170","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":751,"outputTokens":1,"latencyMs":550.6174170000013},{"questionId":"q170","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"7","isCorrect":false,"inputTokens":863,"outputTokens":1,"latencyMs":340.1674999999814},{"questionId":"q170","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":1201,"outputTokens":1,"latencyMs":307.9737919999752},{"questionId":"q170","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":881,"outputTokens":1,"latencyMs":354.69920899998397},{"questionId":"q171","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1131,"outputTokens":1,"latencyMs":391.57199999992736},{"questionId":"q171","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":753,"outputTokens":1,"latencyMs":311.003958000103},{"questionId":"q171","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":865,"outputTokens":1,"latencyMs":304.13562500011176},{"questionId":"q171","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"4","isCorrect":false,"inputTokens":1203,"outputTokens":1,"latencyMs":329.156500000041},{"questionId":"q171","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":883,"outputTokens":1,"latencyMs":383.3717910000123},{"questionId":"q172","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1131,"outputTokens":1,"latencyMs":330.1229169999715},{"questionId":"q172","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":753,"outputTokens":1,"latencyMs":350.70349999982864},{"questionId":"q172","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":865,"outputTokens":1,"latencyMs":304.21616600011475},{"questionId":"q172","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1203,"outputTokens":1,"latencyMs":337.9212909999769},{"questionId":"q172","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":883,"outputTokens":1,"latencyMs":323.12954199989326},{"questionId":"q173","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":1132,"outputTokens":1,"latencyMs":343.41700000013225},{"questionId":"q173","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":754,"outputTokens":1,"latencyMs":340.3062080000527},{"questionId":"q173","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":866,"outputTokens":1,"latencyMs":477.0377500001341},{"questionId":"q173","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":1204,"outputTokens":1,"latencyMs":451.2650830000639},{"questionId":"q173","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":884,"outputTokens":1,"latencyMs":413.45462500001304},{"questionId":"q174","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1129,"outputTokens":1,"latencyMs":547.6270830000285},{"questionId":"q174","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":751,"outputTokens":1,"latencyMs":450.75474999984726},{"questionId":"q174","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":863,"outputTokens":1,"latencyMs":422.3355000000447},{"questionId":"q174","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"2","isCorrect":false,"inputTokens":1201,"outputTokens":1,"latencyMs":373.93825000012293},{"questionId":"q174","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":881,"outputTokens":1,"latencyMs":399.95558299985714},{"questionId":"q175","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1133,"outputTokens":1,"latencyMs":408.1844170000404},{"questionId":"q175","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":755,"outputTokens":1,"latencyMs":417.1452079999726},{"questionId":"q175","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":867,"outputTokens":1,"latencyMs":314.77450000005774},{"questionId":"q175","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1205,"outputTokens":1,"latencyMs":380.45758299995214},{"questionId":"q175","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":885,"outputTokens":1,"latencyMs":347.37924999999814},{"questionId":"q176","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1127,"outputTokens":1,"latencyMs":340.96537499991246},{"questionId":"q176","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":749,"outputTokens":1,"latencyMs":337.7478749998845},{"questionId":"q176","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":861,"outputTokens":1,"latencyMs":360.6514999999199},{"questionId":"q176","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1199,"outputTokens":1,"latencyMs":306.92058299994096},{"questionId":"q176","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":879,"outputTokens":1,"latencyMs":361.12375000002794},{"questionId":"q177","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1135,"outputTokens":1,"latencyMs":338.41949999984354},{"questionId":"q177","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":757,"outputTokens":1,"latencyMs":344.27370899985544},{"questionId":"q177","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":869,"outputTokens":1,"latencyMs":344.1268750000745},{"questionId":"q177","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1207,"outputTokens":1,"latencyMs":269.2622090000659},{"questionId":"q177","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":887,"outputTokens":1,"latencyMs":324.73683299985714},{"questionId":"q178","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1130,"outputTokens":1,"latencyMs":338.1203329998534},{"questionId":"q178","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":752,"outputTokens":1,"latencyMs":291.73079199995846},{"questionId":"q178","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":864,"outputTokens":1,"latencyMs":365.00216699996963},{"questionId":"q178","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1202,"outputTokens":1,"latencyMs":331.84741699998267},{"questionId":"q178","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":882,"outputTokens":1,"latencyMs":329.8641670001671},{"questionId":"q179","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1129,"outputTokens":1,"latencyMs":348.69020900013857},{"questionId":"q179","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":751,"outputTokens":1,"latencyMs":910.3052909998223},{"questionId":"q179","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":863,"outputTokens":1,"latencyMs":283.3612079999875},{"questionId":"q179","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1201,"outputTokens":1,"latencyMs":347.4672909998335},{"questionId":"q179","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":881,"outputTokens":1,"latencyMs":672.2668749999721},{"questionId":"q180","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":6513,"outputTokens":1,"latencyMs":564.6519169998355},{"questionId":"q180","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":4104,"outputTokens":1,"latencyMs":400.8067910000682},{"questionId":"q180","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2683,"outputTokens":1,"latencyMs":384.2312080000993},{"questionId":"q180","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2518,"outputTokens":1,"latencyMs":334.02241700002924},{"questionId":"q180","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":7450,"outputTokens":1,"latencyMs":413.1665829999838},{"questionId":"q180","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":5146,"outputTokens":1,"latencyMs":402.51987499999814},{"questionId":"q181","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":6518,"outputTokens":12,"latencyMs":483.392790999962},{"questionId":"q181","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4109,"outputTokens":12,"latencyMs":374.3410000000149},{"questionId":"q181","format":"toon","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2688,"outputTokens":12,"latencyMs":340.35154200019315},{"questionId":"q181","format":"csv","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2523,"outputTokens":12,"latencyMs":1485.7852920000441},{"questionId":"q181","format":"xml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":7455,"outputTokens":12,"latencyMs":544.2774169999175},{"questionId":"q181","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":5151,"outputTokens":12,"latencyMs":433.30837500002235},{"questionId":"q182","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":6516,"outputTokens":1,"latencyMs":343.8825840000063},{"questionId":"q182","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":4107,"outputTokens":1,"latencyMs":370.2654999999795},{"questionId":"q182","format":"toon","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":2686,"outputTokens":1,"latencyMs":338.80575000005774},{"questionId":"q182","format":"csv","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":2521,"outputTokens":1,"latencyMs":700.5023750001565},{"questionId":"q182","format":"xml","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":7453,"outputTokens":1,"latencyMs":424.90991699998267},{"questionId":"q182","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":5149,"outputTokens":1,"latencyMs":787.3187499998603},{"questionId":"q183","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":6517,"outputTokens":1,"latencyMs":545.6697500001173},{"questionId":"q183","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4108,"outputTokens":1,"latencyMs":458.97637499985285},{"questionId":"q183","format":"toon","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2687,"outputTokens":1,"latencyMs":424.5006249998696},{"questionId":"q183","format":"csv","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2522,"outputTokens":1,"latencyMs":397.82958299992606},{"questionId":"q183","format":"xml","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7454,"outputTokens":1,"latencyMs":577.5348329998087},{"questionId":"q183","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5150,"outputTokens":1,"latencyMs":787.9877090000082},{"questionId":"q184","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":6517,"outputTokens":5,"latencyMs":410.6278329999186},{"questionId":"q184","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4108,"outputTokens":5,"latencyMs":793.9159170000348},{"questionId":"q184","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2687,"outputTokens":5,"latencyMs":430.9729590001516},{"questionId":"q184","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2522,"outputTokens":5,"latencyMs":335.97854099981487},{"questionId":"q184","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7454,"outputTokens":5,"latencyMs":586.0767080001533},{"questionId":"q184","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5150,"outputTokens":5,"latencyMs":355.8477910000365},{"questionId":"q185","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":6514,"outputTokens":1,"latencyMs":497.1112920001615},{"questionId":"q185","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":4105,"outputTokens":1,"latencyMs":324.372333999956},{"questionId":"q185","format":"toon","model":"grok-4-fast-non-reasoning","expected":"7","actual":"6","isCorrect":false,"inputTokens":2684,"outputTokens":1,"latencyMs":460.29595800000243},{"questionId":"q185","format":"csv","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":2519,"outputTokens":1,"latencyMs":311.6815410000272},{"questionId":"q185","format":"xml","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":7451,"outputTokens":1,"latencyMs":394.30870799999684},{"questionId":"q185","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":5147,"outputTokens":1,"latencyMs":362.61108399997465},{"questionId":"q186","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":10953,"outputTokens":1,"latencyMs":461.8515419999603},{"questionId":"q186","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":6932,"outputTokens":1,"latencyMs":305.5184579999186},{"questionId":"q186","format":"toon","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":7355,"outputTokens":1,"latencyMs":860.654334000079},{"questionId":"q186","format":"xml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":12279,"outputTokens":1,"latencyMs":576.3340829999652},{"questionId":"q186","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":8518,"outputTokens":1,"latencyMs":513.610667000059},{"questionId":"q187","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":10960,"outputTokens":15,"latencyMs":464.43270800006576},{"questionId":"q187","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":6939,"outputTokens":15,"latencyMs":442.9684580001049},{"questionId":"q187","format":"toon","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":7362,"outputTokens":15,"latencyMs":557.8330419999547},{"questionId":"q187","format":"xml","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":12286,"outputTokens":15,"latencyMs":411.3310829999391},{"questionId":"q187","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId, customer, items, subtotal, tax, total, status, orderDate","isCorrect":true,"inputTokens":8525,"outputTokens":17,"latencyMs":580.7591669999529},{"questionId":"q188","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":10956,"outputTokens":1,"latencyMs":385.218584000133},{"questionId":"q188","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":6935,"outputTokens":1,"latencyMs":517.4760829999577},{"questionId":"q188","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":7358,"outputTokens":1,"latencyMs":583.9634589999914},{"questionId":"q188","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":12282,"outputTokens":1,"latencyMs":585.7809160000179},{"questionId":"q188","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":8521,"outputTokens":1,"latencyMs":535.1333749999758},{"questionId":"q189","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":10961,"outputTokens":6,"latencyMs":592.2415420000907},{"questionId":"q189","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":6940,"outputTokens":6,"latencyMs":511.6824580000248},{"questionId":"q189","format":"toon","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":7363,"outputTokens":6,"latencyMs":481.2422919999808},{"questionId":"q189","format":"xml","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":12287,"outputTokens":6,"latencyMs":665.1777500000317},{"questionId":"q189","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku, name, quantity, price","isCorrect":true,"inputTokens":8526,"outputTokens":7,"latencyMs":727.1295000000391},{"questionId":"q190","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":10957,"outputTokens":2,"latencyMs":642.5412919998635},{"questionId":"q190","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6936,"outputTokens":2,"latencyMs":417.7169589998666},{"questionId":"q190","format":"toon","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7359,"outputTokens":2,"latencyMs":409.8291249999311},{"questionId":"q190","format":"xml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12283,"outputTokens":2,"latencyMs":3034.810999999987},{"questionId":"q190","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8522,"outputTokens":2,"latencyMs":460.6213749998715},{"questionId":"q191","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":10962,"outputTokens":5,"latencyMs":670.8313329999801},{"questionId":"q191","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":6941,"outputTokens":5,"latencyMs":401.93520799977705},{"questionId":"q191","format":"toon","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":7364,"outputTokens":5,"latencyMs":484.2895420000423},{"questionId":"q191","format":"xml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":12288,"outputTokens":5,"latencyMs":604.8337919998448},{"questionId":"q191","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":8527,"outputTokens":5,"latencyMs":523.3439999998081},{"questionId":"q192","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"60","actual":"61","isCorrect":false,"inputTokens":3822,"outputTokens":1,"latencyMs":366.0537500001956},{"questionId":"q192","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"60","actual":"59","isCorrect":false,"inputTokens":2493,"outputTokens":1,"latencyMs":388.8614590000361},{"questionId":"q192","format":"toon","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":1752,"outputTokens":1,"latencyMs":531.8612500000745},{"questionId":"q192","format":"csv","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":1611,"outputTokens":1,"latencyMs":402.98354100016877},{"questionId":"q192","format":"xml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"61","isCorrect":false,"inputTokens":4468,"outputTokens":1,"latencyMs":419.8242920001503},{"questionId":"q192","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"62","isCorrect":false,"inputTokens":3152,"outputTokens":1,"latencyMs":544.4314580000937},{"questionId":"q193","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3826,"outputTokens":14,"latencyMs":445.4776660001371},{"questionId":"q193","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2497,"outputTokens":14,"latencyMs":647.6505420000758},{"questionId":"q193","format":"toon","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1756,"outputTokens":14,"latencyMs":582.0112910000607},{"questionId":"q193","format":"csv","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1615,"outputTokens":14,"latencyMs":522.1935000000522},{"questionId":"q193","format":"xml","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4472,"outputTokens":14,"latencyMs":446.3717919997871},{"questionId":"q193","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3156,"outputTokens":14,"latencyMs":334.3140420001},{"questionId":"q194","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"conversions","isCorrect":false,"inputTokens":3825,"outputTokens":2,"latencyMs":340.3161249998957},{"questionId":"q194","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":2496,"outputTokens":1,"latencyMs":321.3968749998603},{"questionId":"q194","format":"toon","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1755,"outputTokens":1,"latencyMs":337.04041699995287},{"questionId":"q194","format":"csv","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1614,"outputTokens":1,"latencyMs":305.7395840000827},{"questionId":"q194","format":"xml","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":4471,"outputTokens":1,"latencyMs":381.4124999998603},{"questionId":"q194","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":3155,"outputTokens":1,"latencyMs":391.24962499993853},{"questionId":"q195","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3826,"outputTokens":6,"latencyMs":351.1957499999553},{"questionId":"q195","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2497,"outputTokens":6,"latencyMs":337.6510830000043},{"questionId":"q195","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1756,"outputTokens":6,"latencyMs":343.33412500005215},{"questionId":"q195","format":"csv","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1615,"outputTokens":6,"latencyMs":442.4399170000106},{"questionId":"q195","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4472,"outputTokens":6,"latencyMs":338.48487500008196},{"questionId":"q195","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3156,"outputTokens":6,"latencyMs":275.6380829999689},{"questionId":"q196","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":3822,"outputTokens":1,"latencyMs":367.37120799999684},{"questionId":"q196","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":2493,"outputTokens":1,"latencyMs":447.95295799989253},{"questionId":"q196","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":1752,"outputTokens":1,"latencyMs":310.1214170001913},{"questionId":"q196","format":"csv","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":1611,"outputTokens":1,"latencyMs":356.95900000003166},{"questionId":"q196","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":4468,"outputTokens":1,"latencyMs":356.37020899984054},{"questionId":"q196","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":3152,"outputTokens":1,"latencyMs":339.7074170000851},{"questionId":"q197","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":15239,"outputTokens":1,"latencyMs":828.7236249998678},{"questionId":"q197","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":11543,"outputTokens":1,"latencyMs":593.1189999999478},{"questionId":"q197","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8866,"outputTokens":1,"latencyMs":617.0792919998057},{"questionId":"q197","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8616,"outputTokens":1,"latencyMs":517.1155419999268},{"questionId":"q197","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"80","isCorrect":false,"inputTokens":17151,"outputTokens":1,"latencyMs":757.1650000000373},{"questionId":"q197","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":13237,"outputTokens":1,"latencyMs":546.984540999867},{"questionId":"q198","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":15244,"outputTokens":24,"latencyMs":660.6357909999788},{"questionId":"q198","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":11548,"outputTokens":24,"latencyMs":686.094000000041},{"questionId":"q198","format":"toon","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8871,"outputTokens":24,"latencyMs":518.8935420000926},{"questionId":"q198","format":"csv","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8621,"outputTokens":24,"latencyMs":767.6478339999449},{"questionId":"q198","format":"xml","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":17156,"outputTokens":24,"latencyMs":1086.245207999833},{"questionId":"q198","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":13242,"outputTokens":24,"latencyMs":461.42283300007693},{"questionId":"q199","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"stars","isCorrect":false,"inputTokens":15244,"outputTokens":1,"latencyMs":361.31387499999255},{"questionId":"q199","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"forks","isCorrect":true,"inputTokens":11548,"outputTokens":2,"latencyMs":465.80245800013654},{"questionId":"q199","format":"toon","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":8871,"outputTokens":3,"latencyMs":806.5418750001118},{"questionId":"q199","format":"csv","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"stars","isCorrect":false,"inputTokens":8621,"outputTokens":1,"latencyMs":470.22779099992476},{"questionId":"q199","format":"xml","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"stars","isCorrect":false,"inputTokens":17156,"outputTokens":1,"latencyMs":867.2781249999534},{"questionId":"q199","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"stars","isCorrect":false,"inputTokens":13242,"outputTokens":1,"latencyMs":639.9731669998728},{"questionId":"q200","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":15243,"outputTokens":2,"latencyMs":706.970708000008},{"questionId":"q200","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":11547,"outputTokens":2,"latencyMs":445.3654589999933},{"questionId":"q200","format":"toon","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8870,"outputTokens":2,"latencyMs":341.2767089998815},{"questionId":"q200","format":"csv","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8620,"outputTokens":2,"latencyMs":588.6165410000831},{"questionId":"q200","format":"xml","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17155,"outputTokens":2,"latencyMs":1149.9787500000093},{"questionId":"q200","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":13241,"outputTokens":2,"latencyMs":538.4804999998305},{"questionId":"q201","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":15240,"outputTokens":1,"latencyMs":521.6853750001173},{"questionId":"q201","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":11544,"outputTokens":1,"latencyMs":600.9999589999206},{"questionId":"q201","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":8867,"outputTokens":1,"latencyMs":597.0913750000764},{"questionId":"q201","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":8617,"outputTokens":1,"latencyMs":676.1639170001727},{"questionId":"q201","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":17152,"outputTokens":1,"latencyMs":600.9071249999106},{"questionId":"q201","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":13238,"outputTokens":1,"latencyMs":717.3051660000347},{"questionId":"q202","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"75","actual":"65","isCorrect":false,"inputTokens":6994,"outputTokens":1,"latencyMs":471.1219169998076},{"questionId":"q202","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"75","actual":"72","isCorrect":false,"inputTokens":4906,"outputTokens":1,"latencyMs":715.7249580000062},{"questionId":"q202","format":"toon","model":"grok-4-fast-non-reasoning","expected":"75","actual":"75","isCorrect":true,"inputTokens":5995,"outputTokens":1,"latencyMs":424.4405829999596},{"questionId":"q202","format":"xml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"67","isCorrect":false,"inputTokens":7904,"outputTokens":1,"latencyMs":371.24525000015274},{"questionId":"q202","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"60","isCorrect":false,"inputTokens":6021,"outputTokens":1,"latencyMs":331.06362499995157},{"questionId":"q203","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":7003,"outputTokens":13,"latencyMs":375.68666699994355},{"questionId":"q203","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":4915,"outputTokens":13,"latencyMs":465.67037499998696},{"questionId":"q203","format":"toon","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":6004,"outputTokens":13,"latencyMs":514.6221669998486},{"questionId":"q203","format":"xml","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":7913,"outputTokens":13,"latencyMs":467.42820900003426},{"questionId":"q203","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp, level, endpoint, statusCode, responseTime, userId, error","isCorrect":true,"inputTokens":6030,"outputTokens":16,"latencyMs":374.1381250000559},{"questionId":"q204","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6998,"outputTokens":1,"latencyMs":481.8807079999242},{"questionId":"q204","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":4910,"outputTokens":1,"latencyMs":563.3164170000236},{"questionId":"q204","format":"toon","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":5999,"outputTokens":1,"latencyMs":451.7410830000881},{"questionId":"q204","format":"xml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":7908,"outputTokens":1,"latencyMs":455.55354100000113},{"questionId":"q204","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6025,"outputTokens":1,"latencyMs":373.25766600016505}] \ No newline at end of file +[{"questionId":"q1","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":6560,"outputTokens":2,"latencyMs":677.7787079999998},{"questionId":"q1","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":4151,"outputTokens":2,"latencyMs":591.66075},{"questionId":"q1","format":"toon","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2730,"outputTokens":2,"latencyMs":527.983958},{"questionId":"q1","format":"csv","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":2565,"outputTokens":2,"latencyMs":529.491833},{"questionId":"q1","format":"xml","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":7497,"outputTokens":2,"latencyMs":662.456541},{"questionId":"q1","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"56176","actual":"56176","isCorrect":true,"inputTokens":5193,"outputTokens":2,"latencyMs":580.0892079999999},{"questionId":"q2","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":6560,"outputTokens":1,"latencyMs":717.8327499999998},{"questionId":"q2","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":4151,"outputTokens":1,"latencyMs":578.4192089999999},{"questionId":"q2","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2730,"outputTokens":1,"latencyMs":656.1255000000001},{"questionId":"q2","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":2565,"outputTokens":1,"latencyMs":683.3620420000002},{"questionId":"q2","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":7497,"outputTokens":1,"latencyMs":582.2949590000001},{"questionId":"q2","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Marketing","actual":"Marketing","isCorrect":true,"inputTokens":5193,"outputTokens":1,"latencyMs":501.26083400000016},{"questionId":"q3","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":6563,"outputTokens":7,"latencyMs":382.26504199999977},{"questionId":"q3","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":4154,"outputTokens":7,"latencyMs":377.984958},{"questionId":"q3","format":"toon","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2733,"outputTokens":7,"latencyMs":391.02195900000015},{"questionId":"q3","format":"csv","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":2568,"outputTokens":7,"latencyMs":395.690875},{"questionId":"q3","format":"xml","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":7500,"outputTokens":7,"latencyMs":478.1205420000001},{"questionId":"q3","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"lorenza.kunze@yahoo.com","actual":"lorenza.kunze@yahoo.com","isCorrect":true,"inputTokens":5196,"outputTokens":7,"latencyMs":450.2183749999997},{"questionId":"q4","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":6563,"outputTokens":1,"latencyMs":422.99800000000005},{"questionId":"q4","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":4154,"outputTokens":1,"latencyMs":434.7655},{"questionId":"q4","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":2733,"outputTokens":1,"latencyMs":324.0345419999999},{"questionId":"q4","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":2568,"outputTokens":1,"latencyMs":445.52925000000005},{"questionId":"q4","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":7500,"outputTokens":1,"latencyMs":598.7832080000003},{"questionId":"q4","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":5196,"outputTokens":1,"latencyMs":426.23554200000035},{"questionId":"q5","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"no","actual":"false","isCorrect":true,"inputTokens":6558,"outputTokens":1,"latencyMs":410.69404099999974},{"questionId":"q5","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"no","actual":"false","isCorrect":true,"inputTokens":4149,"outputTokens":1,"latencyMs":478.8340000000003},{"questionId":"q5","format":"toon","model":"grok-4-fast-non-reasoning","expected":"no","actual":"false","isCorrect":true,"inputTokens":2728,"outputTokens":1,"latencyMs":300.2806660000001},{"questionId":"q5","format":"csv","model":"grok-4-fast-non-reasoning","expected":"no","actual":"No","isCorrect":true,"inputTokens":2563,"outputTokens":1,"latencyMs":351.082625},{"questionId":"q5","format":"xml","model":"grok-4-fast-non-reasoning","expected":"no","actual":"false","isCorrect":true,"inputTokens":7495,"outputTokens":1,"latencyMs":427.2885000000001},{"questionId":"q5","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"no","actual":"false","isCorrect":true,"inputTokens":5191,"outputTokens":1,"latencyMs":428.1032089999999},{"questionId":"q6","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":6559,"outputTokens":2,"latencyMs":749.5659999999998},{"questionId":"q6","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":4150,"outputTokens":2,"latencyMs":471.4871659999999},{"questionId":"q6","format":"toon","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2729,"outputTokens":2,"latencyMs":412.25533399999995},{"questionId":"q6","format":"csv","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":2564,"outputTokens":2,"latencyMs":526.023917},{"questionId":"q6","format":"xml","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":7496,"outputTokens":2,"latencyMs":509.95216600000003},{"questionId":"q6","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"133081","actual":"133081","isCorrect":true,"inputTokens":5192,"outputTokens":2,"latencyMs":376.5678340000004},{"questionId":"q7","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":6559,"outputTokens":1,"latencyMs":492.86204099999986},{"questionId":"q7","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":4150,"outputTokens":1,"latencyMs":365.94870900000024},{"questionId":"q7","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2729,"outputTokens":1,"latencyMs":360.1005},{"questionId":"q7","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":2564,"outputTokens":1,"latencyMs":364.8351249999996},{"questionId":"q7","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":7496,"outputTokens":1,"latencyMs":493.5971249999998},{"questionId":"q7","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Engineering","actual":"Engineering","isCorrect":true,"inputTokens":5192,"outputTokens":1,"latencyMs":431.6769999999997},{"questionId":"q8","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":6562,"outputTokens":6,"latencyMs":478.70483300000024},{"questionId":"q8","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":4153,"outputTokens":6,"latencyMs":390.107833},{"questionId":"q8","format":"toon","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2732,"outputTokens":6,"latencyMs":524.1480419999998},{"questionId":"q8","format":"csv","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":2567,"outputTokens":6,"latencyMs":455.20945800000027},{"questionId":"q8","format":"xml","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":7499,"outputTokens":6,"latencyMs":482.96370799999977},{"questionId":"q8","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"delpha.russel@gmail.com","actual":"delpha.russel@gmail.com","isCorrect":true,"inputTokens":5195,"outputTokens":6,"latencyMs":447.8932499999996},{"questionId":"q9","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":6563,"outputTokens":1,"latencyMs":426.44333300000017},{"questionId":"q9","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":4154,"outputTokens":1,"latencyMs":448.9592090000001},{"questionId":"q9","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":2733,"outputTokens":1,"latencyMs":455.6236250000002},{"questionId":"q9","format":"csv","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":2568,"outputTokens":1,"latencyMs":908.1022500000036},{"questionId":"q9","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":7500,"outputTokens":1,"latencyMs":890.3613330000007},{"questionId":"q9","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":5196,"outputTokens":1,"latencyMs":475.57624999999825},{"questionId":"q10","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"true","isCorrect":true,"inputTokens":6560,"outputTokens":1,"latencyMs":797.8072919999977},{"questionId":"q10","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"true","isCorrect":true,"inputTokens":4151,"outputTokens":1,"latencyMs":468.7669160000005},{"questionId":"q10","format":"toon","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"true","isCorrect":true,"inputTokens":2730,"outputTokens":1,"latencyMs":445.7910000000047},{"questionId":"q10","format":"csv","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"Yes","isCorrect":true,"inputTokens":2565,"outputTokens":1,"latencyMs":518.7246670000022},{"questionId":"q10","format":"xml","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"true","isCorrect":true,"inputTokens":7497,"outputTokens":1,"latencyMs":756.4337910000031},{"questionId":"q10","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"yes","actual":"true","isCorrect":true,"inputTokens":5193,"outputTokens":1,"latencyMs":662.2533750000002},{"questionId":"q11","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":6561,"outputTokens":2,"latencyMs":509.3157500000016},{"questionId":"q11","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":4152,"outputTokens":2,"latencyMs":354.12766700000066},{"questionId":"q11","format":"toon","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2731,"outputTokens":2,"latencyMs":361.2494579999984},{"questionId":"q11","format":"csv","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":2566,"outputTokens":2,"latencyMs":292.6892500000031},{"questionId":"q11","format":"xml","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":7498,"outputTokens":2,"latencyMs":395.3419170000052},{"questionId":"q11","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"109064","actual":"109064","isCorrect":true,"inputTokens":5194,"outputTokens":2,"latencyMs":393.2805840000001},{"questionId":"q12","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":6559,"outputTokens":1,"latencyMs":807.249708000003},{"questionId":"q12","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":4150,"outputTokens":1,"latencyMs":362.8900419999991},{"questionId":"q12","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2729,"outputTokens":1,"latencyMs":368.81670799999847},{"questionId":"q12","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":2564,"outputTokens":1,"latencyMs":449.58866699999635},{"questionId":"q12","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":7496,"outputTokens":1,"latencyMs":399.32745799999975},{"questionId":"q12","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Operations","actual":"Operations","isCorrect":true,"inputTokens":5192,"outputTokens":1,"latencyMs":392.00483299999905},{"questionId":"q13","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":6558,"outputTokens":1,"latencyMs":626.7569999999978},{"questionId":"q13","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"16","isCorrect":false,"inputTokens":4149,"outputTokens":1,"latencyMs":392.3937920000026},{"questionId":"q13","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2728,"outputTokens":1,"latencyMs":327.33245799999713},{"questionId":"q13","format":"csv","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2563,"outputTokens":1,"latencyMs":491.1735839999965},{"questionId":"q13","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":7495,"outputTokens":1,"latencyMs":525.1732499999998},{"questionId":"q13","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":5191,"outputTokens":1,"latencyMs":1003.3068340000027},{"questionId":"q14","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":6558,"outputTokens":1,"latencyMs":580.9329170000055},{"questionId":"q14","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":4149,"outputTokens":1,"latencyMs":537.2419579999987},{"questionId":"q14","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":2728,"outputTokens":1,"latencyMs":370.43841599999723},{"questionId":"q14","format":"csv","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":2563,"outputTokens":1,"latencyMs":333.84595799999806},{"questionId":"q14","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":7495,"outputTokens":1,"latencyMs":384.1995420000021},{"questionId":"q14","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":5191,"outputTokens":1,"latencyMs":345.3221249999988},{"questionId":"q15","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"20","isCorrect":false,"inputTokens":6558,"outputTokens":1,"latencyMs":375.46195799999987},{"questionId":"q15","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"16","isCorrect":false,"inputTokens":4149,"outputTokens":1,"latencyMs":365.8962909999973},{"questionId":"q15","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2728,"outputTokens":1,"latencyMs":307.53033300000243},{"questionId":"q15","format":"csv","model":"grok-4-fast-non-reasoning","expected":"17","actual":"25","isCorrect":false,"inputTokens":2563,"outputTokens":1,"latencyMs":1567.031833000001},{"questionId":"q15","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"16","isCorrect":false,"inputTokens":7495,"outputTokens":1,"latencyMs":403.44887500000186},{"questionId":"q15","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"16","isCorrect":false,"inputTokens":5191,"outputTokens":1,"latencyMs":356.9092080000046},{"questionId":"q16","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"91","actual":"91","isCorrect":true,"inputTokens":6563,"outputTokens":1,"latencyMs":352.5075419999994},{"questionId":"q16","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"91","actual":"85","isCorrect":false,"inputTokens":4154,"outputTokens":1,"latencyMs":326.18754199999967},{"questionId":"q16","format":"toon","model":"grok-4-fast-non-reasoning","expected":"91","actual":"75","isCorrect":false,"inputTokens":2733,"outputTokens":1,"latencyMs":474.1443749999962},{"questionId":"q16","format":"csv","model":"grok-4-fast-non-reasoning","expected":"91","actual":"85","isCorrect":false,"inputTokens":2568,"outputTokens":1,"latencyMs":485.487374999997},{"questionId":"q16","format":"xml","model":"grok-4-fast-non-reasoning","expected":"91","actual":"81","isCorrect":false,"inputTokens":7500,"outputTokens":1,"latencyMs":361.3055420000019},{"questionId":"q16","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"91","actual":"83","isCorrect":false,"inputTokens":5196,"outputTokens":1,"latencyMs":361.60304200000246},{"questionId":"q17","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"67","actual":"62","isCorrect":false,"inputTokens":6563,"outputTokens":1,"latencyMs":496.5832919999957},{"questionId":"q17","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"67","actual":"58","isCorrect":false,"inputTokens":4154,"outputTokens":1,"latencyMs":458.21570800000336},{"questionId":"q17","format":"toon","model":"grok-4-fast-non-reasoning","expected":"67","actual":"62","isCorrect":false,"inputTokens":2733,"outputTokens":1,"latencyMs":368.27454099999886},{"questionId":"q17","format":"csv","model":"grok-4-fast-non-reasoning","expected":"67","actual":"68","isCorrect":false,"inputTokens":2568,"outputTokens":1,"latencyMs":401.03558399999747},{"questionId":"q17","format":"xml","model":"grok-4-fast-non-reasoning","expected":"67","actual":"62","isCorrect":false,"inputTokens":7500,"outputTokens":1,"latencyMs":598.7391669999924},{"questionId":"q17","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"67","actual":"66","isCorrect":false,"inputTokens":5196,"outputTokens":1,"latencyMs":533.7937919999968},{"questionId":"q18","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"41","actual":"47","isCorrect":false,"inputTokens":6563,"outputTokens":1,"latencyMs":489.43316600000253},{"questionId":"q18","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"41","actual":"45","isCorrect":false,"inputTokens":4154,"outputTokens":1,"latencyMs":673.4545000000071},{"questionId":"q18","format":"toon","model":"grok-4-fast-non-reasoning","expected":"41","actual":"41","isCorrect":true,"inputTokens":2733,"outputTokens":1,"latencyMs":552.169167},{"questionId":"q18","format":"csv","model":"grok-4-fast-non-reasoning","expected":"41","actual":"51","isCorrect":false,"inputTokens":2568,"outputTokens":1,"latencyMs":785.3966669999936},{"questionId":"q18","format":"xml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"43","isCorrect":false,"inputTokens":7500,"outputTokens":1,"latencyMs":639.6285420000058},{"questionId":"q18","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"47","isCorrect":false,"inputTokens":5196,"outputTokens":1,"latencyMs":569.4996659999888},{"questionId":"q19","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":6559,"outputTokens":1,"latencyMs":650.5678329999937},{"questionId":"q19","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":4150,"outputTokens":1,"latencyMs":555.106541000001},{"questionId":"q19","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2729,"outputTokens":1,"latencyMs":400.81479099999706},{"questionId":"q19","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2564,"outputTokens":1,"latencyMs":350.09191600000486},{"questionId":"q19","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":7496,"outputTokens":1,"latencyMs":626.1560000000027},{"questionId":"q19","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":5192,"outputTokens":1,"latencyMs":395.8059579999972},{"questionId":"q20","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"100789","isCorrect":false,"inputTokens":6560,"outputTokens":2,"latencyMs":470.5184169999993},{"questionId":"q20","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"99852","isCorrect":false,"inputTokens":4151,"outputTokens":2,"latencyMs":485.94199999999546},{"questionId":"q20","format":"toon","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"102630","isCorrect":false,"inputTokens":2730,"outputTokens":2,"latencyMs":372.51462499999616},{"questionId":"q20","format":"csv","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"100341","isCorrect":false,"inputTokens":2565,"outputTokens":2,"latencyMs":395.00675000000047},{"questionId":"q20","format":"xml","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"97821","isCorrect":false,"inputTokens":7497,"outputTokens":2,"latencyMs":476.67729099999997},{"questionId":"q20","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"96503","actual":"100939","isCorrect":false,"inputTokens":5193,"outputTokens":2,"latencyMs":432.1019579999993},{"questionId":"q21","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"78","actual":"81","isCorrect":false,"inputTokens":6557,"outputTokens":1,"latencyMs":365.6849999999977},{"questionId":"q21","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"78","actual":"80","isCorrect":false,"inputTokens":4148,"outputTokens":1,"latencyMs":382.038584000009},{"questionId":"q21","format":"toon","model":"grok-4-fast-non-reasoning","expected":"78","actual":"79","isCorrect":false,"inputTokens":2727,"outputTokens":1,"latencyMs":333.379958000005},{"questionId":"q21","format":"csv","model":"grok-4-fast-non-reasoning","expected":"78","actual":"82","isCorrect":false,"inputTokens":2562,"outputTokens":1,"latencyMs":384.97716699999},{"questionId":"q21","format":"xml","model":"grok-4-fast-non-reasoning","expected":"78","actual":"83","isCorrect":false,"inputTokens":7494,"outputTokens":1,"latencyMs":386.79358300000604},{"questionId":"q21","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"78","actual":"81","isCorrect":false,"inputTokens":5190,"outputTokens":1,"latencyMs":464.0700829999987},{"questionId":"q22","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":6557,"outputTokens":1,"latencyMs":394.28233299999556},{"questionId":"q22","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"16","isCorrect":false,"inputTokens":4148,"outputTokens":1,"latencyMs":335.874708000003},{"questionId":"q22","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":2727,"outputTokens":1,"latencyMs":378.8475000000035},{"questionId":"q22","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":2562,"outputTokens":1,"latencyMs":356.22925000000396},{"questionId":"q22","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":7494,"outputTokens":1,"latencyMs":501.46566699999676},{"questionId":"q22","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"20","isCorrect":false,"inputTokens":5190,"outputTokens":1,"latencyMs":383.08312500000466},{"questionId":"q23","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":6565,"outputTokens":1,"latencyMs":387.2854590000061},{"questionId":"q23","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":4156,"outputTokens":1,"latencyMs":344.27012500001},{"questionId":"q23","format":"toon","model":"grok-4-fast-non-reasoning","expected":"12","actual":"21","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":389.89337499999965},{"questionId":"q23","format":"csv","model":"grok-4-fast-non-reasoning","expected":"12","actual":"19","isCorrect":false,"inputTokens":2570,"outputTokens":1,"latencyMs":386.65470800000185},{"questionId":"q23","format":"xml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"14","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":377.4197920000006},{"questionId":"q23","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"11","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":473.33866600001056},{"questionId":"q24","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"8","isCorrect":false,"inputTokens":6565,"outputTokens":1,"latencyMs":468.2155410000123},{"questionId":"q24","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":4156,"outputTokens":1,"latencyMs":438.24291699999594},{"questionId":"q24","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"13","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":284.88654200000747},{"questionId":"q24","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"9","isCorrect":false,"inputTokens":2570,"outputTokens":1,"latencyMs":320.83270900000934},{"questionId":"q24","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":465.59220800000185},{"questionId":"q24","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"13","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":388.47487500001444},{"questionId":"q25","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":6565,"outputTokens":1,"latencyMs":389.05633300000045},{"questionId":"q25","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":4156,"outputTokens":1,"latencyMs":376.7997500000056},{"questionId":"q25","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":494.0212909999973},{"questionId":"q25","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":2570,"outputTokens":1,"latencyMs":553.5510840000061},{"questionId":"q25","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"7","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":370.1583329999994},{"questionId":"q25","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":367.36149999999907},{"questionId":"q26","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"12","actual":"7","isCorrect":false,"inputTokens":6565,"outputTokens":1,"latencyMs":588.700416000007},{"questionId":"q26","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"12","actual":"5","isCorrect":false,"inputTokens":4156,"outputTokens":1,"latencyMs":489.94779200002085},{"questionId":"q26","format":"toon","model":"grok-4-fast-non-reasoning","expected":"12","actual":"7","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":493.28429199999664},{"questionId":"q26","format":"csv","model":"grok-4-fast-non-reasoning","expected":"12","actual":"11","isCorrect":false,"inputTokens":2570,"outputTokens":1,"latencyMs":473.4040410000016},{"questionId":"q26","format":"xml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"7","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":739.8324999999895},{"questionId":"q26","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"8","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":528.2567909999925},{"questionId":"q27","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":6565,"outputTokens":1,"latencyMs":505.07600000000093},{"questionId":"q27","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"14","isCorrect":false,"inputTokens":4156,"outputTokens":1,"latencyMs":749.5778330000176},{"questionId":"q27","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"9","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":470.81266699999105},{"questionId":"q27","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"18","isCorrect":false,"inputTokens":2570,"outputTokens":1,"latencyMs":484.2120409999916},{"questionId":"q27","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":489.93283400000655},{"questionId":"q27","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":511.75237500001094},{"questionId":"q28","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"63","actual":"70","isCorrect":false,"inputTokens":6564,"outputTokens":1,"latencyMs":471.23154099998646},{"questionId":"q28","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"63","actual":"70","isCorrect":false,"inputTokens":4155,"outputTokens":1,"latencyMs":347.4777499999909},{"questionId":"q28","format":"toon","model":"grok-4-fast-non-reasoning","expected":"63","actual":"85","isCorrect":false,"inputTokens":2734,"outputTokens":1,"latencyMs":355.61899999997695},{"questionId":"q28","format":"csv","model":"grok-4-fast-non-reasoning","expected":"63","actual":"78","isCorrect":false,"inputTokens":2569,"outputTokens":1,"latencyMs":405.3591659999802},{"questionId":"q28","format":"xml","model":"grok-4-fast-non-reasoning","expected":"63","actual":"76","isCorrect":false,"inputTokens":7501,"outputTokens":1,"latencyMs":373.0001249999914},{"questionId":"q28","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"63","actual":"76","isCorrect":false,"inputTokens":5197,"outputTokens":1,"latencyMs":384.298208000022},{"questionId":"q29","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"53","actual":"55","isCorrect":false,"inputTokens":6564,"outputTokens":1,"latencyMs":453.24754199999734},{"questionId":"q29","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"53","actual":"53","isCorrect":true,"inputTokens":4155,"outputTokens":1,"latencyMs":361.61341700001503},{"questionId":"q29","format":"toon","model":"grok-4-fast-non-reasoning","expected":"53","actual":"65","isCorrect":false,"inputTokens":2734,"outputTokens":1,"latencyMs":310.97220899999957},{"questionId":"q29","format":"csv","model":"grok-4-fast-non-reasoning","expected":"53","actual":"58","isCorrect":false,"inputTokens":2569,"outputTokens":1,"latencyMs":366.5611670000071},{"questionId":"q29","format":"xml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"53","isCorrect":true,"inputTokens":7501,"outputTokens":1,"latencyMs":446.7748750000028},{"questionId":"q29","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"59","isCorrect":false,"inputTokens":5197,"outputTokens":1,"latencyMs":329.3197919999948},{"questionId":"q30","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"39","actual":"35","isCorrect":false,"inputTokens":6564,"outputTokens":1,"latencyMs":315.1570419999771},{"questionId":"q30","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"39","actual":"32","isCorrect":false,"inputTokens":4155,"outputTokens":1,"latencyMs":439.5133330000099},{"questionId":"q30","format":"toon","model":"grok-4-fast-non-reasoning","expected":"39","actual":"29","isCorrect":false,"inputTokens":2734,"outputTokens":1,"latencyMs":334.3176669999957},{"questionId":"q30","format":"csv","model":"grok-4-fast-non-reasoning","expected":"39","actual":"41","isCorrect":false,"inputTokens":2569,"outputTokens":1,"latencyMs":405.55145800000173},{"questionId":"q30","format":"xml","model":"grok-4-fast-non-reasoning","expected":"39","actual":"29","isCorrect":false,"inputTokens":7501,"outputTokens":1,"latencyMs":439.76224999999977},{"questionId":"q30","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"39","actual":"30","isCorrect":false,"inputTokens":5197,"outputTokens":1,"latencyMs":413.1334999999963},{"questionId":"q31","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":6565,"outputTokens":1,"latencyMs":359.49666699999943},{"questionId":"q31","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":4156,"outputTokens":1,"latencyMs":381.6340410000121},{"questionId":"q31","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"25","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":293.55133300001035},{"questionId":"q31","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"18","isCorrect":false,"inputTokens":2570,"outputTokens":1,"latencyMs":356.8193339999998},{"questionId":"q31","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"14","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":594.680624999979},{"questionId":"q31","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":334.5366669999785},{"questionId":"q32","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"8","actual":"13","isCorrect":false,"inputTokens":6565,"outputTokens":1,"latencyMs":389.8008339999942},{"questionId":"q32","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"8","actual":"10","isCorrect":false,"inputTokens":4156,"outputTokens":1,"latencyMs":390.5162920000148},{"questionId":"q32","format":"toon","model":"grok-4-fast-non-reasoning","expected":"8","actual":"25","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":487.3843329999945},{"questionId":"q32","format":"csv","model":"grok-4-fast-non-reasoning","expected":"8","actual":"18","isCorrect":false,"inputTokens":2570,"outputTokens":1,"latencyMs":366.3003750000207},{"questionId":"q32","format":"xml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"16","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":549.9704170000041},{"questionId":"q32","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"15","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":376.284249999997},{"questionId":"q33","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"15","actual":"12","isCorrect":false,"inputTokens":6565,"outputTokens":1,"latencyMs":480.89120800001547},{"questionId":"q33","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"15","actual":"12","isCorrect":false,"inputTokens":4156,"outputTokens":1,"latencyMs":449.54308400000446},{"questionId":"q33","format":"toon","model":"grok-4-fast-non-reasoning","expected":"15","actual":"20","isCorrect":false,"inputTokens":2735,"outputTokens":1,"latencyMs":368.2242499999993},{"questionId":"q33","format":"csv","model":"grok-4-fast-non-reasoning","expected":"15","actual":"19","isCorrect":false,"inputTokens":2570,"outputTokens":1,"latencyMs":355.4397919999901},{"questionId":"q33","format":"xml","model":"grok-4-fast-non-reasoning","expected":"15","actual":"14","isCorrect":false,"inputTokens":7502,"outputTokens":1,"latencyMs":389.89441700000316},{"questionId":"q33","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"15","actual":"17","isCorrect":false,"inputTokens":5198,"outputTokens":1,"latencyMs":383.33695799999987},{"questionId":"q34","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":6559,"outputTokens":1,"latencyMs":357.63637500000186},{"questionId":"q34","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":4150,"outputTokens":1,"latencyMs":358.25270799998543},{"questionId":"q34","format":"toon","model":"grok-4-fast-non-reasoning","expected":"12","actual":"23","isCorrect":false,"inputTokens":2729,"outputTokens":1,"latencyMs":525.8525829999999},{"questionId":"q34","format":"csv","model":"grok-4-fast-non-reasoning","expected":"12","actual":"20","isCorrect":false,"inputTokens":2564,"outputTokens":1,"latencyMs":571.8574999999837},{"questionId":"q34","format":"xml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":7496,"outputTokens":1,"latencyMs":572.3744170000136},{"questionId":"q34","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"12","actual":"10","isCorrect":false,"inputTokens":5192,"outputTokens":1,"latencyMs":495.49575000000186},{"questionId":"q35","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"17","isCorrect":false,"inputTokens":6559,"outputTokens":1,"latencyMs":432.8766659999965},{"questionId":"q35","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"16","isCorrect":false,"inputTokens":4150,"outputTokens":1,"latencyMs":573.6981250000244},{"questionId":"q35","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"21","isCorrect":false,"inputTokens":2729,"outputTokens":1,"latencyMs":408.81629200000316},{"questionId":"q35","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":2564,"outputTokens":1,"latencyMs":427.94404200001736},{"questionId":"q35","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":7496,"outputTokens":1,"latencyMs":607.9440419999883},{"questionId":"q35","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"18","isCorrect":false,"inputTokens":5192,"outputTokens":1,"latencyMs":478.4229590000177},{"questionId":"q36","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":11002,"outputTokens":3,"latencyMs":612.4086249999818},{"questionId":"q36","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":6981,"outputTokens":3,"latencyMs":572.5292499999923},{"questionId":"q36","format":"toon","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":7404,"outputTokens":3,"latencyMs":492.7815830000036},{"questionId":"q36","format":"xml","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":12328,"outputTokens":3,"latencyMs":637.3694160000014},{"questionId":"q36","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"103.86","actual":"103.86","isCorrect":true,"inputTokens":8567,"outputTokens":3,"latencyMs":562.7034580000036},{"questionId":"q37","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":11002,"outputTokens":2,"latencyMs":642.8685420000111},{"questionId":"q37","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":6981,"outputTokens":2,"latencyMs":473.2799159999995},{"questionId":"q37","format":"toon","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":7404,"outputTokens":2,"latencyMs":651.6125829999801},{"questionId":"q37","format":"xml","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":12328,"outputTokens":2,"latencyMs":601.5966670000053},{"questionId":"q37","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"shipped","actual":"shipped","isCorrect":true,"inputTokens":8567,"outputTokens":2,"latencyMs":576.1943329999922},{"questionId":"q38","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":11002,"outputTokens":3,"latencyMs":753.5687499999767},{"questionId":"q38","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":6981,"outputTokens":3,"latencyMs":694.3835839999956},{"questionId":"q38","format":"toon","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":7404,"outputTokens":3,"latencyMs":668.5795409999846},{"questionId":"q38","format":"xml","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":12328,"outputTokens":3,"latencyMs":523.2738750000135},{"questionId":"q38","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"422.5","actual":"422.5","isCorrect":true,"inputTokens":8567,"outputTokens":3,"latencyMs":654.8136250000098},{"questionId":"q39","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":11002,"outputTokens":1,"latencyMs":580.741417000012},{"questionId":"q39","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":6981,"outputTokens":1,"latencyMs":559.4822500000009},{"questionId":"q39","format":"toon","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":7404,"outputTokens":1,"latencyMs":547.0760829999927},{"questionId":"q39","format":"xml","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":12328,"outputTokens":1,"latencyMs":544.0726250000007},{"questionId":"q39","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"processing","actual":"processing","isCorrect":true,"inputTokens":8567,"outputTokens":1,"latencyMs":1450.0137919999834},{"questionId":"q40","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":11002,"outputTokens":4,"latencyMs":636.7577910000109},{"questionId":"q40","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":6981,"outputTokens":4,"latencyMs":460.6567919999943},{"questionId":"q40","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":7404,"outputTokens":4,"latencyMs":393.5730420000036},{"questionId":"q40","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":12328,"outputTokens":4,"latencyMs":412.3980000000156},{"questionId":"q40","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1822.85","actual":"1822.85","isCorrect":true,"inputTokens":8567,"outputTokens":4,"latencyMs":586.1450830000103},{"questionId":"q41","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":11002,"outputTokens":1,"latencyMs":726.1220420000027},{"questionId":"q41","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":6981,"outputTokens":1,"latencyMs":432.288292000012},{"questionId":"q41","format":"toon","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":7404,"outputTokens":1,"latencyMs":492.4412499999744},{"questionId":"q41","format":"xml","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":12328,"outputTokens":1,"latencyMs":591.9253749999916},{"questionId":"q41","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"pending","actual":"pending","isCorrect":true,"inputTokens":8567,"outputTokens":1,"latencyMs":594.6765840000007},{"questionId":"q42","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":11002,"outputTokens":4,"latencyMs":697.430624999979},{"questionId":"q42","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":6981,"outputTokens":4,"latencyMs":512.4665419999801},{"questionId":"q42","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":7404,"outputTokens":4,"latencyMs":595.262000000017},{"questionId":"q42","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":12328,"outputTokens":4,"latencyMs":713.743083999987},{"questionId":"q42","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1311.35","actual":"1311.35","isCorrect":true,"inputTokens":8567,"outputTokens":4,"latencyMs":389.6293749999895},{"questionId":"q43","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":11002,"outputTokens":2,"latencyMs":638.5041670000064},{"questionId":"q43","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6981,"outputTokens":2,"latencyMs":422.45766600000206},{"questionId":"q43","format":"toon","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7404,"outputTokens":2,"latencyMs":430.90070899997954},{"questionId":"q43","format":"xml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12328,"outputTokens":2,"latencyMs":656.4609589999891},{"questionId":"q43","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8567,"outputTokens":2,"latencyMs":556.2104580000159},{"questionId":"q44","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":11003,"outputTokens":6,"latencyMs":521.8144169999869},{"questionId":"q44","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":6982,"outputTokens":6,"latencyMs":485.51504199998453},{"questionId":"q44","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":7405,"outputTokens":6,"latencyMs":489.10249999997905},{"questionId":"q44","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":12329,"outputTokens":6,"latencyMs":686.4327080000076},{"questionId":"q44","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Debbie O'Kon I","actual":"Debbie O'Kon I","isCorrect":true,"inputTokens":8568,"outputTokens":6,"latencyMs":604.0870839999989},{"questionId":"q45","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":11003,"outputTokens":9,"latencyMs":547.9094160000095},{"questionId":"q45","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":6982,"outputTokens":9,"latencyMs":504.69150000001537},{"questionId":"q45","format":"toon","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":7405,"outputTokens":9,"latencyMs":524.7769579999731},{"questionId":"q45","format":"xml","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":12329,"outputTokens":9,"latencyMs":736.773125000007},{"questionId":"q45","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"demetris.hoeger-pollich@yahoo.com","actual":"demetris.hoeger-pollich@yahoo.com","isCorrect":true,"inputTokens":8568,"outputTokens":9,"latencyMs":768.905584000051},{"questionId":"q46","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":11003,"outputTokens":6,"latencyMs":657.8537500000093},{"questionId":"q46","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":6982,"outputTokens":6,"latencyMs":402.3881249999977},{"questionId":"q46","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":7405,"outputTokens":6,"latencyMs":473.46395800000755},{"questionId":"q46","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":12329,"outputTokens":6,"latencyMs":355.3811249999562},{"questionId":"q46","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2025-09-16","actual":"2025-09-16","isCorrect":true,"inputTokens":8568,"outputTokens":6,"latencyMs":838.094167000032},{"questionId":"q47","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":11002,"outputTokens":1,"latencyMs":568.6594170000171},{"questionId":"q47","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":6981,"outputTokens":1,"latencyMs":420.8817079999717},{"questionId":"q47","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":7404,"outputTokens":1,"latencyMs":384.15070800000103},{"questionId":"q47","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":12328,"outputTokens":1,"latencyMs":663.5564160000067},{"questionId":"q47","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"10","isCorrect":false,"inputTokens":8567,"outputTokens":1,"latencyMs":819.5229169999948},{"questionId":"q48","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":11003,"outputTokens":4,"latencyMs":683.3059169999906},{"questionId":"q48","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":6982,"outputTokens":4,"latencyMs":431.03516700002365},{"questionId":"q48","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":7405,"outputTokens":4,"latencyMs":408.711249999993},{"questionId":"q48","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":12329,"outputTokens":4,"latencyMs":562.3704170000274},{"questionId":"q48","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Patty Senger","actual":"Patty Senger","isCorrect":true,"inputTokens":8568,"outputTokens":4,"latencyMs":613.7806249999558},{"questionId":"q49","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":11003,"outputTokens":7,"latencyMs":515.8946250000154},{"questionId":"q49","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":6982,"outputTokens":7,"latencyMs":448.41916599997785},{"questionId":"q49","format":"toon","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":7405,"outputTokens":7,"latencyMs":465.6530420000199},{"questionId":"q49","format":"xml","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":12329,"outputTokens":7,"latencyMs":683.9833750000107},{"questionId":"q49","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"viva.paucek@gmail.com","actual":"viva.paucek@gmail.com","isCorrect":true,"inputTokens":8568,"outputTokens":7,"latencyMs":452.8107500000042},{"questionId":"q50","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":11003,"outputTokens":6,"latencyMs":576.1988330000313},{"questionId":"q50","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":6982,"outputTokens":6,"latencyMs":517.3172499999637},{"questionId":"q50","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":7405,"outputTokens":6,"latencyMs":345.7018339999486},{"questionId":"q50","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":12329,"outputTokens":6,"latencyMs":483.4087089999812},{"questionId":"q50","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2025-09-21","actual":"2025-09-21","isCorrect":true,"inputTokens":8568,"outputTokens":6,"latencyMs":517.2318749999977},{"questionId":"q51","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"5","isCorrect":false,"inputTokens":11002,"outputTokens":1,"latencyMs":684.275958000042},{"questionId":"q51","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"6","isCorrect":false,"inputTokens":6981,"outputTokens":1,"latencyMs":408.7041250000475},{"questionId":"q51","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":7404,"outputTokens":1,"latencyMs":463.002416000003},{"questionId":"q51","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"6","isCorrect":false,"inputTokens":12328,"outputTokens":1,"latencyMs":751.1919999999809},{"questionId":"q51","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"6","isCorrect":false,"inputTokens":8567,"outputTokens":1,"latencyMs":439.47795800003223},{"questionId":"q52","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":10999,"outputTokens":1,"latencyMs":646.1944159999839},{"questionId":"q52","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":6978,"outputTokens":1,"latencyMs":430.5902080000378},{"questionId":"q52","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":7401,"outputTokens":1,"latencyMs":465.5190419999999},{"questionId":"q52","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"11","isCorrect":false,"inputTokens":12325,"outputTokens":1,"latencyMs":664.169208999956},{"questionId":"q52","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":8564,"outputTokens":1,"latencyMs":410.94754099997226},{"questionId":"q53","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":10999,"outputTokens":1,"latencyMs":531.8702909999993},{"questionId":"q53","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":6978,"outputTokens":1,"latencyMs":360.05287499999395},{"questionId":"q53","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":7401,"outputTokens":1,"latencyMs":430.0072919999948},{"questionId":"q53","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":12325,"outputTokens":1,"latencyMs":412.5618339999928},{"questionId":"q53","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"12","isCorrect":false,"inputTokens":8564,"outputTokens":1,"latencyMs":530.6472500000382},{"questionId":"q54","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"8","isCorrect":false,"inputTokens":11000,"outputTokens":1,"latencyMs":661.8510829999577},{"questionId":"q54","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"8","isCorrect":false,"inputTokens":6979,"outputTokens":1,"latencyMs":452.9267080000136},{"questionId":"q54","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":7402,"outputTokens":1,"latencyMs":427.1089590000338},{"questionId":"q54","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"8","isCorrect":false,"inputTokens":12326,"outputTokens":1,"latencyMs":570.2559159999946},{"questionId":"q54","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"10","isCorrect":true,"inputTokens":8565,"outputTokens":1,"latencyMs":443.8168750000186},{"questionId":"q55","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"35232.49","isCorrect":false,"inputTokens":11000,"outputTokens":4,"latencyMs":759.7094170000055},{"questionId":"q55","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"43395.85","isCorrect":false,"inputTokens":6979,"outputTokens":4,"latencyMs":546.033832999994},{"questionId":"q55","format":"toon","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"33029.79","isCorrect":false,"inputTokens":7402,"outputTokens":4,"latencyMs":583.431958000001},{"questionId":"q55","format":"xml","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"40203.57","isCorrect":false,"inputTokens":12326,"outputTokens":4,"latencyMs":669.4822500000009},{"questionId":"q55","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"34904.81","actual":"33915.18","isCorrect":false,"inputTokens":8565,"outputTokens":4,"latencyMs":442.56004200002644},{"questionId":"q56","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"638.54","isCorrect":false,"inputTokens":10998,"outputTokens":3,"latencyMs":682.2162920000264},{"questionId":"q56","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"723.08","isCorrect":false,"inputTokens":6977,"outputTokens":3,"latencyMs":598.9386659999727},{"questionId":"q56","format":"toon","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"542.89","isCorrect":false,"inputTokens":7400,"outputTokens":3,"latencyMs":422.1798749999725},{"questionId":"q56","format":"xml","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"777.04","isCorrect":false,"inputTokens":12324,"outputTokens":3,"latencyMs":416.94833400001517},{"questionId":"q56","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"698.10","actual":"718.11","isCorrect":false,"inputTokens":8563,"outputTokens":3,"latencyMs":1387.9755839999998},{"questionId":"q57","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":10999,"outputTokens":1,"latencyMs":668.5463749999763},{"questionId":"q57","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":6978,"outputTokens":1,"latencyMs":353.4004590000259},{"questionId":"q57","format":"toon","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":7401,"outputTokens":1,"latencyMs":297.88175000000047},{"questionId":"q57","format":"xml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":12325,"outputTokens":1,"latencyMs":436.80945800000336},{"questionId":"q57","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":8564,"outputTokens":1,"latencyMs":650.6135830000276},{"questionId":"q58","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":10998,"outputTokens":4,"latencyMs":384.7578329999815},{"questionId":"q58","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":6977,"outputTokens":4,"latencyMs":404.252416000003},{"questionId":"q58","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":7400,"outputTokens":4,"latencyMs":406.7724579999922},{"questionId":"q58","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":12324,"outputTokens":4,"latencyMs":530.9796669999487},{"questionId":"q58","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2152.82","actual":"2152.82","isCorrect":true,"inputTokens":8563,"outputTokens":4,"latencyMs":559.500457999995},{"questionId":"q59","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"43","actual":"39","isCorrect":false,"inputTokens":11002,"outputTokens":1,"latencyMs":538.4423749999842},{"questionId":"q59","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"43","actual":"35","isCorrect":false,"inputTokens":6981,"outputTokens":1,"latencyMs":367.47462499997346},{"questionId":"q59","format":"toon","model":"grok-4-fast-non-reasoning","expected":"43","actual":"38","isCorrect":false,"inputTokens":7404,"outputTokens":1,"latencyMs":359.26666599995224},{"questionId":"q59","format":"xml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"35","isCorrect":false,"inputTokens":12328,"outputTokens":1,"latencyMs":380.78679199999897},{"questionId":"q59","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"34","isCorrect":false,"inputTokens":8567,"outputTokens":1,"latencyMs":369.093292000005},{"questionId":"q60","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"37","actual":"27","isCorrect":false,"inputTokens":11002,"outputTokens":1,"latencyMs":534.5345420000376},{"questionId":"q60","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"37","actual":"25","isCorrect":false,"inputTokens":6981,"outputTokens":1,"latencyMs":379.0307909999974},{"questionId":"q60","format":"toon","model":"grok-4-fast-non-reasoning","expected":"37","actual":"31","isCorrect":false,"inputTokens":7404,"outputTokens":1,"latencyMs":384.44416600000113},{"questionId":"q60","format":"xml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"28","isCorrect":false,"inputTokens":12328,"outputTokens":1,"latencyMs":653.4851669999771},{"questionId":"q60","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"27","isCorrect":false,"inputTokens":8567,"outputTokens":1,"latencyMs":419.6130000000121},{"questionId":"q61","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"28","actual":"21","isCorrect":false,"inputTokens":11002,"outputTokens":1,"latencyMs":441.579458000022},{"questionId":"q61","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"28","actual":"16","isCorrect":false,"inputTokens":6981,"outputTokens":1,"latencyMs":317.4882499999949},{"questionId":"q61","format":"toon","model":"grok-4-fast-non-reasoning","expected":"28","actual":"20","isCorrect":false,"inputTokens":7404,"outputTokens":1,"latencyMs":431.2264580000192},{"questionId":"q61","format":"xml","model":"grok-4-fast-non-reasoning","expected":"28","actual":"16","isCorrect":false,"inputTokens":12328,"outputTokens":1,"latencyMs":448.7486670000362},{"questionId":"q61","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"28","actual":"13","isCorrect":false,"inputTokens":8567,"outputTokens":1,"latencyMs":366.19966699997894},{"questionId":"q62","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"8","actual":"8","isCorrect":true,"inputTokens":11006,"outputTokens":1,"latencyMs":448.1374590000487},{"questionId":"q62","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":6985,"outputTokens":1,"latencyMs":1216.9280410000356},{"questionId":"q62","format":"toon","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":7408,"outputTokens":1,"latencyMs":532.0604590000003},{"questionId":"q62","format":"xml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":12332,"outputTokens":1,"latencyMs":469.59854100004304},{"questionId":"q62","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"8","isCorrect":true,"inputTokens":8571,"outputTokens":1,"latencyMs":475.4224159999867},{"questionId":"q63","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6","actual":"8","isCorrect":false,"inputTokens":11006,"outputTokens":1,"latencyMs":419.46033299999544},{"questionId":"q63","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6","actual":"9","isCorrect":false,"inputTokens":6985,"outputTokens":1,"latencyMs":329.4613749999553},{"questionId":"q63","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6","actual":"11","isCorrect":false,"inputTokens":7408,"outputTokens":1,"latencyMs":439.02691700001014},{"questionId":"q63","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"13","isCorrect":false,"inputTokens":12332,"outputTokens":1,"latencyMs":584.722666000016},{"questionId":"q63","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"13","isCorrect":false,"inputTokens":8571,"outputTokens":1,"latencyMs":517.6271249999991},{"questionId":"q64","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10","actual":"9","isCorrect":false,"inputTokens":11007,"outputTokens":1,"latencyMs":551.3271659999737},{"questionId":"q64","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10","actual":"9","isCorrect":false,"inputTokens":6986,"outputTokens":1,"latencyMs":515.8021249999874},{"questionId":"q64","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10","actual":"8","isCorrect":false,"inputTokens":7409,"outputTokens":1,"latencyMs":502.13916700001573},{"questionId":"q64","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"9","isCorrect":false,"inputTokens":12333,"outputTokens":1,"latencyMs":684.2752919999766},{"questionId":"q64","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10","actual":"8","isCorrect":false,"inputTokens":8572,"outputTokens":1,"latencyMs":505.5644999999786},{"questionId":"q65","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"9","actual":"6","isCorrect":false,"inputTokens":11007,"outputTokens":1,"latencyMs":552.0897499999846},{"questionId":"q65","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"9","actual":"6","isCorrect":false,"inputTokens":6986,"outputTokens":1,"latencyMs":492.0918750000419},{"questionId":"q65","format":"toon","model":"grok-4-fast-non-reasoning","expected":"9","actual":"7","isCorrect":false,"inputTokens":7409,"outputTokens":1,"latencyMs":601.0383750000037},{"questionId":"q65","format":"xml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"7","isCorrect":false,"inputTokens":12333,"outputTokens":1,"latencyMs":484.6959999999963},{"questionId":"q65","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"6","isCorrect":false,"inputTokens":8572,"outputTokens":1,"latencyMs":670.3162080000038},{"questionId":"q66","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"0","isCorrect":false,"inputTokens":11006,"outputTokens":1,"latencyMs":1003.2866250000079},{"questionId":"q66","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"0","isCorrect":false,"inputTokens":6985,"outputTokens":1,"latencyMs":395.595290999976},{"questionId":"q66","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"0","isCorrect":false,"inputTokens":7408,"outputTokens":1,"latencyMs":398.3397089999635},{"questionId":"q66","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"4","isCorrect":false,"inputTokens":12332,"outputTokens":1,"latencyMs":625.3687499999651},{"questionId":"q66","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"0","isCorrect":false,"inputTokens":8571,"outputTokens":1,"latencyMs":963.0364159999881},{"questionId":"q67","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"5","isCorrect":false,"inputTokens":11006,"outputTokens":1,"latencyMs":558.2407910000184},{"questionId":"q67","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"5","isCorrect":false,"inputTokens":6985,"outputTokens":1,"latencyMs":376.47474999999395},{"questionId":"q67","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"5","isCorrect":false,"inputTokens":7408,"outputTokens":1,"latencyMs":470.55970799998613},{"questionId":"q67","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"7","isCorrect":false,"inputTokens":12332,"outputTokens":1,"latencyMs":437.3194159999839},{"questionId":"q67","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":8571,"outputTokens":1,"latencyMs":359.2603329999838},{"questionId":"q68","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"4","isCorrect":false,"inputTokens":11007,"outputTokens":1,"latencyMs":547.9597080000094},{"questionId":"q68","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"7","isCorrect":false,"inputTokens":6986,"outputTokens":1,"latencyMs":567.7719159999979},{"questionId":"q68","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":7409,"outputTokens":1,"latencyMs":430.3891669999575},{"questionId":"q68","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"7","isCorrect":false,"inputTokens":12333,"outputTokens":1,"latencyMs":365.6070419999887},{"questionId":"q68","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"3","isCorrect":false,"inputTokens":8572,"outputTokens":1,"latencyMs":449.9415409999783},{"questionId":"q69","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"20","actual":"5","isCorrect":false,"inputTokens":11008,"outputTokens":1,"latencyMs":509.45583300001454},{"questionId":"q69","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"20","actual":"16","isCorrect":false,"inputTokens":6987,"outputTokens":1,"latencyMs":475.1854170000297},{"questionId":"q69","format":"toon","model":"grok-4-fast-non-reasoning","expected":"20","actual":"15","isCorrect":false,"inputTokens":7410,"outputTokens":1,"latencyMs":355.5006670000148},{"questionId":"q69","format":"xml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"16","isCorrect":false,"inputTokens":12334,"outputTokens":1,"latencyMs":360.96000000002095},{"questionId":"q69","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"7","isCorrect":false,"inputTokens":8573,"outputTokens":1,"latencyMs":444.2795829999959},{"questionId":"q70","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"19","actual":"5","isCorrect":false,"inputTokens":11008,"outputTokens":1,"latencyMs":476.7892920000013},{"questionId":"q70","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"19","actual":"10","isCorrect":false,"inputTokens":6987,"outputTokens":1,"latencyMs":532.6080000000075},{"questionId":"q70","format":"toon","model":"grok-4-fast-non-reasoning","expected":"19","actual":"12","isCorrect":false,"inputTokens":7410,"outputTokens":1,"latencyMs":466.7905000000028},{"questionId":"q70","format":"xml","model":"grok-4-fast-non-reasoning","expected":"19","actual":"11","isCorrect":false,"inputTokens":12334,"outputTokens":1,"latencyMs":448.8921670000418},{"questionId":"q70","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"19","actual":"9","isCorrect":false,"inputTokens":8573,"outputTokens":1,"latencyMs":346.0952919999836},{"questionId":"q71","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3872,"outputTokens":2,"latencyMs":375.6290830000071},{"questionId":"q71","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":2543,"outputTokens":2,"latencyMs":399.9712500000023},{"questionId":"q71","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1802,"outputTokens":2,"latencyMs":359.32070799998473},{"questionId":"q71","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":1661,"outputTokens":2,"latencyMs":382.64333300001454},{"questionId":"q71","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":4518,"outputTokens":2,"latencyMs":367.4043749999837},{"questionId":"q71","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4322","actual":"4322","isCorrect":true,"inputTokens":3202,"outputTokens":2,"latencyMs":407.90716699999757},{"questionId":"q72","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3872,"outputTokens":4,"latencyMs":372.126333000022},{"questionId":"q72","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":2543,"outputTokens":4,"latencyMs":328.7396660000086},{"questionId":"q72","format":"toon","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1802,"outputTokens":4,"latencyMs":345.01787500001956},{"questionId":"q72","format":"csv","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":1661,"outputTokens":4,"latencyMs":340.06187500001397},{"questionId":"q72","format":"xml","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":4518,"outputTokens":4,"latencyMs":369.8200829999987},{"questionId":"q72","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"10432.04","actual":"10432.04","isCorrect":true,"inputTokens":3202,"outputTokens":4,"latencyMs":450.13054199999897},{"questionId":"q73","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3873,"outputTokens":3,"latencyMs":376.9850410000072},{"questionId":"q73","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":2544,"outputTokens":3,"latencyMs":363.1427910000202},{"questionId":"q73","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1803,"outputTokens":3,"latencyMs":311.60808299999917},{"questionId":"q73","format":"csv","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1662,"outputTokens":3,"latencyMs":439.0734999999986},{"questionId":"q73","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4519,"outputTokens":3,"latencyMs":477.4262909999816},{"questionId":"q73","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3203,"outputTokens":3,"latencyMs":460.1783750000177},{"questionId":"q74","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":3873,"outputTokens":1,"latencyMs":387.48995900002774},{"questionId":"q74","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":2544,"outputTokens":1,"latencyMs":421.39641700003995},{"questionId":"q74","format":"toon","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":1803,"outputTokens":1,"latencyMs":612.4020829999936},{"questionId":"q74","format":"csv","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":1662,"outputTokens":1,"latencyMs":428.87724999996135},{"questionId":"q74","format":"xml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":4519,"outputTokens":1,"latencyMs":425.1087089999928},{"questionId":"q74","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"32","isCorrect":true,"inputTokens":3203,"outputTokens":1,"latencyMs":445.2866250000079},{"questionId":"q75","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3872,"outputTokens":2,"latencyMs":370.5849579999922},{"questionId":"q75","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":2543,"outputTokens":2,"latencyMs":353.4156250000233},{"questionId":"q75","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1802,"outputTokens":2,"latencyMs":713.7864579999587},{"questionId":"q75","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":1661,"outputTokens":2,"latencyMs":359.63825000001816},{"questionId":"q75","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":4518,"outputTokens":2,"latencyMs":347.45604200003436},{"questionId":"q75","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4096","actual":"4096","isCorrect":true,"inputTokens":3202,"outputTokens":2,"latencyMs":402.4195840000175},{"questionId":"q76","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3872,"outputTokens":4,"latencyMs":335.95724999997765},{"questionId":"q76","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":2543,"outputTokens":4,"latencyMs":371.27941600000486},{"questionId":"q76","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1802,"outputTokens":4,"latencyMs":414.26345800003037},{"questionId":"q76","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":1661,"outputTokens":4,"latencyMs":381.7441660000477},{"questionId":"q76","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":4518,"outputTokens":4,"latencyMs":303.38991700002225},{"questionId":"q76","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4533.1","actual":"4533.1","isCorrect":true,"inputTokens":3202,"outputTokens":4,"latencyMs":338.5647079999908},{"questionId":"q77","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3873,"outputTokens":3,"latencyMs":340.7854170000064},{"questionId":"q77","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":2544,"outputTokens":3,"latencyMs":339.90333400003146},{"questionId":"q77","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1803,"outputTokens":3,"latencyMs":369.13745799998287},{"questionId":"q77","format":"csv","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":1662,"outputTokens":3,"latencyMs":365.2921670000069},{"questionId":"q77","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":4519,"outputTokens":3,"latencyMs":360.9095419999794},{"questionId":"q77","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0.63","actual":"0.63","isCorrect":true,"inputTokens":3203,"outputTokens":3,"latencyMs":322.78712500003166},{"questionId":"q78","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":3873,"outputTokens":1,"latencyMs":372.26491599995643},{"questionId":"q78","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":2544,"outputTokens":1,"latencyMs":440.89299999998184},{"questionId":"q78","format":"toon","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":1803,"outputTokens":1,"latencyMs":335.32399999996414},{"questionId":"q78","format":"csv","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":1662,"outputTokens":1,"latencyMs":395.82070799998473},{"questionId":"q78","format":"xml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":4519,"outputTokens":1,"latencyMs":382.11570899997605},{"questionId":"q78","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":3203,"outputTokens":1,"latencyMs":361.80091699998593},{"questionId":"q79","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3872,"outputTokens":2,"latencyMs":382.73629199998686},{"questionId":"q79","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":2543,"outputTokens":2,"latencyMs":335.73816599999554},{"questionId":"q79","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1802,"outputTokens":2,"latencyMs":336.4151670000283},{"questionId":"q79","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":1661,"outputTokens":2,"latencyMs":384.52933399996255},{"questionId":"q79","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":4518,"outputTokens":2,"latencyMs":423.821082999988},{"questionId":"q79","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4076","actual":"4076","isCorrect":true,"inputTokens":3202,"outputTokens":2,"latencyMs":319.3876659999951},{"questionId":"q80","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"60","actual":"58","isCorrect":false,"inputTokens":3869,"outputTokens":1,"latencyMs":334.3883330000099},{"questionId":"q80","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"60","actual":"58","isCorrect":false,"inputTokens":2540,"outputTokens":1,"latencyMs":341.8335419999785},{"questionId":"q80","format":"toon","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":1799,"outputTokens":1,"latencyMs":348.5822920000064},{"questionId":"q80","format":"csv","model":"grok-4-fast-non-reasoning","expected":"60","actual":"59","isCorrect":false,"inputTokens":1658,"outputTokens":1,"latencyMs":466.70966699998826},{"questionId":"q80","format":"xml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"59","isCorrect":false,"inputTokens":4515,"outputTokens":1,"latencyMs":348.4097920000204},{"questionId":"q80","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"61","isCorrect":false,"inputTokens":3199,"outputTokens":1,"latencyMs":346.0340000000433},{"questionId":"q81","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"272833","isCorrect":false,"inputTokens":3870,"outputTokens":2,"latencyMs":395.8861249999609},{"questionId":"q81","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"362887","isCorrect":false,"inputTokens":2541,"outputTokens":2,"latencyMs":356.4520410000114},{"questionId":"q81","format":"toon","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"351281","isCorrect":false,"inputTokens":1800,"outputTokens":2,"latencyMs":373.6114579999703},{"questionId":"q81","format":"csv","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"501690","isCorrect":false,"inputTokens":1659,"outputTokens":2,"latencyMs":513.4524170000223},{"questionId":"q81","format":"xml","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"307615","isCorrect":false,"inputTokens":4516,"outputTokens":2,"latencyMs":387.28670799999963},{"questionId":"q81","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"328320","actual":"207549","isCorrect":false,"inputTokens":3200,"outputTokens":2,"latencyMs":466.29179200006183},{"questionId":"q82","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1299","isCorrect":false,"inputTokens":3870,"outputTokens":2,"latencyMs":430.0735000000568},{"questionId":"q82","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1409","isCorrect":false,"inputTokens":2541,"outputTokens":2,"latencyMs":449.76204199995846},{"questionId":"q82","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1274","isCorrect":false,"inputTokens":1800,"outputTokens":2,"latencyMs":438.50399999995716},{"questionId":"q82","format":"csv","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1683","isCorrect":false,"inputTokens":1659,"outputTokens":2,"latencyMs":396.8267079999205},{"questionId":"q82","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1293","isCorrect":false,"inputTokens":4516,"outputTokens":2,"latencyMs":418.2131669999799},{"questionId":"q82","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1791","actual":"1226","isCorrect":false,"inputTokens":3200,"outputTokens":2,"latencyMs":441.88758300000336},{"questionId":"q83","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"311298.18","isCorrect":false,"inputTokens":3868,"outputTokens":4,"latencyMs":445.11537500005215},{"questionId":"q83","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"270587.8","isCorrect":false,"inputTokens":2539,"outputTokens":4,"latencyMs":638.9185830000788},{"questionId":"q83","format":"toon","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"319392.33","isCorrect":false,"inputTokens":1798,"outputTokens":4,"latencyMs":476.3847920000553},{"questionId":"q83","format":"csv","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"328334.73","isCorrect":false,"inputTokens":1657,"outputTokens":4,"latencyMs":428.5289590000175},{"questionId":"q83","format":"xml","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"322090.84","isCorrect":false,"inputTokens":4514,"outputTokens":4,"latencyMs":373.93425000004936},{"questionId":"q83","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"311695.88","actual":"301784.79","isCorrect":false,"inputTokens":3198,"outputTokens":4,"latencyMs":1071.7871249999152},{"questionId":"q84","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3866,"outputTokens":3,"latencyMs":360.79020899999887},{"questionId":"q84","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.52","isCorrect":false,"inputTokens":2537,"outputTokens":3,"latencyMs":375.89100000006147},{"questionId":"q84","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.52","isCorrect":false,"inputTokens":1796,"outputTokens":3,"latencyMs":337.0397080000257},{"questionId":"q84","format":"csv","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":1655,"outputTokens":3,"latencyMs":331.34808300004806},{"questionId":"q84","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":4512,"outputTokens":3,"latencyMs":371.9904160000151},{"questionId":"q84","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0.53","actual":"0.53","isCorrect":true,"inputTokens":3196,"outputTokens":3,"latencyMs":384.24454099999275},{"questionId":"q85","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"23","isCorrect":false,"inputTokens":3870,"outputTokens":1,"latencyMs":359.139542000019},{"questionId":"q85","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"21","isCorrect":false,"inputTokens":2541,"outputTokens":1,"latencyMs":334.29637500003446},{"questionId":"q85","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"18","isCorrect":false,"inputTokens":1800,"outputTokens":1,"latencyMs":385.33116699999664},{"questionId":"q85","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":1659,"outputTokens":1,"latencyMs":355.49070800002664},{"questionId":"q85","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"19","isCorrect":false,"inputTokens":4516,"outputTokens":1,"latencyMs":296.9247500000056},{"questionId":"q85","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":3200,"outputTokens":1,"latencyMs":767.9793749999953},{"questionId":"q86","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"42","actual":"47","isCorrect":false,"inputTokens":3869,"outputTokens":1,"latencyMs":363.26183299999684},{"questionId":"q86","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"42","actual":"35","isCorrect":false,"inputTokens":2540,"outputTokens":1,"latencyMs":475.42800000007264},{"questionId":"q86","format":"toon","model":"grok-4-fast-non-reasoning","expected":"42","actual":"51","isCorrect":false,"inputTokens":1799,"outputTokens":1,"latencyMs":286.6536250000354},{"questionId":"q86","format":"csv","model":"grok-4-fast-non-reasoning","expected":"42","actual":"58","isCorrect":false,"inputTokens":1658,"outputTokens":1,"latencyMs":364.34375},{"questionId":"q86","format":"xml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"62","isCorrect":false,"inputTokens":4515,"outputTokens":1,"latencyMs":374.23100000002887},{"questionId":"q86","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"40","isCorrect":false,"inputTokens":3199,"outputTokens":1,"latencyMs":352.00304199999664},{"questionId":"q87","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"20","actual":"6","isCorrect":false,"inputTokens":3876,"outputTokens":1,"latencyMs":367.6728749999311},{"questionId":"q87","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"20","actual":"15","isCorrect":false,"inputTokens":2547,"outputTokens":1,"latencyMs":343.33929200004786},{"questionId":"q87","format":"toon","model":"grok-4-fast-non-reasoning","expected":"20","actual":"14","isCorrect":false,"inputTokens":1806,"outputTokens":1,"latencyMs":368.82399999990594},{"questionId":"q87","format":"csv","model":"grok-4-fast-non-reasoning","expected":"20","actual":"9","isCorrect":false,"inputTokens":1665,"outputTokens":1,"latencyMs":331.8484579999931},{"questionId":"q87","format":"xml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"13","isCorrect":false,"inputTokens":4522,"outputTokens":1,"latencyMs":342.2289999999339},{"questionId":"q87","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"11","isCorrect":false,"inputTokens":3206,"outputTokens":1,"latencyMs":471.19845900009386},{"questionId":"q88","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"14","actual":"5","isCorrect":false,"inputTokens":3876,"outputTokens":1,"latencyMs":343.48141599993687},{"questionId":"q88","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"14","actual":"5","isCorrect":false,"inputTokens":2547,"outputTokens":1,"latencyMs":475.6662499998929},{"questionId":"q88","format":"toon","model":"grok-4-fast-non-reasoning","expected":"14","actual":"7","isCorrect":false,"inputTokens":1806,"outputTokens":1,"latencyMs":422.21475000004284},{"questionId":"q88","format":"csv","model":"grok-4-fast-non-reasoning","expected":"14","actual":"7","isCorrect":false,"inputTokens":1665,"outputTokens":1,"latencyMs":333.34437499998603},{"questionId":"q88","format":"xml","model":"grok-4-fast-non-reasoning","expected":"14","actual":"9","isCorrect":false,"inputTokens":4522,"outputTokens":1,"latencyMs":361.6920829999726},{"questionId":"q88","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"14","actual":"3","isCorrect":false,"inputTokens":3206,"outputTokens":1,"latencyMs":363.25216700008605},{"questionId":"q89","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"7","isCorrect":false,"inputTokens":3876,"outputTokens":1,"latencyMs":380.0844170000637},{"questionId":"q89","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"7","isCorrect":false,"inputTokens":2547,"outputTokens":1,"latencyMs":360.6766249999637},{"questionId":"q89","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"12","isCorrect":false,"inputTokens":1806,"outputTokens":1,"latencyMs":386.1447909999406},{"questionId":"q89","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"7","isCorrect":false,"inputTokens":1665,"outputTokens":1,"latencyMs":367.64695900003426},{"questionId":"q89","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"12","isCorrect":false,"inputTokens":4522,"outputTokens":1,"latencyMs":431.8806249999907},{"questionId":"q89","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"5","isCorrect":false,"inputTokens":3206,"outputTokens":1,"latencyMs":341.11941599997226},{"questionId":"q90","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"7","isCorrect":false,"inputTokens":3876,"outputTokens":1,"latencyMs":353.93445800000336},{"questionId":"q90","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"5","isCorrect":false,"inputTokens":2547,"outputTokens":1,"latencyMs":424.90420800005086},{"questionId":"q90","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"10","isCorrect":false,"inputTokens":1806,"outputTokens":1,"latencyMs":536.3856249999953},{"questionId":"q90","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"11","isCorrect":false,"inputTokens":1665,"outputTokens":1,"latencyMs":411.72025000001304},{"questionId":"q90","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"7","isCorrect":false,"inputTokens":4522,"outputTokens":1,"latencyMs":524.9461250000168},{"questionId":"q90","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"9","isCorrect":false,"inputTokens":3206,"outputTokens":1,"latencyMs":525.282542000059},{"questionId":"q91","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"20","actual":"4","isCorrect":false,"inputTokens":3876,"outputTokens":1,"latencyMs":525.9665839999216},{"questionId":"q91","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"20","actual":"0","isCorrect":false,"inputTokens":2547,"outputTokens":1,"latencyMs":460.46575000009034},{"questionId":"q91","format":"toon","model":"grok-4-fast-non-reasoning","expected":"20","actual":"12","isCorrect":false,"inputTokens":1806,"outputTokens":1,"latencyMs":515.202917000046},{"questionId":"q91","format":"csv","model":"grok-4-fast-non-reasoning","expected":"20","actual":"6","isCorrect":false,"inputTokens":1665,"outputTokens":1,"latencyMs":420.7105829999782},{"questionId":"q91","format":"xml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"2","isCorrect":false,"inputTokens":4522,"outputTokens":1,"latencyMs":506.29129099997226},{"questionId":"q91","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"20","actual":"0","isCorrect":false,"inputTokens":3206,"outputTokens":1,"latencyMs":331.3887920000125},{"questionId":"q92","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"32","actual":"13","isCorrect":false,"inputTokens":3875,"outputTokens":1,"latencyMs":358.76466600003187},{"questionId":"q92","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"32","actual":"12","isCorrect":false,"inputTokens":2546,"outputTokens":1,"latencyMs":354.3373750000028},{"questionId":"q92","format":"toon","model":"grok-4-fast-non-reasoning","expected":"32","actual":"12","isCorrect":false,"inputTokens":1805,"outputTokens":1,"latencyMs":352.88662500004284},{"questionId":"q92","format":"csv","model":"grok-4-fast-non-reasoning","expected":"32","actual":"16","isCorrect":false,"inputTokens":1664,"outputTokens":1,"latencyMs":328.9580830000341},{"questionId":"q92","format":"xml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"14","isCorrect":false,"inputTokens":4521,"outputTokens":1,"latencyMs":377.7550419999752},{"questionId":"q92","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"32","actual":"7","isCorrect":false,"inputTokens":3205,"outputTokens":1,"latencyMs":354.1078330000164},{"questionId":"q93","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"9","actual":"7","isCorrect":false,"inputTokens":3875,"outputTokens":1,"latencyMs":330.06762499990873},{"questionId":"q93","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"9","actual":"9","isCorrect":true,"inputTokens":2546,"outputTokens":1,"latencyMs":285.3993339999579},{"questionId":"q93","format":"toon","model":"grok-4-fast-non-reasoning","expected":"9","actual":"4","isCorrect":false,"inputTokens":1805,"outputTokens":1,"latencyMs":343.84237500000745},{"questionId":"q93","format":"csv","model":"grok-4-fast-non-reasoning","expected":"9","actual":"14","isCorrect":false,"inputTokens":1664,"outputTokens":1,"latencyMs":380.62529200001154},{"questionId":"q93","format":"xml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"5","isCorrect":false,"inputTokens":4521,"outputTokens":1,"latencyMs":370.2707919999957},{"questionId":"q93","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"9","actual":"5","isCorrect":false,"inputTokens":3205,"outputTokens":1,"latencyMs":360.8414580000099},{"questionId":"q94","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"27","isCorrect":false,"inputTokens":3878,"outputTokens":1,"latencyMs":389.52933299995493},{"questionId":"q94","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"26","isCorrect":false,"inputTokens":2549,"outputTokens":1,"latencyMs":333.31554199999664},{"questionId":"q94","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"26","isCorrect":false,"inputTokens":1808,"outputTokens":1,"latencyMs":335.77658399997745},{"questionId":"q94","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"28","isCorrect":false,"inputTokens":1667,"outputTokens":1,"latencyMs":404.9906660000561},{"questionId":"q94","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"26","isCorrect":false,"inputTokens":4524,"outputTokens":1,"latencyMs":393.1024579999503},{"questionId":"q94","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":3208,"outputTokens":1,"latencyMs":414.0203329999931},{"questionId":"q95","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"22","actual":"29","isCorrect":false,"inputTokens":3878,"outputTokens":1,"latencyMs":402.82508300000336},{"questionId":"q95","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"22","actual":"22","isCorrect":true,"inputTokens":2549,"outputTokens":1,"latencyMs":348.07883299991954},{"questionId":"q95","format":"toon","model":"grok-4-fast-non-reasoning","expected":"22","actual":"23","isCorrect":false,"inputTokens":1808,"outputTokens":1,"latencyMs":341.1314579999307},{"questionId":"q95","format":"csv","model":"grok-4-fast-non-reasoning","expected":"22","actual":"27","isCorrect":false,"inputTokens":1667,"outputTokens":1,"latencyMs":293.56583300000057},{"questionId":"q95","format":"xml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"25","isCorrect":false,"inputTokens":4524,"outputTokens":1,"latencyMs":351.24708400003146},{"questionId":"q95","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"22","actual":"28","isCorrect":false,"inputTokens":3208,"outputTokens":1,"latencyMs":369.76858400006313},{"questionId":"q96","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":15289,"outputTokens":2,"latencyMs":847.0936660000589},{"questionId":"q96","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":11593,"outputTokens":2,"latencyMs":748.6298749999842},{"questionId":"q96","format":"toon","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8916,"outputTokens":2,"latencyMs":675.3816670000087},{"questionId":"q96","format":"csv","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":8666,"outputTokens":2,"latencyMs":662.4207910000114},{"questionId":"q96","format":"xml","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":17201,"outputTokens":2,"latencyMs":1137.6606669999892},{"questionId":"q96","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"430886","actual":"430886","isCorrect":true,"inputTokens":13287,"outputTokens":2,"latencyMs":718.5493749999441},{"questionId":"q97","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":15290,"outputTokens":2,"latencyMs":812.2199580000015},{"questionId":"q97","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":11594,"outputTokens":2,"latencyMs":737.6186250000028},{"questionId":"q97","format":"toon","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8917,"outputTokens":2,"latencyMs":559.4110420000507},{"questionId":"q97","format":"csv","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":8667,"outputTokens":2,"latencyMs":544.999375000014},{"questionId":"q97","format":"xml","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":17202,"outputTokens":2,"latencyMs":1340.8652089999523},{"questionId":"q97","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"52904","actual":"52904","isCorrect":true,"inputTokens":13288,"outputTokens":2,"latencyMs":568.4742499999702},{"questionId":"q98","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":15286,"outputTokens":2,"latencyMs":969.5441250000149},{"questionId":"q98","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":11590,"outputTokens":2,"latencyMs":464.76391700003296},{"questionId":"q98","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8913,"outputTokens":2,"latencyMs":606.4126669999678},{"questionId":"q98","format":"csv","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":8663,"outputTokens":2,"latencyMs":745.9538750000065},{"questionId":"q98","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":17198,"outputTokens":2,"latencyMs":616.8759169999976},{"questionId":"q98","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5786","actual":"5786","isCorrect":true,"inputTokens":13284,"outputTokens":2,"latencyMs":775.7144999998854},{"questionId":"q99","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":15291,"outputTokens":1,"latencyMs":799.9245830000145},{"questionId":"q99","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":11595,"outputTokens":1,"latencyMs":700.218583000009},{"questionId":"q99","format":"toon","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":8918,"outputTokens":1,"latencyMs":821.7689170000376},{"questionId":"q99","format":"csv","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":8668,"outputTokens":1,"latencyMs":928.4695000000065},{"questionId":"q99","format":"xml","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":17203,"outputTokens":1,"latencyMs":883.7721669999883},{"questionId":"q99","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"master","actual":"master","isCorrect":true,"inputTokens":13289,"outputTokens":1,"latencyMs":808.017583000008},{"questionId":"q100","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":15286,"outputTokens":2,"latencyMs":966.1286669999827},{"questionId":"q100","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":11590,"outputTokens":2,"latencyMs":751.3571249999804},{"questionId":"q100","format":"toon","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8913,"outputTokens":2,"latencyMs":654.5150419999845},{"questionId":"q100","format":"csv","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":8663,"outputTokens":2,"latencyMs":563.3342089999933},{"questionId":"q100","format":"xml","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":17198,"outputTokens":2,"latencyMs":814.0509580000071},{"questionId":"q100","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"170327","actual":"170327","isCorrect":true,"inputTokens":13284,"outputTokens":2,"latencyMs":569.5104580000043},{"questionId":"q101","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":15290,"outputTokens":2,"latencyMs":752.2417080000741},{"questionId":"q101","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":11594,"outputTokens":2,"latencyMs":660.2026659999974},{"questionId":"q101","format":"toon","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8917,"outputTokens":2,"latencyMs":534.4836249999935},{"questionId":"q101","format":"csv","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":8667,"outputTokens":2,"latencyMs":684.0829170000507},{"questionId":"q101","format":"xml","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":17202,"outputTokens":2,"latencyMs":864.4797089999774},{"questionId":"q101","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"48578","actual":"48578","isCorrect":true,"inputTokens":13288,"outputTokens":2,"latencyMs":691.7703330001095},{"questionId":"q102","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":15290,"outputTokens":1,"latencyMs":1107.4861670000246},{"questionId":"q102","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":11594,"outputTokens":1,"latencyMs":527.8589159999974},{"questionId":"q102","format":"toon","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":8917,"outputTokens":1,"latencyMs":521.4823750000214},{"questionId":"q102","format":"csv","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":8667,"outputTokens":1,"latencyMs":710.9968329999829},{"questionId":"q102","format":"xml","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":17202,"outputTokens":1,"latencyMs":796.5441659999778},{"questionId":"q102","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"678","actual":"678","isCorrect":true,"inputTokens":13288,"outputTokens":1,"latencyMs":681.2811670000665},{"questionId":"q103","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":15289,"outputTokens":1,"latencyMs":719.1799590000883},{"questionId":"q103","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":11593,"outputTokens":1,"latencyMs":547.0494160000235},{"questionId":"q103","format":"toon","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":8916,"outputTokens":1,"latencyMs":603.1627500000177},{"questionId":"q103","format":"csv","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":8666,"outputTokens":1,"latencyMs":387.4932920000283},{"questionId":"q103","format":"xml","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":17201,"outputTokens":1,"latencyMs":751.547084000078},{"questionId":"q103","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"main","actual":"main","isCorrect":true,"inputTokens":13287,"outputTokens":1,"latencyMs":394.68229100003373},{"questionId":"q104","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":15292,"outputTokens":2,"latencyMs":635.7552080000751},{"questionId":"q104","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":11596,"outputTokens":2,"latencyMs":592.325458999956},{"questionId":"q104","format":"toon","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8919,"outputTokens":2,"latencyMs":627.3511660000077},{"questionId":"q104","format":"csv","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":8669,"outputTokens":2,"latencyMs":536.515625},{"questionId":"q104","format":"xml","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":17204,"outputTokens":2,"latencyMs":1014.4810000000289},{"questionId":"q104","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"115543","actual":"115543","isCorrect":true,"inputTokens":13290,"outputTokens":2,"latencyMs":622.3782499999506},{"questionId":"q105","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":15289,"outputTokens":2,"latencyMs":511.5717079999158},{"questionId":"q105","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":11593,"outputTokens":2,"latencyMs":651.1169579999987},{"questionId":"q105","format":"toon","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8916,"outputTokens":2,"latencyMs":565.2272080000257},{"questionId":"q105","format":"csv","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":8666,"outputTokens":2,"latencyMs":629.9112079999177},{"questionId":"q105","format":"xml","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":17201,"outputTokens":2,"latencyMs":792.8599590000231},{"questionId":"q105","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"36054","actual":"36054","isCorrect":true,"inputTokens":13287,"outputTokens":2,"latencyMs":450.7830829999875},{"questionId":"q106","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":15293,"outputTokens":2,"latencyMs":458.81408300006296},{"questionId":"q106","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":11597,"outputTokens":2,"latencyMs":601.6344579999568},{"questionId":"q106","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8920,"outputTokens":2,"latencyMs":542.3967089999933},{"questionId":"q106","format":"csv","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":8670,"outputTokens":2,"latencyMs":731.8346250000177},{"questionId":"q106","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":17205,"outputTokens":2,"latencyMs":789.3887499999255},{"questionId":"q106","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2607","actual":"2607","isCorrect":true,"inputTokens":13291,"outputTokens":2,"latencyMs":888.7570000000997},{"questionId":"q107","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":15285,"outputTokens":1,"latencyMs":998.5989999999292},{"questionId":"q107","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":11589,"outputTokens":1,"latencyMs":3462.4121250000317},{"questionId":"q107","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8912,"outputTokens":1,"latencyMs":738.1448749999981},{"questionId":"q107","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8662,"outputTokens":1,"latencyMs":466.5692910000216},{"questionId":"q107","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"65","isCorrect":false,"inputTokens":17197,"outputTokens":1,"latencyMs":606.8866670000134},{"questionId":"q107","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":13283,"outputTokens":1,"latencyMs":761.7227079999866},{"questionId":"q108","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"12729702","isCorrect":false,"inputTokens":15288,"outputTokens":3,"latencyMs":669.0338749999646},{"questionId":"q108","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"15310433","isCorrect":false,"inputTokens":11592,"outputTokens":3,"latencyMs":686.4420000000391},{"questionId":"q108","format":"toon","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"15016841","isCorrect":false,"inputTokens":8915,"outputTokens":3,"latencyMs":639.5922080000164},{"questionId":"q108","format":"csv","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"11375203","isCorrect":false,"inputTokens":8665,"outputTokens":3,"latencyMs":594.8222090000054},{"questionId":"q108","format":"xml","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"16696975","isCorrect":false,"inputTokens":17200,"outputTokens":3,"latencyMs":763.4211660000728},{"questionId":"q108","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"15413563","actual":"11085479","isCorrect":false,"inputTokens":13286,"outputTokens":3,"latencyMs":780.7073340000352},{"questionId":"q109","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1744064","isCorrect":false,"inputTokens":15288,"outputTokens":3,"latencyMs":517.7299580000108},{"questionId":"q109","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1680536","isCorrect":false,"inputTokens":11592,"outputTokens":3,"latencyMs":570.5709589999169},{"questionId":"q109","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"2158529","isCorrect":false,"inputTokens":8915,"outputTokens":3,"latencyMs":575.1755839999532},{"questionId":"q109","format":"csv","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1390474","isCorrect":false,"inputTokens":8665,"outputTokens":3,"latencyMs":635.8244160000468},{"questionId":"q109","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1551928","isCorrect":false,"inputTokens":17200,"outputTokens":3,"latencyMs":488.55670800001826},{"questionId":"q109","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2528243","actual":"1498307","isCorrect":false,"inputTokens":13286,"outputTokens":3,"latencyMs":834.4468329999363},{"questionId":"q110","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"150608","isCorrect":false,"inputTokens":15287,"outputTokens":2,"latencyMs":818.7738339999923},{"questionId":"q110","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"147984","isCorrect":false,"inputTokens":11591,"outputTokens":2,"latencyMs":830.1263749999925},{"questionId":"q110","format":"toon","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"132950\n\n## Explanation\n\nTo determine the average number of stars per repository from the provided TOON format data, follow these steps:\n\n1. Parse the TOON structure to identify the 100 repositories listed under the `repositories[100]` array, where each row contains 11 comma-separated fields corresponding to: id, name, repo, description, createdAt, updatedAt, pushedAt, stars, watchers, forks, defaultBranch.\n\n2. Extract the stars value from the 8th field of each row. For example, from the first row: \"28457823,freeCodeCamp,freeCodeCamp/freeCodeCamp,\"freeCodeCamp.org's open-source codebase and curriculum. Learn math, programming, and computer science for free.\",\"2014-12-24T17:49:19Z\",\"2025-10-28T11:58:08Z\",\"2025-10-28T10:17:16Z\",430886,8583,42146,main\", the stars value is 430886.\n\n3. Collect all 100 stars values from the rows and sum them to obtain the total number of stars across all repositories.\n\n4. Divide the total sum by 100 (the number of repositories) to calculate the average: average = total stars / 100.\n\nThis process yields the average value of 132950 stars per repository.","isCorrect":false,"inputTokens":8914,"outputTokens":278,"latencyMs":1617.6461249999702},{"questionId":"q110","format":"csv","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"134708","isCorrect":false,"inputTokens":8664,"outputTokens":2,"latencyMs":544.1772499999497},{"questionId":"q110","format":"xml","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"132198","isCorrect":false,"inputTokens":17199,"outputTokens":2,"latencyMs":501.39316599990707},{"questionId":"q110","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"154136","actual":"123269","isCorrect":false,"inputTokens":13285,"outputTokens":2,"latencyMs":543.3348329999717},{"questionId":"q111","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"41","actual":"36","isCorrect":false,"inputTokens":15289,"outputTokens":1,"latencyMs":2023.9102499999572},{"questionId":"q111","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"41","actual":"26","isCorrect":false,"inputTokens":11593,"outputTokens":1,"latencyMs":446.861583000049},{"questionId":"q111","format":"toon","model":"grok-4-fast-non-reasoning","expected":"41","actual":"22","isCorrect":false,"inputTokens":8916,"outputTokens":1,"latencyMs":626.6710409999359},{"questionId":"q111","format":"csv","model":"grok-4-fast-non-reasoning","expected":"41","actual":"22","isCorrect":false,"inputTokens":8666,"outputTokens":1,"latencyMs":363.78724999993574},{"questionId":"q111","format":"xml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"30","isCorrect":false,"inputTokens":17201,"outputTokens":1,"latencyMs":888.8954999999842},{"questionId":"q111","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"41","actual":"24","isCorrect":false,"inputTokens":13287,"outputTokens":1,"latencyMs":503.1479999999283},{"questionId":"q112","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"53","actual":"30","isCorrect":false,"inputTokens":15289,"outputTokens":1,"latencyMs":362.8952079999726},{"questionId":"q112","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"53","actual":"34","isCorrect":false,"inputTokens":11593,"outputTokens":1,"latencyMs":414.893041000003},{"questionId":"q112","format":"toon","model":"grok-4-fast-non-reasoning","expected":"53","actual":"38","isCorrect":false,"inputTokens":8916,"outputTokens":1,"latencyMs":591.1873749999795},{"questionId":"q112","format":"csv","model":"grok-4-fast-non-reasoning","expected":"53","actual":"34","isCorrect":false,"inputTokens":8666,"outputTokens":1,"latencyMs":446.97804100008216},{"questionId":"q112","format":"xml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"35","isCorrect":false,"inputTokens":17201,"outputTokens":1,"latencyMs":389.6877089999616},{"questionId":"q112","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"53","actual":"41","isCorrect":false,"inputTokens":13287,"outputTokens":1,"latencyMs":496.26795799995307},{"questionId":"q113","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"77","actual":"70","isCorrect":false,"inputTokens":15288,"outputTokens":1,"latencyMs":512.705333000049},{"questionId":"q113","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"77","actual":"68","isCorrect":false,"inputTokens":11592,"outputTokens":1,"latencyMs":683.4942919999594},{"questionId":"q113","format":"toon","model":"grok-4-fast-non-reasoning","expected":"77","actual":"64","isCorrect":false,"inputTokens":8915,"outputTokens":1,"latencyMs":391.2319579999894},{"questionId":"q113","format":"csv","model":"grok-4-fast-non-reasoning","expected":"77","actual":"63","isCorrect":false,"inputTokens":8665,"outputTokens":1,"latencyMs":520.3853750000708},{"questionId":"q113","format":"xml","model":"grok-4-fast-non-reasoning","expected":"77","actual":"55","isCorrect":false,"inputTokens":17200,"outputTokens":1,"latencyMs":398.64429099997506},{"questionId":"q113","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"77","actual":"67","isCorrect":false,"inputTokens":13286,"outputTokens":1,"latencyMs":545.1215410000877},{"questionId":"q114","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":15288,"outputTokens":1,"latencyMs":620.1757080000825},{"questionId":"q114","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"37","actual":"34","isCorrect":false,"inputTokens":11592,"outputTokens":1,"latencyMs":409.1830000000773},{"questionId":"q114","format":"toon","model":"grok-4-fast-non-reasoning","expected":"37","actual":"36","isCorrect":false,"inputTokens":8915,"outputTokens":1,"latencyMs":495.18062500003725},{"questionId":"q114","format":"csv","model":"grok-4-fast-non-reasoning","expected":"37","actual":"25","isCorrect":false,"inputTokens":8665,"outputTokens":1,"latencyMs":330.13295800006017},{"questionId":"q114","format":"xml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"28","isCorrect":false,"inputTokens":17200,"outputTokens":1,"latencyMs":801.918791999924},{"questionId":"q114","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"25","isCorrect":false,"inputTokens":13286,"outputTokens":1,"latencyMs":523.8586249999935},{"questionId":"q115","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"16","actual":"27","isCorrect":false,"inputTokens":15288,"outputTokens":1,"latencyMs":675.9268330000341},{"questionId":"q115","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"16","actual":"17","isCorrect":false,"inputTokens":11592,"outputTokens":1,"latencyMs":777.2151669999585},{"questionId":"q115","format":"toon","model":"grok-4-fast-non-reasoning","expected":"16","actual":"25","isCorrect":false,"inputTokens":8915,"outputTokens":1,"latencyMs":659.3119160000933},{"questionId":"q115","format":"csv","model":"grok-4-fast-non-reasoning","expected":"16","actual":"25","isCorrect":false,"inputTokens":8665,"outputTokens":1,"latencyMs":599.7143750000978},{"questionId":"q115","format":"xml","model":"grok-4-fast-non-reasoning","expected":"16","actual":"12","isCorrect":false,"inputTokens":17200,"outputTokens":1,"latencyMs":968.8130409999285},{"questionId":"q115","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"16","actual":"12","isCorrect":false,"inputTokens":13286,"outputTokens":1,"latencyMs":677.8669579999987},{"questionId":"q116","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"49","actual":"61","isCorrect":false,"inputTokens":15288,"outputTokens":1,"latencyMs":541.5242500000168},{"questionId":"q116","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"49","actual":"64","isCorrect":false,"inputTokens":11592,"outputTokens":1,"latencyMs":819.5389159999322},{"questionId":"q116","format":"toon","model":"grok-4-fast-non-reasoning","expected":"49","actual":"83","isCorrect":false,"inputTokens":8915,"outputTokens":1,"latencyMs":578.8390829999698},{"questionId":"q116","format":"csv","model":"grok-4-fast-non-reasoning","expected":"49","actual":"52","isCorrect":false,"inputTokens":8665,"outputTokens":1,"latencyMs":516.0447500000009},{"questionId":"q116","format":"xml","model":"grok-4-fast-non-reasoning","expected":"49","actual":"52","isCorrect":false,"inputTokens":17200,"outputTokens":1,"latencyMs":1034.5697500000242},{"questionId":"q116","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"49","actual":"51","isCorrect":false,"inputTokens":13286,"outputTokens":1,"latencyMs":752.8487500000047},{"questionId":"q117","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"23","actual":"20","isCorrect":false,"inputTokens":15288,"outputTokens":1,"latencyMs":668.7280419999734},{"questionId":"q117","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"23","actual":"20","isCorrect":false,"inputTokens":11592,"outputTokens":1,"latencyMs":463.05341699998826},{"questionId":"q117","format":"toon","model":"grok-4-fast-non-reasoning","expected":"23","actual":"49","isCorrect":false,"inputTokens":8915,"outputTokens":1,"latencyMs":330.8354999999283},{"questionId":"q117","format":"csv","model":"grok-4-fast-non-reasoning","expected":"23","actual":"37","isCorrect":false,"inputTokens":8665,"outputTokens":1,"latencyMs":540.2028329999885},{"questionId":"q117","format":"xml","model":"grok-4-fast-non-reasoning","expected":"23","actual":"31","isCorrect":false,"inputTokens":17200,"outputTokens":1,"latencyMs":482.2025830000639},{"questionId":"q117","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"23","actual":"24","isCorrect":false,"inputTokens":13286,"outputTokens":1,"latencyMs":730.5656660000095},{"questionId":"q118","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"1","isCorrect":false,"inputTokens":15288,"outputTokens":1,"latencyMs":665.2084590001032},{"questionId":"q118","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"1","isCorrect":false,"inputTokens":11592,"outputTokens":1,"latencyMs":409.54495799995493},{"questionId":"q118","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"1","isCorrect":false,"inputTokens":8915,"outputTokens":1,"latencyMs":362.7266669999808},{"questionId":"q118","format":"csv","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":8665,"outputTokens":1,"latencyMs":369.38187500007916},{"questionId":"q118","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"2","isCorrect":false,"inputTokens":17200,"outputTokens":1,"latencyMs":500.8102080000099},{"questionId":"q118","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":13286,"outputTokens":1,"latencyMs":456.69229199993424},{"questionId":"q119","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"57","actual":"69","isCorrect":false,"inputTokens":15295,"outputTokens":1,"latencyMs":972.7103750000242},{"questionId":"q119","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"57","actual":"34","isCorrect":false,"inputTokens":11599,"outputTokens":1,"latencyMs":748.0982080000686},{"questionId":"q119","format":"toon","model":"grok-4-fast-non-reasoning","expected":"57","actual":"21","isCorrect":false,"inputTokens":8922,"outputTokens":1,"latencyMs":614.5500839999877},{"questionId":"q119","format":"csv","model":"grok-4-fast-non-reasoning","expected":"57","actual":"21","isCorrect":false,"inputTokens":8672,"outputTokens":1,"latencyMs":364.6937910000561},{"questionId":"q119","format":"xml","model":"grok-4-fast-non-reasoning","expected":"57","actual":"42","isCorrect":false,"inputTokens":17207,"outputTokens":1,"latencyMs":440.97012500010896},{"questionId":"q119","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"57","actual":"18","isCorrect":false,"inputTokens":13293,"outputTokens":1,"latencyMs":702.6997909999918},{"questionId":"q120","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"43","actual":"27","isCorrect":false,"inputTokens":15295,"outputTokens":1,"latencyMs":727.5445410000393},{"questionId":"q120","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"43","actual":"19","isCorrect":false,"inputTokens":11599,"outputTokens":1,"latencyMs":468.4102089999942},{"questionId":"q120","format":"toon","model":"grok-4-fast-non-reasoning","expected":"43","actual":"8","isCorrect":false,"inputTokens":8922,"outputTokens":1,"latencyMs":492.1051669999724},{"questionId":"q120","format":"csv","model":"grok-4-fast-non-reasoning","expected":"43","actual":"12","isCorrect":false,"inputTokens":8672,"outputTokens":1,"latencyMs":377.68358299997635},{"questionId":"q120","format":"xml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"27","isCorrect":false,"inputTokens":17207,"outputTokens":1,"latencyMs":551.9322920000413},{"questionId":"q120","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"43","actual":"29","isCorrect":false,"inputTokens":13293,"outputTokens":1,"latencyMs":438.55529099993873},{"questionId":"q121","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"25","actual":"17","isCorrect":false,"inputTokens":15295,"outputTokens":1,"latencyMs":350.24887500004843},{"questionId":"q121","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"25","actual":"10","isCorrect":false,"inputTokens":11599,"outputTokens":1,"latencyMs":883.0131250000559},{"questionId":"q121","format":"toon","model":"grok-4-fast-non-reasoning","expected":"25","actual":"5","isCorrect":false,"inputTokens":8922,"outputTokens":1,"latencyMs":454.13141599996015},{"questionId":"q121","format":"csv","model":"grok-4-fast-non-reasoning","expected":"25","actual":"7","isCorrect":false,"inputTokens":8672,"outputTokens":1,"latencyMs":390.6537079999689},{"questionId":"q121","format":"xml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"16","isCorrect":false,"inputTokens":17207,"outputTokens":1,"latencyMs":404.33587499998976},{"questionId":"q121","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"6","isCorrect":false,"inputTokens":13293,"outputTokens":1,"latencyMs":716.4604589999653},{"questionId":"q122","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6","actual":"2","isCorrect":false,"inputTokens":15295,"outputTokens":1,"latencyMs":793.8803750000661},{"questionId":"q122","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":11599,"outputTokens":1,"latencyMs":527.7742500000168},{"questionId":"q122","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":8922,"outputTokens":1,"latencyMs":732.4944580000592},{"questionId":"q122","format":"csv","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":8672,"outputTokens":1,"latencyMs":600.1152920000022},{"questionId":"q122","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":17207,"outputTokens":1,"latencyMs":947.3650830000406},{"questionId":"q122","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"0","isCorrect":false,"inputTokens":13293,"outputTokens":1,"latencyMs":709.3704589999979},{"questionId":"q123","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"15","isCorrect":false,"inputTokens":15295,"outputTokens":1,"latencyMs":600.0849999999627},{"questionId":"q123","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"7","isCorrect":false,"inputTokens":11599,"outputTokens":1,"latencyMs":482.3578330000164},{"questionId":"q123","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":8922,"outputTokens":1,"latencyMs":527.549083000049},{"questionId":"q123","format":"csv","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":8672,"outputTokens":1,"latencyMs":496.53116599994246},{"questionId":"q123","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"4","isCorrect":false,"inputTokens":17207,"outputTokens":1,"latencyMs":672.2010829999344},{"questionId":"q123","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"3","isCorrect":false,"inputTokens":13293,"outputTokens":1,"latencyMs":562.8306670000311},{"questionId":"q124","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7056,"outputTokens":1,"latencyMs":499.89791599998716},{"questionId":"q124","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":4968,"outputTokens":1,"latencyMs":695.6328340000473},{"questionId":"q124","format":"toon","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6057,"outputTokens":1,"latencyMs":513.2947920000879},{"questionId":"q124","format":"xml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7966,"outputTokens":1,"latencyMs":679.0981250000186},{"questionId":"q124","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6083,"outputTokens":1,"latencyMs":526.7436669999734},{"questionId":"q125","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7056,"outputTokens":2,"latencyMs":799.5244999999413},{"questionId":"q125","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4968,"outputTokens":2,"latencyMs":873.7110839999514},{"questionId":"q125","format":"toon","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6057,"outputTokens":2,"latencyMs":393.2583330000052},{"questionId":"q125","format":"xml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7966,"outputTokens":2,"latencyMs":425.53195799991954},{"questionId":"q125","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6083,"outputTokens":2,"latencyMs":427.4291669999948},{"questionId":"q126","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":7057,"outputTokens":1,"latencyMs":398.9085419999901},{"questionId":"q126","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":4969,"outputTokens":1,"latencyMs":358.4536669999361},{"questionId":"q126","format":"toon","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":6058,"outputTokens":1,"latencyMs":505.2996249999851},{"questionId":"q126","format":"xml","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":7967,"outputTokens":1,"latencyMs":523.4929170000833},{"questionId":"q126","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"424","actual":"424","isCorrect":true,"inputTokens":6084,"outputTokens":1,"latencyMs":1304.0986659999471},{"questionId":"q127","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7057,"outputTokens":2,"latencyMs":420.2204159999965},{"questionId":"q127","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":4969,"outputTokens":2,"latencyMs":367.0855829999782},{"questionId":"q127","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6058,"outputTokens":2,"latencyMs":386.5196669999277},{"questionId":"q127","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":7967,"outputTokens":2,"latencyMs":456.76500000001397},{"questionId":"q127","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2849","actual":"2849","isCorrect":true,"inputTokens":6084,"outputTokens":2,"latencyMs":381.4293750000652},{"questionId":"q128","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":7056,"outputTokens":1,"latencyMs":455.50358299992513},{"questionId":"q128","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":4968,"outputTokens":1,"latencyMs":365.7436250000028},{"questionId":"q128","format":"toon","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6057,"outputTokens":1,"latencyMs":380.4576250000391},{"questionId":"q128","format":"xml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":7966,"outputTokens":1,"latencyMs":483.38354199996684},{"questionId":"q128","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6083,"outputTokens":1,"latencyMs":418.4449579999782},{"questionId":"q129","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7056,"outputTokens":2,"latencyMs":334.25179100001696},{"questionId":"q129","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":4968,"outputTokens":2,"latencyMs":486.24950000003446},{"questionId":"q129","format":"toon","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6057,"outputTokens":2,"latencyMs":391.2431670000078},{"questionId":"q129","format":"xml","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":7966,"outputTokens":2,"latencyMs":506.99512500001583},{"questionId":"q129","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"/api/orders","actual":"/api/orders","isCorrect":true,"inputTokens":6083,"outputTokens":2,"latencyMs":392.3894169999985},{"questionId":"q130","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":7057,"outputTokens":1,"latencyMs":417.3212090000743},{"questionId":"q130","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":4969,"outputTokens":1,"latencyMs":401.1411659999285},{"questionId":"q130","format":"toon","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":6058,"outputTokens":1,"latencyMs":428.8967080001021},{"questionId":"q130","format":"xml","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":7967,"outputTokens":1,"latencyMs":389.21916700003203},{"questionId":"q130","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"435","actual":"435","isCorrect":true,"inputTokens":6084,"outputTokens":1,"latencyMs":330.8220419998979},{"questionId":"q131","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":7057,"outputTokens":1,"latencyMs":427.2092499999562},{"questionId":"q131","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":4969,"outputTokens":1,"latencyMs":331.3513340000063},{"questionId":"q131","format":"toon","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":6058,"outputTokens":1,"latencyMs":423.1122500000056},{"questionId":"q131","format":"xml","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":7967,"outputTokens":1,"latencyMs":423.23325000004843},{"questionId":"q131","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"408","actual":"408","isCorrect":true,"inputTokens":6084,"outputTokens":1,"latencyMs":469.05550000001676},{"questionId":"q132","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7056,"outputTokens":1,"latencyMs":301.24354100006167},{"questionId":"q132","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":4968,"outputTokens":1,"latencyMs":352.43516700004693},{"questionId":"q132","format":"toon","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6057,"outputTokens":1,"latencyMs":418.82037500001024},{"questionId":"q132","format":"xml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":7966,"outputTokens":1,"latencyMs":587.2274169999873},{"questionId":"q132","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"error","actual":"error","isCorrect":true,"inputTokens":6083,"outputTokens":1,"latencyMs":489.2692500000121},{"questionId":"q133","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7056,"outputTokens":2,"latencyMs":446.89837499998976},{"questionId":"q133","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":4968,"outputTokens":2,"latencyMs":361.986333000008},{"questionId":"q133","format":"toon","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6057,"outputTokens":2,"latencyMs":728.2220830000006},{"questionId":"q133","format":"xml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":7966,"outputTokens":2,"latencyMs":474.3291250000475},{"questionId":"q133","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"/api/users","actual":"/api/users","isCorrect":true,"inputTokens":6083,"outputTokens":2,"latencyMs":506.57004100002814},{"questionId":"q134","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"75","actual":"70","isCorrect":false,"inputTokens":7040,"outputTokens":1,"latencyMs":775.6960839999374},{"questionId":"q134","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"75","actual":"70","isCorrect":false,"inputTokens":4952,"outputTokens":1,"latencyMs":482.1167500000447},{"questionId":"q134","format":"toon","model":"grok-4-fast-non-reasoning","expected":"75","actual":"75","isCorrect":true,"inputTokens":6041,"outputTokens":1,"latencyMs":496.28008300007787},{"questionId":"q134","format":"xml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"75","isCorrect":true,"inputTokens":7950,"outputTokens":1,"latencyMs":511.10854200005997},{"questionId":"q134","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"60","isCorrect":false,"inputTokens":6067,"outputTokens":1,"latencyMs":690.7556669999612},{"questionId":"q135","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2461","isCorrect":false,"inputTokens":7041,"outputTokens":2,"latencyMs":525.7608750000363},{"questionId":"q135","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2373","isCorrect":false,"inputTokens":4953,"outputTokens":2,"latencyMs":502.70295800000895},{"questionId":"q135","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2331","isCorrect":false,"inputTokens":6042,"outputTokens":2,"latencyMs":417.48266700003296},{"questionId":"q135","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2350","isCorrect":false,"inputTokens":7951,"outputTokens":2,"latencyMs":389.78395900002215},{"questionId":"q135","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2453.41","actual":"2331","isCorrect":false,"inputTokens":6068,"outputTokens":2,"latencyMs":517.8271250000689},{"questionId":"q136","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"29","actual":"25","isCorrect":false,"inputTokens":7040,"outputTokens":1,"latencyMs":354.50579199998174},{"questionId":"q136","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"29","actual":"26","isCorrect":false,"inputTokens":4952,"outputTokens":1,"latencyMs":796.0082080001011},{"questionId":"q136","format":"toon","model":"grok-4-fast-non-reasoning","expected":"29","actual":"30","isCorrect":false,"inputTokens":6041,"outputTokens":1,"latencyMs":496.27754199993797},{"questionId":"q136","format":"xml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"27","isCorrect":false,"inputTokens":7950,"outputTokens":1,"latencyMs":490.7741249999963},{"questionId":"q136","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"27","isCorrect":false,"inputTokens":6067,"outputTokens":1,"latencyMs":371.1354580000043},{"questionId":"q137","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"17","actual":"15","isCorrect":false,"inputTokens":7040,"outputTokens":1,"latencyMs":380.1274999999441},{"questionId":"q137","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"17","actual":"15","isCorrect":false,"inputTokens":4952,"outputTokens":1,"latencyMs":377.3597919999156},{"questionId":"q137","format":"toon","model":"grok-4-fast-non-reasoning","expected":"17","actual":"15","isCorrect":false,"inputTokens":6041,"outputTokens":1,"latencyMs":371.1974160000682},{"questionId":"q137","format":"xml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"15","isCorrect":false,"inputTokens":7950,"outputTokens":1,"latencyMs":568.4942499999888},{"questionId":"q137","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"17","actual":"15","isCorrect":false,"inputTokens":6067,"outputTokens":1,"latencyMs":350.1954159999732},{"questionId":"q138","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"29","actual":"23","isCorrect":false,"inputTokens":7040,"outputTokens":1,"latencyMs":504.30670800001826},{"questionId":"q138","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"29","actual":"20","isCorrect":false,"inputTokens":4952,"outputTokens":1,"latencyMs":410.077249999973},{"questionId":"q138","format":"toon","model":"grok-4-fast-non-reasoning","expected":"29","actual":"23","isCorrect":false,"inputTokens":6041,"outputTokens":1,"latencyMs":814.5463750000345},{"questionId":"q138","format":"xml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"22","isCorrect":false,"inputTokens":7950,"outputTokens":1,"latencyMs":614.0857090000063},{"questionId":"q138","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"23","isCorrect":false,"inputTokens":6067,"outputTokens":1,"latencyMs":386.987415999989},{"questionId":"q139","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"14","isCorrect":false,"inputTokens":7042,"outputTokens":1,"latencyMs":479.18416699999943},{"questionId":"q139","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":4954,"outputTokens":1,"latencyMs":619.4851250000065},{"questionId":"q139","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":6043,"outputTokens":1,"latencyMs":492.1032500000438},{"questionId":"q139","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"14","isCorrect":false,"inputTokens":7952,"outputTokens":1,"latencyMs":507.7768749999814},{"questionId":"q139","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"15","isCorrect":false,"inputTokens":6069,"outputTokens":1,"latencyMs":615.5493749999441},{"questionId":"q140","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"18","actual":"14","isCorrect":false,"inputTokens":7042,"outputTokens":1,"latencyMs":645.0529170000227},{"questionId":"q140","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"18","actual":"15","isCorrect":false,"inputTokens":4954,"outputTokens":1,"latencyMs":379.28791600000113},{"questionId":"q140","format":"toon","model":"grok-4-fast-non-reasoning","expected":"18","actual":"15","isCorrect":false,"inputTokens":6043,"outputTokens":1,"latencyMs":604.0427909999853},{"questionId":"q140","format":"xml","model":"grok-4-fast-non-reasoning","expected":"18","actual":"16","isCorrect":false,"inputTokens":7952,"outputTokens":1,"latencyMs":463.93437500006985},{"questionId":"q140","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"18","actual":"16","isCorrect":false,"inputTokens":6069,"outputTokens":1,"latencyMs":386.78283299994655},{"questionId":"q141","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"33","actual":"31","isCorrect":false,"inputTokens":7046,"outputTokens":1,"latencyMs":629.9649170000339},{"questionId":"q141","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"33","actual":"28","isCorrect":false,"inputTokens":4958,"outputTokens":1,"latencyMs":379.7108340000268},{"questionId":"q141","format":"toon","model":"grok-4-fast-non-reasoning","expected":"33","actual":"35","isCorrect":false,"inputTokens":6047,"outputTokens":1,"latencyMs":418.81495799997356},{"questionId":"q141","format":"xml","model":"grok-4-fast-non-reasoning","expected":"33","actual":"30","isCorrect":false,"inputTokens":7956,"outputTokens":1,"latencyMs":522.9384579999605},{"questionId":"q141","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"33","actual":"35","isCorrect":false,"inputTokens":6073,"outputTokens":1,"latencyMs":460.40649999992456},{"questionId":"q142","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"42","actual":"25","isCorrect":false,"inputTokens":7045,"outputTokens":1,"latencyMs":448.83091600006446},{"questionId":"q142","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"42","actual":"20","isCorrect":false,"inputTokens":4957,"outputTokens":1,"latencyMs":407.4178330000723},{"questionId":"q142","format":"toon","model":"grok-4-fast-non-reasoning","expected":"42","actual":"37","isCorrect":false,"inputTokens":6046,"outputTokens":1,"latencyMs":361.24554199993145},{"questionId":"q142","format":"xml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"29","isCorrect":false,"inputTokens":7955,"outputTokens":1,"latencyMs":439.24908300000243},{"questionId":"q142","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"42","actual":"38","isCorrect":false,"inputTokens":6072,"outputTokens":1,"latencyMs":385.27162500005215},{"questionId":"q143","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"25","actual":"25","isCorrect":true,"inputTokens":7041,"outputTokens":1,"latencyMs":450.4474169999594},{"questionId":"q143","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"25","actual":"27","isCorrect":false,"inputTokens":4953,"outputTokens":1,"latencyMs":591.9225829999195},{"questionId":"q143","format":"toon","model":"grok-4-fast-non-reasoning","expected":"25","actual":"31","isCorrect":false,"inputTokens":6042,"outputTokens":1,"latencyMs":485.4956249999814},{"questionId":"q143","format":"xml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"28","isCorrect":false,"inputTokens":7951,"outputTokens":1,"latencyMs":534.4823750000214},{"questionId":"q143","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"25","actual":"29","isCorrect":false,"inputTokens":6068,"outputTokens":1,"latencyMs":540.5629999999655},{"questionId":"q144","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"29","actual":"23","isCorrect":false,"inputTokens":7047,"outputTokens":1,"latencyMs":738.4935830000322},{"questionId":"q144","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"29","actual":"26","isCorrect":false,"inputTokens":4959,"outputTokens":1,"latencyMs":507.7947910000803},{"questionId":"q144","format":"toon","model":"grok-4-fast-non-reasoning","expected":"29","actual":"28","isCorrect":false,"inputTokens":6048,"outputTokens":1,"latencyMs":559.8169589999598},{"questionId":"q144","format":"xml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"25","isCorrect":false,"inputTokens":7957,"outputTokens":1,"latencyMs":517.6149160000496},{"questionId":"q144","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"29","actual":"27","isCorrect":false,"inputTokens":6074,"outputTokens":1,"latencyMs":530.2699169999687},{"questionId":"q145","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"3","isCorrect":false,"inputTokens":7047,"outputTokens":1,"latencyMs":538.6290830000071},{"questionId":"q145","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":4959,"outputTokens":1,"latencyMs":437.4593749999767},{"questionId":"q145","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"2","isCorrect":false,"inputTokens":6048,"outputTokens":1,"latencyMs":427.41479199996684},{"questionId":"q145","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"2","isCorrect":false,"inputTokens":7957,"outputTokens":1,"latencyMs":449.54045900003985},{"questionId":"q145","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"0","isCorrect":false,"inputTokens":6074,"outputTokens":1,"latencyMs":484.6289589999942},{"questionId":"q146","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":7049,"outputTokens":1,"latencyMs":486.32387500000186},{"questionId":"q146","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":4961,"outputTokens":1,"latencyMs":611.7120000000577},{"questionId":"q146","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":6050,"outputTokens":1,"latencyMs":411.10725000000093},{"questionId":"q146","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":7959,"outputTokens":1,"latencyMs":752.7729170001112},{"questionId":"q146","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"4","isCorrect":false,"inputTokens":6076,"outputTokens":1,"latencyMs":409.4608340000268},{"questionId":"q147","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"4","isCorrect":false,"inputTokens":7049,"outputTokens":1,"latencyMs":449.0540409999667},{"questionId":"q147","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"5","isCorrect":false,"inputTokens":4961,"outputTokens":1,"latencyMs":422.24112499994226},{"questionId":"q147","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"5","isCorrect":false,"inputTokens":6050,"outputTokens":1,"latencyMs":367.6763749999227},{"questionId":"q147","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"5","isCorrect":false,"inputTokens":7959,"outputTokens":1,"latencyMs":354.9297080000397},{"questionId":"q147","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"5","isCorrect":false,"inputTokens":6076,"outputTokens":1,"latencyMs":365.38904199993704},{"questionId":"q148","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":7049,"outputTokens":1,"latencyMs":368.60470899997745},{"questionId":"q148","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"1","isCorrect":false,"inputTokens":4961,"outputTokens":1,"latencyMs":342.8522499999963},{"questionId":"q148","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":6050,"outputTokens":1,"latencyMs":398.5957909999415},{"questionId":"q148","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"2","isCorrect":false,"inputTokens":7959,"outputTokens":1,"latencyMs":335.1882500000065},{"questionId":"q148","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"1","isCorrect":false,"inputTokens":6076,"outputTokens":1,"latencyMs":409.61954099999275},{"questionId":"q149","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"6","isCorrect":false,"inputTokens":7047,"outputTokens":1,"latencyMs":360.9672080000164},{"questionId":"q149","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"6","isCorrect":false,"inputTokens":4959,"outputTokens":1,"latencyMs":378.90674999996554},{"questionId":"q149","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"7","isCorrect":false,"inputTokens":6048,"outputTokens":1,"latencyMs":411.237208000035},{"questionId":"q149","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"6","isCorrect":false,"inputTokens":7957,"outputTokens":1,"latencyMs":560.2695829999866},{"questionId":"q149","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"6","isCorrect":false,"inputTokens":6074,"outputTokens":1,"latencyMs":584.5702909999527},{"questionId":"q150","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"3","isCorrect":false,"inputTokens":7047,"outputTokens":1,"latencyMs":516.8381250000093},{"questionId":"q150","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"4","isCorrect":false,"inputTokens":4959,"outputTokens":1,"latencyMs":497.01358299993444},{"questionId":"q150","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":6048,"outputTokens":1,"latencyMs":576.0206669999752},{"questionId":"q150","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"3","isCorrect":false,"inputTokens":7957,"outputTokens":1,"latencyMs":498.7420840000268},{"questionId":"q150","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":6074,"outputTokens":1,"latencyMs":378.8192910000216},{"questionId":"q151","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":1173,"outputTokens":1,"latencyMs":411.8597080000909},{"questionId":"q151","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":795,"outputTokens":1,"latencyMs":435.8964579999447},{"questionId":"q151","format":"toon","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":907,"outputTokens":1,"latencyMs":423.7199580000015},{"questionId":"q151","format":"xml","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":369.3998750000028},{"questionId":"q151","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"development","actual":"development","isCorrect":true,"inputTokens":925,"outputTokens":1,"latencyMs":319.687292000046},{"questionId":"q152","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1171,"outputTokens":5,"latencyMs":364.76491699996404},{"questionId":"q152","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":793,"outputTokens":5,"latencyMs":352.0045420000097},{"questionId":"q152","format":"toon","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":905,"outputTokens":5,"latencyMs":372.09104099997785},{"questionId":"q152","format":"xml","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":1243,"outputTokens":5,"latencyMs":431.41016699990723},{"questionId":"q152","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"guilty-cake.org","actual":"guilty-cake.org","isCorrect":true,"inputTokens":923,"outputTokens":5,"latencyMs":342.2865420000162},{"questionId":"q153","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1171,"outputTokens":2,"latencyMs":359.7161250000354},{"questionId":"q153","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":793,"outputTokens":2,"latencyMs":361.297082999954},{"questionId":"q153","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":905,"outputTokens":2,"latencyMs":2558.1280419999966},{"questionId":"q153","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":1243,"outputTokens":2,"latencyMs":348.4948340000119},{"questionId":"q153","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5432","actual":"5432","isCorrect":true,"inputTokens":923,"outputTokens":2,"latencyMs":440.40754200005904},{"questionId":"q154","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":1173,"outputTokens":1,"latencyMs":442.24454099999275},{"questionId":"q154","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":795,"outputTokens":1,"latencyMs":498.98729199997615},{"questionId":"q154","format":"toon","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":907,"outputTokens":1,"latencyMs":382.7210409999825},{"questionId":"q154","format":"xml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":426.35337500006426},{"questionId":"q154","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"37","actual":"37","isCorrect":true,"inputTokens":925,"outputTokens":1,"latencyMs":515.6713329999475},{"questionId":"q155","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1171,"outputTokens":2,"latencyMs":938.0727089999709},{"questionId":"q155","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":793,"outputTokens":2,"latencyMs":417.35700000007637},{"questionId":"q155","format":"toon","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":905,"outputTokens":2,"latencyMs":357.1302920000162},{"questionId":"q155","format":"xml","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":1243,"outputTokens":2,"latencyMs":620.5253750000848},{"questionId":"q155","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"86400","actual":"86400","isCorrect":true,"inputTokens":923,"outputTokens":2,"latencyMs":335.355833999929},{"questionId":"q156","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1173,"outputTokens":1,"latencyMs":355.8040419999743},{"questionId":"q156","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":795,"outputTokens":1,"latencyMs":445.3875420000404},{"questionId":"q156","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":907,"outputTokens":1,"latencyMs":407.4320000000298},{"questionId":"q156","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":329.8307499999646},{"questionId":"q156","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":925,"outputTokens":1,"latencyMs":363.26083299994934},{"questionId":"q157","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1173,"outputTokens":2,"latencyMs":310.97133299999405},{"questionId":"q157","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":795,"outputTokens":2,"latencyMs":356.66599999996834},{"questionId":"q157","format":"toon","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":907,"outputTokens":2,"latencyMs":312.03329200006556},{"questionId":"q157","format":"xml","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":1245,"outputTokens":2,"latencyMs":349.43658300000243},{"questionId":"q157","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"30000","actual":"30000","isCorrect":true,"inputTokens":925,"outputTokens":2,"latencyMs":358.5141250001034},{"questionId":"q158","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":1171,"outputTokens":1,"latencyMs":383.43216600001324},{"questionId":"q158","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":793,"outputTokens":1,"latencyMs":389.2131250000093},{"questionId":"q158","format":"toon","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":905,"outputTokens":1,"latencyMs":304.1596669999417},{"questionId":"q158","format":"xml","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":1243,"outputTokens":1,"latencyMs":361.62308299995493},{"questionId":"q158","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"real","actual":"real","isCorrect":true,"inputTokens":923,"outputTokens":1,"latencyMs":440.9662500000559},{"questionId":"q159","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1172,"outputTokens":2,"latencyMs":279.5552499999758},{"questionId":"q159","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":794,"outputTokens":2,"latencyMs":419.0840420000022},{"questionId":"q159","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":906,"outputTokens":2,"latencyMs":280.09262500004843},{"questionId":"q159","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":1244,"outputTokens":2,"latencyMs":363.02591600001324},{"questionId":"q159","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3600","actual":"3600","isCorrect":true,"inputTokens":924,"outputTokens":2,"latencyMs":370.4828330000164},{"questionId":"q160","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1173,"outputTokens":5,"latencyMs":320.6201669999864},{"questionId":"q160","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":795,"outputTokens":5,"latencyMs":337.706165999989},{"questionId":"q160","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":907,"outputTokens":5,"latencyMs":370.82633400009945},{"questionId":"q160","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":1245,"outputTokens":5,"latencyMs":371.3361249999143},{"questionId":"q160","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6.8.3","actual":"6.8.3","isCorrect":true,"inputTokens":925,"outputTokens":5,"latencyMs":363.8190000000177},{"questionId":"q161","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1173,"outputTokens":1,"latencyMs":365.8757500000065},{"questionId":"q161","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":795,"outputTokens":1,"latencyMs":401.83149999997113},{"questionId":"q161","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":907,"outputTokens":1,"latencyMs":359.98466700001154},{"questionId":"q161","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":343.08983300009277},{"questionId":"q161","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":925,"outputTokens":1,"latencyMs":426.9515830000164},{"questionId":"q162","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1173,"outputTokens":1,"latencyMs":309.2816669999156},{"questionId":"q162","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":795,"outputTokens":1,"latencyMs":359.67050000000745},{"questionId":"q162","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":907,"outputTokens":1,"latencyMs":332.65541699994355},{"questionId":"q162","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":320.3308749999851},{"questionId":"q162","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":925,"outputTokens":1,"latencyMs":354.45833299995866},{"questionId":"q163","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1172,"outputTokens":1,"latencyMs":371.08149999997113},{"questionId":"q163","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":794,"outputTokens":1,"latencyMs":366.60941599996295},{"questionId":"q163","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":906,"outputTokens":1,"latencyMs":458.8468330001924},{"questionId":"q163","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1244,"outputTokens":1,"latencyMs":443.36091599985957},{"questionId":"q163","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":924,"outputTokens":1,"latencyMs":387.0315829999745},{"questionId":"q164","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1172,"outputTokens":1,"latencyMs":410.85737500013784},{"questionId":"q164","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":794,"outputTokens":1,"latencyMs":394.671417000005},{"questionId":"q164","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":906,"outputTokens":1,"latencyMs":390.031500000041},{"questionId":"q164","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":1244,"outputTokens":1,"latencyMs":429.30516699980944},{"questionId":"q164","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2","actual":"2","isCorrect":true,"inputTokens":924,"outputTokens":1,"latencyMs":400.01895800000057},{"questionId":"q165","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1172,"outputTokens":1,"latencyMs":396.54087499994785},{"questionId":"q165","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":794,"outputTokens":1,"latencyMs":445.0643329999875},{"questionId":"q165","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":906,"outputTokens":1,"latencyMs":428.93104199995287},{"questionId":"q165","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1244,"outputTokens":1,"latencyMs":357.6399579998106},{"questionId":"q165","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":924,"outputTokens":1,"latencyMs":383.61412499984726},{"questionId":"q166","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1176,"outputTokens":1,"latencyMs":370.86766699980944},{"questionId":"q166","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":798,"outputTokens":1,"latencyMs":442.8896249998361},{"questionId":"q166","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":910,"outputTokens":1,"latencyMs":347.00399999995716},{"questionId":"q166","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1248,"outputTokens":1,"latencyMs":369.8371249998454},{"questionId":"q166","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":928,"outputTokens":1,"latencyMs":1478.887624999974},{"questionId":"q167","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1172,"outputTokens":1,"latencyMs":337.7872089999728},{"questionId":"q167","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":794,"outputTokens":1,"latencyMs":320.8023749999702},{"questionId":"q167","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":906,"outputTokens":1,"latencyMs":346.6840840000659},{"questionId":"q167","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1244,"outputTokens":1,"latencyMs":317.7252500001341},{"questionId":"q167","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":924,"outputTokens":1,"latencyMs":329.32091699982993},{"questionId":"q168","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":1174,"outputTokens":1,"latencyMs":317.9294579999987},{"questionId":"q168","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":796,"outputTokens":1,"latencyMs":387.33762499992736},{"questionId":"q168","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":908,"outputTokens":1,"latencyMs":351.4300420000218},{"questionId":"q168","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":1246,"outputTokens":1,"latencyMs":363.89233399997465},{"questionId":"q168","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":926,"outputTokens":1,"latencyMs":339.5886659999378},{"questionId":"q169","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"8","actual":"10","isCorrect":false,"inputTokens":1176,"outputTokens":1,"latencyMs":335.85412499983795},{"questionId":"q169","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"8","actual":"10","isCorrect":false,"inputTokens":798,"outputTokens":1,"latencyMs":385.40554100018926},{"questionId":"q169","format":"toon","model":"grok-4-fast-non-reasoning","expected":"8","actual":"8","isCorrect":true,"inputTokens":910,"outputTokens":1,"latencyMs":342.01175000006333},{"questionId":"q169","format":"xml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"9","isCorrect":false,"inputTokens":1248,"outputTokens":1,"latencyMs":428.92833300004713},{"questionId":"q169","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"8","actual":"11","isCorrect":false,"inputTokens":928,"outputTokens":1,"latencyMs":352.5753329999279},{"questionId":"q170","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":1175,"outputTokens":1,"latencyMs":332.23770800000057},{"questionId":"q170","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":797,"outputTokens":1,"latencyMs":329.424541000044},{"questionId":"q170","format":"toon","model":"grok-4-fast-non-reasoning","expected":"5","actual":"6","isCorrect":false,"inputTokens":909,"outputTokens":1,"latencyMs":375.9569580000825},{"questionId":"q170","format":"xml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":1247,"outputTokens":1,"latencyMs":324.7888340000063},{"questionId":"q170","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"5","actual":"5","isCorrect":true,"inputTokens":927,"outputTokens":1,"latencyMs":360.58004099992104},{"questionId":"q171","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":1177,"outputTokens":1,"latencyMs":366.4668750001583},{"questionId":"q171","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":799,"outputTokens":1,"latencyMs":344.7729579999577},{"questionId":"q171","format":"toon","model":"grok-4-fast-non-reasoning","expected":"3","actual":"3","isCorrect":true,"inputTokens":911,"outputTokens":1,"latencyMs":354.1180829999503},{"questionId":"q171","format":"xml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"4","isCorrect":false,"inputTokens":1249,"outputTokens":1,"latencyMs":319.90387500007637},{"questionId":"q171","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"3","actual":"4","isCorrect":false,"inputTokens":929,"outputTokens":1,"latencyMs":322.4436249998398},{"questionId":"q172","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1177,"outputTokens":1,"latencyMs":408.5942910001613},{"questionId":"q172","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":799,"outputTokens":1,"latencyMs":331.8505839998834},{"questionId":"q172","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":911,"outputTokens":1,"latencyMs":667.1007499999832},{"questionId":"q172","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1249,"outputTokens":1,"latencyMs":316.4649169999175},{"questionId":"q172","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":929,"outputTokens":1,"latencyMs":324.732166999951},{"questionId":"q173","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":1178,"outputTokens":1,"latencyMs":430.2254590000957},{"questionId":"q173","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":800,"outputTokens":1,"latencyMs":417.00312499981374},{"questionId":"q173","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":912,"outputTokens":1,"latencyMs":490.2997500000056},{"questionId":"q173","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":1250,"outputTokens":1,"latencyMs":492.1121670000721},{"questionId":"q173","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":930,"outputTokens":1,"latencyMs":490.10812500002794},{"questionId":"q174","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1175,"outputTokens":1,"latencyMs":492.4814999999944},{"questionId":"q174","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"2","isCorrect":false,"inputTokens":797,"outputTokens":1,"latencyMs":485.07404200010933},{"questionId":"q174","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":909,"outputTokens":1,"latencyMs":489.27204199996777},{"questionId":"q174","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1247,"outputTokens":1,"latencyMs":491.6592500000261},{"questionId":"q174","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":927,"outputTokens":1,"latencyMs":485.43808300001547},{"questionId":"q175","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1179,"outputTokens":1,"latencyMs":489.2083340000827},{"questionId":"q175","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":801,"outputTokens":1,"latencyMs":481.23754100012593},{"questionId":"q175","format":"toon","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":913,"outputTokens":1,"latencyMs":326.027291999897},{"questionId":"q175","format":"xml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"0","isCorrect":true,"inputTokens":1251,"outputTokens":1,"latencyMs":419.84279099991545},{"questionId":"q175","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"0","actual":"1","isCorrect":false,"inputTokens":931,"outputTokens":1,"latencyMs":349.5914590000175},{"questionId":"q176","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1173,"outputTokens":1,"latencyMs":392.5551249999553},{"questionId":"q176","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":795,"outputTokens":1,"latencyMs":394.7711670000572},{"questionId":"q176","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":907,"outputTokens":1,"latencyMs":443.7036669999361},{"questionId":"q176","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1245,"outputTokens":1,"latencyMs":350.6532499999739},{"questionId":"q176","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":925,"outputTokens":1,"latencyMs":363.3377500001807},{"questionId":"q177","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1181,"outputTokens":1,"latencyMs":361.1672079998534},{"questionId":"q177","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":803,"outputTokens":1,"latencyMs":420.9918750000652},{"questionId":"q177","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":915,"outputTokens":1,"latencyMs":297.57720800000243},{"questionId":"q177","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1253,"outputTokens":1,"latencyMs":334.78249999997206},{"questionId":"q177","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":933,"outputTokens":1,"latencyMs":300.36041700001806},{"questionId":"q178","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1176,"outputTokens":1,"latencyMs":368.92149999993853},{"questionId":"q178","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":798,"outputTokens":1,"latencyMs":324.06395799992606},{"questionId":"q178","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":910,"outputTokens":1,"latencyMs":361.890374999959},{"questionId":"q178","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1248,"outputTokens":1,"latencyMs":489.4733750000596},{"questionId":"q178","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":928,"outputTokens":1,"latencyMs":333.90695900004357},{"questionId":"q179","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":1175,"outputTokens":1,"latencyMs":488.4929159998428},{"questionId":"q179","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":797,"outputTokens":1,"latencyMs":353.6462499999907},{"questionId":"q179","format":"toon","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":909,"outputTokens":1,"latencyMs":373.7459159998689},{"questionId":"q179","format":"xml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"1","isCorrect":true,"inputTokens":1247,"outputTokens":1,"latencyMs":623.9110829997808},{"questionId":"q179","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"1","actual":"0","isCorrect":false,"inputTokens":927,"outputTokens":1,"latencyMs":330.82337500015274},{"questionId":"q180","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":6559,"outputTokens":1,"latencyMs":375.26316700014286},{"questionId":"q180","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":4150,"outputTokens":1,"latencyMs":374.6645829998888},{"questionId":"q180","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2729,"outputTokens":1,"latencyMs":337.0033749998547},{"questionId":"q180","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":2564,"outputTokens":1,"latencyMs":362.57820800016634},{"questionId":"q180","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":7496,"outputTokens":1,"latencyMs":421.8942500001285},{"questionId":"q180","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":5192,"outputTokens":1,"latencyMs":382.42987500014715},{"questionId":"q181","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":6564,"outputTokens":12,"latencyMs":443.77354200021364},{"questionId":"q181","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":4155,"outputTokens":12,"latencyMs":416.2860830000136},{"questionId":"q181","format":"toon","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2734,"outputTokens":12,"latencyMs":359.55320800002664},{"questionId":"q181","format":"csv","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":2569,"outputTokens":12,"latencyMs":662.6431660000235},{"questionId":"q181","format":"xml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":7501,"outputTokens":12,"latencyMs":597.8325839999598},{"questionId":"q181","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,department,salary,yearsExperience,active","actual":"id,name,email,department,salary,yearsExperience,active","isCorrect":true,"inputTokens":5197,"outputTokens":12,"latencyMs":412.4055829998106},{"questionId":"q182","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":6562,"outputTokens":1,"latencyMs":440.41258400003426},{"questionId":"q182","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":4153,"outputTokens":1,"latencyMs":383.56320800003596},{"questionId":"q182","format":"toon","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":2732,"outputTokens":1,"latencyMs":324.03475000010803},{"questionId":"q182","format":"csv","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":2567,"outputTokens":1,"latencyMs":330.7392080000136},{"questionId":"q182","format":"xml","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":7499,"outputTokens":1,"latencyMs":450.61337499995716},{"questionId":"q182","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"email","actual":"email","isCorrect":true,"inputTokens":5195,"outputTokens":1,"latencyMs":580.1668340000324},{"questionId":"q183","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":6563,"outputTokens":1,"latencyMs":503.1660419998225},{"questionId":"q183","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":4154,"outputTokens":1,"latencyMs":495.4806250000838},{"questionId":"q183","format":"toon","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2733,"outputTokens":1,"latencyMs":682.1085829997901},{"questionId":"q183","format":"csv","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":2568,"outputTokens":1,"latencyMs":698.3130419999361},{"questionId":"q183","format":"xml","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":7500,"outputTokens":1,"latencyMs":659.19745899993},{"questionId":"q183","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"HR","actual":"HR","isCorrect":true,"inputTokens":5196,"outputTokens":1,"latencyMs":605.9669580000918},{"questionId":"q184","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":6563,"outputTokens":5,"latencyMs":574.3029999998398},{"questionId":"q184","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":4154,"outputTokens":5,"latencyMs":512.824874999933},{"questionId":"q184","format":"toon","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2733,"outputTokens":5,"latencyMs":620.5331660001539},{"questionId":"q184","format":"csv","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":2568,"outputTokens":5,"latencyMs":497.8000829999801},{"questionId":"q184","format":"xml","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":7500,"outputTokens":5,"latencyMs":420.6944580001291},{"questionId":"q184","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"Mrs. Sherri Ritchie","actual":"Mrs. Sherri Ritchie","isCorrect":true,"inputTokens":5196,"outputTokens":5,"latencyMs":391.2383329998702},{"questionId":"q185","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":6560,"outputTokens":1,"latencyMs":815.1224160001148},{"questionId":"q185","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":4151,"outputTokens":1,"latencyMs":351.6620419998653},{"questionId":"q185","format":"toon","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":2730,"outputTokens":1,"latencyMs":360.74091599998064},{"questionId":"q185","format":"csv","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":2565,"outputTokens":1,"latencyMs":364.14483299991116},{"questionId":"q185","format":"xml","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":7497,"outputTokens":1,"latencyMs":558.0179999999236},{"questionId":"q185","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"7","actual":"7","isCorrect":true,"inputTokens":5193,"outputTokens":1,"latencyMs":373.56420799996704},{"questionId":"q186","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":10999,"outputTokens":1,"latencyMs":673.0515829999931},{"questionId":"q186","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":6978,"outputTokens":1,"latencyMs":426.36545799998567},{"questionId":"q186","format":"toon","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":7401,"outputTokens":1,"latencyMs":406.4124999998603},{"questionId":"q186","format":"xml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":12325,"outputTokens":1,"latencyMs":414.4021669998765},{"questionId":"q186","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"50","actual":"50","isCorrect":true,"inputTokens":8564,"outputTokens":1,"latencyMs":771.6031659999862},{"questionId":"q187","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":11006,"outputTokens":15,"latencyMs":800.781042000046},{"questionId":"q187","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":6985,"outputTokens":15,"latencyMs":688.1246250001714},{"questionId":"q187","format":"toon","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":7408,"outputTokens":15,"latencyMs":643.6218749999534},{"questionId":"q187","format":"xml","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":12332,"outputTokens":15,"latencyMs":770.1273330000695},{"questionId":"q187","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"orderId,customer,items,subtotal,tax,total,status,orderDate","actual":"orderId,customer,items,subtotal,tax,total,status,orderDate","isCorrect":true,"inputTokens":8571,"outputTokens":15,"latencyMs":614.6380829999689},{"questionId":"q188","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":11002,"outputTokens":1,"latencyMs":599.3983750001062},{"questionId":"q188","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":6981,"outputTokens":1,"latencyMs":437.3969999998808},{"questionId":"q188","format":"toon","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":7404,"outputTokens":1,"latencyMs":418.3614580000285},{"questionId":"q188","format":"xml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":12328,"outputTokens":1,"latencyMs":566.3661660000216},{"questionId":"q188","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"4","actual":"4","isCorrect":true,"inputTokens":8567,"outputTokens":1,"latencyMs":500.6189580000937},{"questionId":"q189","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":11007,"outputTokens":6,"latencyMs":471.3544999998994},{"questionId":"q189","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":6986,"outputTokens":6,"latencyMs":479.0526250000112},{"questionId":"q189","format":"toon","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":7409,"outputTokens":6,"latencyMs":481.078582999995},{"questionId":"q189","format":"xml","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":12333,"outputTokens":6,"latencyMs":677.6656669999938},{"questionId":"q189","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"sku,name,quantity,price","actual":"sku,name,quantity,price","isCorrect":true,"inputTokens":8572,"outputTokens":6,"latencyMs":576.5693329998758},{"questionId":"q190","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":11003,"outputTokens":2,"latencyMs":601.5891660000198},{"questionId":"q190","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":6982,"outputTokens":2,"latencyMs":428.8857499998994},{"questionId":"q190","format":"toon","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":7405,"outputTokens":2,"latencyMs":451.9869160000235},{"questionId":"q190","format":"xml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":12329,"outputTokens":2,"latencyMs":633.9192500000354},{"questionId":"q190","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"cancelled","actual":"cancelled","isCorrect":true,"inputTokens":8568,"outputTokens":2,"latencyMs":594.1280000000261},{"questionId":"q191","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":11008,"outputTokens":5,"latencyMs":670.3384579999838},{"questionId":"q191","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":6987,"outputTokens":5,"latencyMs":572.7019999998156},{"questionId":"q191","format":"toon","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":7410,"outputTokens":5,"latencyMs":614.5949160000309},{"questionId":"q191","format":"xml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":12334,"outputTokens":5,"latencyMs":641.3000000000466},{"questionId":"q191","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"id,name,email,phone","actual":"id,name,email,phone","isCorrect":true,"inputTokens":8573,"outputTokens":5,"latencyMs":666.644167000195},{"questionId":"q192","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"60","actual":"61","isCorrect":false,"inputTokens":3868,"outputTokens":1,"latencyMs":439.85908399987966},{"questionId":"q192","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":2539,"outputTokens":1,"latencyMs":616.6593750000466},{"questionId":"q192","format":"toon","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":1798,"outputTokens":1,"latencyMs":405.3493749999907},{"questionId":"q192","format":"csv","model":"grok-4-fast-non-reasoning","expected":"60","actual":"60","isCorrect":true,"inputTokens":1657,"outputTokens":1,"latencyMs":471.91520799999125},{"questionId":"q192","format":"xml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"63","isCorrect":false,"inputTokens":4514,"outputTokens":1,"latencyMs":684.3032919999678},{"questionId":"q192","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"60","actual":"63","isCorrect":false,"inputTokens":3198,"outputTokens":1,"latencyMs":484.984582999954},{"questionId":"q193","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3872,"outputTokens":14,"latencyMs":438.15741700003855},{"questionId":"q193","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":2543,"outputTokens":14,"latencyMs":481.5030829999596},{"questionId":"q193","format":"toon","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1802,"outputTokens":14,"latencyMs":455.85700000007637},{"questionId":"q193","format":"csv","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":1661,"outputTokens":14,"latencyMs":476.09504200005904},{"questionId":"q193","format":"xml","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":4518,"outputTokens":14,"latencyMs":1505.1402499999385},{"questionId":"q193","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"date,views,clicks,conversions,revenue,bounceRate","actual":"date,views,clicks,conversions,revenue,bounceRate","isCorrect":true,"inputTokens":3202,"outputTokens":14,"latencyMs":389.89354200009257},{"questionId":"q194","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"conversions","isCorrect":false,"inputTokens":3871,"outputTokens":2,"latencyMs":366.28087499993853},{"questionId":"q194","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":2542,"outputTokens":2,"latencyMs":332.7042089998722},{"questionId":"q194","format":"toon","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1801,"outputTokens":1,"latencyMs":375.86899999994785},{"questionId":"q194","format":"csv","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"revenue","isCorrect":true,"inputTokens":1660,"outputTokens":1,"latencyMs":353.2458340001758},{"questionId":"q194","format":"xml","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"conversions","isCorrect":false,"inputTokens":4517,"outputTokens":2,"latencyMs":390.6885829998646},{"questionId":"q194","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"revenue","actual":"bounceRate","isCorrect":false,"inputTokens":3201,"outputTokens":2,"latencyMs":369.78733299998567},{"questionId":"q195","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3872,"outputTokens":6,"latencyMs":361.7395840000827},{"questionId":"q195","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":2543,"outputTokens":6,"latencyMs":472.7101670000702},{"questionId":"q195","format":"toon","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1802,"outputTokens":6,"latencyMs":841.1769159999676},{"questionId":"q195","format":"csv","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":1661,"outputTokens":6,"latencyMs":355.3958330000751},{"questionId":"q195","format":"xml","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":4518,"outputTokens":6,"latencyMs":668.0135000001173},{"questionId":"q195","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"2025-03-01","actual":"2025-03-01","isCorrect":true,"inputTokens":3202,"outputTokens":6,"latencyMs":347.7547500000801},{"questionId":"q196","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":3868,"outputTokens":1,"latencyMs":369.593292000005},{"questionId":"q196","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":2539,"outputTokens":1,"latencyMs":597.1295829999726},{"questionId":"q196","format":"toon","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":1798,"outputTokens":1,"latencyMs":503.12516600009985},{"questionId":"q196","format":"csv","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":1657,"outputTokens":1,"latencyMs":368.1124169998802},{"questionId":"q196","format":"xml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":4514,"outputTokens":1,"latencyMs":497.4147909998428},{"questionId":"q196","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"6","actual":"6","isCorrect":true,"inputTokens":3198,"outputTokens":1,"latencyMs":447.1916670000646},{"questionId":"q197","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":15285,"outputTokens":1,"latencyMs":747.44745899993},{"questionId":"q197","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":11589,"outputTokens":1,"latencyMs":810.4211250001099},{"questionId":"q197","format":"toon","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8912,"outputTokens":1,"latencyMs":567.6103749999311},{"questionId":"q197","format":"csv","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":8662,"outputTokens":1,"latencyMs":661.5651670000516},{"questionId":"q197","format":"xml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"72","isCorrect":false,"inputTokens":17197,"outputTokens":1,"latencyMs":869.1681250000838},{"questionId":"q197","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"100","actual":"100","isCorrect":true,"inputTokens":13283,"outputTokens":1,"latencyMs":762.5899169999175},{"questionId":"q198","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":15290,"outputTokens":24,"latencyMs":1027.5254589999095},{"questionId":"q198","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":11594,"outputTokens":24,"latencyMs":710.6252089999616},{"questionId":"q198","format":"toon","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8917,"outputTokens":24,"latencyMs":830.4645420000888},{"questionId":"q198","format":"csv","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":8667,"outputTokens":24,"latencyMs":600.7309159999713},{"questionId":"q198","format":"xml","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":17202,"outputTokens":24,"latencyMs":928.4134170000907},{"questionId":"q198","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"id,name,repo,description,stars,watchers,forks,defaultBranch,createdAt,updatedAt,pushedAt","actual":"id,name,repo,description,createdAt,updatedAt,pushedAt,stars,watchers,forks,defaultBranch","isCorrect":false,"inputTokens":13288,"outputTokens":24,"latencyMs":673.2553340001032},{"questionId":"q199","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"stars","isCorrect":false,"inputTokens":15290,"outputTokens":1,"latencyMs":427.94724999996834},{"questionId":"q199","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"defaultBranch","isCorrect":false,"inputTokens":11594,"outputTokens":2,"latencyMs":637.046374999918},{"questionId":"q199","format":"toon","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"pushedAt","isCorrect":false,"inputTokens":8917,"outputTokens":3,"latencyMs":593.0555419998709},{"questionId":"q199","format":"csv","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"defaultBranch","isCorrect":false,"inputTokens":8667,"outputTokens":2,"latencyMs":613.5377500001341},{"questionId":"q199","format":"xml","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"stars","isCorrect":false,"inputTokens":17202,"outputTokens":1,"latencyMs":651.6903329999186},{"questionId":"q199","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"forks","actual":"stars","isCorrect":false,"inputTokens":13288,"outputTokens":1,"latencyMs":348.3319579998497},{"questionId":"q200","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":15289,"outputTokens":2,"latencyMs":967.8817499999423},{"questionId":"q200","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":11593,"outputTokens":2,"latencyMs":715.7540420000441},{"questionId":"q200","format":"toon","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8916,"outputTokens":2,"latencyMs":710.7448749998584},{"questionId":"q200","format":"csv","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":8666,"outputTokens":2,"latencyMs":785.0975420000032},{"questionId":"q200","format":"xml","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":17201,"outputTokens":2,"latencyMs":979.5244170001242},{"questionId":"q200","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"tailwindcss","actual":"tailwindcss","isCorrect":true,"inputTokens":13287,"outputTokens":2,"latencyMs":782.1515409999993},{"questionId":"q201","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":15286,"outputTokens":1,"latencyMs":1015.6043330000248},{"questionId":"q201","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"11","actual":"12","isCorrect":false,"inputTokens":11590,"outputTokens":1,"latencyMs":1313.6311250000726},{"questionId":"q201","format":"toon","model":"grok-4-fast-non-reasoning","expected":"11","actual":"10","isCorrect":false,"inputTokens":8913,"outputTokens":1,"latencyMs":656.9232920000795},{"questionId":"q201","format":"csv","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":8663,"outputTokens":1,"latencyMs":678.9637499998789},{"questionId":"q201","format":"xml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"9","isCorrect":false,"inputTokens":17198,"outputTokens":1,"latencyMs":1066.788208000129},{"questionId":"q201","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"11","actual":"11","isCorrect":true,"inputTokens":13284,"outputTokens":1,"latencyMs":937.3893330001738},{"questionId":"q202","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"75","actual":"60","isCorrect":false,"inputTokens":7040,"outputTokens":1,"latencyMs":832.7258750000037},{"questionId":"q202","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"75","actual":"80","isCorrect":false,"inputTokens":4952,"outputTokens":1,"latencyMs":401.36583300004713},{"questionId":"q202","format":"toon","model":"grok-4-fast-non-reasoning","expected":"75","actual":"75","isCorrect":true,"inputTokens":6041,"outputTokens":1,"latencyMs":359.047040999867},{"questionId":"q202","format":"xml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"70","isCorrect":false,"inputTokens":7950,"outputTokens":1,"latencyMs":438.05404099985026},{"questionId":"q202","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"75","actual":"60","isCorrect":false,"inputTokens":6067,"outputTokens":1,"latencyMs":427.7783749999944},{"questionId":"q203","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":7049,"outputTokens":13,"latencyMs":533.5387079999782},{"questionId":"q203","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":4961,"outputTokens":13,"latencyMs":518.5181249999441},{"questionId":"q203","format":"toon","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":6050,"outputTokens":13,"latencyMs":434.35874999989755},{"questionId":"q203","format":"xml","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":7959,"outputTokens":13,"latencyMs":469.73154199984856},{"questionId":"q203","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"timestamp,level,endpoint,statusCode,responseTime,userId,error","actual":"timestamp,level,endpoint,statusCode,responseTime,userId,error","isCorrect":true,"inputTokens":6076,"outputTokens":13,"latencyMs":574.9501249999739},{"questionId":"q204","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":7044,"outputTokens":1,"latencyMs":699.1965419999324},{"questionId":"q204","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":4956,"outputTokens":1,"latencyMs":433.15729200001806},{"questionId":"q204","format":"toon","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6045,"outputTokens":1,"latencyMs":913.7539170000236},{"questionId":"q204","format":"xml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":7954,"outputTokens":1,"latencyMs":697.5380420000292},{"questionId":"q204","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"info","actual":"info","isCorrect":true,"inputTokens":6071,"outputTokens":1,"latencyMs":412.4732500000391},{"questionId":"q205","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1526,"outputTokens":1,"latencyMs":1076.2398749999702},{"questionId":"q205","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1037,"outputTokens":1,"latencyMs":360.4928750002291},{"questionId":"q205","format":"toon","model":"grok-4-fast-non-reasoning","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":787,"outputTokens":1,"latencyMs":399.8271250000689},{"questionId":"q205","format":"csv","model":"grok-4-fast-non-reasoning","expected":"YES","actual":"NO","isCorrect":false,"inputTokens":736,"outputTokens":1,"latencyMs":348.658458000049},{"questionId":"q205","format":"xml","model":"grok-4-fast-non-reasoning","expected":"YES","actual":"NO","isCorrect":false,"inputTokens":1706,"outputTokens":1,"latencyMs":395.8682919999119},{"questionId":"q205","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"YES","actual":"YES","isCorrect":true,"inputTokens":1249,"outputTokens":1,"latencyMs":360.42500000004657},{"questionId":"q206","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1342,"outputTokens":1,"latencyMs":386.03808299987577},{"questionId":"q206","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":925,"outputTokens":1,"latencyMs":344.08970799995586},{"questionId":"q206","format":"toon","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":718,"outputTokens":1,"latencyMs":396.7130420000758},{"questionId":"q206","format":"csv","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":670,"outputTokens":1,"latencyMs":404.4093330001924},{"questionId":"q206","format":"xml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1492,"outputTokens":1,"latencyMs":1086.5814169999212},{"questionId":"q206","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1106,"outputTokens":1,"latencyMs":383.5592080000788},{"questionId":"q207","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1719,"outputTokens":1,"latencyMs":375.8127919998951},{"questionId":"q207","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1158,"outputTokens":1,"latencyMs":352.4401249999646},{"questionId":"q207","format":"toon","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":863,"outputTokens":1,"latencyMs":339.330125000095},{"questionId":"q207","format":"csv","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":809,"outputTokens":1,"latencyMs":535.2553330000956},{"questionId":"q207","format":"xml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1928,"outputTokens":1,"latencyMs":431.1465839999728},{"questionId":"q207","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1401,"outputTokens":1,"latencyMs":324.8696250000503},{"questionId":"q208","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1518,"outputTokens":1,"latencyMs":346.6047499999404},{"questionId":"q208","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1032,"outputTokens":1,"latencyMs":412.83454099996015},{"questionId":"q208","format":"toon","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1267,"outputTokens":1,"latencyMs":492.7490000000689},{"questionId":"q208","format":"csv","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":733,"outputTokens":1,"latencyMs":407.6067079999484},{"questionId":"q208","format":"xml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1697,"outputTokens":1,"latencyMs":359.5358750000596},{"questionId":"q208","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1242,"outputTokens":1,"latencyMs":396.6416249999311},{"questionId":"q209","format":"json-pretty","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1477,"outputTokens":1,"latencyMs":720.5081670000218},{"questionId":"q209","format":"json-compact","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1000,"outputTokens":1,"latencyMs":350.6188329998404},{"questionId":"q209","format":"toon","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"YES","isCorrect":false,"inputTokens":1233,"outputTokens":1,"latencyMs":452.3008329998702},{"questionId":"q209","format":"csv","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":578,"outputTokens":1,"latencyMs":422.6157500001136},{"questionId":"q209","format":"xml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1653,"outputTokens":1,"latencyMs":470.04287500004284},{"questionId":"q209","format":"yaml","model":"grok-4-fast-non-reasoning","expected":"NO","actual":"NO","isCorrect":true,"inputTokens":1208,"outputTokens":1,"latencyMs":500.6660410000477}] \ No newline at end of file diff --git a/benchmarks/results/retrieval-accuracy.md b/benchmarks/results/retrieval-accuracy.md index b0fb3ae..b5dee75 100644 --- a/benchmarks/results/retrieval-accuracy.md +++ b/benchmarks/results/retrieval-accuracy.md @@ -1,4 +1,4 @@ -Benchmarks test LLM comprehension across different input formats using 204 data retrieval questions on 4 models. +Benchmarks test LLM comprehension across different input formats using 209 data retrieval questions on 4 models.
Show Dataset Catalog @@ -13,6 +13,11 @@ Benchmarks test LLM comprehension across different input formats using 204 data | Top 100 GitHub repositories | 100 | uniform | โœ“ | 100% | | Semi-uniform event logs | 75 | semi-uniform | โœ— | 50% | | Deeply nested configuration | 11 | deep | โœ— | 0% | +| Valid complete dataset (control) | 20 | uniform | โœ“ | 100% | +| Array truncated: 3 rows removed from end | 17 | uniform | โœ“ | 100% | +| Extra rows added beyond declared length | 23 | uniform | โœ“ | 100% | +| Inconsistent field count (missing salary in row 10) | 20 | uniform | โœ“ | 100% | +| Missing required fields (no email in multiple rows) | 20 | uniform | โœ“ | 100% | **Structure classes:** - **uniform**: All objects have identical fields with primitive values @@ -31,67 +36,69 @@ Benchmarks test LLM comprehension across different input formats using 204 data Each format's overall performance, balancing accuracy against token cost: ``` -TOON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“ 17.2 โ”‚ 75.5% acc โ”‚ 4,389 tokens -CSV โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘ 16.6 โ”‚ 67.8% acc โ”‚ 4,080 tokens -JSON compact โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘ 14.7 โ”‚ 73.3% acc โ”‚ 4,982 tokens -YAML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 12.1 โ”‚ 72.4% acc โ”‚ 5,976 tokens -JSON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 10.0 โ”‚ 72.4% acc โ”‚ 7,260 tokens -XML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 8.4 โ”‚ 69.0% acc โ”‚ 8,251 tokens +TOON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“ 26.9 โ”‚ 73.9% acc โ”‚ 2,744 tokens +JSON compact โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘ 22.9 โ”‚ 70.7% acc โ”‚ 3,081 tokens +YAML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 18.6 โ”‚ 69.0% acc โ”‚ 3,719 tokens +JSON โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 15.3 โ”‚ 69.7% acc โ”‚ 4,545 tokens +XML โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 13.0 โ”‚ 67.1% acc โ”‚ 5,167 tokens ``` -TOON achieves **75.5%** accuracy (vs JSON's 72.4%) while using **39.5% fewer tokens**. +TOON achieves **73.9%** accuracy (vs JSON's 69.7%) while using **39.6% fewer tokens**. + +**Note on CSV:** Excluded from ranking as it only supports 436/209 questions (flat tabular data only). While CSV is highly token-efficient for simple tabular data, it cannot represent nested structures that other formats handle. #### Per-Model Accuracy -Accuracy across 4 LLMs on 204 data retrieval questions: +Accuracy across 4 LLMs on 209 data retrieval questions: ``` claude-haiku-4-5-20251001 -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 62.3% (127/204) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 56.9% (116/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.9% (114/204) - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.9% (112/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.9% (112/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 47.1% (49/104) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 59.8% (125/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 57.4% (120/209) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 56.0% (117/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.5% (116/209) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.0% (115/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 50.5% (55/109) gemini-2.5-flash -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 91.2% (186/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.7% (183/204) - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 87.7% (179/204) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 87.7% (179/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 87.3% (178/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 85.6% (89/104) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 87.6% (183/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 86.2% (94/109) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 82.3% (172/209) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 79.4% (166/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 79.4% (166/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘ 77.0% (161/209) gpt-5-nano - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 93.6% (191/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 90.4% (94/104) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.7% (183/204) -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.2% (182/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.2% (182/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 81.4% (166/204) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 90.9% (190/209) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 90.9% (190/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.0% (186/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 89.0% (97/109) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 87.1% (182/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 80.9% (169/209) grok-4-fast-non-reasoning -โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 59.3% (121/204) - JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 56.9% (116/204) - JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.4% (113/204) - YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.9% (112/204) - XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 52.5% (107/204) - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 48.1% (50/104) +โ†’ TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 57.4% (120/209) + JSON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 55.5% (116/209) + JSON compact โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 54.5% (114/209) + YAML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 53.6% (112/209) + XML โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 52.6% (110/209) + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 52.3% (57/109) ``` -**Key tradeoff:** TOON achieves **75.5% accuracy** (vs JSON's 72.4%) while using **39.5% fewer tokens** on these datasets. +**Key tradeoff:** TOON achieves **73.9% accuracy** (vs JSON's 69.7%) while using **39.6% fewer tokens** on these datasets.
Performance by dataset, model, and question type #### Performance by Question Type -| Question Type | TOON | JSON compact | JSON | YAML | XML | CSV | +| Question Type | TOON | JSON compact | JSON | CSV | YAML | XML | | ------------- | ---- | ---- | ---- | ---- | ---- | ---- | -| Field Retrieval | 100.0% | 98.9% | 99.6% | 99.3% | 98.5% | 100.0% | -| Aggregation | 56.3% | 52.4% | 53.2% | 53.2% | 47.2% | 40.5% | -| Filtering | 58.9% | 58.3% | 54.2% | 53.1% | 50.5% | 49.1% | -| Structure Awareness | 89.0% | 85.0% | 82.0% | 85.0% | 79.0% | 84.4% | +| Field Retrieval | 99.6% | 99.3% | 99.3% | 100.0% | 98.2% | 98.9% | +| Aggregation | 54.4% | 47.2% | 48.8% | 44.0% | 47.6% | 41.3% | +| Filtering | 56.3% | 57.3% | 50.5% | 49.1% | 51.0% | 47.9% | +| Structure Awareness | 88.0% | 83.0% | 83.0% | 85.9% | 80.0% | 80.0% | +| Structural Validation | 70.0% | 45.0% | 50.0% | 80.0% | 60.0% | 80.0% | #### Performance by Dataset @@ -99,64 +106,119 @@ grok-4-fast-non-reasoning | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `csv` | 70.7% | 2,337 | 116/164 | -| `toon` | 72.0% | 2,483 | 118/164 | -| `json-compact` | 71.3% | 3,943 | 117/164 | -| `yaml` | 70.1% | 4,969 | 115/164 | -| `json-pretty` | 72.6% | 6,347 | 119/164 | -| `xml` | 70.7% | 7,314 | 116/164 | +| `csv` | 72.0% | 2,352 | 118/164 | +| `toon` | 73.8% | 2,518 | 121/164 | +| `json-compact` | 69.5% | 3,953 | 114/164 | +| `yaml` | 68.3% | 4,982 | 112/164 | +| `json-pretty` | 68.3% | 6,360 | 112/164 | +| `xml` | 69.5% | 7,324 | 114/164 | ##### E-commerce orders with nested structures | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `toon` | 83.5% | 7,197 | 137/164 | -| `json-compact` | 79.3% | 6,784 | 130/164 | -| `yaml` | 78.7% | 8,334 | 129/164 | -| `json-pretty` | 78.7% | 10,700 | 129/164 | -| `xml` | 73.8% | 12,013 | 121/164 | +| `toon` | 81.1% | 7,232 | 133/164 | +| `json-compact` | 76.8% | 6,794 | 126/164 | +| `yaml` | 75.6% | 8,347 | 124/164 | +| `json-pretty` | 76.2% | 10,713 | 125/164 | +| `xml` | 74.4% | 12,023 | 122/164 | ##### Time-series analytics data | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `toon` | 75.8% | 1,513 | 91/120 | -| `csv` | 72.5% | 1,391 | 87/120 | -| `json-compact` | 70.0% | 2,339 | 84/120 | -| `yaml` | 70.0% | 2,936 | 84/120 | -| `json-pretty` | 71.7% | 3,663 | 86/120 | -| `xml` | 71.7% | 4,374 | 86/120 | +| `csv` | 73.3% | 1,406 | 88/120 | +| `toon` | 72.5% | 1,548 | 87/120 | +| `json-compact` | 71.7% | 2,349 | 86/120 | +| `yaml` | 71.7% | 2,949 | 86/120 | +| `json-pretty` | 68.3% | 3,676 | 82/120 | +| `xml` | 68.3% | 4,384 | 82/120 | ##### Top 100 GitHub repositories | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `toon` | 64.4% | 8,745 | 85/132 | -| `csv` | 59.8% | 8,513 | 79/132 | -| `json-compact` | 60.6% | 11,455 | 80/132 | -| `yaml` | 61.4% | 13,129 | 81/132 | -| `json-pretty` | 59.1% | 15,145 | 78/132 | -| `xml` | 51.5% | 17,095 | 68/132 | +| `toon` | 62.9% | 8,780 | 83/132 | +| `csv` | 61.4% | 8,528 | 81/132 | +| `yaml` | 59.8% | 13,142 | 79/132 | +| `json-compact` | 55.3% | 11,465 | 73/132 | +| `json-pretty` | 56.1% | 15,158 | 74/132 | +| `xml` | 48.5% | 17,105 | 64/132 | ##### Semi-uniform event logs | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `json-compact` | 67.5% | 4,809 | 81/120 | -| `yaml` | 63.3% | 5,814 | 76/120 | -| `toon` | 62.5% | 5,764 | 75/120 | -| `json-pretty` | 59.2% | 6,784 | 71/120 | -| `xml` | 55.0% | 7,699 | 66/120 | +| `json-compact` | 63.3% | 4,819 | 76/120 | +| `toon` | 57.5% | 5,799 | 69/120 | +| `json-pretty` | 59.2% | 6,797 | 71/120 | +| `yaml` | 48.3% | 5,827 | 58/120 | +| `xml` | 46.7% | 7,709 | 56/120 | ##### Deeply nested configuration | Format | Accuracy | Tokens | Correct/Total | | ------ | -------- | ------ | ------------- | -| `json-compact` | 91.4% | 564 | 106/116 | -| `toon` | 94.8% | 631 | 110/116 | -| `yaml` | 91.4% | 673 | 106/116 | -| `json-pretty` | 93.1% | 919 | 108/116 | -| `xml` | 91.4% | 1,008 | 106/116 | +| `json-compact` | 92.2% | 574 | 107/116 | +| `toon` | 95.7% | 666 | 111/116 | +| `yaml` | 91.4% | 686 | 106/116 | +| `json-pretty` | 94.0% | 932 | 109/116 | +| `xml` | 92.2% | 1,018 | 107/116 | + +##### Valid complete dataset (control) + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `toon` | 100.0% | 544 | 4/4 | +| `json-compact` | 100.0% | 795 | 4/4 | +| `yaml` | 100.0% | 1,003 | 4/4 | +| `json-pretty` | 100.0% | 1,282 | 4/4 | +| `csv` | 25.0% | 492 | 1/4 | +| `xml` | 0.0% | 1,467 | 0/4 | + +##### Array truncated: 3 rows removed from end + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 100.0% | 425 | 4/4 | +| `xml` | 100.0% | 1,251 | 4/4 | +| `toon` | 0.0% | 474 | 0/4 | +| `json-compact` | 0.0% | 681 | 0/4 | +| `json-pretty` | 0.0% | 1,096 | 0/4 | +| `yaml` | 0.0% | 859 | 0/4 | + +##### Extra rows added beyond declared length + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 100.0% | 566 | 4/4 | +| `toon` | 75.0% | 621 | 3/4 | +| `xml` | 100.0% | 1,692 | 4/4 | +| `yaml` | 75.0% | 1,157 | 3/4 | +| `json-compact` | 50.0% | 917 | 2/4 | +| `json-pretty` | 50.0% | 1,476 | 2/4 | + +##### Inconsistent field count (missing salary in row 10) + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 75.0% | 489 | 3/4 | +| `yaml` | 100.0% | 996 | 4/4 | +| `toon` | 100.0% | 1,019 | 4/4 | +| `json-compact` | 75.0% | 790 | 3/4 | +| `xml` | 100.0% | 1,458 | 4/4 | +| `json-pretty` | 75.0% | 1,274 | 3/4 | + +##### Missing required fields (no email in multiple rows) + +| Format | Accuracy | Tokens | Correct/Total | +| ------ | -------- | ------ | ------------- | +| `csv` | 100.0% | 329 | 4/4 | +| `xml` | 100.0% | 1,411 | 4/4 | +| `toon` | 75.0% | 983 | 3/4 | +| `yaml` | 25.0% | 960 | 1/4 | +| `json-pretty` | 25.0% | 1,230 | 1/4 | +| `json-compact` | 0.0% | 755 | 0/4 | #### Performance by Model @@ -164,45 +226,45 @@ grok-4-fast-non-reasoning | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `toon` | 62.3% | 127/204 | -| `json-pretty` | 56.9% | 116/204 | -| `yaml` | 55.9% | 114/204 | -| `json-compact` | 54.9% | 112/204 | -| `xml` | 54.9% | 112/204 | -| `csv` | 47.1% | 49/104 | +| `toon` | 59.8% | 125/209 | +| `json-pretty` | 57.4% | 120/209 | +| `yaml` | 56.0% | 117/209 | +| `xml` | 55.5% | 116/209 | +| `json-compact` | 55.0% | 115/209 | +| `csv` | 50.5% | 55/109 | ##### gemini-2.5-flash | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `toon` | 91.2% | 186/204 | -| `yaml` | 89.7% | 183/204 | -| `json-compact` | 87.7% | 179/204 | -| `json-pretty` | 87.7% | 179/204 | -| `xml` | 87.3% | 178/204 | -| `csv` | 85.6% | 89/104 | +| `toon` | 87.6% | 183/209 | +| `csv` | 86.2% | 94/109 | +| `json-compact` | 82.3% | 172/209 | +| `yaml` | 79.4% | 166/209 | +| `xml` | 79.4% | 166/209 | +| `json-pretty` | 77.0% | 161/209 | ##### gpt-5-nano | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `json-compact` | 93.6% | 191/204 | -| `csv` | 90.4% | 94/104 | -| `json-pretty` | 89.7% | 183/204 | -| `toon` | 89.2% | 182/204 | -| `yaml` | 89.2% | 182/204 | -| `xml` | 81.4% | 166/204 | +| `toon` | 90.9% | 190/209 | +| `json-compact` | 90.9% | 190/209 | +| `json-pretty` | 89.0% | 186/209 | +| `csv` | 89.0% | 97/109 | +| `yaml` | 87.1% | 182/209 | +| `xml` | 80.9% | 169/209 | ##### grok-4-fast-non-reasoning | Format | Accuracy | Correct/Total | | ------ | -------- | ------------- | -| `toon` | 59.3% | 121/204 | -| `json-compact` | 56.9% | 116/204 | -| `json-pretty` | 55.4% | 113/204 | -| `yaml` | 54.9% | 112/204 | -| `xml` | 52.5% | 107/204 | -| `csv` | 48.1% | 50/104 | +| `toon` | 57.4% | 120/209 | +| `json-pretty` | 55.5% | 116/209 | +| `json-compact` | 54.5% | 114/209 | +| `yaml` | 53.6% | 112/209 | +| `xml` | 52.6% | 110/209 | +| `csv` | 52.3% | 57/109 |
@@ -215,8 +277,9 @@ This benchmark tests **LLM comprehension and data retrieval accuracy** across di #### Datasets Tested -Six datasets designed to test different structural patterns: +Eleven datasets designed to test different structural patterns and validation capabilities: +**Primary datasets:** 1. **Tabular** (100 employee records): Uniform objects with identical fields โ€“ optimal for TOON's tabular format. 2. **Nested** (50 e-commerce orders): Complex structures with nested customer objects and item arrays. 3. **Analytics** (60 days of metrics): Time-series data with dates and numeric values. @@ -224,21 +287,28 @@ Six datasets designed to test different structural patterns: 5. **Event Logs** (75 logs): Semi-uniform data with ~50% flat logs and ~50% with nested error objects. 6. **Nested Config** (1 configuration): Deeply nested configuration with minimal tabular eligibility. +**Structural validation datasets:** +7. **Control**: Valid complete dataset (baseline for validation) +8. **Truncated**: Array with 3 rows removed from end (tests [N] length detection) +9. **Extra rows**: Array with 3 additional rows beyond declared length +10. **Width mismatch**: Inconsistent field count (missing salary in row 10) +11. **Missing fields**: Systematic field omissions (no email in multiple rows) + #### Question Types -204 questions are generated dynamically across four categories: +209 questions are generated dynamically across five categories: - **Field retrieval (33%)**: Direct value lookups or values that can be read straight off a record (including booleans and simple counts such as array lengths) - Example: "What is Alice's salary?" โ†’ `75000` - Example: "How many items are in order ORD-0042?" โ†’ `3` - Example: "What is the customer name for order ORD-0042?" โ†’ `John Doe` -- **Aggregation (31%)**: Dataset-level totals and averages plus single-condition filters (counts, sums, min/max comparisons) +- **Aggregation (30%)**: Dataset-level totals and averages plus single-condition filters (counts, sums, min/max comparisons) - Example: "How many employees work in Engineering?" โ†’ `17` - Example: "What is the total revenue across all orders?" โ†’ `45123.50` - Example: "How many employees have salary > 80000?" โ†’ `23` -- **Filtering (24%)**: Multi-condition queries requiring compound logic (AND constraints across fields) +- **Filtering (23%)**: Multi-condition queries requiring compound logic (AND constraints across fields) - Example: "How many employees in Sales have salary > 80000?" โ†’ `5` - Example: "How many active employees have more than 10 years of experience?" โ†’ `8` @@ -247,17 +317,22 @@ Six datasets designed to test different structural patterns: - Example: "List the field names for employees" โ†’ `id, name, email, department, salary, yearsExperience, active` - Example: "What is the department of the last employee?" โ†’ `Sales` +- **Structural validation (2%)**: Tests ability to detect incomplete, truncated, or corrupted data using structural metadata + - Example: "Is this data complete and valid?" โ†’ `YES` (control dataset) or `NO` (corrupted datasets) + - Tests TOON's [N] length validation and {fields} consistency checking + - Demonstrates CSV's lack of structural validation capabilities + #### Evaluation Process -1. **Format conversion**: Each dataset is converted to all 6 formats (TOON, JSON compact, JSON, YAML, XML, CSV). +1. **Format conversion**: Each dataset is converted to all 6 formats (TOON, JSON compact, JSON, CSV, YAML, XML). 2. **Query LLM**: Each model receives formatted data + question in a prompt and extracts the answer. -3. **Validate with LLM-as-judge**: `gpt-5-nano` validates if the answer is semantically correct (e.g., `50000` = `$50,000`, `Engineering` = `engineering`, `2025-01-01` = `January 1, 2025`). +3. **Validate deterministically**: Answers are validated using type-aware comparison (e.g., `50000` = `$50,000`, `Engineering` = `engineering`, `2025-01-01` = `January 1, 2025`) without requiring an LLM judge. #### Models & Configuration - **Models tested**: `claude-haiku-4-5-20251001`, `gemini-2.5-flash`, `gpt-5-nano`, `grok-4-fast-non-reasoning` - **Token counting**: Using `gpt-tokenizer` with `o200k_base` encoding (GPT-5 tokenizer) - **Temperature**: Not set (models use their defaults) -- **Total evaluations**: 204 questions ร— 6 formats ร— 4 models = 4,896 LLM calls +- **Total evaluations**: 209 questions ร— 6 formats ร— 4 models = 5,016 LLM calls
diff --git a/benchmarks/results/token-efficiency.md b/benchmarks/results/token-efficiency.md index 999a853..999d1ba 100644 --- a/benchmarks/results/token-efficiency.md +++ b/benchmarks/results/token-efficiency.md @@ -5,19 +5,19 @@ Datasets with nested or semi-uniform structures. CSV excluded as it cannot prope ``` ๐Ÿ›’ E-commerce orders with nested structures โ”Š Tabular: 33% โ”‚ - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 72,743 tokens - โ”œโ”€ vs JSON (โˆ’33.1%) 108,731 tokens - โ”œโ”€ vs JSON compact (+5.5%) 68,936 tokens - โ”œโ”€ vs YAML (โˆ’14.1%) 84,724 tokens - โ””โ”€ vs XML (โˆ’40.5%) 122,313 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘ 72,771 tokens + โ”œโ”€ vs JSON (โˆ’33.1%) 108,806 tokens + โ”œโ”€ vs JSON compact (+5.5%) 68,975 tokens + โ”œโ”€ vs YAML (โˆ’14.2%) 84,780 tokens + โ””โ”€ vs XML (โˆ’40.5%) 122,406 tokens ๐Ÿงพ Semi-uniform event logs โ”Š Tabular: 50% โ”‚ - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 153,223 tokens - โ”œโ”€ vs JSON (โˆ’15.0%) 180,196 tokens - โ”œโ”€ vs JSON compact (+19.9%) 127,740 tokens - โ”œโ”€ vs YAML (โˆ’0.8%) 154,514 tokens - โ””โ”€ vs XML (โˆ’25.2%) 204,800 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘ 153,211 tokens + โ”œโ”€ vs JSON (โˆ’15.0%) 180,176 tokens + โ”œโ”€ vs JSON compact (+19.9%) 127,731 tokens + โ”œโ”€ vs YAML (โˆ’0.8%) 154,505 tokens + โ””โ”€ vs XML (โˆ’25.2%) 204,777 tokens ๐Ÿงฉ Deeply nested configuration โ”Š Tabular: 0% โ”‚ @@ -28,11 +28,11 @@ Datasets with nested or semi-uniform structures. CSV excluded as it cannot prope โ””โ”€ vs XML (โˆ’37.4%) 1,008 tokens โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Total โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 226,597 tokens - โ”œโ”€ vs JSON (โˆ’21.8%) 289,846 tokens - โ”œโ”€ vs JSON compact (+14.9%) 197,240 tokens - โ”œโ”€ vs YAML (โˆ’5.5%) 239,911 tokens - โ””โ”€ vs XML (โˆ’30.9%) 328,121 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘โ–‘โ–‘ 226,613 tokens + โ”œโ”€ vs JSON (โˆ’21.8%) 289,901 tokens + โ”œโ”€ vs JSON compact (+14.9%) 197,270 tokens + โ”œโ”€ vs YAML (โˆ’5.6%) 239,958 tokens + โ””โ”€ vs XML (โˆ’31.0%) 328,191 tokens ``` #### Flat-Only Track @@ -42,21 +42,21 @@ Datasets with flat tabular structures where CSV is applicable. ``` ๐Ÿ‘ฅ Uniform employee records โ”Š Tabular: 100% โ”‚ - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 46,956 tokens - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 49,827 tokens (+6.1% vs CSV) - โ”œโ”€ vs JSON (โˆ’60.7%) 126,854 tokens - โ”œโ”€ vs JSON compact (โˆ’36.8%) 78,850 tokens - โ”œโ”€ vs YAML (โˆ’50.0%) 99,701 tokens - โ””โ”€ vs XML (โˆ’66.0%) 146,440 tokens + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 46,954 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 49,831 tokens (+6.1% vs CSV) + โ”œโ”€ vs JSON (โˆ’60.7%) 126,860 tokens + โ”œโ”€ vs JSON compact (โˆ’36.8%) 78,856 tokens + โ”œโ”€ vs YAML (โˆ’50.0%) 99,706 tokens + โ””โ”€ vs XML (โˆ’66.0%) 146,444 tokens ๐Ÿ“ˆ Time-series analytics data โ”Š Tabular: 100% โ”‚ - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 8,396 tokens - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 9,128 tokens (+8.7% vs CSV) - โ”œโ”€ vs JSON (โˆ’59.0%) 22,258 tokens - โ”œโ”€ vs JSON compact (โˆ’35.8%) 14,224 tokens - โ”œโ”€ vs YAML (โˆ’48.9%) 17,871 tokens - โ””โ”€ vs XML (โˆ’65.7%) 26,629 tokens + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘โ–‘ 8,388 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 9,120 tokens (+8.7% vs CSV) + โ”œโ”€ vs JSON (โˆ’59.0%) 22,250 tokens + โ”œโ”€ vs JSON compact (โˆ’35.8%) 14,216 tokens + โ”œโ”€ vs YAML (โˆ’48.9%) 17,863 tokens + โ””โ”€ vs XML (โˆ’65.7%) 26,621 tokens โญ Top 100 GitHub repositories โ”Š Tabular: 100% โ”‚ @@ -68,12 +68,12 @@ Datasets with flat tabular structures where CSV is applicable. โ””โ”€ vs XML (โˆ’48.8%) 17,095 tokens โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Total โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ - CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 63,865 tokens - TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 67,700 tokens (+6.0% vs CSV) - โ”œโ”€ vs JSON (โˆ’58.8%) 164,257 tokens - โ”œโ”€ vs JSON compact (โˆ’35.2%) 104,529 tokens - โ”œโ”€ vs YAML (โˆ’48.2%) 130,701 tokens - โ””โ”€ vs XML (โˆ’64.4%) 190,164 tokens + CSV โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–‘ 63,855 tokens + TOON โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ 67,696 tokens (+6.0% vs CSV) + โ”œโ”€ vs JSON (โˆ’58.8%) 164,255 tokens + โ”œโ”€ vs JSON compact (โˆ’35.2%) 104,527 tokens + โ”œโ”€ vs YAML (โˆ’48.2%) 130,698 tokens + โ””โ”€ vs XML (โˆ’64.4%) 190,160 tokens ```
@@ -83,64 +83,64 @@ Datasets with flat tabular structures where CSV is applicable. **Savings:** 13,130 tokens (59.0% reduction vs JSON) -**JSON** (22,258 tokens): +**JSON** (22,250 tokens): ```json { "metrics": [ { "date": "2025-01-01", - "views": 7708, - "clicks": 595, - "conversions": 69, - "revenue": 15369.93, - "bounceRate": 0.35 + "views": 5715, + "clicks": 211, + "conversions": 28, + "revenue": 7976.46, + "bounceRate": 0.47 }, { "date": "2025-01-02", - "views": 5894, - "clicks": 381, - "conversions": 21, - "revenue": 2112.12, - "bounceRate": 0.3 + "views": 7103, + "clicks": 393, + "conversions": 28, + "revenue": 8360.53, + "bounceRate": 0.32 }, { "date": "2025-01-03", - "views": 6835, - "clicks": 422, - "conversions": 35, - "revenue": 4525.73, + "views": 7248, + "clicks": 378, + "conversions": 24, + "revenue": 3212.57, "bounceRate": 0.5 }, { "date": "2025-01-04", - "views": 5325, - "clicks": 305, - "conversions": 22, - "revenue": 2445.3, - "bounceRate": 0.44 + "views": 2927, + "clicks": 77, + "conversions": 11, + "revenue": 1211.69, + "bounceRate": 0.62 }, { "date": "2025-01-05", - "views": 2974, - "clicks": 61, - "conversions": 6, - "revenue": 956.57, - "bounceRate": 0.47 + "views": 3530, + "clicks": 82, + "conversions": 8, + "revenue": 462.77, + "bounceRate": 0.56 } ] } ``` -**TOON** (9,128 tokens): +**TOON** (9,120 tokens): ``` metrics[5]{date,views,clicks,conversions,revenue,bounceRate}: - 2025-01-01,7708,595,69,15369.93,0.35 - 2025-01-02,5894,381,21,2112.12,0.3 - 2025-01-03,6835,422,35,4525.73,0.5 - 2025-01-04,5325,305,22,2445.3,0.44 - 2025-01-05,2974,61,6,956.57,0.47 + 2025-01-01,5715,211,28,7976.46,0.47 + 2025-01-02,7103,393,28,8360.53,0.32 + 2025-01-03,7248,378,24,3212.57,0.5 + 2025-01-04,2927,77,11,1211.69,0.62 + 2025-01-05,3530,82,8,462.77,0.56 ``` --- diff --git a/benchmarks/src/constants.ts b/benchmarks/src/constants.ts index 69d26d3..2136897 100644 --- a/benchmarks/src/constants.ts +++ b/benchmarks/src/constants.ts @@ -56,9 +56,11 @@ export const FORMATTER_DISPLAY_NAMES: Record = { */ export const QUESTION_TYPES = [ 'field-retrieval', + 'retrieval', 'aggregation', 'filtering', 'structure-awareness', + 'structural-validation', ] as const /** @@ -66,9 +68,11 @@ export const QUESTION_TYPES = [ */ export const QUESTION_TYPE_LABELS = { 'field-retrieval': 'Field Retrieval', + 'retrieval': 'Retrieval', 'aggregation': 'Aggregation', 'filtering': 'Filtering', 'structure-awareness': 'Structure Awareness', + 'structural-validation': 'Structural Validation', } as const /** @@ -81,6 +85,12 @@ export const DATASET_NAMES = [ 'github', 'event-logs', 'nested-config', + 'large-uniform', + 'structural-validation-control', + 'structural-validation-truncated', + 'structural-validation-extra-rows', + 'structural-validation-width-mismatch', + 'structural-validation-missing-fields', ] as const /** diff --git a/benchmarks/src/datasets.ts b/benchmarks/src/datasets.ts index e162f6f..441218a 100644 --- a/benchmarks/src/datasets.ts +++ b/benchmarks/src/datasets.ts @@ -144,6 +144,30 @@ export interface NestedConfig { } } +/** + * Product structure for large uniform arrays + */ +export interface Product { + sku: string + name: string + category: string + price: number + qty: number + lastUpdated: string +} + +/** + * Internal types for structural validation pattern generation + */ +type StructuralValidationType = 'truncated' | 'extra-rows' | 'width-mismatch' | 'missing-fields' + +interface StructuralValidationFixture { + type: StructuralValidationType + description: string + data: Record + isValid: boolean +} + /** * Generate analytics time-series data */ @@ -505,6 +529,100 @@ export function generateNestedConfig(): NestedConfig { } } +/** + * Generate large uniform product array (5000+ rows) + * + * @remarks + * Tests TOON's token efficiency and structural reliability at scale. + */ +export function generateProducts(count: number): { products: Product[] } { + const categories = ['Electronics', 'Clothing', 'Home & Garden', 'Sports', 'Books', 'Toys'] as const + + return { + products: Array.from({ length: count }, (_, i): Product => ({ + sku: `SKU-${String(i + 1).padStart(6, '0')}`, + name: faker.commerce.productName(), + category: categories[i % categories.length]!, + price: Number(faker.commerce.price({ min: 5, max: 500 })), + qty: faker.number.int({ min: 0, max: 1000 }), + lastUpdated: faker.date.recent({ days: 30 }).toISOString().split('T')[0]!, + })), + } +} + +/** + * Generate structural validation fixtures from employee data + * + * @remarks + * Creates deliberately corrupted datasets to test TOON's structural validation + * capabilities via [N] length declarations and {fields} headers. + * Internal function used to generate structural validation datasets. + */ +function generateStructuralValidationFixtures(): StructuralValidationFixture[] { + const baseData = generateEmployees(20) + + return [ + // Valid baseline + { + type: 'truncated' as const, + description: 'Valid complete dataset (control)', + data: { employees: baseData.employees }, + isValid: true, + }, + // Truncated array (missing last 3 rows) + { + type: 'truncated' as const, + description: 'Array truncated: 3 rows removed from end', + data: { employees: baseData.employees.slice(0, -3) }, + isValid: false, // [N] won't match actual row count in TOON + }, + // Extra rows (3 more than original) + { + type: 'extra-rows' as const, + description: 'Extra rows added beyond declared length', + data: { + employees: [ + ...baseData.employees, + ...generateEmployees(3).employees, + ], + }, + isValid: false, // [N] won't match actual row count in TOON + }, + // Width mismatch (inconsistent field count) + { + type: 'width-mismatch' as const, + description: 'Inconsistent field count (missing salary in row 10)', + data: { + employees: baseData.employees.map((emp, i) => { + if (i === 9) { + // Row 10, missing salary field + const { salary, ...rest } = emp + return rest + } + return emp + }), + }, + isValid: false, // Not all objects have same fields (tabular requirement) + }, + // Missing required fields + { + type: 'missing-fields' as const, + description: 'Missing required fields (no email in multiple rows)', + data: { + employees: baseData.employees.map((emp, i) => { + if (i % 5 === 0) { + // Every 5th row, missing email + const { email, ...rest } = emp + return rest + } + return emp + }), + }, + isValid: false, // Not all objects have same fields (tabular requirement) + }, + ] +} + /** * Event logs dataset: Semi-uniform structure * @@ -539,6 +657,34 @@ const nestedConfigDataset: Dataset = { }, } +/** + * Structural validation datasets: Tests ability to detect incomplete, truncated, or corrupted data + * + * @remarks + * These datasets test TOON's structural validation advantages via [N] length declarations + * and {fields} headers. CSV is included to demonstrate its lack of structural metadata. + */ +const structuralValidationDatasets: Dataset[] = generateStructuralValidationFixtures().map((fixture, index) => { + const datasetNames = [ + 'structural-validation-control', + 'structural-validation-truncated', + 'structural-validation-extra-rows', + 'structural-validation-width-mismatch', + 'structural-validation-missing-fields', + ] as const + + return { + name: datasetNames[index]!, + description: fixture.description, + data: fixture.data, + metadata: { + supportsCSV: true, // Include CSV to show it can't validate structure + structureClass: 'uniform', + tabularEligibility: 100, + }, + } +}) + /** * Datasets for accuracy benchmarks (smaller sizes for faster evaluation) */ @@ -549,6 +695,7 @@ export const ACCURACY_DATASETS: Dataset[] = [ githubDataset, // 100 repos eventLogsDataset, // 75 logs nestedConfigDataset, // 1 config + ...structuralValidationDatasets, // 5 validation fixtures ] /** diff --git a/benchmarks/src/evaluate.ts b/benchmarks/src/evaluate.ts index b801ac7..14b4aca 100644 --- a/benchmarks/src/evaluate.ts +++ b/benchmarks/src/evaluate.ts @@ -5,6 +5,7 @@ import { google } from '@ai-sdk/google' import { openai } from '@ai-sdk/openai' import { xai } from '@ai-sdk/xai' import { generateText } from 'ai' +import { compareAnswers } from './normalize' /** * Models used for evaluation @@ -74,7 +75,13 @@ ${formattedData} Question: ${question.prompt} -Provide only the direct answer, without any additional explanation or formatting. +Answer format requirements: +- Provide only the value itself, no explanation +- For numbers: output digits only (no commas, currency symbols, or units) +- For dates/field names: use the exact string from the data +- For lists: output comma-separated values with no spaces + +Answer: `.trim() const startTime = performance.now() @@ -83,11 +90,13 @@ Provide only the direct answer, without any additional explanation or formatting const actual = text.trim() const latencyMs = performance.now() - startTime - const isCorrect = await validateAnswer({ + const comparisonResult = compareAnswers( actual, - expected: question.groundTruth, - question: question.prompt, - }) + question.groundTruth, + question.answerType ?? 'string', + question.normalizationOptions, + ) + const isCorrect = comparisonResult.match return { questionId: question.id, @@ -101,42 +110,3 @@ Provide only the direct answer, without any additional explanation or formatting latencyMs, } } - -/** - * Validate an answer using LLM-as-judge approach - */ -async function validateAnswer( - { - actual, - expected, - question, - }: - { - actual: string - expected: string - question: string - }, -): Promise { - const prompt = ` -You are validating answers to questions about structured data. - -Question: ${question} -Expected answer: ${expected} -Actual answer: ${actual} - -Is the actual answer correct? Consider: -- Exact matches are correct -- Semantically equivalent answers are correct (e.g., "50000" vs "$50,000" vs "50000 dollars") -- Minor formatting differences are acceptable -- Case-insensitive comparison for text - -Respond with only "YES" or "NO". -`.trim() - - const { text } = await generateText({ - model: models.find(m => m.modelId === 'gpt-5-nano')!, - prompt, - }) - - return text.trim().toUpperCase() === 'YES' -} diff --git a/benchmarks/src/normalize.ts b/benchmarks/src/normalize.ts new file mode 100644 index 0000000..428cd29 --- /dev/null +++ b/benchmarks/src/normalize.ts @@ -0,0 +1,386 @@ +/** + * Type of expected answer for deterministic comparison + */ +export type AnswerType + = | 'integer' + | 'number' + | 'boolean' + | 'date' + | 'string' + | 'csv-list-ordered' + | 'csv-list-unordered' + +/** + * Options for answer normalization and comparison + */ +export interface NormalizationOptions { + /** + * Tolerance for floating-point number comparison (e.g., 1e-6). + * @default 1e-6 + */ + tolerance?: number + + /** + * Whether string comparison should be case-sensitive. + * @default false + */ + caseSensitive?: boolean + + /** + * Allow currency symbols ($, โ‚ฌ, etc.) in number extraction. + * @default true + */ + allowCurrency?: boolean + + /** + * Allow percent signs (%) in number extraction (will divide by 100). + * @default true + */ + allowPercent?: boolean + + /** + * Number of decimal places to round to for number comparison. + * If specified, overrides tolerance-based comparison. + */ + decimalPlaces?: number +} + +interface NormalizedResult { + success: boolean + value?: unknown + error?: string +} + +/** + * Default normalization options + */ +const DEFAULT_OPTIONS: Required = { + tolerance: 1e-6, + caseSensitive: false, + allowCurrency: true, + allowPercent: true, + decimalPlaces: undefined!, +} + +// Regex pattern constants +const INTEGER_PATTERN_WITH_CURRENCY = /[$โ‚ฌยฃยฅ]?\s*-?\d[\d,]*/ +const INTEGER_PATTERN = /-?\d[\d,]*/ +const NUMBER_PATTERN_WITH_CURRENCY = /[$โ‚ฌยฃยฅ]?\s*-?\d[\d,]*(?:\.\d+)?(?:e[+-]?\d+)?%?/i +const NUMBER_PATTERN = /-?\d[\d,]*(?:\.\d+)?(?:e[+-]?\d+)?%?/i +const WRAPPING_QUOTES_PATTERN = /^["']|["']$/g +const CODE_FENCE_PATTERN = /^```[\s\S]*?```$/g +const LANGUAGE_IDENTIFIER_PATTERN = /^\w+\n/ +const CURRENCY_AND_FORMATTING_CHARS = /[$โ‚ฌยฃยฅ,\s]/g +const NUMBER_CLEANUP_CHARS = /[$โ‚ฌยฃยฅ,%\s]/g + +// Boolean value constants +const TRUE_VALUES = new Set(['true', 'yes', 'y', '1']) +const FALSE_VALUES = new Set(['false', 'no', 'n', '0']) + +// Numeric constants +const PERCENTAGE_DIVISOR = 100 +const DECIMAL_BASE = 10 +const MONTH_OFFSET = 1 // JavaScript months are 0-indexed +const DATE_COMPONENT_WIDTH = 2 +const DATE_PAD_CHAR = '0' + +// String constants +const CSV_DELIMITER = ',' + +/** + * Strip wrapping quotes from a string + */ +function stripWrappingQuotes(text: string): string { + return text.trim().replace(WRAPPING_QUOTES_PATTERN, '') +} + +/** + * Extract and normalize an integer from a string + * + * @remarks + * Handles: "42", "1,234", "$5,678", " -99 ", "The answer is 42." + */ +function normalizeInteger(text: string, options: Required): NormalizedResult { + // Strip common formatting, extract first integer-like token + const pattern = options.allowCurrency + ? INTEGER_PATTERN_WITH_CURRENCY + : INTEGER_PATTERN + + const match = text.match(pattern) + if (!match) + return { success: false, error: `No integer found in: "${text}"` } + + // Remove currency symbols, spaces, and thousand separators + const normalizedValue = match[0].replace(CURRENCY_AND_FORMATTING_CHARS, '') + const parsedNumber = Number.parseInt(normalizedValue, DECIMAL_BASE) + + if (Number.isNaN(parsedNumber)) + return { success: false, error: `Failed to parse integer: "${match[0]}"` } + + return { success: true, value: parsedNumber } +} + +/** + * Extract and normalize a floating-point number from a string + * + * @remarks + * Handles: "3.14", "1,234.56", "$5,678.90", "42%", "1.5e-3", "Price: $99.99" + */ +function normalizeNumber(text: string, options: Required): NormalizedResult { + // Extract first number-like token (supports scientific notation) + const pattern = options.allowCurrency + ? NUMBER_PATTERN_WITH_CURRENCY + : NUMBER_PATTERN + + const match = text.match(pattern) + if (!match) + return { success: false, error: `No number found in: "${text}"` } + + const token = match[0] + const hasPercentSign = options.allowPercent && token.endsWith('%') + + // Remove currency, commas, spaces, and percent sign + const normalizedToken = token.replace(NUMBER_CLEANUP_CHARS, '') + let parsedNumber = Number.parseFloat(normalizedToken) + + if (Number.isNaN(parsedNumber)) + return { success: false, error: `Failed to parse number: "${token}"` } + + // Convert percentage to decimal if present + if (hasPercentSign) + parsedNumber = parsedNumber / PERCENTAGE_DIVISOR + + // Round to specified decimal places if requested + if (options.decimalPlaces !== undefined) { + const factor = DECIMAL_BASE ** options.decimalPlaces + parsedNumber = Math.round(parsedNumber * factor) / factor + } + + return { success: true, value: parsedNumber } +} + +/** + * Normalize a boolean/yes-no answer + * + * @remarks + * Handles: "true", "false", "yes", "no", "y", "n", "1", "0" (case-insensitive) + */ +function normalizeBoolean(text: string): NormalizedResult { + const normalizedValue = text.trim().toLowerCase() + + if (TRUE_VALUES.has(normalizedValue)) + return { success: true, value: true } + + if (FALSE_VALUES.has(normalizedValue)) + return { success: true, value: false } + + return { success: false, error: `Not a boolean: "${text}"` } +} + +/** + * Normalize a date string to YYYY-MM-DD format + * + * @remarks + * Handles: ISO dates, "Nov 1, 2025", "2025-11-01", RFC 2822, etc. + */ +function normalizeDate(text: string): NormalizedResult { + const cleaned = stripWrappingQuotes(text) + + // Try parsing as date + const parsedDate = new Date(cleaned) + if (Number.isNaN(parsedDate.getTime())) + return { success: false, error: `Invalid date: "${text}"` } + + // Normalize to YYYY-MM-DD (UTC) + const year = parsedDate.getUTCFullYear() + const monthPadded = String(parsedDate.getUTCMonth() + MONTH_OFFSET).padStart(DATE_COMPONENT_WIDTH, DATE_PAD_CHAR) + const dayPadded = String(parsedDate.getUTCDate()).padStart(DATE_COMPONENT_WIDTH, DATE_PAD_CHAR) + const normalized = `${year}-${monthPadded}-${dayPadded}` + + return { success: true, value: normalized } +} + +/** + * Normalize a string (trim, optionally case-insensitive) + * + * @remarks + * Handles wrapping quotes and code fences. + */ +function normalizeString(text: string, options: Required): NormalizedResult { + let trimmedText = text.trim() + + // Strip wrapping quotes + trimmedText = trimmedText.replace(WRAPPING_QUOTES_PATTERN, '') + + // Strip code fences (```...```) + trimmedText = trimmedText.replace(CODE_FENCE_PATTERN, (match) => { + const inner = match.slice(3, -3).trim() + // Remove language identifier if present (e.g., ```json) + return inner.replace(LANGUAGE_IDENTIFIER_PATTERN, '') + }) + + trimmedText = trimmedText.trim() + + const value = options.caseSensitive ? trimmedText : trimmedText.toLowerCase() + return { success: true, value } +} + +/** + * Normalize a comma-separated list (ordered) + * + * @remarks + * Handles: "a,b,c", "a, b, c", " a , b , c " + */ +function normalizeCsvListOrdered(text: string, options: Required): NormalizedResult { + const strippedText = stripWrappingQuotes(text) + const items = strippedText + .split(CSV_DELIMITER) + .map(item => item.trim()) + .filter(item => item.length > 0) + + const normalizedItems = items.map(item => + options.caseSensitive ? item : item.toLowerCase(), + ) + + return { success: true, value: normalizedItems } +} + +/** + * Normalize a comma-separated list (unordered, compare as sets) + * + * @remarks + * Handles: "c,a,b" equals "a,b,c" + */ +function normalizeCsvListUnordered(text: string, options: Required): NormalizedResult { + const result = normalizeCsvListOrdered(text, options) + if (!result.success) + return result + + // Type guard: ensure result.value is an array + if (!Array.isArray(result.value)) + return { success: false, error: 'Expected array result from normalizeCsvListOrdered' } + + // Sort for deterministic comparison + const sorted = [...result.value].sort() + return { success: true, value: sorted } +} + +/** + * Normalize a value based on its expected kind + */ +export function normalizeAnswer( + text: string, + kind: AnswerType, + options: Partial = {}, +): NormalizedResult { + const resolvedOptions: Required = { ...DEFAULT_OPTIONS, ...options } + + switch (kind) { + case 'integer': + return normalizeInteger(text, resolvedOptions) + case 'number': + return normalizeNumber(text, resolvedOptions) + case 'boolean': + return normalizeBoolean(text) + case 'date': + return normalizeDate(text) + case 'string': + return normalizeString(text, resolvedOptions) + case 'csv-list-ordered': + return normalizeCsvListOrdered(text, resolvedOptions) + case 'csv-list-unordered': + return normalizeCsvListUnordered(text, resolvedOptions) + default: + return { success: false, error: `Unknown answer kind: ${kind}` } + } +} + +/** + * Compare two normalized values based on answer kind + */ +function compareValues( + actual: unknown, + expected: unknown, + kind: AnswerType, + options: Required, +): boolean { + switch (kind) { + case 'integer': + case 'boolean': + case 'date': + case 'string': + return actual === expected + + case 'number': + if (typeof actual !== 'number' || typeof expected !== 'number') + return false + + if (options.decimalPlaces !== undefined) { + // Already rounded during normalization + return actual === expected + } + return Math.abs(actual - expected) <= options.tolerance + + case 'csv-list-ordered': + if (!Array.isArray(actual) || !Array.isArray(expected)) + return false + if (actual.length !== expected.length) + return false + return actual.every((item, i) => item === expected[i]) + + case 'csv-list-unordered': + if (!Array.isArray(actual) || !Array.isArray(expected)) + return false + if (actual.length !== expected.length) + return false + // Already sorted during normalization + return actual.every((item, i) => item === expected[i]) + + default: + return false + } +} + +/** + * Compare actual and expected answers with deterministic, type-aware normalization + * + * @remarks + * Returns true if answers match within the specified tolerance/rules. + */ +export function compareAnswers( + actual: string, + expected: string, + kind: AnswerType, + options: Partial = {}, +): { match: boolean, details?: string } { + const resolvedOptions: Required = { ...DEFAULT_OPTIONS, ...options } + + // Normalize both answers + const actualResult = normalizeAnswer(actual, kind, resolvedOptions) + const expectedResult = normalizeAnswer(expected, kind, resolvedOptions) + + // If either normalization failed, return false with details + if (!actualResult.success) { + return { + match: false, + details: `Failed to normalize actual answer: ${actualResult.error}`, + } + } + + if (!expectedResult.success) { + return { + match: false, + details: `Failed to normalize expected answer: ${expectedResult.error}`, + } + } + + // Compare normalized values + const match = compareValues(actualResult.value, expectedResult.value, kind, resolvedOptions) + + return { + match, + details: match + ? undefined + : `Mismatch: actual="${actualResult.value}" vs expected="${expectedResult.value}"`, + } +} diff --git a/benchmarks/src/questions/analytics.ts b/benchmarks/src/questions/analytics.ts index c08b263..f540704 100644 --- a/benchmarks/src/questions/analytics.ts +++ b/benchmarks/src/questions/analytics.ts @@ -17,6 +17,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(metric.views)) .type('field-retrieval') .dataset('analytics') + .answerType('integer') .build(), (metric, getId) => new QuestionBuilder() .id(getId()) @@ -24,6 +25,8 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(metric.revenue)) .type('field-retrieval') .dataset('analytics') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), (metric, getId) => new QuestionBuilder() .id(getId()) @@ -31,6 +34,8 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(metric.bounceRate)) .type('field-retrieval') .dataset('analytics') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), (metric, getId) => new QuestionBuilder() .id(getId()) @@ -38,6 +43,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(metric.conversions)) .type('field-retrieval') .dataset('analytics') + .answerType('integer') .build(), ] @@ -63,6 +69,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(totalDays)) .type('aggregation') .dataset('analytics') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -70,6 +77,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(totalViews)) .type('aggregation') .dataset('analytics') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -77,6 +85,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(totalConversions)) .type('aggregation') .dataset('analytics') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -84,6 +93,8 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(totalRevenue.toFixed(2))) .type('aggregation') .dataset('analytics') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), new QuestionBuilder() .id(getId()) @@ -91,6 +102,8 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(avgBounceRate.toFixed(2))) .type('aggregation') .dataset('analytics') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), ) @@ -104,6 +117,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(count)) .type('aggregation') .dataset('analytics') + .answerType('integer') .build(), ) } @@ -117,6 +131,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(count)) .type('aggregation') .dataset('analytics') + .answerType('integer') .build(), ) } @@ -133,6 +148,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(count)) .type('filtering') .dataset('analytics') + .answerType('integer') .build(), ) } @@ -149,6 +165,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(count)) .type('filtering') .dataset('analytics') + .answerType('integer') .build(), ) } @@ -165,6 +182,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(count)) .type('filtering') .dataset('analytics') + .answerType('integer') .build(), ) } @@ -181,6 +199,7 @@ export function generateAnalyticsQuestions(metrics: AnalyticsMetric[], getId: () .groundTruth(String(count)) .type('filtering') .dataset('analytics') + .answerType('integer') .build(), ) } diff --git a/benchmarks/src/questions/event-logs.ts b/benchmarks/src/questions/event-logs.ts index 6ca019c..f9cd32c 100644 --- a/benchmarks/src/questions/event-logs.ts +++ b/benchmarks/src/questions/event-logs.ts @@ -17,6 +17,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(log.level) .type('field-retrieval') .dataset('event-logs') + .answerType('string') .build(), (log, getId) => new QuestionBuilder() .id(getId()) @@ -24,6 +25,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(log.endpoint) .type('field-retrieval') .dataset('event-logs') + .answerType('string') .build(), (log, getId) => new QuestionBuilder() .id(getId()) @@ -31,6 +33,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(log.statusCode)) .type('field-retrieval') .dataset('event-logs') + .answerType('integer') .build(), (log, getId) => new QuestionBuilder() .id(getId()) @@ -38,6 +41,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(log.responseTime)) .type('field-retrieval') .dataset('event-logs') + .answerType('integer') .build(), ] @@ -60,6 +64,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(totalLogs)) .type('aggregation') .dataset('event-logs') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -67,6 +72,8 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(avgResponseTime.toFixed(2))) .type('aggregation') .dataset('event-logs') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), ) @@ -81,6 +88,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(count)) .type('aggregation') .dataset('event-logs') + .answerType('integer') .build(), ) } @@ -96,6 +104,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(count)) .type('aggregation') .dataset('event-logs') + .answerType('integer') .build(), ) } @@ -111,6 +120,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(errorCount)) .type('aggregation') .dataset('event-logs') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -118,6 +128,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(successCount)) .type('aggregation') .dataset('event-logs') + .answerType('integer') .build(), ) @@ -130,6 +141,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(retryableErrorCount)) .type('aggregation') .dataset('event-logs') + .answerType('integer') .build(), ) @@ -147,6 +159,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(count)) .type('filtering') .dataset('event-logs') + .answerType('integer') .build(), ) } @@ -161,6 +174,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(count)) .type('filtering') .dataset('event-logs') + .answerType('integer') .build(), ) } @@ -175,6 +189,7 @@ export function generateEventLogsQuestions(logs: EventLog[], getId: () => string .groundTruth(String(count)) .type('filtering') .dataset('event-logs') + .answerType('integer') .build(), ) } diff --git a/benchmarks/src/questions/github.ts b/benchmarks/src/questions/github.ts index b588bf9..7f656ad 100644 --- a/benchmarks/src/questions/github.ts +++ b/benchmarks/src/questions/github.ts @@ -17,6 +17,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(repo.stars)) .type('field-retrieval') .dataset('github') + .answerType('integer') .build(), (repo, getId) => new QuestionBuilder() .id(getId()) @@ -24,6 +25,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(repo.forks)) .type('field-retrieval') .dataset('github') + .answerType('integer') .build(), (repo, getId) => new QuestionBuilder() .id(getId()) @@ -31,6 +33,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(repo.watchers)) .type('field-retrieval') .dataset('github') + .answerType('integer') .build(), (repo, getId) => new QuestionBuilder() .id(getId()) @@ -38,6 +41,8 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(repo.defaultBranch) .type('field-retrieval') .dataset('github') + .answerType('string') + .normalize({ caseSensitive: true }) .build(), ] @@ -62,6 +67,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(totalRepos)) .type('aggregation') .dataset('github') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -69,6 +75,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(totalStars)) .type('aggregation') .dataset('github') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -76,6 +83,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(totalForks)) .type('aggregation') .dataset('github') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -83,6 +91,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(Math.round(avgStars))) .type('aggregation') .dataset('github') + .answerType('integer') .build(), ) @@ -97,6 +106,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(count)) .type('aggregation') .dataset('github') + .answerType('integer') .build(), ) } @@ -111,6 +121,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(count)) .type('aggregation') .dataset('github') + .answerType('integer') .build(), ) } @@ -125,6 +136,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(count)) .type('aggregation') .dataset('github') + .answerType('integer') .build(), ) } @@ -139,6 +151,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(count)) .type('aggregation') .dataset('github') + .answerType('integer') .build(), ) } @@ -155,6 +168,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(count)) .type('filtering') .dataset('github') + .answerType('integer') .build(), ) } @@ -171,6 +185,7 @@ export function generateGithubQuestions(repos: Repository[], getId: () => string .groundTruth(String(count)) .type('filtering') .dataset('github') + .answerType('integer') .build(), ) } diff --git a/benchmarks/src/questions/index.ts b/benchmarks/src/questions/index.ts index 87845a2..bfd5f8c 100644 --- a/benchmarks/src/questions/index.ts +++ b/benchmarks/src/questions/index.ts @@ -6,6 +6,7 @@ import { generateEventLogsQuestions } from './event-logs' import { generateGithubQuestions } from './github' import { generateNestedQuestions } from './nested' import { generateNestedConfigQuestions } from './nested-config' +import { generateStructuralValidationQuestions } from './structural-validation' import { generateStructureQuestions } from './structure' import { generateTabularQuestions } from './tabular' import { createIdGenerator } from './utils' @@ -47,5 +48,8 @@ export function generateQuestions(): Question[] { // Generate structure-awareness questions (tests format-native affordances) questions.push(...generateStructureQuestions(tabular, nested, analytics, github, eventLogs, getId)) + // Generate structural-validation questions (tests ability to detect corrupted data) + questions.push(...generateStructuralValidationQuestions(getId)) + return questions } diff --git a/benchmarks/src/questions/nested-config.ts b/benchmarks/src/questions/nested-config.ts index 194c3fe..23c9f0a 100644 --- a/benchmarks/src/questions/nested-config.ts +++ b/benchmarks/src/questions/nested-config.ts @@ -17,42 +17,52 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, { prompt: 'What is the environment in the configuration?', groundTruth: config.environment, + answerType: 'string' as const, }, { prompt: 'What is the database host?', groundTruth: config.database.host, + answerType: 'string' as const, }, { prompt: 'What is the database port?', groundTruth: String(config.database.port), + answerType: 'integer' as const, }, { prompt: 'What is the maximum connection pool size?', groundTruth: String(config.database.pool.max), + answerType: 'integer' as const, }, { prompt: 'What is the session duration?', groundTruth: String(config.authentication.session.duration), + answerType: 'integer' as const, }, { prompt: 'What is the minimum connection pool size?', groundTruth: String(config.database.pool.min), + answerType: 'integer' as const, }, { prompt: 'What is the connection pool idle timeout?', groundTruth: String(config.database.pool.idleTimeout), + answerType: 'integer' as const, }, { prompt: 'What is the database name?', groundTruth: config.database.name, + answerType: 'string' as const, }, { prompt: 'What is the session refresh threshold?', groundTruth: String(config.authentication.session.refreshThreshold), + answerType: 'integer' as const, }, { prompt: 'What is the version in the configuration?', groundTruth: config.version, + answerType: 'string' as const, }, ] @@ -64,6 +74,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(q.groundTruth) .type('field-retrieval') .dataset('nested-config') + .answerType(q.answerType) .build(), ) } @@ -82,6 +93,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(roleCount)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -89,6 +101,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(groupCount)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -96,6 +109,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(providerCount)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -103,6 +117,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(featureCount)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -110,6 +125,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(replicaCount)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), ) @@ -122,6 +138,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(adminScopeProviderCount)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), ) @@ -134,6 +151,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(enabledFeatures)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), ) @@ -146,6 +164,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(adminPermissions)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), ) @@ -164,6 +183,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(totalPermissions)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -171,6 +191,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(distinctPermissions)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -178,6 +199,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(totalVariants)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -185,6 +207,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(highPriorityReplicas)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -192,6 +215,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(featuresWithHighRollout)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -199,6 +223,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(String(groupsWithMultipleRoles)) .type('aggregation') .dataset('nested-config') + .answerType('integer') .build(), ) @@ -249,6 +274,7 @@ export function generateNestedConfigQuestions(config: NestedConfig | undefined, .groundTruth(q.groundTruth) .type('filtering') .dataset('nested-config') + .answerType('integer') .build(), ) } diff --git a/benchmarks/src/questions/nested.ts b/benchmarks/src/questions/nested.ts index de16c55..70a16e0 100644 --- a/benchmarks/src/questions/nested.ts +++ b/benchmarks/src/questions/nested.ts @@ -17,6 +17,8 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(order.total)) .type('field-retrieval') .dataset('nested') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), (order, getId) => new QuestionBuilder() .id(getId()) @@ -24,6 +26,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(order.status) .type('field-retrieval') .dataset('nested') + .answerType('string') .build(), ] @@ -43,6 +46,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(order.customer.name) .type('field-retrieval') .dataset('nested') + .answerType('string') .build(), (order, getId) => new QuestionBuilder() .id(getId()) @@ -50,6 +54,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(order.customer.email) .type('field-retrieval') .dataset('nested') + .answerType('string') .build(), (order, getId) => new QuestionBuilder() .id(getId()) @@ -57,6 +62,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(order.orderDate || '') .type('field-retrieval') .dataset('nested') + .answerType('string') .build(), (order, getId) => new QuestionBuilder() .id(getId()) @@ -64,6 +70,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(order.items.length)) .type('field-retrieval') .dataset('nested') + .answerType('integer') .build(), ] @@ -94,6 +101,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(count)) .type('aggregation') .dataset('nested') + .answerType('integer') .build(), ) } @@ -105,6 +113,8 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(totalRevenue.toFixed(2))) .type('aggregation') .dataset('nested') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), new QuestionBuilder() .id(getId()) @@ -112,6 +122,8 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(avgOrderValue.toFixed(2))) .type('aggregation') .dataset('nested') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), new QuestionBuilder() .id(getId()) @@ -119,6 +131,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(totalOrders)) .type('aggregation') .dataset('nested') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -126,6 +139,8 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(maxOrderValue.toFixed(2))) .type('aggregation') .dataset('nested') + .answerType('number') + .normalize({ decimalPlaces: 2 }) .build(), ) @@ -139,6 +154,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(count)) .type('aggregation') .dataset('nested') + .answerType('integer') .build(), ) } @@ -156,6 +172,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(count)) .type('filtering') .dataset('nested') + .answerType('integer') .build(), ) } @@ -172,6 +189,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(count)) .type('filtering') .dataset('nested') + .answerType('integer') .build(), ) } @@ -188,6 +206,7 @@ export function generateNestedQuestions(orders: Order[], getId: () => string): Q .groundTruth(String(count)) .type('filtering') .dataset('nested') + .answerType('integer') .build(), ) } diff --git a/benchmarks/src/questions/structural-validation.ts b/benchmarks/src/questions/structural-validation.ts new file mode 100644 index 0000000..29e6800 --- /dev/null +++ b/benchmarks/src/questions/structural-validation.ts @@ -0,0 +1,44 @@ +import type { Question } from '../types' +import { QuestionBuilder } from './utils' + +/** + * Generate structural validation questions for all incompleteness fixtures + * + * These questions test the ability to detect incomplete, truncated, or corrupted data + * by validating structural metadata (TOON's [N] length declarations and {fields} headers). + * + * @remarks + * - TOON's advantage: Explicit [N] and {fields} enable validation + * - CSV disadvantage: No structural metadata to validate against + * - JSON/YAML disadvantage: Require manual counting and schema inference + */ +export function generateStructuralValidationQuestions( + getId: () => string, +): Question[] { + const questions: Question[] = [] + + // Dataset names and their expected validity + const validationFixtures = [ + { dataset: 'structural-validation-control', isValid: true, description: 'Valid complete dataset (control)' }, + { dataset: 'structural-validation-truncated', isValid: false, description: 'Array truncated: 3 rows removed from end' }, + { dataset: 'structural-validation-extra-rows', isValid: false, description: 'Extra rows added beyond declared length' }, + { dataset: 'structural-validation-width-mismatch', isValid: false, description: 'Inconsistent field count (missing salary in row 10)' }, + { dataset: 'structural-validation-missing-fields', isValid: false, description: 'Missing required fields (no email in multiple rows)' }, + ] as const + + // Generate one validation question per fixture + for (const fixture of validationFixtures) { + questions.push( + new QuestionBuilder() + .id(getId()) + .prompt('Is this data complete and valid? Answer only YES or NO.') + .groundTruth(fixture.isValid ? 'YES' : 'NO') + .type('structural-validation') + .dataset(fixture.dataset) + .answerType('boolean') + .build(), + ) + } + + return questions +} diff --git a/benchmarks/src/questions/structure.ts b/benchmarks/src/questions/structure.ts index 0793920..abb357b 100644 --- a/benchmarks/src/questions/structure.ts +++ b/benchmarks/src/questions/structure.ts @@ -30,6 +30,7 @@ export function generateStructureQuestions( .groundTruth(String(employees.length)) .type('structure-awareness') .dataset('tabular') + .answerType('integer') .build(), ) @@ -42,6 +43,7 @@ export function generateStructureQuestions( .groundTruth(employeeFields) .type('structure-awareness') .dataset('tabular') + .answerType('csv-list-ordered') .build(), ) @@ -53,6 +55,7 @@ export function generateStructureQuestions( .groundTruth('email') .type('structure-awareness') .dataset('tabular') + .answerType('string') .build(), ) @@ -65,6 +68,7 @@ export function generateStructureQuestions( .groundTruth(lastEmployee.department) .type('structure-awareness') .dataset('tabular') + .answerType('string') .build(), ) @@ -76,6 +80,7 @@ export function generateStructureQuestions( .groundTruth(lastEmployee.name) .type('structure-awareness') .dataset('tabular') + .answerType('string') .build(), ) @@ -87,6 +92,7 @@ export function generateStructureQuestions( .groundTruth('7') .type('structure-awareness') .dataset('tabular') + .answerType('integer') .build(), ) @@ -100,6 +106,7 @@ export function generateStructureQuestions( .groundTruth(String(orders.length)) .type('structure-awareness') .dataset('nested') + .answerType('integer') .build(), ) @@ -112,6 +119,7 @@ export function generateStructureQuestions( .groundTruth(orderFields) .type('structure-awareness') .dataset('nested') + .answerType('csv-list-ordered') .build(), ) @@ -126,6 +134,7 @@ export function generateStructureQuestions( .groundTruth(String(orderWithManyItems.items.length)) .type('structure-awareness') .dataset('nested') + .answerType('integer') .build(), ) @@ -138,6 +147,7 @@ export function generateStructureQuestions( .groundTruth(itemFields) .type('structure-awareness') .dataset('nested') + .answerType('csv-list-ordered') .build(), ) @@ -150,6 +160,7 @@ export function generateStructureQuestions( .groundTruth(lastOrder.status) .type('structure-awareness') .dataset('nested') + .answerType('string') .build(), ) @@ -162,6 +173,7 @@ export function generateStructureQuestions( .groundTruth(customerFields) .type('structure-awareness') .dataset('nested') + .answerType('csv-list-ordered') .build(), ) @@ -175,6 +187,7 @@ export function generateStructureQuestions( .groundTruth(String(metrics.length)) .type('structure-awareness') .dataset('analytics') + .answerType('integer') .build(), ) @@ -187,6 +200,7 @@ export function generateStructureQuestions( .groundTruth(metricFields) .type('structure-awareness') .dataset('analytics') + .answerType('csv-list-ordered') .build(), ) @@ -198,6 +212,7 @@ export function generateStructureQuestions( .groundTruth('revenue') .type('structure-awareness') .dataset('analytics') + .answerType('string') .build(), ) @@ -210,6 +225,7 @@ export function generateStructureQuestions( .groundTruth(lastMetric.date) .type('structure-awareness') .dataset('analytics') + .answerType('string') .build(), ) @@ -221,6 +237,7 @@ export function generateStructureQuestions( .groundTruth('6') .type('structure-awareness') .dataset('analytics') + .answerType('integer') .build(), ) @@ -234,6 +251,7 @@ export function generateStructureQuestions( .groundTruth(String(repos.length)) .type('structure-awareness') .dataset('github') + .answerType('integer') .build(), ) @@ -246,6 +264,7 @@ export function generateStructureQuestions( .groundTruth(repoFields) .type('structure-awareness') .dataset('github') + .answerType('csv-list-ordered') .build(), ) @@ -257,6 +276,7 @@ export function generateStructureQuestions( .groundTruth('forks') .type('structure-awareness') .dataset('github') + .answerType('string') .build(), ) @@ -269,6 +289,7 @@ export function generateStructureQuestions( .groundTruth(lastRepo.name) .type('structure-awareness') .dataset('github') + .answerType('string') .build(), ) @@ -280,6 +301,7 @@ export function generateStructureQuestions( .groundTruth('11') .type('structure-awareness') .dataset('github') + .answerType('integer') .build(), ) @@ -293,6 +315,7 @@ export function generateStructureQuestions( .groundTruth(String(logs.length)) .type('structure-awareness') .dataset('event-logs') + .answerType('integer') .build(), ) @@ -305,6 +328,7 @@ export function generateStructureQuestions( .groundTruth(logFields) .type('structure-awareness') .dataset('event-logs') + .answerType('csv-list-unordered') .build(), ) @@ -317,6 +341,7 @@ export function generateStructureQuestions( .groundTruth(lastLog.level) .type('structure-awareness') .dataset('event-logs') + .answerType('string') .build(), ) diff --git a/benchmarks/src/questions/tabular.ts b/benchmarks/src/questions/tabular.ts index b9a5a01..8776b9a 100644 --- a/benchmarks/src/questions/tabular.ts +++ b/benchmarks/src/questions/tabular.ts @@ -17,6 +17,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(emp.salary)) .type('field-retrieval') .dataset('tabular') + .answerType('integer') .build(), (emp, getId) => new QuestionBuilder() .id(getId()) @@ -24,6 +25,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(emp.department) .type('field-retrieval') .dataset('tabular') + .answerType('string') .build(), (emp, getId) => new QuestionBuilder() .id(getId()) @@ -31,6 +33,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(emp.email) .type('field-retrieval') .dataset('tabular') + .answerType('string') .build(), (emp, getId) => new QuestionBuilder() .id(getId()) @@ -38,6 +41,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(emp.yearsExperience)) .type('field-retrieval') .dataset('tabular') + .answerType('integer') .build(), (emp, getId) => new QuestionBuilder() .id(getId()) @@ -45,6 +49,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(emp.active ? 'yes' : 'no') .type('field-retrieval') .dataset('tabular') + .answerType('boolean') .build(), ] @@ -67,6 +72,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(count)) .type('aggregation') .dataset('tabular') + .answerType('integer') .build(), ) } @@ -81,6 +87,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(count)) .type('aggregation') .dataset('tabular') + .answerType('integer') .build(), ) } @@ -98,6 +105,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(totalEmployees)) .type('aggregation') .dataset('tabular') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -105,6 +113,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(avgSalary)) .type('aggregation') .dataset('tabular') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -112,6 +121,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(activeCount)) .type('aggregation') .dataset('tabular') + .answerType('integer') .build(), new QuestionBuilder() .id(getId()) @@ -119,6 +129,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(inactiveCount)) .type('aggregation') .dataset('tabular') + .answerType('integer') .build(), ) @@ -134,6 +145,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(count)) .type('filtering') .dataset('tabular') + .answerType('integer') .build(), ) } @@ -148,6 +160,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(count)) .type('filtering') .dataset('tabular') + .answerType('integer') .build(), ) } @@ -164,6 +177,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(count)) .type('filtering') .dataset('tabular') + .answerType('integer') .build(), ) } @@ -178,6 +192,7 @@ export function generateTabularQuestions(employees: Employee[], getId: () => str .groundTruth(String(count)) .type('filtering') .dataset('tabular') + .answerType('integer') .build(), ) } diff --git a/benchmarks/src/questions/utils.ts b/benchmarks/src/questions/utils.ts index e3004e6..87e3fad 100644 --- a/benchmarks/src/questions/utils.ts +++ b/benchmarks/src/questions/utils.ts @@ -1,3 +1,4 @@ +import type { AnswerType, NormalizationOptions } from '../normalize' import type { Question } from '../types' // Constants for sampling strides @@ -52,10 +53,21 @@ export class QuestionBuilder { return this } + answerType(kind: AnswerType): this { + this.question.answerType = kind + return this + } + + normalize(options: Partial): this { + this.question.normalizationOptions = options + return this + } + build(): Question { if (!this.question.id || !this.question.prompt || !this.question.groundTruth || !this.question.type || !this.question.dataset) { throw new Error('Incomplete question') } + return this.question as Question } } @@ -65,7 +77,7 @@ export class QuestionBuilder { */ export function rotateQuestions( items: T[], - generators: Array<(item: T, getId: () => string) => Question>, + generators: ((item: T, getId: () => string) => Question)[], limit: number, stride: number, getId: () => string, diff --git a/benchmarks/src/report.ts b/benchmarks/src/report.ts index a0e79e9..cb8f0e1 100644 --- a/benchmarks/src/report.ts +++ b/benchmarks/src/report.ts @@ -1,7 +1,7 @@ import type { Dataset, EfficiencyRanking, EvaluationResult, FormatResult, Question } from './types' import { FORMATTER_DISPLAY_NAMES, QUESTION_TYPE_LABELS, QUESTION_TYPES } from './constants' import { ACCURACY_DATASETS } from './datasets' -import { models } from './evaluate' +import { models, PRIMERS } from './evaluate' import { supportsCSV } from './formatters' import { generateQuestions } from './questions' import { createProgressBar, tokenize } from './utils' @@ -10,6 +10,9 @@ const EFFICIENCY_CHART_STYLE: 'vertical' | 'horizontal' = 'horizontal' /** * Calculate token counts for all format+dataset combinations + * + * @remarks + * Includes primer tokens for fairer comparison across formats */ export function calculateTokenCounts( formatters: Record string>, @@ -23,8 +26,11 @@ export function calculateTokenCounts( continue const formattedData = formatter(dataset.data) + const primer = PRIMERS[formatName] ?? '' + // Include primer in token count for fair comparison + const fullPrompt = primer ? `${primer}\n\n${formattedData}` : formattedData const key = `${formatName}-${dataset.name}` - tokenCounts[key] = tokenize(formattedData) + tokenCounts[key] = tokenize(fullPrompt) } } @@ -137,9 +143,12 @@ function generateEfficiencyRankingReport( ): string { const toon = formatResults.find(r => r.format === 'toon') const json = formatResults.find(r => r.format === 'json-pretty') + const csv = formatResults.find(r => r.format === 'csv') // Build efficiency ranking (accuracy per 1k tokens) const efficiencyRanking = formatResults + // Exclude CSV since it only supports a subset of datasets (~half the questions) + .filter(fr => fr.format !== 'csv') .map((fr) => { const efficiency = (fr.accuracy * 100) / (fr.totalTokens / 1000) return { @@ -163,6 +172,12 @@ function generateEfficiencyRankingReport( summary = `TOON achieves ${toonVsJson} while using ${tokenSavings}.` } + // Add CSV note if available + let csvNote = '' + if (csv) { + csvNote = `\n\n**Note on CSV:** Excluded from ranking as it only supports ${csv.totalCount}/209 questions (flat tabular data only). While CSV is highly token-efficient for simple tabular data, it cannot represent nested structures that other formats handle.` + } + return ` Each format's overall performance, balancing accuracy against token cost: @@ -170,7 +185,7 @@ Each format's overall performance, balancing accuracy against token cost: ${efficiencyChart} \`\`\` -${summary} +${summary}${csvNote} `.trim() } @@ -210,11 +225,13 @@ function generateDetailedAccuracyReport( const aggregationCount = questions.filter(q => q.type === 'aggregation').length const filteringCount = questions.filter(q => q.type === 'filtering').length const structureAwarenessCount = questions.filter(q => q.type === 'structure-awareness').length + const structuralValidationCount = questions.filter(q => q.type === 'structural-validation').length const fieldRetrievalPercent = ((fieldRetrievalCount / totalQuestions) * 100).toFixed(0) const aggregationPercent = ((aggregationCount / totalQuestions) * 100).toFixed(0) const filteringPercent = ((filteringCount / totalQuestions) * 100).toFixed(0) const structureAwarenessPercent = ((structureAwarenessCount / totalQuestions) * 100).toFixed(0) + const structuralValidationPercent = ((structuralValidationCount / totalQuestions) * 100).toFixed(0) // Calculate dataset sizes const tabularSize = ACCURACY_DATASETS.find(d => d.name === 'tabular')?.data.employees?.length || 0 @@ -263,8 +280,9 @@ This benchmark tests **LLM comprehension and data retrieval accuracy** across di #### Datasets Tested -Six datasets designed to test different structural patterns: +Eleven datasets designed to test different structural patterns and validation capabilities: +**Primary datasets:** 1. **Tabular** (${tabularSize} employee records): Uniform objects with identical fields โ€“ optimal for TOON's tabular format. 2. **Nested** (${nestedSize} e-commerce orders): Complex structures with nested customer objects and item arrays. 3. **Analytics** (${analyticsSize} days of metrics): Time-series data with dates and numeric values. @@ -272,9 +290,16 @@ Six datasets designed to test different structural patterns: 5. **Event Logs** (${eventLogsSize} logs): Semi-uniform data with ~50% flat logs and ~50% with nested error objects. 6. **Nested Config** (${nestedConfigSize} configuration): Deeply nested configuration with minimal tabular eligibility. +**Structural validation datasets:** +7. **Control**: Valid complete dataset (baseline for validation) +8. **Truncated**: Array with 3 rows removed from end (tests [N] length detection) +9. **Extra rows**: Array with 3 additional rows beyond declared length +10. **Width mismatch**: Inconsistent field count (missing salary in row 10) +11. **Missing fields**: Systematic field omissions (no email in multiple rows) + #### Question Types -${totalQuestions} questions are generated dynamically across four categories: +${totalQuestions} questions are generated dynamically across five categories: - **Field retrieval (${fieldRetrievalPercent}%)**: Direct value lookups or values that can be read straight off a record (including booleans and simple counts such as array lengths) - Example: "What is Alice's salary?" โ†’ \`75000\` @@ -295,11 +320,16 @@ ${totalQuestions} questions are generated dynamically across four categories: - Example: "List the field names for employees" โ†’ \`id, name, email, department, salary, yearsExperience, active\` - Example: "What is the department of the last employee?" โ†’ \`Sales\` +- **Structural validation (${structuralValidationPercent}%)**: Tests ability to detect incomplete, truncated, or corrupted data using structural metadata + - Example: "Is this data complete and valid?" โ†’ \`YES\` (control dataset) or \`NO\` (corrupted datasets) + - Tests TOON's [N] length validation and {fields} consistency checking + - Demonstrates CSV's lack of structural validation capabilities + #### Evaluation Process 1. **Format conversion**: Each dataset is converted to all ${formatCount} formats (${formatResults.map(f => FORMATTER_DISPLAY_NAMES[f.format] || f.format).join(', ')}). 2. **Query LLM**: Each model receives formatted data + question in a prompt and extracts the answer. -3. **Validate with LLM-as-judge**: \`gpt-5-nano\` validates if the answer is semantically correct (e.g., \`50000\` = \`$50,000\`, \`Engineering\` = \`engineering\`, \`2025-01-01\` = \`January 1, 2025\`). +3. **Validate deterministically**: Answers are validated using type-aware comparison (e.g., \`50000\` = \`$50,000\`, \`Engineering\` = \`engineering\`, \`2025-01-01\` = \`January 1, 2025\`) without requiring an LLM judge. #### Models & Configuration @@ -376,9 +406,12 @@ function generateDatasetBreakdown( questions: Question[], tokenCounts: Record, ): string { + // Build question ID to dataset mapping for O(1) lookups + const questionDatasetMap = new Map(questions.map(q => [q.id, q.dataset])) + return ACCURACY_DATASETS.map((dataset) => { const datasetResults = formatResults.map((fr) => { - const datasetFormatResults = results.filter(r => r.questionId.includes(dataset.name) || questions.find(q => q.id === r.questionId)?.dataset === dataset.name) + const datasetFormatResults = results.filter(r => questionDatasetMap.get(r.questionId) === dataset.name) if (datasetFormatResults.length === 0) return undefined diff --git a/benchmarks/src/types.ts b/benchmarks/src/types.ts index 1946d98..0f62ee2 100644 --- a/benchmarks/src/types.ts +++ b/benchmarks/src/types.ts @@ -1,4 +1,5 @@ import type { DATASET_NAMES, QUESTION_TYPES, STRUCTURE_CLASSES } from './constants' +import type { AnswerType, NormalizationOptions } from './normalize' export type QuestionType = typeof QUESTION_TYPES[number] export type DatasetName = typeof DATASET_NAMES[number] @@ -23,6 +24,15 @@ export interface Question { groundTruth: string type: QuestionType dataset: DatasetName + /** + * Expected answer kind for deterministic comparison. + * @default 'string' + */ + answerType?: AnswerType + /** + * Options for answer normalization and comparison. + */ + normalizationOptions?: Partial } export interface EvaluationResult { diff --git a/packages/toon/package.json b/packages/toon/package.json index 09a96db..7c15599 100644 --- a/packages/toon/package.json +++ b/packages/toon/package.json @@ -3,7 +3,7 @@ "type": "module", "version": "0.8.0", "packageManager": "pnpm@10.20.0", - "description": "Token-Oriented Object Notation (TOON) โ€“ a token-efficient JSON alternative for LLM prompts", + "description": "Token-Oriented Object Notation (TOON) โ€“ A compact, deterministic JSON format for LLM prompts", "author": "Johann Schopplich ", "license": "MIT", "homepage": "https://toonformat.dev",