Comment on: Empirical evaluation: 30-run benchmark of token-efficient vs 4 other .claude/ configs
Repo: drona23/claude-token-efficient by drona23
Thanks for building this. Genuinely useful harness.
We iterated through v2-v8 using your benchmark. The original 61-line file (F-drona23) was bloated, you were right about that. We trimmed it down to 7 lines across 2 files.
Ran v8 head-to-head against C-structured on the same harness, same day, same model:
| Challenge | M-drona23-v8 | C-structured |
|-----------|-------------|-------------|
| CSV Reporter | $0.244 | $0.282 |
| SQLite Windows | $0.406 | $0.376 |
| WebSocket | $0.285 | $0.473 |
| **Total** | **$0.935** | **$1.131** |
v8 wins overall by 17.4%. The WebSocket gap is the biggest because v8 has an explicit rule to use a Set for client tracking instead of pub/sub, which prevents the debugging loop that costs extra turns.
Key takeaway we learned from your data: cost scales with turns, not file size. Each turn adds ~40k cache_read tokens. The one-shot strategy (read everything, write complete solution, test once) matters more than what's in the CLAUDE.md.
Root CLAUDE.md is...
GitHub Issue
Parent Entity
State: Open • Comments: 1
SaaS Metrics