ROIpad ← Back to Search
github.com › repository issue

Empirical evaluation: 30-run benchmark of token-efficient vs 4 other .claude/ configs

drona23/claude-token-efficient
Status: Open
Opened: Mar 31, 2026
Comments: 1
## Summary I built a deterministic evaluation harness to test whether aggressive output-reduction rules actually save total tokens in agentic coding tasks. Your repo's actual CLAUDE.md was tested directly alongside 5 other configurations across 3 coding challenges. **Each agent gets a test file and must make all tests pass.** All configs pass 100%. The comparison is purely tokens to green. ## The 6 Configs Tested | Config | What's in `.claude/` | Size | |--------|---------------------|------| | A-baseline | "A coding project." | 1 line | | B-token-efficient | Our 12-line summary of token-reduction ideas | 12 lines | | C-structured | CLAUDE.md + rules + agents + reference | 4 files | | D-workflow | CLAUDE.md + rules + skills + hooks | 4 files | | E-hybrid | CLAUDE.md + rules + agents | 3 files | | **F-drona23** | **Your actual CLAUDE.md from this repo** | **61 lines** | ## Results — All Pass, Token Cost Varies ### CSV Reporter | Config | Avg Tokens | Avg Cost | |--------|------------|----------| | E-hybrid | 1,012 | $0.068 | | C-structured | 1,016 | $0.067 | | A-baseline | 1,088 | $0.078 | | B-token-efficient | 1,096 | $0.093 | | **F-drona23** | **1,137** | **$0.084** | | D-workflow | 1,199 | $0.083 | ### SQLite Window Functions | Config | Avg Tokens | Avg Cost | |--------|------------|----------| | E-hybrid | 1,230 | $0.108 | | A-baseline | 1,255 | $0.120 | | C-structured | 1,287 | $0.116 | | B-token-efficient | 1,339 | $0.116 | | D-workflow | 1,374 | $0.123 | | **F-...
Unknown
View on GitHub ↗
Related Content