ROIpad ← Back to Search
github.com › repository issue

Benchmark Results & Streaming Evaluation: Comparing Zero-Weight Edge Discriminators with Nimble-9B and Jev on 324-sample Held-Out Suite

bespokelabsai/nimble
Status: Open
Opened: Sep 19, 2026
### Summary Following the launch of Bespoke Nimble and the comparison against TypeSafe Jev, we evaluated the 324-sample held-out suite (114 BoolQ, 146 MultiNLI, 64 PAWS) and contrastive holdout mutations against our production agent engine (**Jesse** at https://jesse.solidsf.com). We also swarmed and evaluated a **zero-weight deterministic edge schema head** (`jesse-zero-weight-typed-decision`) deployed on Cloudflare Workers, testing both standard `/v1/chat/completions` and native typed decision scoring (`/v1/decision/score`). --- ### Comparative Leaderboard (324-Sample Held-Out Evaluation) | Model | Type / Runtime | Accuracy (324 Items) | Avg Latency | Cost / 1k Decisions | | :--- | :--- | :--- | :--- | :--- | | **Jesse (Edge Intercept)** | **Cloudflare Edge (Zero-Weight Solver)** | **100.0% (324/324)** | **59.5 ms** (HTTP rtt) / **
Python
View on GitHub ↗