Performance Benchmark & Regression Analysis: Deltafin Engine

# Performance Benchmark & Regression Analysis: Deltafin Engine **Date:** August 3, 2026 **Hardware Environment:** Intel Core Ultra 9 285K | NVIDIA GeForce RTX 5090 (32 GB) | PCIe Gen5 **Target Model:** Kimi K3 (2.8T Parameter MoE, 93 Layers) **Test Sequence:** Prompt: *"The capital of France is"* --- ## Executive Summary A performance comparison was conducted between the optimized **Python/CUDA v4 pipeline** and the newly implemented **Rust Engine** for the Deltafin MoE inference harness. While the Rust rewrite succeeded in lowering low-level language overhead, it introduced a severe memory management regression by omitting the VRAM-resident LRU Expert Cache present in Python v4. As a result, the Rust implementation relies on continuous Host-to-Device (H2D) tensor streaming over PCIe, resulting in a **2.25× slowdown in decode latency** and a **55.4% drop in generation throughput**. --- ## Macro Performance Metrics | Engine / Architecture | Generation Speed | Decode Latency | Total Compute Time | Memory Strategy | | :--- | :--- | :--- | :--- | :--- | | **CPU Baseline (AVX2)** | 0.0735 tk/s | 13.60 s/tok | 62.27s (10 tok) | System RAM | | **Python CUDA v1 (No Cache)** | 0.0793 tk/s | 12.60 s/tok | 123.75s (16 tok) | Streaming PCIe H2D | | **Python CUDA v4 (512-Entry Cache)** | **0.1053 tk/s** | **9.50 s/tok** | **45.43s (10 tok)** | **On-GPU LRU Cache (~9 GB)** | | **New Rust Engine** | **0.0469 tk/s** | **21.33 s/tok** | **16.20s kernel (3 tok)** | **Un-cached ...