The `harness2-judge`'s grade persistence logic during task re-runs and judge failures, specifically the `_merge` operation's handling of run IDs and rollouts.
Raw Developer Origin & Technical Request
GitHub Issue
Oct 18, 2026
## Description
If a task is re-run and the next `harness2-judge` call fails for that task, `judged.json` keeps the grade of the rollout that was replaced. The report then shows a score for a deliverable that no longer exists.
## Problem
Run ids are fixed labels (`//`), so a re-run task writes its new rollout under the same run id. A partial task re-runs on resume, for example. `_merge` takes "same run id" to mean "same rollout". When the new judge call fails, it keeps the old graded row in place of the new ungraded one.
This repro uses `DemoAdapter`, a judge that grades the answer on disk, and the judge CLI's own collect / judge / merge / write steps. No network is used:
```
attempt 1: partial # sequential mode hit a transient 429
judge 1: demo-oc-agg1/ledger-1/agg1 1/1
attempt 2: ok # resume re-runs the task; the new rollout is wrong
on disk: demo-oc-agg1/ledger-1/agg1 -> 'Total: 105.25'
judge 2: demo-oc-agg1/ledger-1/agg1 1/1 graded=True # judge was down; old 1/1 kept
```
Repro script
```python
import json, tempfile
from pathlib import Path
from unittest.mock import patch
from harness2.cli import judge as JCLI
from harness2.core import judge as J, recursion, schedule
from harness2.core.proposer import Reply
from harness2.demo import DemoAdapter, DemoProposer
class Adapter(DemoAdapter):
judge_down = False
def score(self, run_...
Developer Debate & Comments
No active discussions extracted for this entry yet.
Adjacent Repository Pain Points
Other highly discussed features and pain points extracted from google-research/harness2.
Frequently Asked Questions
Market intelligence mapped to The `harness2-judge`'s grade persistence logic during task re-runs and judge failures, specifically the `_merge` operation's handling of run IDs and rollouts..
What problem does The `harness2-judge`'s grade persistence logic during task re-runs and judge failures, specifically the `_merge` operation's handling of run IDs and rollouts. solve?
What architecture is tied to The `harness2-judge`'s grade persistence logic during task re-runs and judge failures, specifically the `_merge` operation's handling of run IDs and rollouts.?
Engagement Signals
Cross-Market Term Frequency
Quantifies the cross-market adoption of foundational terms like _merge and rollout by tracking occurrence frequency across active SaaS architectures and enterprise developer debates.
SaaS Metrics