Contrastive-LM/CLM

Same result on a third backend: Hugging Face transformers on a Mac mini M4 Pro (MPS, bf16) gives `1.999`, `{"0": 0.001, "1": 0.0, "2": 0.999}` for the exact request above. vLLM/CUDA, MLX and PyTorch-MPS all agree, so it's the head, not the serving stack. Reversing the level order still picks "Very angry" here too.

Something I noticed along the way: the README's example outputs don't match the repo's own playground screenshot for the same request.

| | README | `assets/playground.png` (RTX 4090) | my Mac run |
|---|---|---|---|
| urgency (noul) | 0.410 | 0.848 | 0.841 |
| billing | 0.939 | 0.988 | 0.987 |
| frustration | 1.984 | 2.00 | 2.000 |
| tides, Moon | 0.997 (model card: 0.993) | – | 0.993 |

The screenshot and independent runs agree with each other, so maybe the README numbers came from an earlier head? If that checkpoint exists, it'd be interesting to see whether it has the same score behavior.

(Ran this with Claude Code's help; the numbers are from my machine.)

I get your exact numbers, `1.999` with `{"0": 0.001, "1": 0.0, "2": 0.999}`, from a Qwen3-8B encoder running on MLX (bf16 and 8-bit), so it isn't your vLLM setup.

It doesn't look like the model reading the vibe either. The score question returns "Very angry" at 1.00 for every state I tried, including a happy one like your "Have a nice day" example, and it still picks "Very angry" when the levels are listed in reverse:

| Levels | "I'm very calm." | "thanks so much, everything works perfectly now!" | "hmm, that's a bit annoying." | "this is the THIRD time I've called, I am furious!!" |
|---|---|---|---|---|
| `Calm / Frustrated / Very angry` | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 |
| same levels, reversed order | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 |
| levels as sentences (`The customer is calm.` ...) | somewhat frustrated, 0.46 | somewhat frustrated, 0.78 | somewhat frustrated, 0.86 | somewhat frustrated, 0.63 ...

> > Literally what you said when you angry, so the model is too good that it check the vibe, not the statement
>
> I'm not entirely convinced by this theory. ^^ Because "Customer: Thanks for the quick help yesterday, everything works now. Have a nice day!" also gives me a "Very angry."

Im being satire here so dont take it serious lol, but yes we need more prove like this. But maybe you can try to test if it actually able to catch the vibe by provide the list of vibe, including vibe "faking happy" or something to test if my point is true.

Just my 2cents but first glance about beat the DeepSWE and tbench 2.1 is something unreal for things like this, because this kind of System One models are not for coding.

> Literally what you said when you angry, so the model is too good that it check the vibe, not the statement

I'm not entirely convinced by this theory. ^^
Because "Customer: Thanks for the quick help yesterday, everything works now. Have a nice day!" also gives me a "Very angry."

Literally what you said when you angry, so the model is too good that it check the vibe, not the statement