Gemini Executive Synthesis

Contrastive-LM (CLM) for sentiment/frustration scoring, specifically the performance and consistency of the `CLM_v0.1-8B.pt` model head with the `Qwen/Qwen3-8B` encoder.

Technical Positioning

The project aims to deliver accurate, context-aware sentiment scoring, positioning itself for complex, non-coding tasks by leveraging LLMs. The developers are attempting to validate the model's ability to reliably interpret nuanced sentiment, demonstrating its capability for consistent output across various inputs and environments, potentially outperforming benchmarks like DeepSWE and tbench 2.1.

SaaS Insight & Market Implications

This issue exposes a critical functional defect: the CLM model consistently misclassifies sentiment, defaulting to 'Very angry' irrespective of input. This is not an environmental anomaly, as multiple independent backend setups reproduce the error, indicating a fundamental flaw within the model's 'head' or the specific `CLM_v0.1-8B.pt` checkpoint. Furthermore, discrepancies between published README examples, playground screenshots, and actual model outputs reveal poor version control and documentation. For B2B SaaS, this defect renders the product unusable for its stated purpose, directly impacting its viability. Enterprise clients demand predictable performance, accurate documentation, and robust release processes. The failure to deliver basic functionality and maintain consistent collateral severely undermines the product's credibility and market potential.

Proprietary Technical Taxonomy

Raw Developer Origin & Technical Request

GitHub Issue Sep 24, 2026

Repo: Contrastive-LM/CLM

Unexpected scores

Hi there.
I tried to run the repo on an RTX 4090 and can reproduce some of the README but for different queries I'm getting seemingly wrong scores.
For example "I'm very calm." gives a score corresponding to "Very angry".

Requst
```
curl -s localhost:8700/v1/systemone -H 'content-type: application/json' -d '{
"state": "Customer: I'\''m very calm.",
"questions": {
"frustration": {"type": "score", "instructions": "How frustrated is the customer?",
"criteria": ["Calm", "Frustrated", "Very angry"]}
}
}'
```
Returns `"score": 1.999, "probabilities": {"0": 0.001, "1": 0.000, "2": 0.999}`, i.e. "Very angry" with 99.9%.
Is this expected with the published head?

Setup (CUDA, following the README Quickstart)
- Repo: Contrastive-LM/CLM @ 7956937 ("Initial Commit")
- Head: Contrastive-LM/CLM-v0.1-8B / CLM_v0.1-8B.pt, downloaded automatically by clm-serve
- Encoder: Qwen/Qwen3-8B (latest on the Hub)
- vLLM 0.30.0, torch 2.13.0+cu130, 1× RTX 4090 (24 GB), driver 580.126.09

```
vllm serve Qwen/Qwen3-8B --served-model-name qwen3-8b --runner pooling \
--enable-prefix-caching --max-model-len 2048 --gpu-memory-utilization 0.9 --port 8090
clm-serve --port 8700 --emb-url 127.0.0.1/v1/embeddings
```

View Raw Source

Developer Debate & Comments

dathtd119 • Sep 24, 2026

Literally what you said when you angry, so the model is too good that it check the vibe, not the statement

brookman • Sep 24, 2026

> Literally what you said when you angry, so the model is too good that it check the vibe, not the statement I'm not entirely convinced by this theory. ^^ Because "Customer: Thanks for the quick help yesterday, everything works now. Have a nice day!" also gives me a "Very angry."

dathtd119 • Sep 24, 2026

> > Literally what you said when you angry, so the model is too good that it check the vibe, not the statement > > I'm not entirely convinced by this theory. ^^ Because "Customer: Thanks for the quick help yesterday, everything works now. Have a nice day!" also gives me a "Very angry." Im being satire here so dont take it serious lol, but yes we need more prove like this. But maybe you can try to test if it actually able to catch the vibe by provide the list of vibe, including vibe "faking happy" or something to test if my point is true. Just my 2cents but first glance about beat the DeepSWE and tbench 2.1 is something unreal for things like this, because this kind of System One models are not for coding.

ssrajadh • Sep 24, 2026

I get your exact numbers, `1.999` with `{"0": 0.001, "1": 0.0, "2": 0.999}`, from a Qwen3-8B encoder running on MLX (bf16 and 8-bit), so it isn't your vLLM setup. It doesn't look like the model reading the vibe either. The score question returns "Very angry" at 1.00 for every state I tried, including a happy one like your "Have a nice day" example, and it still picks "Very angry" when the levels are listed in reverse: | Levels | "I'm very calm." | "thanks so much, everything works perfectly now!" | "hmm, that's a bit annoying." | "this is the THIRD time I've called, I am furious!!" | |---|---|---|---|---| | `Calm / Frustrated / Very angry` | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 | | same levels, reversed order | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 | Very angry, 1.00 | | levels as sentences (`The customer is calm.` ...) | somewhat frustrated, 0.46 | somewhat frustrated, 0.78 | somewhat frustrated, 0.86 | somewhat frustrated, 0.63 ...

moltenkaizen • Sep 24, 2026

Same result on a third backend: Hugging Face transformers on a Mac mini M4 Pro (MPS, bf16) gives `1.999`, `{"0": 0.001, "1": 0.0, "2": 0.999}` for the exact request above. vLLM/CUDA, MLX and PyTorch-MPS all agree, so it's the head, not the serving stack. Reversing the level order still picks "Very angry" here too. Something I noticed along the way: the README's example outputs don't match the repo's own playground screenshot for the same request. | | README | `assets/playground.png` (RTX 4090) | my Mac run | |---|---|---|---| | urgency (noul) | 0.410 | 0.848 | 0.841 | | billing | 0.939 | 0.988 | 0.987 | | frustration | 1.984 | 2.00 | 2.000 | | tides, Moon | 0.997 (model card: 0.993) | – | 0.993 | The screenshot and independent runs agree with each other, so maybe the README numbers came from an earlier head? If that checkpoint exists, it'd be interesting to see whether it has the same score behavior. (Ran this with Claude Code's help; the numbers are from my machine.)

Adjacent Repository Pain Points

Other highly discussed features and pain points extracted from Contrastive-LM/CLM.

State encoder's input format in Super Mario

Extracted Positioning

Contrastive-LM (CLM) for state encoding in reinforcement learning, specifically the input format for its LLM-based state encoder when processing non-textual data like Super Mario frames.

The project positions LLMs for novel applications beyond traditional text processing, extending their utility to visual state encoding in environments like Super Mario. The developer seeks clarity on how this non-standard input is handled to understand the model's architectural design and potential for broader adaptation.

Frequently Asked Questions

Market intelligence mapped to Contrastive-LM (CLM) for sentiment/frustration scoring, specifically the performance and consistency of the `CLM_v0.1-8B.pt` model head with the `Qwen/Qwen3-8B` encoder..

What problem does Contrastive-LM (CLM) for sentiment/frustration scoring, specifically the performance and consistency of the `CLM_v0.1-8B.pt` model head with the `Qwen/Qwen3-8B` encoder. solve?

Based on our AI analysis of the original developer request, its primary technical positioning is: The project aims to deliver accurate, context-aware sentiment scoring, positioning itself for complex, non-coding tasks by leveraging LLMs. The developers are attempting to validate the model's ability to reliably interpret nuanced sentiment, demonstrating its capability for consistent output across various inputs and environments, potentially outperforming benchmarks like DeepSWE and tbench 2.1.

Are engineers actively discussing Contrastive-LM (CLM) for sentiment/frustration scoring, specifically the performance and consistency of the `CLM_v0.1-8B.pt` model head with the `Qwen/Qwen3-8B` encoder.?

Yes, we have tracked 5 direct responses and active debates regarding this specific topic originating from GitHub Issue.

Which technical concepts are associated with Contrastive-LM (CLM) for sentiment/frustration scoring, specifically the performance and consistency of the `CLM_v0.1-8B.pt` model head with the `Qwen/Qwen3-8B` encoder.?

Our proprietary extraction maps Contrastive-LM (CLM) for sentiment/frustration scoring, specifically the performance and consistency of the `CLM_v0.1-8B.pt` model head with the `Qwen/Qwen3-8B` encoder. to adjacent architectural concepts including RTX 4090, README, queries, scores.

Engagement Signals

Replies

open

Issue Status

Cross-Market Term Frequency

Quantifies the cross-market adoption of foundational terms like state and score by tracking occurrence frequency across active SaaS architectures and enterprise developer debates.