NandhaKishorM/laya

running it directly via Transformers.js is definitely the dream for edge/local development
the main reason we went with the REST API approach for PR #3 is GPU acceleration and scaling.
It's much easier to leverage CUDA via PyTorch in a dedicated Python microservice (which can run on heavy GPU nodes) while keeping the Node.js/Bun consumer lightweight on CPU nodes. Plus, the Python SDK handles all of Laya's custom routing, confidence math, and tensor shapes, which we'd have to completely rewrite in JS to use the raw ONNX logits

Why not run [Laya ONNX](https://huggingface.co/Mattepiu/laya-onnx) (or multilingual variant) using Transformers.js on server?

Hey @NVentimiglia - this is genuinely awesome, thank you for building this and for being so thoughtful about attribution!

The MCP server is exactly the kind of ecosystem integration I was hoping PR #3 would unlock. The whole point of adding laya serve was to make Laya callable from *any* language or tool and an MCP server is a perfect example of that. So in a way, your repo is the proof that the REST API was worth building.

On your benchmark question your instinct is probably right. The default Laya checkpoint (convaiinnovations/laya) is a ModernBERT-large fine-tuned specifically on **structured decision workflows** (routing, triage, classification with explicit criteria). Benchmarks that treat it like a general-purpose NLU model or throw open-ended reasoning at it will make it look weaker than it is.

A fairer benchmark setup would be:
- **Give it explicit criteria** for every question (the criteria field in the request) - that's when it really shines vs Jev and generative models
-...

Hey @NandhaKishorM just confirmed it's clean!

The branch was rebased on v0.3.3 earlier today. GitHub API shows:
- **mergeable: true**
- **mergeable_state: clean**
- Branch is **0 commits behind** main

Also updated in this rebase:
- server.py now uses laya.Router + preload() instead of laya.load() to match your new v0.3.3 API
- Test mocking fixed so CI doesn't download the 421MB model on every test run (~5s test suite now)
- Ruff lint is clean (removed unused List, Tuple imports from cli.py)

Should be ready to merge whenever you are!

Closing as it goes now

> Hey [@gggiiia](https://github.com/gggiiia)! We actually _just_ built almost exactly this today in PR [#3](https://github.com/NandhaKishorM/laya/pull/3)!
>
> While it doesn't load the safetensors natively in Node via ONNX/WASM (which is tricky for CUDA performance), PR [#3](https://github.com/NandhaKishorM/laya/pull/3) introduces a complete, heavily-typed **TypeScript SDK** (laya-sdk) that connects to the new laya serve REST API.
>
> It supports exactly what you need for your PDF SaaS:
>
> * Native Node.js & Bun.js compatibility (uses standard etch)
> * Full TypeScript types for inputs and structured JSON results
> * Async inference
> * A dedicated client.decideBatch() method for blasting through hundreds of PDF passages concurrently.
>
> You can check out the implementation in the js/ folder of PR [#3](https://github.com/NandhaKishorM/laya/pull/3): [#3](https://github.com/NandhaKishorM/laya/pull/3)

Can you check the conflict in your pr

Will check it

there are some dynamic changes need to be pushed in this repo just a new model coming up with mmbert base, thats why I am just holding up

Hey @gggiiia! We actually *just* built almost exactly this today in PR #3!

While it doesn't load the safetensors natively in Node via ONNX/WASM (which is tricky for CUDA performance), PR #3 introduces a complete, heavily-typed **TypeScript SDK** (laya-sdk) that connects to the new laya serve REST API.

It supports exactly what you need for your PDF SaaS:
- Native Node.js & Bun.js compatibility (uses standard etch)
- Full TypeScript types for inputs and structured JSON results
- Async inference
- A dedicated client.decideBatch() method for blasting through hundreds of PDF passages concurrently.

You can check out the implementation in the js/ folder of PR #3: https://github.com/NandhaKishorM/laya/pull/3