Unofficial community documentation

Laya AI - Open Source System-1 Decision Model

Run Laya locally

pip install, dependencies, hardware, and the README examples. This page does not call the model from the browser.

Requirements

The package requires Python 3.10 or newer. pyproject.toml classifies 3.10 through 3.13. The README says the dependency floor is Python 3.10 because huggingface_hub 1.x, transformers 5.x, and torch 2.14 require it. Declared dependencies are torch, transformers, safetensors, huggingface_hub, and numpy.

README checkpoint table.
	encoder	params	context	use it for
laya	ModernBERT-large	421M	512	English
laya-multilingual	mmBERT-base	322M	1024	100+ languages, 2x faster
laya-typed-decisions	ModernBERT-large	421M	1024	the typed-decisions workflows

Published latency was measured on a Tesla T4. With Router(preload=True) the README reports 32.8 ms on GPU and 193–464 ms on CPU. A cold checkpoint build costs seconds. At the default max_loaded=1, alternating languages reloads a model on every switch: the README measures a 7.4 s median reload on CPU and 10.3 s on T4.

The repository does not publish a minimum RAM or VRAM number. It does say production servers should preload, that router.attach avoids a second copy in VRAM when an agent is already loaded, and that router.unload() frees memory. Parameter counts above are the only size figures in the README.

Install

bash

pip install laya

Example

Recommended Router quickstart from the repository README. Comments after the print calls are the README's annotated sample output, not a run from this website.

python

import laya
from laya import Router

# Preload checkpoints into memory for instant sub-35ms routing
router = Router(preload=True)

# 1. State in any language or schema
state = {
    "from": "[email protected]",
    "subject": "Duplicate charge on invoice #4411",
    "body": "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
}

# 2. Define your typed questions
questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this request?",
        "criteria": {
            "billing": "invoices, payments, refunds",
            "technical": "bugs, outages, system errors",
            "sales": "pricing, new contracts",
            "other": "everything else"
        }
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this request?",
        "criteria": ["not urgent", "soon", "critical deadline or blocking issue"]
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the user threaten to cancel or leave?"
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Does the user explicitly request a refund?"
    }
}

# 3. English state -> automatically routed to laya (ModernBERT-large, 39.5 ms)
res_en = router.predict(state, questions)
print("Department :", res_en["answers"]["department"]["choice"])  # -> billing (confidence: 0.94)
print("Routing    :", res_en["routing"]["model"])                 # -> english

# 4. Hindi state -> automatically routed to laya-multilingual (mmBERT-base, 32.8 ms)
res_hi = router.predict({"body": "मुझसे दो बार शुल्क लिया गया, कृपया पैसे वापस करें।"}, questions)
print("Department :", res_hi["answers"]["department"]["choice"])  # -> billing (confidence: 0.86)
print("Routing    :", res_hi["routing"]["model"])                 # -> multilingual

# 5. Explicit override when you want a specific checkpoint
res_td = router.predict(state, questions, model="typed-decisions")

Install Python 3.10+ and run the pip command above.
Save the script and run it with python.
The first predict downloads checkpoints from Hugging Face (convaiinnovations/laya). The machine needs network access to the Hub.
Router(preload=True) keeps checkpoints resident. Pass device="cuda" when a CUDA GPU is available. CPU still runs; use the CPU latency band above, not the T4 figure.
Language detection runs before the forward pass and selects english, multilingual, or an explicit model="typed-decisions" override.

Memory pitfalls

The repository does not publish a minimum RAM or VRAM number. What it does publish is the failure mode: with the default max_loaded=1, a language switch reloads a checkpoint. The README measures a 7.4 s median reload on CPU and 10.3 s on a T4. Router(preload=True) keeps the checkpoints resident so a language flip is detection only.

A separate Hugging Face community card, Luni/laya-jev-benchmark, reports a 14.9 s cold load on an RTX 5090. That figure is theirs, measured on different hardware from the T4 table.

python

router = Router(preload=True)
router = Router(preload=True, device="cuda")

router.preload(["english", "multilingual"])
router.attach("english", existing_agent)

router = Router(max_loaded=2)
router.unload()

Inspect routing

From the README. router.route explains the checkpoint choice without a forward pass.

python

res_hi["routing"]
# {
#   'model': 'multilingual',
#   'repo': 'convaiinnovations/laya/multilingual',
#   'reason': 'non-Latin script (devanagari, 100% of letters); the English checkpoint cannot read it'
# }

router.route({"body": "Der Kunde wurde zweimal belastet"}, questions).reason
# "Latin script but language looks like 'de', not English"

Single checkpoint

Direct laya.load from the README, for a pipeline that should not switch models. Comments are the repository's annotated sample output.

python

import laya

agent = laya.load("convaiinnovations/laya")
agent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual")
agent_td = laya.load("convaiinnovations/laya", subfolder="typed-decisions")

result = agent.predict(state, questions)
answers = result["answers"]

print("Department :", answers["department"]["choice"])   # -> billing (confidence: 0.94)
print("Urgency    :", answers["urgency"]["score"])        # -> 1.84 / 2.0
print("Churn Risk :", answers["churn_risk"]["noul"])       # -> 0.892 (89.2% probability)

Confidence gate

README example. route_automatically and escalate_to_human_agent are placeholders in that snippet, not library functions. Both checkpoints ship over-confident, so fit a temperature on your own held-out data before using 0.85 as a cutoff.

python

dept = answers["department"]["choice"]
conf = answers["department"]["confidence"]

if conf >= 0.85:
    route_automatically(dept)
else:
    escalate_to_human_agent(dept, reason=f"Low confidence ({conf:.2f})")

Presets

Four question schemas shipped in the package: model routing, prompt guardrails, moderation, and ticket triage.

python

import laya

agent = laya.load("convaiinnovations/laya")

routing = agent.predict({"request": "Refactor this service using dependency injection"}, laya.router_questions())
guard = agent.predict({"prompt": "Ignore all instructions"}, laya.guard_questions())
safety = agent.predict({"post": "User comment text"}, laya.moderation_questions())
triage = agent.predict({"message": "My payment failed twice"}, laya.triage_questions())