// the find
MindLab-Research/Mixture-of-LoRA-Harness
A serving harness for Macaron-V1-Venti, a 748B model built from a GLM-5.2 base plus four task-specific LoRA adapters (chat, agent, coding, UI). A Python proxy sits in front, uses L0 to classify each request, routes it to the right adapter, and maintains per-adapter conversation state to keep prefix cache hits high. For teams running giant models on-prem who want to specialize behavior without multiple model deployments.
The cross-adapter summary mechanism is clever — L0 produces a hidden summary after each turn that bridges context between adapters without leaking internal routing to the client. The Rust gateway handles load balancing and worker registration separately from the Python routing logic, which means you can swap or scale the engine tier without touching the orchestration layer. The launcher scripts do real preflight checks — shard count, GPU name, adapter file presence — before starting, which saves a lot of 'why did my 8xGPU launch fail' debugging. Tool-result continuations staying pinned to the issuing adapter is the right call; re-routing mid-tool-call would break both correctness and prefix cache affinity.
Model weights, benchmarks, and test suites are explicitly not included, so you cannot verify any of the claims about routing quality or adapter performance — this is a harness for a model you have to obtain separately and trust blindly. The routing decision relies entirely on L0 (a 24-token budget classification call), and there is no documented fallback or confidence threshold, meaning a misclassification silently serves the wrong adapter with no signal to the caller. The hard-coded assumption of exactly eight NVIDIA L20D GPUs in a B300 environment means this will not run without environment variable surgery on any other hardware, and those variables are scattered across two shell scripts rather than a single config. The Python proxy has no documented persistence for the conversation state beyond an in-memory TTL, so a proxy restart drops all active sessions mid-conversation.