Local model execution
Run the engine and model weights in infrastructure the operator controls rather than depending on an external inference API for every request.
Why MER
Model capacity should not stop at VRAM.
Sparse Mixture-of-Experts models activate only some expert networks for each token. MER is building around that property: the active working set fits usable VRAM while the complete expert pool can extend through RAM to NVMe.
The model-fit equation changes.
Fully resident accelerator inference typically requires the complete model footprint to fit available accelerator memory. That remains the preferred deployment when the model fits and maximum latency or throughput is the priority.
Routed MoE models expose a different possibility: keep the transformer’s active state and highest-value expert working set in VRAM, while virtualizing the larger expert pool across RAM and NVMe. More VRAM can improve residency and reduce misses without changing that capacity architecture. MER does not claim that misses perform like fully resident weights.
Deployment tradeoff
Fully resident when it fits. Virtualized when capacity matters.
| Factor | Fully resident accelerator | MER virtualized residency |
|---|---|---|
| Model-fit requirement | Complete model fits accelerator memory | Active working set fits usable VRAM; expert pool may exceed it |
| Expert residency | All expert weights resident | Demand-managed VRAM hot set |
| Storage participation | Primarily startup and loading | NVMe/SSD backs misses through RAM staging |
| When VRAM increases | More room for model, KV, or concurrency | Larger expert hot set and potentially fewer misses |
| Miss cost | No expert residency miss in the steady state | Workload-dependent storage, staging, and upload latency |
| Expected operating point | Best latency and throughput when the model fits | Capacity-oriented inference with measurable miss tradeoffs |
More VRAM changes the hot set, not the thesis.
A 24 GB, 48 GB, or 80 GB GPU gives MER a different safe expert-residency budget and therefore a different likely miss rate and performance profile. The intent is for those devices to share a capacity architecture, not equal performance. No universal model-to-VRAM multiplier is claimed until model-, artifact-, and hardware-specific measurements exist.
Data control
Keep execution inside the operator-controlled boundary.
Deployment flexibility
Relevant target environments include on-premises systems, private cloud, disconnected research networks, edge sites, and sovereign infrastructure.
MER does not claim security, privacy, or compliance certifications that have not been completed.
Target use cases
Where this operating point may matter.
MoE models larger than accelerator memory
Intended evaluation area; not presented as a completed customer deployment.
Constrained-VRAM deployment
Intended evaluation area; not presented as a completed customer deployment.
Local and private inference
Intended evaluation area; not presented as a completed customer deployment.
NVMe-backed expert virtualization research
Intended evaluation area; not presented as a completed customer deployment.
Existing operator-owned GPU hardware
Intended evaluation area; not presented as a completed customer deployment.
Environmental positioning