Run models beyond VRAM
Keep the most useful experts in GPU memory while RAM and NVMe hold the much larger expert pool.
See the memory modelNext-Generation AI Infrastructure
Use all your VRAM. Run beyond it.
Mixture-of-Experts models use only a small subset of their expert networks for each token. MER is building an inference runtime that keeps the most useful experts in GPU memory and uses RAM and NVMe for the larger pool, allowing model capacity to extend beyond VRAM.
Input tokens enter a central router. Four active paths lead to one selected expert in each surrounding expert pool while the other experts remain inactive.
Partnerships & Programs
MER engine features
Keep active work on the GPU
The target execution path keeps transformer state and resident-expert work on the accelerator instead of moving each layer through CPU memory.
Review integration statusMove only what is needed
Use a resident expert immediately; stage a missing expert from RAM or NVMe toward VRAM as demand changes.
Explore hit and miss flowProve every architecture step
Deterministic output checks and NVIDIA L4 hardware tests establish correctness while the complete GPU-native path is built.
Inspect the evidenceHow MER works
Route precisely. Move less.
When the model needs an expert, MER checks the GPU first. A resident expert can run immediately; a missing expert moves from RAM or NVMe toward VRAM. The architecture is also designed to anticipate likely demand while keeping foreground inference higher priority.
Select
A sparse MoE router chooses the small subset of expert networks needed for each token.
Use or move
Use an expert immediately if it is in VRAM; otherwise move it from RAM or load it from NVMe through RAM.
Execute and anticipate
Run the selected expert on the GPU. The architecture is designed to start moving likely next experts early when confidence and resource budgets allow.
Why MER
Use the VRAM you have. Virtualize the rest.
VRAM remains the fastest place to work. MER aims to fill it safely with the active expert set while using operator-controlled RAM and local storage to extend model capacity beyond the GPU.
Use the VRAM you have
Reserve safe space for the model core, context, active requests, temporary GPU work, and transfers.
Virtualize the rest
Use nearly all remaining safe VRAM for active experts while RAM and NVMe hold the larger pool.
Measure every miss
More VRAM should reduce miss frequency and improve performance; storage latency remains visible and workload-dependent.
Measured evidence
A real engine, not a concept deck.
MER has demonstrated deterministic output agreement, real NVIDIA L4 execution, and measurable reductions in host synchronization. The complete GPU-native token loop and oversized-model throughput target are still in development.
Deterministic output agreement
The qualification reference and GPU expert execution generated identical output across the fixed test corpus. Correctness evidence, not speed.
Review the correctness scopeHardware-validated foundation
GPU state, dense execution, attention preparation, and per-request context storage were exercised on real hardware.
Review the validated scopeFewer host synchronizations
A controlled earlier warm-path batching test reduced synchronization operations from 6,144 to 768.
See the batching evidenceIntermediate CPU readbacks
The hardware-qualified foundation keeps intermediate state on the GPU instead of reading it back to the CPU between qualified operations.
See the GPU-native foundationCorrectness
Fixed-corpus generated output matched exactly across the qualification paths.
Real hardware
NVIDIA L4 tests validate the current GPU-native foundation.
Controlled diagnostic
An earlier warm-path test reduced host synchronizations by 87.5%.
Not yet claimed
Full GPU-native generation and the ≥10 real generated tokens/s target.
Early access
Build the next test with us.
Bring a model, hardware profile, and capacity constraint. We will determine whether the current engine is ready for a useful, bounded evaluation.