2026-08-05 06:08:13
📦 Loads HF safetensors directly, plus native GGUF for Gemma-4. MoE backends: offload, triton or hybrid — ft bench bw calibrates the host-RAM vs PCIe split per machine
🌐 Serves the OpenAI API (/v1/chat/completions, /v1/responses) and the Anthropic API (/v1/messages) — existing clients just point at the server