mlx-serve

Native Zig inference server for Apple Silicon — no Python, no conda. One binary. 35%+ faster decode than LM Studio on Gemma 4 E4B 4-bit. Drop-in replacement for Ollama (/api/chat, /api/generate), plus full OpenAI and Anthropic APIs on the same port. Beyond chat: local image gen (FLUX.2 + Krea-2), video gen with audio (LTX-Video), voice cloning (Qwen3-TTS + ECAPA-TDNN), and an agent that runs in an isolated Linux VM on Virtualization.framework. Free macOS menu-bar app included.