oMLX
oMLX is a macOS-native MLX server designed for high-performance local LLM inference on Apple Silicon.
Free tierVisit site ↗
Our verdict
Who it's for
Fast local LLM inference on Apple Silicon with persistent SSD cache
Who should look elsewhere
Requires macOS 15+ and Apple Silicon, limiting compatibility to recent Mac hardware
Pros
- +Dramatically reduces TTFT on long contexts for coding agents by persisting KV cache to SSD
- +Significant throughput improvements with continuous batching at high concurrency
- +Seamless integration with popular coding tools via OpenAI/Anthropic compatible APIs
Cons
- −Requires macOS 15+ and Apple Silicon, limiting compatibility to recent Mac hardware
- −Large models demand substantial RAM (64GB+ recommended), making it less accessible on lower-end Macs
Key features
- ✓Paged SSD KV caching with two-tier RAM/SSD architecture and LRU eviction policy
- ✓Continuous batching via mlx-lm's BatchGenerator for concurrent request handling
- ✓Native macOS menu bar app with web dashboard for model management and real-time metrics
- ✓Multi-model serving supporting LLM, VLM, embedding, and reranker models simultaneously
- ✓OpenAI and Anthropic drop-in API endpoints with one-click config generator for tools like Claude Code and Cursor
- ✓Tool calling support for JSON, Qwen, Gemma, GLM, and MiniMax formats with MCP integration
- ✓Automatic model discovery from standard Hugging Face cache (~/.cache/huggingface/hub)
- ✓Signed and notarized app with in-app auto-update (not Electron-based)
Pricing
No public pricing plans recorded.