jundot/omlx
Daily ranking profile for jundot/omlx in AI Rank.
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
- Stars: 14039
- Forks: 1188
- Language: Python
- Category: Infrastructure
- Subcategory: Deploy & Serve,Inference service optimization
- Keywords: DevOps, LocalAI, Memory management, Retrieval