Run bigger models on the
Mac you already have.
Live Apple Silicon KV-cache quantization studio. Configure your hardware and target model on the left to see the optimal quantization strategy, live memory compression, and Metal kernel speedup.
New here? What this actually does
As an LLM generates tokens, it stores key-value pairs (the KV cache) in unified RAM so it doesn't recompute previous context. The longer the conversation, the more memory the KV cache consumes — quickly exceeding unified RAM.
VeloxQuant-MLX compresses these cache entries up to 8× with specialized Apple Silicon Metal kernels. This studio calculates the best algorithm, memory headroom, and context limits for your exact hardware setup.
Where these numbers come from: Sizing formulas and recommendations run the same logic that powers the veloxquant recommend CLI tool (veloxquant_mlx/tools/mac_recommender.py). Benchmark curves in Step 3 are exact Metal kernel measurements committed under figures/. To run native profiling on your machine, run python -m veloxquant_mlx benchmark.