Skip to main content

VeloxQuant-MLXGetting Started Algorithms Guides API Reference Changelog Blog

All posts

2026

Build Local AI Apps with the VeloxQuant Node.js SDK
Batching Decode Attention Across Requests Got Us 3.83x Real Throughput — Across Layers Got Us Nothing
Batching Decode Attention Across Layers Sounded Great — Until the Residual Stream Said No
Chasing the Mac-vs-CUDA Prefill Gap — and Finding a Wall Instead
Fusing Quantize and Pack Into One Metal Dispatch
The ROM Chip That Wasn't, and the 230× Speedup That Was
The Sign Was the Whole Paper: Debugging a KV Cache Compressor on Real Models
A 5.65× Metal Kernel
TensorOps Research: What We Learned Optimizing KV Caches
KIVI: The Most-Cited KV Cache Baseline, Implemented in MLX
Hands-On: Compressing Your First LLM with VeloxQuant-MLX
I Wrote a Metal Kernel to Stop My Mac From OOMing
TurboQuant + Metal Kernels: The Combined Writeup
Benchmark Results: 10 Models, 8 Compression Configs
I Ported a Google Research Paper to Apple Silicon
VeloxQuant-MLX: Fast KV Cache Quantization for Apple Silicon

Tags

A

apple-silicon16
attention3

B

benchmarking5
benchmarks3

G

gpu5

J

javascript1

K

kivi2
kv-cache8

L

llm1

M

memory1
metal10
mlx16

N

nodejs1

P

performance4

Q

q-filters1
quantization4

R

research1
results1
roofline2
rvq1

T

turboquant2
tutorial1
typescript1

W

weights1

Docs

Getting Started
Algorithms
Guides
API Reference

Community

GitHub
X (Twitter)
LinkedIn
Discord
Issues
☕ Buy me a coffee

More

Blog
Changelog
Home

MIT License © 2026 VeloxQuant-MLX. Built with Docusaurus.