Implementing TurboQuant in gguf-runner cuts KV-cache memory roughly in half while staying close to Q8 throughput.
gguf-runner gained vision support, ships GitHub release binaries, and received many usability and performance improvements.