NVIDIA's KVPress library offers practical KV cache compression for long-context LLM inference. A new end-to-end coding tutorial demonstrates how to implement multiple compression strategies, benchmark their performance, and achieve significant memory savings during generation.
New theoretical analysis frames deep neural networks as geometric systems, revealing why ReLU's preservation of spatial distance information gives it a decisive edge over sigmoid for deep inference. The geometric perspective offers a principled framework for understanding activation function choices and their real-world cost implications.
