Skip to main content
← Back to all tags

quantization

2 articles tagged with “quantization”

quantization
ai
· ~8 min read

Serving Frontier MoE Models at Scale: FP8 KV Caches, INT4 Weights, and KV Cache Integrity

Frontier mixture-of-experts models are wonderful to serve and very hard to fit into GPU memory. In August 2026, Cloudflare published the operating de

ai
· ~4 min read

The On-Device LLM Revolution: Edge AI in 2026

The AI world is experiencing a fundamental shift. After years of cloud-centric inference dominated by massive data center GPUs, language models are m