Skip to main content
← Back to all tags

sglang

2 articles tagged with “sglang”

gpusglanginference
ai
· ~8 min read

Serving Frontier MoE Models at Scale: FP8 KV Caches, INT4 Weights, and KV Cache Integrity

Frontier mixture-of-experts models are wonderful to serve and very hard to fit into GPU memory. In August 2026, Cloudflare published the operating de

ai
· ~7 min read

vLLM vs SGLang: Choosing an LLM Inference Framework in 2026

Serving large language models at production scale boils down to one problem: getting the most tokens out of your GPU per second, per dollar. Two open