ChipokiaTech news, without the noise
← All stories
InfoQ·

Presentation: Scaling LLM-Based Ranking Systems for Latency-Critical Search & Recommendation Workloads

Sundara Raman Ramachandran shares how LinkedIn productionizes LLM ranking under strict P99 latency SLAs. He discusses distilling a 7B model down to 400M parameters, compressing job contexts into single-token embeddings for a 10x throughput leap, and…

Preview courtesy of InfoQ. The full article opens on their site.

More from InfoQ