Quantization from the ground up

Published by hadi on

A thorough explainer on how quantization makes LLMs 4x smaller and 2x faster while losing only 5-10% accuracy. Covers floating point precision, compression techniques, and how to measure quality loss, with interactive examples throughout.

Read more

Categories: PHP
Tags:

Related Posts

PHP

★ Detecting spam and auto-replies with Jev and the Laravel AI SDK

Yesterday Taylor announced that Jev support landed in the 1.x branch of the Laravel AI SDK. We started using it that same day for spam detection in There There. Let’s take a look at what Read more…

PHP

What’s New in PHP 8.6

A concise overview of what’s coming in PHP 8.6, including partial function application, clamp(), the new Duration class, and several smaller language and standard library improvements. Read more

PHP

Logarithmic auto-scaling for Laravel Horizon

A thoughtful write-up on adding logarithmic auto-scaling to Laravel Horizon. It shows how log-based weighting keeps huge queue spikes from starving smaller realtime queues while still giving large backlogs enough workers. Read more