Quantization isn’t a shortcut. It’s a test. More training leads to better compre...

		acchow on May 31, 2024 \| parent \| context \| favorite \| on: “Imprecise” language models are smaller, speedier,... Quantization isn’t a shortcut. It’s a test. More training leads to better compression. Trimming precision off the integers reveals that there was more compression that could be had.