That's not how it works. LLMs were only trained on text, so this is new data it'... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Legend2440 on Sept 28, 2023 \| parent \| context \| favorite \| on: AI language models can exceed PNG and FLAC in loss... That's not how it works. LLMs were only trained on text, so this is new data it's never seen before. There's no train-test leakage.

nuancebydefault on Sept 28, 2023 [–]

You are right but I am sure there are patterns in the weights that (accidentally or not) predict patterns commonly seen in media.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact