I tried 3 times the "Which is heavier, a 10.01kg block of steel on or a 9.99kg b... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		1209412comb 10 months ago \| parent \| context \| favorite \| on: 30% drop in O1-preview accuracy when Putnam proble... I tried 3 times the "Which is heavier, a 10.01kg block of steel on or a 9.99kg bag of feathers?" and ChatGPT keep converting kg to pound and saying the 9.99kg is heavier.

fragmede 10 months ago [–]

Which model? On the paid plus tier, GPT-4o, GPT-o1, and GPT-o1mini all successfully got the 10.1. I did not try any other models.

gpt-4o: https://chatgpt.com/share/67768221-6c60-8009-9988-671beadb5a...

o1-mini: https://chatgpt.com/share/67768231-6490-8009-89a6-f758f0116c...

o1: https://chatgpt.com/share/67768254-1280-8009-aac9-1a3b75ccb4...

mycall 10 months ago | [–]

Did you try it more than once per model? I could see temperature or other similar gradients providing wrong answers.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact