Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Will this run on an M3 48GB?


You’ll need 44GB just for the weights

By default only 75% of unified memory is available to GPU if you have >36GB. So with 48 total only 36 is available for GPU with is lower than 44.

tldr; without quantization you will not be able to run it.


https://medium.com/@tedisaacs/codestral-for-ai-coding-models...

This article says 88GB without quantization. Though it then goes on to make a ridiculous claim that if you had 128GB of RAM, then using up 88GB of 128GB would make everything else really slow because I guess the think the remaining 40GB of RAM somehow isn't enough for your OS and desktop apps.

So it’s probably not a very authoritative source.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: