Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I wonder if the mlx optimized would run on 64gb mac


LM Studio's heuristics (which I've found to be pretty reliable) suggest that a 3-bit quantization (~50 GB) should work fine.


You can fine tune the amount of unified memory reserved for the system vs GPU, just search up `sysctl iogpu.wired_limit_mb`. On my 64gb mac mini the default out of the box is only like ~44gb available to the GPU (i forget the exact number), but tuning this parameter should help you run models that are a little larger than that.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: