Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Why do you need such high cross card bandwidth for inference? Are you hosting for a lot of users at once?


The Epyc boards make things way easier (I have 4 epyc boards of various generations) because they have loads of x16 slots and you’re not screwing around with bifurcation and sketchy PCI splitters. Another oft-forgotten item that consumes lanes is 25 or 40Gb NICs which you might fine you want if you’re pushing big model files around to other machines or storage.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: