Do you mostly just hit the session limits? If so I know it's not ideal but you could wait an hour or two for that to reset. Not sure if that would work for you but just a suggestion
Consumer CPUs don't have enough PCIE lanes to do that. Even if they had physical x16 slots, at most two of them would be x16.
What's cheap to you? You can find Epyc 7002/7003 boards on ebay in the $400 range and those will do it. That's probably the best deal for 4x PCIE 4.0 x16 and DDR4. Probably $500 range with a CPU. That's in the ballpark of a mid to high end consumer setup these days.
That's the path I'm taking.
All with all those PCI-e lanes available.
If I had more income, I would also buy 4x 96g Optane drives of p0 swap disks and a few ssd's for p1 swap disks. To evaluate how well you can get a 1T model running in these absurd ram prices.
Even if you're fine with risers, that might not be enough. If the bridge lanes are PCIe Gen 3, as many consumer boards have, your Gen 5 card might not init. I extensively tested several motherboards to try and get my AM5 CPU talking to a triple Radeon AI Pro 9700 XT setup, and they absolutely refuse to come up on PCIe3. I was using dummy EDID plugs for them, so they think they have a display, ruling out that issue.
What I eventually had to do was buy a used Threadripper box to run those cards, because PCIe Gen 4 definitely works.
I want to spend $1500 for a card that can run a proper large model, even if it only can do 25 tk/s.
Intel is squandering a golden opportunity to knee-cap AMD and Nvdia, under the totally delusional pretense that intel enterprise cards still have a fighting chance.
Can confirm doing so is awesome. Get some slightly bigger ones and partition them for additional use as zil. They're extremely satisfying to use, and depressing to remember that we'll never see their like again.
Sure! This is more or less how I'm using Optane in my storage box:
Two of U.2x4 to PCIe x16 riser cards, one loaded with 960GB Intel-branded Optanes, one with 1.5TB IBM-branded. PCIe bifurcation is set up in the BIOS to let them all come up properly, where they just show as regular NVMe. Riser cards like this can easily be substituted for PCIe to SAS/Oculink to U.2 cables, if that would be more accommodating to your chassis.
Once they all come up, partition them for your preferred split of swap and ZFS special. Swap should have them all mounted with the same priority and discard=pages, I also recommend setting up zswap (not zram swap) with lz4 as an additional layer of fast, evictable memory pool, as well as `vm.overcommit_memory=2` and `vm.swappiness=150`. This will effectively give you really good memory tiering for workloads and file cache.
When adding the other partitions to ZFS, use `-ashift=12 special mirror dev dev special mirror dev dev ...`. ZFS special covers all metadata, the intent log (sorta write cache), and optionally small files. I like to set it up so <= 8k small files get sent there, but you can probably go higher depending on how much capacity you allocate. My ~24T of allocated data ended up being ~150GB special with 8k small file, and that's with the whole pool configured with deduplication and blake3 for all hashes. Blake3 is fast as heck, but has very long hashes, so from a metadata standpoint, I'm using the most expensive option. I mitigate that a bit my setting metadata redundancy to `some`, since my metadata is effectively RAID10 anyway.
With some extra NVMe/Optane allocated to regular ZFS read cache, and all my spinning-rust data VDEVs also as RAID10, it's almost like having the whole array in memory, or at least on fast flash. Eliminating metadata from your drives seeking and letting them be written nearly instantly with Optane does wonderful things for spinning rust :)
reply