If that was available when I started to build my homelab server, I'd have bought it. My requirements were a low-power but modern and punchy mini-ITX board with an AMD processor in a very compact build with a 48v DC power supply and SmartOS.
That was basically unobtainium and I've compromised down to a AliExpress mini-ITX motherboard with a mobile AMD CPU, a AM5 heatsink and firmware that is... flavorful, powered by a screaming TFX power supply crammed in an absurdly tight 3.8L noname case running on Proxmox (when you start reconfiguring PCI Express bridges through the serial port kernel debugger because that's just about the only device the Illumos kernel enumerated at all on what is supposed to be your main server, it's time to give up on SmartOS).
It works, but it's a little box of pure hatred and heresy that's quite far off from what I've wanted initially. It is actually an improvement over my previous main server, somehow.
No offense but your requirements make very little sense for that use case, unless you really needed PCI-Express.
I’d have bought a Beelink or similar mini PC if I wanted small size and low power along with low price. You lose some modularity compared to ITX boards but I am almost certain you spent more money and deal with more noise and maybe even more power consumption.
For me personally my homelab PC is just an ATX mid tower in the closet because those parts are dirt cheap and you can get lots of performance with essentially infinite modularity.
Originally, the homelab was supposed to be located inside a 6U, 30cm deep, encased 19" rack. That would fit flush and very snugly just below my encased electrical panel, located right at the entrance inside my apartment.
That makes noise (a glass panel separates the rack from the entrance), power (can't risk overheating in an enclosed closet) and space (6U and 25cm of usable depth is an exceedingly small volume for a homelab) all important factors at the same time. I also wanted at least 2.5 Gib Ethernet, as much DDR5 RAM and CPU cores as I could chuck into it and two NVMe slots. I rejected a mini PC due to noise concerns and I wanted standardized parts and modularity.
I have a AMD Ryzen 9 7940HS, 64 GiB of RAM and terabytes of NVMe in less than 4 liters of volume without sacrificing modularity. I've mounted an AMD Wraith Stealth, which is a comically oversized CPU cooling fan for a 35W TDP thermal load. It's fast, small, silent, overkill and still fits standard parts.
I do want to get rid of the ridiculously noisy TFX power supply at some point, most likely for a 48v DC ATX power supply. Unfortunately, since I like stupid ideas I want a fully 48v DC powered rack and Power-over-Ethernet throughout my apartment to get rid of as many power supplies and fans as I can. I've started accumulating parts for it (like a 48v 2.5 Gib 10 port PoE switch, PoE powered gigabit switches and PoE-to-12VDC adapters), but I'm still not sure how I'll pull it off in the end since I'm way off the beaten path.
At this point you're probably wondering why I do things that way. I'm not sure myself, but seeing what I tend to do for fun it's probably best not to ask, lest the universe stops its suspension of disbelief while I'm in the middle of performing yet another heretical act on some poor unsuspecting piece of hardware or software.
Actually the more I think about it, it really just sounds like you should move to a mini-ITX case that uses an SFX power supply.
Those specs all seem totally attainable in a mini PC, although something with an equivalent chip and dual nvme slots is up in the $800-1000 range.
Maybe it would be louder? I don’t think they have a reputation for being loud though? I certainly can’t hear the one I’m using as a router, but it has a lower load profile than my main server.
I have a small apartment too but I might be lucky with my deep half height closet where I can shove equipment back there.
It boils down to a pet peeve of mine that requires more context.
A former sysadmin job gave me a deep disdain for 40mm fans and modern 1U/2U off-the-shelf AC powered servers. It's a technological dead-end that is a legacy of old datacenter architectures that no longer make sense to continue. A holistic blade-based DC powered datacenter eliminates much of the inefficiencies inherent to that legacy AC design.
Now, I'm not aiming to do a high-density homelab and regardless how cool they are, an Oxide Computer rack is about three orders of magnitude out of reach across several metrics. That being said, I think a DC powered homelab can enjoy a lot of the benefits.
For example, I have one 10 Gib switch, three gigabit switches (one per room), a Synology NAS and a Android TV player. With AC that's six power supplies, each taking a dedicated outlet for less than 60W combined. With PoE I can power all of that directly from my 10 Gib switch. That's five less power supplies and also gets rid of the power strips.
Or take a 250W ATX power supply. On AC it requires transformers, full bridge rectifiers, big bulky capacitors, DC-to-DC converters and most likely a fan. On DC it requires just the DC-to-DC converters, with modern power electronics that's doable at a fraction of the volume and without the fan because the actual power supply is upstream and potentially shared across multiple equipments.
I might have an affinity for doing stupid and insane projects, but it does come from a thoughtful position. If I ever pull it off, hopefully it can show that we don't have to endure outdated and legacy designs forever even within the comfort of our homes. Alas, I'm busy doing other cursed things at the moment so I just put that angry little server in a spare room for now.
You can get a M3/M4 Max with 128 GB of RAM as well. The Studio will give you > 128 GB.
I have a max with 64GB RAM, which is good enough for 70b models with a 3 bit quant. Even if I had more RAM to run larger models, my GPU would be the bottleneck.
> You can get a M3/M4 Max with 128 GB of RAM as well.
To get an M4 Max so you can have 128GB, you need a macbook pro. The cheapest macbook pro with 128GB is $4700.
M4 Max does have the benefit of more memory controllers, so it has twice as much memory bandwidth as Ryzen AI Max. But that's a lot of money to pay for it.
It seems to be squaring up directly against the mac studio with its efficient APU and big memory bandwidth use cases with a cheaper price tag. At least that's the loose sense that I got based on their keynote.
It's a bit different. Digits is based on the Tegra CPU, which is an ARM chip with integrated nvidia GPU. It's nearly COTS (commercial off the shelf), but not quite. Tegra CPU support isn't in mainline linux, so you have to run their fork of Ubuntu or build your own kernel. The integrated GPU is a special class in nvidia drivers, and some things just don't work on it (they only work on a discrete GPU) for seemingly no reason too.
Keep in mind the digits is part of their server line and server OS, not the random embedded dev kits often used for developers targeting car entertainment systems and often with abandoned kernel + driver.
The nvidia digits uses DGOS, same as their grace+hopper and similar enterprise/cloud products.
Can you expand on this? If I want to run open-weight LLMs and image generation models in the next few years, how likely is it that I will be able to run the "most popular" models (whatever they will be 12 months from now) if I buy Digits?
Will it? I'm not aware of any other than the HP workstation. Maybe one or two of the Chinese mini PC manufacturers. But nothing you can buy as a mini-ITX motherboard.
Of course you will. Asus et al have heard the buzz, have the connections, and can spin up a product far quicker than the 5-9 or months before Strix Halo is available.
It's soldered RAM, which is a fantastic trade-off for specific scenarios (inference). If all you care about is local inference, this thing is basically the same price as a 5090 (I think?) with multiple times the memory, and no need to purchase "everything else" (mobo, CPU, etc.) alongside. And given that home inference will typically be serving a single user at once, a handful at worst, you really have no need for a GPU. I'm guessing that this product will be uniquely positioned for quite a long time.
For every other use-case? Yeah, just get a desktop.
Right - there was no major market gap here. With laptops there was, but not desktops. Not sure the point of this. I hope they didn't spend too much eng time on it, rather than on their laptops. The F16 could use a new rev...
