ZFS sort of moved inside the NVMe controller - it also checksums and scrubs things all the time, you just don't see it. This does not, however, support multi-device redundant storage, but that is not a concern for Apple - the vast majority of their devices have only one storage device.

Everyone seems to focus on the checksumming and scrubbing.

But the real win (at least for me) - every device i have - laptops, desktops, server, even PLC's (they now use freebsd under the covers + ZFS) all backup using zfs snapshots and replication.

I do not ever worry about finding an old file i accidentally deleted. Or restoring a backup to a new machine and "did it really include everything" or anything else.

The machine storing backups is itself replicated to another machine in my detached garage.

If i wanted even more security, i could trivially further replicate it to offsite storage in the same manner.

All of this takes ~0 time to set up, and require 0 maintenance to keep working.

Meanwhile, Apple has gone backwards - time machine can't even make actual full system backups anymore.

Strange specs table - it seems to ignore the tensor core FLOPs, which is what you'd be using most of the time if you're interested in computational throughput.

I think they can be, except of course the problems need to be much harder, and impossible to solve via vibe coding alone. Like it or not, AI assistance is going to stay with us. This is the "new baseline" against which engineers will be judged.

I think the same about education. Assignments must be harder now, it is impossible and undesirable to ban AI assistants.

Students might complain that their AI assistant was acting unusually dumb just before the assignment deadline.

Note that this is _way_ slower at small batch sizes you'd need for interactive use. At batch size 1 this seems to run at 1/3rd the speed of bf16 (so about 1/6th the speed of fp8 you'd realistically be using) if figure 5 is to be believed. This is actually a pretty impressive feat in itself if you know anything about GPU kernel programming, but it is much slower nevertheless. For this to work at "wire speed" it'd need hardware support, which takes years. Their "baseline" elsewhere in the paper is CPU offloading, which is dog slow and can't be made fast due to PCIe bottleneck.

It's perfectly possible to run LLMs quickly on CPUs. An Epyc or Xeon with 12 memory channels achieves similar memory bandwidth to a 4090, which is the limiting factor. Engineering sample Epycs in kits with motherboard and RAM are available on Aliexpress for reasonable prices even.

Did I say it wasn't? If your context is short and your model is small, it is possible to run LLMs on high-end CPUs able to support 12 channels of high-spec DDR5 RDIMMs. It's not possible to run them as fast as they'd run on a GPU equipped with HBM though. Nor would it be even remotely as energy efficient. Also, it's not possible to run LLMs quickly on CPU if your context is long, because CPUs do not have the requisite FLOPS to process long context quickly. And before you bring MoE into the conversation, MoE only affects the feedforward part of each transformer block, and full memory bandwidth and compute savings are only realized at batch size 1, sequence length 1, AKA the most inefficient mode that nobody other than Ollama users use in practice. Sequence length 8 (common for speculative decoding) could be using up to 8x37B parameters (assuming you want to run DeepSeek - the strongest available open weights model). Batch size of even 2 with sequence length 8 could use almost all parameters if you're particularly unlucky. Prompt will almost certainly use all parameters, and will slam into the FLOPS wall of your EPYC's ALUs. So can LLMs (with an emphasis on "Large") be run on CPUs? Yes. Are you going to have a good time running them this way? No.

llamafile contains specific optimizations for prompt processing using AVX512 for dealing with just this issue: https://justine.lol/matmul/ (about a 10x speedup over llama.cpp)

Somewhere between 8 and 192 cores I'm sure there's enough AVX512 to get the job done. And we've managed to reinvent Intel's Larrabee / Knights concept.

Sadly, the highly optimized AVX512 kernels of llamafile don't support these exotic floats yet as far as I know.

Yes, energy efficiency per query will be terrible compared to a hyperscaler. However privacy will be perfect. Flexibility will be higher than other options - as running on the CPU is almost always possible. Even with new algorithms and experimental models.

Hi! one of the contributors to the paper — we have kernels not released yet that can shave down decoding latency by >20%.

Also when we ran experiments for streaming with the current kernels, we were median ~1.3x slower at inference

Thanks for chiming in! How do you explain the top-most graph in Figure 5? Am I misreading it?

Depends on what kind of exercise. A few heavy sets of barbell squats or deadlifts will put you out pretty good. Especially the deadlifts.

> Read between the lines, sort of like people used to read Pravda in the Soviet Union.

To be fair, you're not far off at this point.

This is why I don't believe 90% of what they say about Trump. As soon as he took office in 2017 it was so obvious that MSM was paid to absolutely destroy him. Then in 2021 suddenly everyone single problem wasn't due to the president anymore.

I woke up to this in 2016 when theretofore beloved public figure Donald Trump turned into literally Hitler immediately after he descended down that elevator in the Trump tower, all without changing a single opinion he'd ever held. And it's been unrelenting ever since.

and he was a former democrat! I'm not saying he's perfect, I'm not saying he's sophisticated, or that he hasn't said some stupid things. No way, there are way better republican candidates. But TDS is real.

Are you one of those people who somehow believe that, because the pop culture "chemical imbalance" ideology was never factual, SSRIs don't work.

They are continually prescribed because their actual mechanism doesn't matter, they demonstrably work. That is a matter of statistics, not science.

Anti-science types always point to the same EXTREMELY FEW examples of how science "fails", like Galileo (which had nothing to do with science) and ulcers.

They never seem to point to the much more common examples where people became convinced of something scientifically untrue for decades despite plenty of evidence otherwise. The British recognized a link between citrus and scurvy well before they were even called "Limeys"! They then screwed themselves over by changing some variables (cooking lime juice) and instead turned to a quack ("respected doctor" from a time when most people recognized doctors were worse than the sickness they treated) who insisted on alternative treatment. For about a hundred years, British sailors suffered and died due to one quacks ego.

Phrenology was always, from day one, unscientific. You STILL find morons pushing it's claims, using it to justify their godawful, hateful, and murderous world views.

Ivermectin is a great example, since you can create a "study" in Africa to show Ivermectin cures anything you want, because it is a parasite killer and most people in impoverished areas suffer from parasites, so will improve if they take it. It's entirely unrelated to the illness you claim to treat, but nobody on Facebook will ever understand that, because they tuned out science education decades ago.

How many people have died from alternative medicine quacks pushing outright disproven pseudoscience on people who have been told not to trust scientists by people pushing an agenda?

How much money is made selling sugarpills to idiots who have been told to distrust science, not just "be skeptical of any paper" but outright, scientists are in a conspiracy to lie to you!

SSRIs may work, but the science isn't settled that they work better than a placebo: https://bmjopen.bmj.com/content/9/6/e024886.full . And they come with side effects like sexual dysfunction that other treatments (like therapy) don't face.

not to mention the sexual dysfunction can be permanent.

As far as I can tell, it's "$20K" the same way Cybertruck was "$39K". It's not available for purchase yet, and when it is, it'll be twice as much, because Bezos also likes money.

Why though? Does anyone use it? I tried to use it on my iPad Pro and couldn't get used to it. This would make sense if some iPhone model was a tri-fold foldable with a huge screen, but I don't think that's going to happen in the foreseeable future.