> Native hardware acceleration is supported on Macs with M1 and Intel-based Macs through Apple’s ML Compute framework.

This lead me to believe that it supported hardware acceleration for the architecture (because it lists Intel too), which made it seem ambiguous if it supports the specific accelerator abilities of the M1.

What "native hardware acceleration" is available on Intel-based macs?

ML Compute is GPU-accelerated on both Intel and M1, as well as natively supporting each CPU's respective vector instructions via the BNNS API in Accelerate.framework:

https://developer.apple.com/documentation/mlcompute

I don't think Apple have stated whether MLCompute also utilises the M1's "Neural Engine", but geohot did manage to target it in tinygrad:

https://github.com/geohot/tinygrad/tree/master/ane

The MLCompute API supports three options "CPU", "GPU" or "any", which is GPU sometimes and CPU sometimes. Not the neural engine.

There's some MLComputeANE code in the framework, but afaik there's no way to use it yet. Also, from looking at it, I believe the neural engine only supports float16.

As far as my understanding goes, ML Compute is for training models via CPU or GPU and not ever supposed to use the ANE: https://developer.apple.com/documentation/mlcompute

Core ML on the other hand is designed to use trained models and runs automatically on CPU, GPU or ANE depending on what fits the currently executed model layer best: https://developer.apple.com/documentation/coreml

Have you tried using Zimmer?

But that’s a CoreML model so that might not be possible with regular tensor flow models?

I agree. That is a terribly confusing sentence.