That’s not at all how this works. It doesn’t scan for images of arbitrary subjects. It scans for exact matches of known CP. Your vacation pics are in no danger of being flagged.
No, it uses perceptual hashing which is inexact, and a fuzzy metric like hamming distance between hashes to determine whether or not two images come from the same source image.
Not only is it entirely possible for two images to have the same perceptual hash, it's even more likely that two unrelated images have similar hashes, which would indicate to the system that one image is likely an edited version of a source image in their database.
It’s possible, but very unlikely. Then of course you need many matches to flag the account. And then of course there’s the manual review. The likelihood that an innocent person would get caught up in this at all is zero.
I have built products in this space. It is entirely likely, and in fact, it is incredibly common. You can look at literally any reverse image search engine's results and see this, because they use the same perceptual hashing techniques to do reverse image lookups.
And you don’t think the threshold for a match will be a lot tighter for this use case compared to an image search engine? And you’re ignoring all the other guards I mentioned? Come on. You may not like Apple, but they’re not stupid.
In the real world, exact matching of image hash values won't work.
It's routine for images to change in small ways over their lifetimes as they're shared. According to the model you suggest, modifying a single pixel in the image by even the smallest amount would cause a hash mismatch against the original. If Apple's system is truly that inflexible, it will be trivial to circumvent in no time. Just increment / decrement a random RGB pixel in each of your images, and voila, your porn is scot free.
Of course this countermeasure will be employed almost instantly by miscreants, so how will the FBI respond? Will they give up? Certainly not. They already have a blank check to spy on our phones. So they will devise a clumsier match algorithm that scans more sources of data on your phone and your cloud accounts and your backups, and produces more false positives. Why wouldn't they do this?
Once any telecom service provider opens a door which compromises security or privacy, they will have a much harder time closing it.
You’re correct that it’s not a pixel-by-pixel hash, but it’s still a hash of that specific image. It’s not analyzing the image subject and trying to identify it as CSAM.
