I don't know what happened there, because it should be disabled by default. But if you accidentally enabled it, you can change that in Settings > Accessibility > Voice Control.

Not enabled.

But it keeps happening if I accidentally press my headset button for a millisecond too long. Had been an issue at least since iPhone 5.

You can't, wired headphones and maybe other devices can still activate it even if you disabled it in the settings :(