That was my very first concern when I thought about WebUSB too, but Chromium has a block list of Vendor IDs which includes various security keys.

https://chromium.googlesource.com/chromium/src/+/967d11212c9...