As with all things measured from a wrist worn sensor, and ove the course of the order of a decade of essentially continuous wearing, I've found that the device may be wrong (and completely off the mark at times) but is also mostly right most of the time.
I don't expect anything else for a device I wear loose most of the time (and snug+proper placement when working out) yet it still gives me eerily accurate metrics to work with.
When I mean unconsciously, I mean being unaware in that moment. However reflective hindsight reveals not only correlations in habit, behaviour, and so on but also in hindsight from a calmer frame of mind you can be aware of just how anxious you had become.
Solely based on HRV would be daft. I'm saying it correlates strongly with many other indicators which together are an indicator of my anxiety, realised or not, at that moment.
Ad HRV is a number and other indicators are not, its a great way to have a more quantitative measure which I've found is very helpful. Anecdata and qualitative measures sure, but it works for me.