> though as i just mentioned, the ways of measuring that are via poor proxies at best.
Goodhart's law comes to mind:
> Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
I don't think we should measure it – at least, not obsessively. We should research and study it, and put in place measures that should theoretically work, and pay attention to whether what we're doing is working, but we shouldn't look at the measurements and go "this is making the numbers go down; we must be doing something wrong!" without at least double-checking that the numbers make sense.
Goodhart's law comes to mind:
> Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
I don't think we should measure it – at least, not obsessively. We should research and study it, and put in place measures that should theoretically work, and pay attention to whether what we're doing is working, but we shouldn't look at the measurements and go "this is making the numbers go down; we must be doing something wrong!" without at least double-checking that the numbers make sense.