> Could it be psychological?

I think so. If you strongly associate the taste and smell of coffee with its stimulating effects, your body might react similarly to decaf coffee due to conditioned responses, even if the caffeine content is much lower