It's one of those things that I still tilt my head slightly at when thinking about how the human brain struggles with really large numbers. Its just an odd thing.
In a case like this, I suspect it has more to do with visualizing the solar system rather than how large something is. When we speak in AU, we can create a mental model with the Sun, Earth, and other object since everything is normalized to the size of Earth's orbit. When we speak in kilometers, a bit of arithmetic has to be done before creating the model. Regardless of the units though, we aren't directly visualizing the distances since they will be outside the scope of human experience until we actively start traveling the solar system.