You read me correctly, I don't see why load balancing should be taking place down at the app server level.

Aren't you going to have multiple machines each with their own blessing of unicorns? You're still going to have to use some kind of load balancer in front of independent sockets.