Usually IPv6 addresses are enclosed in square brackets when a port number is involved. But it's true that in many configuration files IPv6s are a nightmare to put in, especially because you never remember what syntax you have to use: sometimes you even have to duplicate semicolons (that's what Exim does...). But I think this is rather a convention problem for the config files, rather a problem of IPv6 addresses themselves.
The IPv6 people must have known that : was a common way to separate IPv4 and port numbers. IPv6 was standardized 4 years after the URL format which used a colon to denote a port number.
