Only to the extent of the environmental impact being greater than the alternative activity. Like another commenter said, wealth is the ability to get people to do things. You're not wealthy if you crawl into a cave and just sit there doing nothing (ignoring that even then you are impacting the environment with your methane emissions).
For one thing it's clearly a problem if you include the environmental impact of constructing those bridges to nowhere.