Wait so adding a robots.txt exclusion for the Wayback Machine makes all previous archives of the site inaccessible? That's very odd behaviour, and really not the point of a robots.txt file... I would expect a robots.txt to control a bot's visits / scraping behaviour, not a site's history.
Yes. They use the robots.txt file to essentially ascertain ownership or control of the domain. Archive.org doesn't want to delete the content they have (for whatever reason), so the compromise they came up with is to read the robots.txt and then hide the content they have archived if the present domain owner/controller wants it to be that way.
If you remove the robots.txt setting, the archives become available again.
