> (Because the Wayback Machine starts from scratch each time?)
I don't know what "start from scratch" would mean – the point is that each site is sampled many times throughout history. That said, it is very odd that a current change in robots.txt would prevent looking at old samples. And that's indeed what it looks like [1]:
I don't know what "start from scratch" would mean – the point is that each site is sampled many times throughout history. That said, it is very odd that a current change in robots.txt would prevent looking at old samples. And that's indeed what it looks like [1]:
> Page cannot be displayed due to robots.txt.
[1] https://web.archive.org/web/*/blog.reidreport.com