They still won't let you look at pages if some domainer has aquired the domain and installed a robots.txt that disallows crawling.

They really should look at the date on the robots.txt and only apply it to pages retrieved while it is in effect.

Show us the pages from before the robots.txt became so restrictive!