Companies like Google love kicking down the ladder. You can bet that the Google crawler will have its own "attestation token" but if you want to crawl the Web with your own code you'll be SOL.
All these billion-dollar tech companies got their start thanks to open, accessible, hackable systems. Now it's all being locked down so only the big guys can play, and the rest of us have to pay a fee just to put our "apps" into their walled gardens, and if we do anything they don't like (or are just unlucky) then we get banned forever.
"You can bet that the Google crawler will have its own "attestation token" but if you want to crawl the Web with your own code you'll be SOL."
Let's be real here and note that while most web properties welcome Google crawlers, there are many, many other scrapers/crawlers that offer zero value to web operators while costing resources.
Companies like Google love kicking down the ladder. You can bet that the Google crawler will have its own "attestation token" but if you want to crawl the Web with your own code you'll be SOL.
All these billion-dollar tech companies got their start thanks to open, accessible, hackable systems. Now it's all being locked down so only the big guys can play, and the rest of us have to pay a fee just to put our "apps" into their walled gardens, and if we do anything they don't like (or are just unlucky) then we get banned forever.