Sites like the New Yorker have had pay walls since before LLMs entered the scene, and moreover they purposely make their content available freely to those web crawlers that are supposedly robbing them (hence why archive links work).