Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's probably for the best that chat interfaces avoid making direct HTTP calls to sources at run-time to confirm that they don't 404 - imagine how much extra traffic that could add to an internet ecosystem which is suffering from badly written crawlers already.

(Not to mention plenty of sites have added robots.txt rules deliberately excluding known AI user-agents now.)



Wouldn't it be the same amount of requests as a regular person researching something the old way?


If you watch the thinking panel in ChatGPT with GPT-5 Thinking it often consults dozens of pages in response to a single prompt.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: