Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
breakingcups
3 months ago
|
parent
|
context
|
favorite
| on:
Nepenthes is a tarpit to catch AI web crawlers
Such as OpenAI, who will ignore robots.txt and change their user agent to evade blocks, apparently[1]
1:
https://www.reddit.com/r/selfhosted/comments/1i154h7/openai_...
Consider applying for YC's Summer 2025 batch! Applications are open till May 13
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
1: https://www.reddit.com/r/selfhosted/comments/1i154h7/openai_...