Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Funnily enough, web scraping was actually the motivating use-case that started my co-founder and I building what is now openpipe.ai. GPT-4 is really good at it, but extremely expensive. But it's actually pretty easy to distill its skill at scraping a specific class of site down to a fine-tuned model that's way cheaper and also really good at scraping that class of site reliably.


Wow, Kyle, you should have mentioned it earlier!

We've been working on this for quite a while. I'll contact you to show how far we've gotten




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: