Can confirm. My website is flooded with AI bots despite attempts to block crawlers to certain parts of it.

Correct me if I am wrong, but these implementations are all CPU bound?, i.e. if I have a good GPU, I should look for alternatives.

You are correct. This project is "on the CPU", so it will not utilize your GPU for computation. If you would like to try out a Rust framework that does support GPUs, Candle https://github.com/huggingface/candle/tree/main may be worth exploring

It's all implemented on the CPU, yes, there's no GPU acceleration whatsoever (at the moment at least).

> if I have a good GPU, I should look for alternatives.

If you actually want to run it, even just on the CPU, you should look for an alternative (and the alternative is called llama.cpp) this is more of an educational resource about how things work when you remove all the layers of complexity in the ecosystem.

LLM are somewhat magic in how effective they can be, but in terms of code it's really simple.

Yes. Depending on gpu 10-20x difference.

For rust you have the llama.cpp wrappers like llm_client (mine), and the candle based projects mistral.rs, and Kalosm.

Although, my project does try and provide an implementation of mistral.rs, I haven’t fully migrated from llama.cpp. A full rust implementation would be nice for quick install times (among other reasons). Right now my crate has to clone and build. It’s automated for mac, pc, and Linux but it adds about a minute of build time.

CPU, yes, but more importantly memory bandwidth.

An RTX 3090 (as one example) has nearly 1TB/s of memory bandwidth. You'd need at least 12 channels of the fastest proof-of-concept DDR5 on the planet to equal that.

If you have a discrete GPU, use an implementation that utilizes it because it's a completely different story.

Apple Silicon boasts impressive numbers on LLM inference because it has a unified CPU-GPU high-bandwidth (400GB/s IIRC) memory architecture.

Depends. Good models are big, and require a lot of memory. Even the 4090 doesn't have that much memory in an LLM context. So your GPU will be faster, but likely can't fit the big models.

Woah, this is very _fun_!

Something similar I worked on in the past https://github.com/lucgagan/auto-playwright/

Does it use ChatGPT every time you run the test or only when a test fails (to check if the selector has changed)?

Very cool. I've been using a similar approach to summarize videos on my website https://ray.run/videos/112-playwright-installation-2022 Doesn't cost much and getting great feedback about it.

Is there a way to try this before I buy it?

I am building something that would probably benefit from this, but with that price tag (solo indy dev) that's going to be a big ask! might be worth it, just no way of knowing without trying it first

Thanks for the response. I paused for a day responding to everyone to think through all the feedback I've received, and shared an update https://github.com/lucgagan/auto-playwright/issues/15

Theirs is a commercial project with MIT client. I made an open-source version of their project. (Their backend is closed source. I have no knowledge of it.)

If I was the creator of the project, I'd be happy with it. Most companies would not have donated anything.

Most people I would imagine

Would you use Facebook login with other apps? Or Google or Apple login? Which one do you prefer?

Google

Second to trad email/pass