I've found LLMs to be very useful for, well, text-based things. I've not found the "bot" implementations useful, but they're better tech for summarization, highlighting important sections (i.e. what sentence of this product review should I show in bold, given the search term), and entity recognition (what are all the products mentioned here).
They are expensive to run, in terms of GPU cycles, but they are noticeably better than the previous models.
It's also hard to constrain them well. If you want 95% accuracy, it takes some tuning work. If you also want to avoid 1% total batshit nonsense (repeat "chicken" 50 times), then you have to check for that. Earlier models were sometimes wrong, but they were not quite so aggressively wrong as the 1% case of LLMs.
That's just my anecdotal experience, but it leaves me both optimistic about applications in the right spaces and worried that people are just shipping something that's OK 75% of the time and calling it a product.
They are expensive to run, in terms of GPU cycles, but they are noticeably better than the previous models.
It's also hard to constrain them well. If you want 95% accuracy, it takes some tuning work. If you also want to avoid 1% total batshit nonsense (repeat "chicken" 50 times), then you have to check for that. Earlier models were sometimes wrong, but they were not quite so aggressively wrong as the 1% case of LLMs.
That's just my anecdotal experience, but it leaves me both optimistic about applications in the right spaces and worried that people are just shipping something that's OK 75% of the time and calling it a product.