I gave it a PDF recently and asked it to help me generate some tables based on the information there in. I thought I'd be saving myself time. I spent easily twice as long as I would have if it I had done it myself. It kept making trivial mistakes, misunderstanding what was in the PDF, hallucinating, etc.
Last summer I used one of the models to help translate a few German wikipedia pages to English, hoping it would make things easier by keeping all the formatting etc. that I'd lose if I copy-pasted mere content via Google Translate.
I did check the translations were correct as part of this — while my German isn't great, it was sufficient for this — and it was fine up until reaching a long table about the timeline of events relevant to the subject, at which point it couldn't help but make stuff up.
Still useful, but when you find the limits of their competence, there's no point attempting to cajole them to go further. They'll save you whatever % of the task in effort, now you have to do all the rest; it's a waste of effort to think either carrot or stick will get them to succeed if they can't do it in the first few tries.
