Like every model I've seen there is something like this:
>>A decoder is trained to predict the corresponding text...
Prediction of expected text in the context of the previous text.
While this is valuable in casual transcription, it can be extremely dangerous in serious contexts.
From personal experience, having given a deposition with an "AI" transcription, it will literally reverse the meanings of sentences.
This is because it produces the EXPECTED output in a context, and NOT THE ACTUAL OUTPUT.
Like a speaker that clips the output, these types of systems 'clip' the really valuable information out of a transcription. Worse yet, this is a completely silent failure, as the transcript LOOKS really good.
Basic info theory shows that there is more information contained in 'surprising' chunks of data than in expected ones. These systems actively work to substitute 'expected' speech to overwrite 'surprising' speech.
The transcript I got was utter trash, multiple pages of errata I had to submit when the normal is a couple of lines. And as I said, some literally reversed the meaning in a consequential way, and yet completely silently.
This kind of silent active failure mode is terrifying. Unless it is solved, and I see no way to solve it without removing ALL predictive algos from the system, these types of systems must not be used in any situation of serious consequence, at least not without real redundancy and backup.
I've been saying this for years. Current "AI" algorithm are fundamentally flawed because they rely on a statistical approach. This works moderately well for some use cases but it will rarely give you 100% confidence.
Good luck with self-flying planes or self-running nuclear power plants.
>>Current "AI" algorithms are fundamentally flawed because they rely on a statistical approach.
YES! The old joke about "Artificial Stupidity" is actually more true than anyone realized.
These statistical so-called-AI systems actually work to actively REMOVE or sanitize out any unexpected information, making it all conform with the EXPECTED results from the training set.
This not only REMOVES the most high-information 'surprising' or unexpected nuggets, it actively HIDES them. When something unexpected comes up, it gets force fit into the expected prediction algorithms and output as if it were good.
I'm not saying that there are no useful things that can be done with this technology — there is a LOT of mundane work out there to be done.
But, we will never get this type of "AI" saying "Huh, that's odd, I wonder why that is?", which is exactly the kind of observation that leads a prepared and fertile mind to great discoveries.
One item I remember was that I said "Dr Kemeny" in relation to Dartmouth College (he was a famous mathematician, invented the BASIC programming language and was president of the college). It replaced those instances with "Jack Kennedy".
In another instance, I said that "Evidently, you have a reading comprehension problem.". It replaced it with "Evidently, I have a ...", completely reversing the meaning.
There was zero problems with the microphones or audio, and it was not rushed or mumbled talk. There were 80+ other examples over a few hours of talking, and some from other speakers. And those were just the obvious ones I could catch.
Another massive problem with this technology is that a human stenographer can notice when s/he missed something and didn't hear and ask the speaker to repeat or clarify what was said, and will often during a pause request clarification on spelling of names, addresses, etc. In contrast, this "AI" technology just barges ahead ASSuming that it knows what it is doing and inserts literally whatever sounds good in the transcript, completely silent that it doesn't have a clue.
Having seen this up close, I'm of the strong opinion that anyone foisting this software on the market without huge warnings that this is not usable for any critical functions is, basically a fraud. They know or certainly should know that these failures not only exist but are common and systemic, yet they barge along like it is OK. It is not.
>>A decoder is trained to predict the corresponding text...
Prediction of expected text in the context of the previous text.
While this is valuable in casual transcription, it can be extremely dangerous in serious contexts.
From personal experience, having given a deposition with an "AI" transcription, it will literally reverse the meanings of sentences.
This is because it produces the EXPECTED output in a context, and NOT THE ACTUAL OUTPUT.
Like a speaker that clips the output, these types of systems 'clip' the really valuable information out of a transcription. Worse yet, this is a completely silent failure, as the transcript LOOKS really good.
Basic info theory shows that there is more information contained in 'surprising' chunks of data than in expected ones. These systems actively work to substitute 'expected' speech to overwrite 'surprising' speech.
The transcript I got was utter trash, multiple pages of errata I had to submit when the normal is a couple of lines. And as I said, some literally reversed the meaning in a consequential way, and yet completely silently.
This kind of silent active failure mode is terrifying. Unless it is solved, and I see no way to solve it without removing ALL predictive algos from the system, these types of systems must not be used in any situation of serious consequence, at least not without real redundancy and backup.