>Turnitin advertises the feature as "98% accurate". That's a pretty terrible accuracy rate for potentially destroying someone's life.
Assuming the 98% applies to both false negatives and positives:
For a given 20k person school, assuming they each write only one paper a year that's run through this, an administrator gets to expel 400 students a year!
The problem is, that's actually sort of the best-case scenario. Obviously, the protests and insanity of that situation would cause it to be quickly scrapped.
The more insidious issues is you have a sub-population of students that write in the "wrong" way. That means the average student might never get a strike, but this unlucky sub-population might get multiple strikes. That would seem to point to the results being "correct" -- after all, you'd expect a cheater to cheat multiple times.
If this population skew is significant enough -- say, 1% of the population writes in the "wrong" way, and the multiplier effect is 10x as strong for them -- you might never get a flag for a normal student. But what you're measuring is not cheating, but deviation from the norm in writing style.
I'll just point out that college students papers are exactly the sort of thing these AI chat things also do. Take a body of preexisting material and distill it into an essay. Really seems to me this is moving the bar as to what counts as cheating rather dramatically.
Memory that pops up is Barton Fink being told to write a script for a wrestling film.
Also sort of topical in high school had a quiz on a short story that I had not read. But I had read some of the authors other short stories. So I started out with 'While I failed to read 'story title'. Based on the author and the title I think the story is about bla bla. And the teacher took off 2 pts for not reading the story first.
Assuming the 98% applies to both false negatives and positives:
For a given 20k person school, assuming they each write only one paper a year that's run through this, an administrator gets to expel 400 students a year!
What fun!