These are very valid concerns and this is a thoughtful essay! My expectation is that this is more of a mission statement and that the process to detect AI will be ongoing and subject to refinement. Medium has already been doing very important work to combat AI by using human curators in the Boost program. Recently, my wife submitted a paper for her Master's program. The professor ran it through some plagarism tool and it scored like a 4%. We asked him to clarify what that meant, and he said it was a confirmation that the paper hadn't been plagariazed (which we already knew). There is some danger in assigning numbers to things. People see a number like 4% and believe it means something, even if the "real" number for plagarism on that paper was ZERO! I use things like spellcheck, but I switched to painting my own featured images because I suspect there's a desire out there to see something human (even if it is rudimentary art compared to what some people can achieve). The inescapable reality is that writers need protections against AI writing. If Medium is at the forefront of this effort, I guess that makes us the guinea pigs and we have to afford them some due consideration. So, I guess it's right to be concerned that our work will be incorrectly flagged, but we have to accept that's the new normal and be prepared to stay calm and defend ourself it it happens. This process, as clumsy as it might be in the beginning, is preferable to having our work swallowed by a tidal wave of AI generated "writing." Thanks again for this lovely post!
I too think this is a step in the right direction. But, yes, you bring up some valid concerns.
My experience as a college instructor has left me so frustrated with all of it!! They added AI detection software to the plagiarism software almost exactly one year ago. They took it out last month. So many issues. The system of identifying AI was flawed. It potentially flagged any use of Grammarly as AI generated. Students caught onto the quickly and would say “I used Grammarly.” But, the biggest issue? There is no way to prove anything is AI generated. It definitely takes a human to look through the AI flags to determine whether AI was used or not. There are definitely hallmarks - no use of specifics or examples, lots of vague statements. Ultimately without being able to provide proof of the use of AI, there is little academic honesty departments can do.
I run everything through a free detection site anyway. I would think that anything under 30% is likely fine, but that’s me. Even if it shows as 30% down to 20%, I’m scrutinizing.
I saw that email - I feel it is the step in the right direction - the recent bot purge has done wonders in eliminating those pesky WhatsApp and Telegram scams and improving Read Ratios - however there are some concerns - will genuine writers be caught up in this latest effort as they did in the bot purge and how does one identify which is AI given the huge number of stories published - I wrote a short form about it raising these questions where a number of writers weighed in - in case you want to check it out here's the link - https://medium.com/p/3d23f7dd5c28 - thanks
Better late than never. If they had kept letting this go on without any changes, we could have worried, but they know that, without actual human writers to provide content, their business model would collapse.
This is similar in my mind to a publication deciding to have a submission fee for writers. Because, as bad as AI can write English, the people running the machines want to make money rather than give it away even more.
On the surface, I like it. It’s the right action to take— especially given their initial embrace of AI.
My concerns echo about of what you’ve written here. A good idea poorly executed leaves a lot of pitfalls.
Who determines what is/what isn’t AI? Will writers receive any notifications, or will their stories just be picked off in the dead of night. If a story is flagged, what recourse will readers have? Who will they address? What role (if any) will editors play here?
Did they tease out any of these 2nd order effects?
An unintended consequence is that the ethical writers will fret over whether their writing will pass some arbitrary metric while the unscrupulous ones will figure out a way around the new policy.
It's a positive response in a general sense, but leaves too much ambiguity for both writers and readers. If my ability to publish episodic creative fiction is threatened, I'll do Substack as my sole publishing platform. I already use Medium as a funnel to bring people to my newsletter here by offering readers the ability to look at all of my archive and read the story installments a day earlier.
I love that they're doing this. It's troubling though that my post could be considered AI when it's not. I mostly write opinions and personal essays. I wonder if my opinion piece is in trouble. Like you said, only time can tell...
I really appreciate Medium for cracking down on fakes and doing its best to kick out purely AI-generated content masquerading as human writing. But as you point out, the unintended consequences could be a problem for all of us. If people would just play nice . . . !
What I think is why did I not get this email? I’m fact, why do I NEVER get these administrative emails? Yes, I have the setting turned on. More random BS from the Medium overloads that isn’t even reaching the inboxes of many writers. If it weren’t for you, I’d never know what’s going on over there. (PS, I hate AI.)
Your intro says AI content "and images". Do you have info about this aspect? Some writers have thrown themselves into the artificial image generators whole hog.
Linda, while I applaud Medium's efforts in this regard, I share your concerns. Not only are the detection tools inadequate, it's hard to see how they can ever be otherwise, since the whole point of AI is to make its output indistinguishable from humans'.
You bring up some very good points. Do you think when a writer consistently scores 98% or 100% AI written it is AI written? 25% AI seems to be still human-written. I have found that when I point out to a writer that I read five of their pieces and all scored 97 to 100% AI, they suddenly start being human. Tricky tricky tricky.
These are very valid concerns and this is a thoughtful essay! My expectation is that this is more of a mission statement and that the process to detect AI will be ongoing and subject to refinement. Medium has already been doing very important work to combat AI by using human curators in the Boost program. Recently, my wife submitted a paper for her Master's program. The professor ran it through some plagarism tool and it scored like a 4%. We asked him to clarify what that meant, and he said it was a confirmation that the paper hadn't been plagariazed (which we already knew). There is some danger in assigning numbers to things. People see a number like 4% and believe it means something, even if the "real" number for plagarism on that paper was ZERO! I use things like spellcheck, but I switched to painting my own featured images because I suspect there's a desire out there to see something human (even if it is rudimentary art compared to what some people can achieve). The inescapable reality is that writers need protections against AI writing. If Medium is at the forefront of this effort, I guess that makes us the guinea pigs and we have to afford them some due consideration. So, I guess it's right to be concerned that our work will be incorrectly flagged, but we have to accept that's the new normal and be prepared to stay calm and defend ourself it it happens. This process, as clumsy as it might be in the beginning, is preferable to having our work swallowed by a tidal wave of AI generated "writing." Thanks again for this lovely post!
I too think this is a step in the right direction. But, yes, you bring up some valid concerns.
My experience as a college instructor has left me so frustrated with all of it!! They added AI detection software to the plagiarism software almost exactly one year ago. They took it out last month. So many issues. The system of identifying AI was flawed. It potentially flagged any use of Grammarly as AI generated. Students caught onto the quickly and would say “I used Grammarly.” But, the biggest issue? There is no way to prove anything is AI generated. It definitely takes a human to look through the AI flags to determine whether AI was used or not. There are definitely hallmarks - no use of specifics or examples, lots of vague statements. Ultimately without being able to provide proof of the use of AI, there is little academic honesty departments can do.
I run everything through a free detection site anyway. I would think that anything under 30% is likely fine, but that’s me. Even if it shows as 30% down to 20%, I’m scrutinizing.
I saw that email - I feel it is the step in the right direction - the recent bot purge has done wonders in eliminating those pesky WhatsApp and Telegram scams and improving Read Ratios - however there are some concerns - will genuine writers be caught up in this latest effort as they did in the bot purge and how does one identify which is AI given the huge number of stories published - I wrote a short form about it raising these questions where a number of writers weighed in - in case you want to check it out here's the link - https://medium.com/p/3d23f7dd5c28 - thanks
Better late than never. If they had kept letting this go on without any changes, we could have worried, but they know that, without actual human writers to provide content, their business model would collapse.
This is similar in my mind to a publication deciding to have a submission fee for writers. Because, as bad as AI can write English, the people running the machines want to make money rather than give it away even more.
I've never used AI. Not even for first drafts. But I'm still concerned about being falsely accused.
How can a writer prove the writing isn't AI?
I'll breathe easier once that question is answered.
Excellent article, Linda.
On the surface, I like it. It’s the right action to take— especially given their initial embrace of AI.
My concerns echo about of what you’ve written here. A good idea poorly executed leaves a lot of pitfalls.
Who determines what is/what isn’t AI? Will writers receive any notifications, or will their stories just be picked off in the dead of night. If a story is flagged, what recourse will readers have? Who will they address? What role (if any) will editors play here?
Did they tease out any of these 2nd order effects?
There really is nothing for me to think until I get some transparency about their methods.
I’m encouraged that the scammers will be discouraged and go away. I am also concerned about false accusations.
An unintended consequence is that the ethical writers will fret over whether their writing will pass some arbitrary metric while the unscrupulous ones will figure out a way around the new policy.
It's a positive response in a general sense, but leaves too much ambiguity for both writers and readers. If my ability to publish episodic creative fiction is threatened, I'll do Substack as my sole publishing platform. I already use Medium as a funnel to bring people to my newsletter here by offering readers the ability to look at all of my archive and read the story installments a day earlier.
It's a brave new world for writers.
I love that they're doing this. It's troubling though that my post could be considered AI when it's not. I mostly write opinions and personal essays. I wonder if my opinion piece is in trouble. Like you said, only time can tell...
Thank God. Why would we ever allow it?
But yes, they need to be careful.
I really appreciate Medium for cracking down on fakes and doing its best to kick out purely AI-generated content masquerading as human writing. But as you point out, the unintended consequences could be a problem for all of us. If people would just play nice . . . !
What I think is why did I not get this email? I’m fact, why do I NEVER get these administrative emails? Yes, I have the setting turned on. More random BS from the Medium overloads that isn’t even reaching the inboxes of many writers. If it weren’t for you, I’d never know what’s going on over there. (PS, I hate AI.)
Your intro says AI content "and images". Do you have info about this aspect? Some writers have thrown themselves into the artificial image generators whole hog.
Linda, while I applaud Medium's efforts in this regard, I share your concerns. Not only are the detection tools inadequate, it's hard to see how they can ever be otherwise, since the whole point of AI is to make its output indistinguishable from humans'.
You bring up some very good points. Do you think when a writer consistently scores 98% or 100% AI written it is AI written? 25% AI seems to be still human-written. I have found that when I point out to a writer that I read five of their pieces and all scored 97 to 100% AI, they suddenly start being human. Tricky tricky tricky.