Better late than never. If they had kept letting this go on without any changes, we could have worried, but they know that, without actual human writers to provide content, their business model would collapse.
This is similar in my mind to a publication deciding to have a submission fee for writers. Because, as bad as AI can write English, the people running the machines want to make money rather than give it away even more.
I saw that email - I feel it is the step in the right direction - the recent bot purge has done wonders in eliminating those pesky WhatsApp and Telegram scams and improving Read Ratios - however there are some concerns - will genuine writers be caught up in this latest effort as they did in the bot purge and how does one identify which is AI given the huge number of stories published - I wrote a short form about it raising these questions where a number of writers weighed in - in case you want to check it out here's the link - https://medium.com/p/3d23f7dd5c28 - thanks
These are very valid concerns and this is a thoughtful essay! My expectation is that this is more of a mission statement and that the process to detect AI will be ongoing and subject to refinement. Medium has already been doing very important work to combat AI by using human curators in the Boost program. Recently, my wife submitted a paper for her Master's program. The professor ran it through some plagarism tool and it scored like a 4%. We asked him to clarify what that meant, and he said it was a confirmation that the paper hadn't been plagariazed (which we already knew). There is some danger in assigning numbers to things. People see a number like 4% and believe it means something, even if the "real" number for plagarism on that paper was ZERO! I use things like spellcheck, but I switched to painting my own featured images because I suspect there's a desire out there to see something human (even if it is rudimentary art compared to what some people can achieve). The inescapable reality is that writers need protections against AI writing. If Medium is at the forefront of this effort, I guess that makes us the guinea pigs and we have to afford them some due consideration. So, I guess it's right to be concerned that our work will be incorrectly flagged, but we have to accept that's the new normal and be prepared to stay calm and defend ourself it it happens. This process, as clumsy as it might be in the beginning, is preferable to having our work swallowed by a tidal wave of AI generated "writing." Thanks again for this lovely post!
Even with plagiarism checks, it has to be looked at - every academic paper *should* have a small percentage of matches - that means your references/work cited list is done correctly. But, it also picks up popular phrases, so that needs to be looked at as well. As an instructor, I usually don’t check if it’s less than 20% because that’s usually just popular phrases and references. Red flags for me are 0% or anything over like 25-30%.
I was wondering if the plagarism checker makes accomodations for direct quotations that are properly cited. I feel as if it's a misleading tool that's assigning a number that's not scientifically significant, but I see the need for it. It's all part of a process that needs to be refined. Both students and teachers these days are facing some real challenges.
One of the settings is to “exclude quotes,” so anything between quotation marks will not be flagged as a match. I believe that is the default, but it can be changed to where it will flag quotes as well.
Could you insert quote marks at the beginning and the end and make the font color the same color as the page so the professor didn't notice? Is that why you're suspicious if it scores zero?
Oooh that’s a new one! Hahaha! But, I’ve had students miss an end quote and it excluded a few paragraphs…
It’s amazing the ways students find to plagiarize!! The best so far, which I had to email my department head about - was a student who most likely bought a paper, ran it through a translator into another language and then back through to translate it back into English. It was impossible to read!! I’ve had students buy or copy/paste papers and use a thesaurus to change every other word - it’s usually obvious, but still mostly readable… when I got the one that wasn’t readable, I initially thought- it took more time for them to use a thesaurus changing words than it would have to actually write the paper!! I had no clue a student would do such a thing. That was another case where they couldn’t prove anything- so very frustrating!
I was seriously considering quitting over the AI issue though.
I always leave the setting to exclude quotes because that will up the percentage and be a waste of time to look through if it pushes it over 20%. I think I’ve had to set it to include them a few times when students have had misplaced quotation marks, or to verify their quote was accurate. But, I’m sure most instructors set it so it doesn’t flag quotes.
And, no, the number is not significant- which is why instructors need to look and review all of the flags. It will flag portions of the title page - student’s name and class for second assignments, if they use headings or still have some of the paper template “placeholders” in the paper, that will be flagged… so, it is a time consuming, but necessary process to look at every flag.
These stories would make an excellent article on AI writing for Medium. Do you write there as well? I know Linda has some publications. I have one called Write and Review and it would fit in there. Maybe a project for the summer when the semester has ended? :)
Seriously I was just thinking yesterday that I need to write an article about it LOL 😂 I haven’t. I have a substack and write on Medium, but haven’t had the time to focus on either lately! I will work on one though and let you know!
On the surface, I like it. It’s the right action to take— especially given their initial embrace of AI.
My concerns echo about of what you’ve written here. A good idea poorly executed leaves a lot of pitfalls.
Who determines what is/what isn’t AI? Will writers receive any notifications, or will their stories just be picked off in the dead of night. If a story is flagged, what recourse will readers have? Who will they address? What role (if any) will editors play here?
Did they tease out any of these 2nd order effects?
I too think this is a step in the right direction. But, yes, you bring up some valid concerns.
My experience as a college instructor has left me so frustrated with all of it!! They added AI detection software to the plagiarism software almost exactly one year ago. They took it out last month. So many issues. The system of identifying AI was flawed. It potentially flagged any use of Grammarly as AI generated. Students caught onto the quickly and would say “I used Grammarly.” But, the biggest issue? There is no way to prove anything is AI generated. It definitely takes a human to look through the AI flags to determine whether AI was used or not. There are definitely hallmarks - no use of specifics or examples, lots of vague statements. Ultimately without being able to provide proof of the use of AI, there is little academic honesty departments can do.
I run everything through a free detection site anyway. I would think that anything under 30% is likely fine, but that’s me. Even if it shows as 30% down to 20%, I’m scrutinizing.
Unfortunately, as of right now, there is no way to prove it is or isn’t AI. It’s impossible to recreate the same AI generated text. That’s why the college I work for took the AI detection tool out of the plagiarism detection software… they can’t put an academic dishonesty case on a student without proof.
In addition, to everything you said, Linda, is that there is no formal appeal process. It's case specific with no recourse if Medium powers-that-be decide the author is a cheat.
I really appreciate Medium for cracking down on fakes and doing its best to kick out purely AI-generated content masquerading as human writing. But as you point out, the unintended consequences could be a problem for all of us. If people would just play nice . . . !
Interesting. I do like running paragraphs/articles through AI to just give me ideas to tweak, or I'll ask for suggestions. I don't use what they rewrite word for word but have taken a few of their suggestions. I've also completely rejected many of their changes. I compared what they rewrote on a specific article and got 35-80% AI with Scribbr. When I put in my finished article that I had rewritten with a word changed here or there, it came out 5%. I am going to rewrite it and see if I can get it down further. One of my articles came in as 1%. I wonder if there is ever a 0% and what is really acceptable?
These are tricky times. I think in a post by Tony it did mention percentage of AI content that would be allowed perhaps to inspire an article, or broach a topic. (Sorry, I don't recall the details). It seems expedient for Medium to suspend the writers rather than contact them and ask if their work was wrongly tagged as A1 to write them back. Maybe they don't have the staffing to handle this type of communication, or want to save time, but being suspended must leave a bad taste I writers' pens. They might find Substack more accommodating -- I don't know. LinkedIn is encouraging their posters to use AI, and I'm afraid they are asking me questions and putting them in some AI bank. There's a lot of AI going on.
Linda, thanks very much for writing about Medium's notice. I don't seem to get their emails or perhaps I'm inundated with emails so much that I miss them. It's frightening to think that authentic writers face being kicked out of the program. I've seen through Medium friends in real time the ramifications wrongful accusations can cause.
An unintended consequence is that the ethical writers will fret over whether their writing will pass some arbitrary metric while the unscrupulous ones will figure out a way around the new policy.
It's a positive response in a general sense, but leaves too much ambiguity for both writers and readers. If my ability to publish episodic creative fiction is threatened, I'll do Substack as my sole publishing platform. I already use Medium as a funnel to bring people to my newsletter here by offering readers the ability to look at all of my archive and read the story installments a day earlier.
Interesting article, Linda. Thought provoking to say the least. Personally I'm not certain there will be a solution by Medium or anyone else. AI by some standards is a false concept in my opinion. To have a program that sorts out "imperfect" Oreo cookies coming off the line is a great use of AI. To have a program to track 1000 miles of fencing in the Outback and make repairs without human hands is a good use of AI. But using AI to replace human creativity is not something that's needed or in demand. Only to those focused on money. In my experience Grammarly provides a "bad answer" about 40% of the time. It scrapes the humanity right out of a sentence or paragraph and inserts something neat and sterile. - it's good at that. But is that what we want? AI generated content IS human content by design. Without pre-existing Human words, it fails because it can't write a love poem on its own - IT has no point of reference. Any effort Medium installs to detect AI generated content will fail a reasonable percentage of the time. Then humans will come up behind it and repair the damage. Like an AI run lawnmower that keeps mowing the neighbor's begonias because their garden starts 4 inches from where yours ends. It's a problem detecting AI content because AI generated content is a problem all on its own. A thought experiment: Remove money or revenue of any kind from the equation - now why do we need AI generated content? Who does it serve?
Better late than never. If they had kept letting this go on without any changes, we could have worried, but they know that, without actual human writers to provide content, their business model would collapse.
This is similar in my mind to a publication deciding to have a submission fee for writers. Because, as bad as AI can write English, the people running the machines want to make money rather than give it away even more.
I saw that email - I feel it is the step in the right direction - the recent bot purge has done wonders in eliminating those pesky WhatsApp and Telegram scams and improving Read Ratios - however there are some concerns - will genuine writers be caught up in this latest effort as they did in the bot purge and how does one identify which is AI given the huge number of stories published - I wrote a short form about it raising these questions where a number of writers weighed in - in case you want to check it out here's the link - https://medium.com/p/3d23f7dd5c28 - thanks
These are very valid concerns and this is a thoughtful essay! My expectation is that this is more of a mission statement and that the process to detect AI will be ongoing and subject to refinement. Medium has already been doing very important work to combat AI by using human curators in the Boost program. Recently, my wife submitted a paper for her Master's program. The professor ran it through some plagarism tool and it scored like a 4%. We asked him to clarify what that meant, and he said it was a confirmation that the paper hadn't been plagariazed (which we already knew). There is some danger in assigning numbers to things. People see a number like 4% and believe it means something, even if the "real" number for plagarism on that paper was ZERO! I use things like spellcheck, but I switched to painting my own featured images because I suspect there's a desire out there to see something human (even if it is rudimentary art compared to what some people can achieve). The inescapable reality is that writers need protections against AI writing. If Medium is at the forefront of this effort, I guess that makes us the guinea pigs and we have to afford them some due consideration. So, I guess it's right to be concerned that our work will be incorrectly flagged, but we have to accept that's the new normal and be prepared to stay calm and defend ourself it it happens. This process, as clumsy as it might be in the beginning, is preferable to having our work swallowed by a tidal wave of AI generated "writing." Thanks again for this lovely post!
Even with plagiarism checks, it has to be looked at - every academic paper *should* have a small percentage of matches - that means your references/work cited list is done correctly. But, it also picks up popular phrases, so that needs to be looked at as well. As an instructor, I usually don’t check if it’s less than 20% because that’s usually just popular phrases and references. Red flags for me are 0% or anything over like 25-30%.
I was wondering if the plagarism checker makes accomodations for direct quotations that are properly cited. I feel as if it's a misleading tool that's assigning a number that's not scientifically significant, but I see the need for it. It's all part of a process that needs to be refined. Both students and teachers these days are facing some real challenges.
One of the settings is to “exclude quotes,” so anything between quotation marks will not be flagged as a match. I believe that is the default, but it can be changed to where it will flag quotes as well.
Could you insert quote marks at the beginning and the end and make the font color the same color as the page so the professor didn't notice? Is that why you're suspicious if it scores zero?
Oooh that’s a new one! Hahaha! But, I’ve had students miss an end quote and it excluded a few paragraphs…
It’s amazing the ways students find to plagiarize!! The best so far, which I had to email my department head about - was a student who most likely bought a paper, ran it through a translator into another language and then back through to translate it back into English. It was impossible to read!! I’ve had students buy or copy/paste papers and use a thesaurus to change every other word - it’s usually obvious, but still mostly readable… when I got the one that wasn’t readable, I initially thought- it took more time for them to use a thesaurus changing words than it would have to actually write the paper!! I had no clue a student would do such a thing. That was another case where they couldn’t prove anything- so very frustrating!
I was seriously considering quitting over the AI issue though.
I always leave the setting to exclude quotes because that will up the percentage and be a waste of time to look through if it pushes it over 20%. I think I’ve had to set it to include them a few times when students have had misplaced quotation marks, or to verify their quote was accurate. But, I’m sure most instructors set it so it doesn’t flag quotes.
And, no, the number is not significant- which is why instructors need to look and review all of the flags. It will flag portions of the title page - student’s name and class for second assignments, if they use headings or still have some of the paper template “placeholders” in the paper, that will be flagged… so, it is a time consuming, but necessary process to look at every flag.
These stories would make an excellent article on AI writing for Medium. Do you write there as well? I know Linda has some publications. I have one called Write and Review and it would fit in there. Maybe a project for the summer when the semester has ended? :)
Seriously I was just thinking yesterday that I need to write an article about it LOL 😂 I haven’t. I have a substack and write on Medium, but haven’t had the time to focus on either lately! I will work on one though and let you know!
Just fully read this - was out and about and just read the first part lol! Yes, I will check out your publication! Thanks!
On the surface, I like it. It’s the right action to take— especially given their initial embrace of AI.
My concerns echo about of what you’ve written here. A good idea poorly executed leaves a lot of pitfalls.
Who determines what is/what isn’t AI? Will writers receive any notifications, or will their stories just be picked off in the dead of night. If a story is flagged, what recourse will readers have? Who will they address? What role (if any) will editors play here?
Did they tease out any of these 2nd order effects?
I too think this is a step in the right direction. But, yes, you bring up some valid concerns.
My experience as a college instructor has left me so frustrated with all of it!! They added AI detection software to the plagiarism software almost exactly one year ago. They took it out last month. So many issues. The system of identifying AI was flawed. It potentially flagged any use of Grammarly as AI generated. Students caught onto the quickly and would say “I used Grammarly.” But, the biggest issue? There is no way to prove anything is AI generated. It definitely takes a human to look through the AI flags to determine whether AI was used or not. There are definitely hallmarks - no use of specifics or examples, lots of vague statements. Ultimately without being able to provide proof of the use of AI, there is little academic honesty departments can do.
I run everything through a free detection site anyway. I would think that anything under 30% is likely fine, but that’s me. Even if it shows as 30% down to 20%, I’m scrutinizing.
I've never used AI. Not even for first drafts. But I'm still concerned about being falsely accused.
How can a writer prove the writing isn't AI?
I'll breathe easier once that question is answered.
Excellent article, Linda.
i too have worried Medium will mistakenly flag me/delete me as AI. If I knew how to avoid it, I sure would!
Unfortunately, as of right now, there is no way to prove it is or isn’t AI. It’s impossible to recreate the same AI generated text. That’s why the college I work for took the AI detection tool out of the plagiarism detection software… they can’t put an academic dishonesty case on a student without proof.
I'm concerned, too, Linda.
:-/
In addition, to everything you said, Linda, is that there is no formal appeal process. It's case specific with no recourse if Medium powers-that-be decide the author is a cheat.
I really appreciate Medium for cracking down on fakes and doing its best to kick out purely AI-generated content masquerading as human writing. But as you point out, the unintended consequences could be a problem for all of us. If people would just play nice . . . !
I’m encouraged that the scammers will be discouraged and go away. I am also concerned about false accusations.
Interesting. I do like running paragraphs/articles through AI to just give me ideas to tweak, or I'll ask for suggestions. I don't use what they rewrite word for word but have taken a few of their suggestions. I've also completely rejected many of their changes. I compared what they rewrote on a specific article and got 35-80% AI with Scribbr. When I put in my finished article that I had rewritten with a word changed here or there, it came out 5%. I am going to rewrite it and see if I can get it down further. One of my articles came in as 1%. I wonder if there is ever a 0% and what is really acceptable?
Thank God. Why would we ever allow it?
But yes, they need to be careful.
These are tricky times. I think in a post by Tony it did mention percentage of AI content that would be allowed perhaps to inspire an article, or broach a topic. (Sorry, I don't recall the details). It seems expedient for Medium to suspend the writers rather than contact them and ask if their work was wrongly tagged as A1 to write them back. Maybe they don't have the staffing to handle this type of communication, or want to save time, but being suspended must leave a bad taste I writers' pens. They might find Substack more accommodating -- I don't know. LinkedIn is encouraging their posters to use AI, and I'm afraid they are asking me questions and putting them in some AI bank. There's a lot of AI going on.
Linda, thanks very much for writing about Medium's notice. I don't seem to get their emails or perhaps I'm inundated with emails so much that I miss them. It's frightening to think that authentic writers face being kicked out of the program. I've seen through Medium friends in real time the ramifications wrongful accusations can cause.
"So basically, AI replicates humans, but if we write like the content that AI is trained on, we get flagged as AI. Ouroboros. Snake eating it’s tail."
I have a headache!
There really is nothing for me to think until I get some transparency about their methods.
Good luck with that.
Haha, yeah…
An unintended consequence is that the ethical writers will fret over whether their writing will pass some arbitrary metric while the unscrupulous ones will figure out a way around the new policy.
It's a positive response in a general sense, but leaves too much ambiguity for both writers and readers. If my ability to publish episodic creative fiction is threatened, I'll do Substack as my sole publishing platform. I already use Medium as a funnel to bring people to my newsletter here by offering readers the ability to look at all of my archive and read the story installments a day earlier.
It's a brave new world for writers.
Interesting article, Linda. Thought provoking to say the least. Personally I'm not certain there will be a solution by Medium or anyone else. AI by some standards is a false concept in my opinion. To have a program that sorts out "imperfect" Oreo cookies coming off the line is a great use of AI. To have a program to track 1000 miles of fencing in the Outback and make repairs without human hands is a good use of AI. But using AI to replace human creativity is not something that's needed or in demand. Only to those focused on money. In my experience Grammarly provides a "bad answer" about 40% of the time. It scrapes the humanity right out of a sentence or paragraph and inserts something neat and sterile. - it's good at that. But is that what we want? AI generated content IS human content by design. Without pre-existing Human words, it fails because it can't write a love poem on its own - IT has no point of reference. Any effort Medium installs to detect AI generated content will fail a reasonable percentage of the time. Then humans will come up behind it and repair the damage. Like an AI run lawnmower that keeps mowing the neighbor's begonias because their garden starts 4 inches from where yours ends. It's a problem detecting AI content because AI generated content is a problem all on its own. A thought experiment: Remove money or revenue of any kind from the equation - now why do we need AI generated content? Who does it serve?