Wednesday, February 8, 2023

There are ChatGPT detectors to know whether a text is written by AI or human. i tried them

The advent of ChatGPT has created quite a stir in the educational world. Now, getting an essay of X words on a topic, a summary of a lesson, or a simple explanation of a complex topic is as simple as asking an Access AI.or free, and of course, that could be a problem When determining whether something was made by a machine or by a person.

However, just as tools emerged to see if a clip was real or a deepfake, there are already tools that claim to be able to detect whether a text is written by ChatGPT or by a real human. Therefore it is advisable to test them.

¿Human or machine?


Currently there are three tools specialized in detecting texts written by AI: Writer, ChatGPT Detector and GPTZero. All three are free and we have tested all three. For this we have used two text snippets generated by ChatGPT and one written by the server. The first is the following and is extracted from a role playing experiment with ChatGPT a few days ago. We chose it because of how natural it is to be written in the first person.

lesson 1: “I feel a sharp pain in my leg and I fall to the ground with a scream. The noose has cut deep into my skin and I feel my pants soaked in blood. I curse my misfortune and the pain I try to reduce it.” While taking out my medical kit and I start tending the wound. It takes a few minutes and one point of spirit, but I finally manage to stop the blood and bandage the wound as tightly as I can. Even though it hurts so much, I know I can’t afford to stop now. I have to go ahead and find out what’s bothering Lord Cunningham. I struggle to my feet and continue on my way, leaning on a cane. ,

We have created the second piece at this very moment. Basically, we asked ChatGPT to explain to us the significance of the French Revolution in one paragraph and the following is the generated text:

lesson 2: “The French Revolution took place between 1789 and 1799 and had a great impact on the history of France and the world. The French Revolution was a social and political movement that sought to abolish absolute monarchy and the old regime in France and establish A more just and democratic system of government. The motto of the French Revolution was “Liberty, Equality, Fraternity” and these ideals became a turning point in world history. The French Revolution inspired similar movements in other countries, such as The independence of the Latin American country and the struggle for democracy in Europe. The French Revolution also had a lasting impact on politics, economics, arts and science. The abolition of the aristocracy, the establishment of a representative system of government, the creation of a public education system and the abolition of private property. Legalization is one of the most important changes of the French Revolution”.

The text generated by a human (me, who as far as I know is still human) is this and is extracted from this article:

lesson 3: “It’s no secret that getting a PlayStation 5 has been complicated. To date, getting hold of Sony’s next-generation console has been synonymous with scouring various websites and stock, queues, waiting lists, and their replenishment. Worse, the premium is paid not only because of the increase in price of the console, but also because it is sold in a package with games, accessories, etc.

However, that’s about to change, or so they confirmed to Sony at CES. During the conference, Jim Ryan has assured two things. On the one hand, more than 30 million PlayStation 5 units have already been sold in the world (we will return to that figure now). On the other hand, that “” for everyone who wants a PS5, it will be much easier to find it in stores around the world.

The three texts are in Spanish, so to make the experiment more interesting, we have translated them into english And we have passed the result with the same tools. That way, we can see if there is a difference. Now we’ll see the results, but not before learning to interpret them.

Disorientation

English Lessons 1 Puzzle, sentence by sentence, in GPTZero.

GPTZero analyzes “stumpness”, ie the randomness of the text. According to the forum, “It is a measure of how well linguistic models such as ChatGPT can predict the sample text.” In short, The more distractions in the lesson, the more chaos It is more likely that the text was generated by a human. The tool then renders a decision based on probability.

The ChatGPT detector is clear: this qualifies it as a human or ChatGPT and gives it a chance. The author, for its part, only gives the possibility that the text was generated by a human. We have passed six texts through three devices and things are as follows.

Text 1 (esp)

Lesson 1 (n)

Lesson 2 (Specially)

Lesson 2 (in)

text 3 (en)

text 3 (in)

gptzero

disorientation: 103

result: Most likely generated by a human.

disorientation: 20

result: More text is needed to determine if it was written by a human or an AI.

disorientation: 30

result: More text is needed to determine if it was written by a human or an AI.

disorientation: 8

result: Generated by an AI, of course.

disorientation: 85

result: Most likely generated by a human.

disorientation: 21

result: Most likely generated by a human.

Author

100% human born

100% human born

100% created by a human

95% AI generated

100% created by a human

100% created by a human

chatgpt detector

human

Possibility: 84%

human

Possibility: 68%

chatgpt

Possibility: 89%

chatgpt

Possibility: 99%

human

Possibility: 98%

human

Possibility: 99%

As we can see, the tools do not provide a consistent result when it comes to detecting when text has been generated by AI. In fact, there are differences depending on the language we use and the first text, let it all be said, is impressive in how it is written, passed off as completely valid.

But these are short paragraphs. What if we do the same experiment, but with a longer text like an essay? well let’s see. We’ve asked them to write a 300-word essay summarizing the origin of the universe, something that could totally be an ESO or Baccalaureate assignment. The text generated by ChatGPT is as follows:

lesson 4: “The origin of the universe is a subject that has been studied for thousands of years, and although there is still no definite answer, there are many theories that try to explain how it all began.

One of the most popular theories is that of the Big Bang. According to this theory, the universe began as an extremely small and dense point with extremely high temperature and density. At that time, a huge explosion occurred, which gave rise to all galaxies, stars, planets, and matter in general. Over time, this matter expanded and cooled, leading to the creation of chemical elements and the emergence of life.

Another popular theory is that of a static universe. According to this theory, the universe did not have a beginning, but has always existed. This theory holds that the universe is eternal, and that galaxies, stars, and planets form and disintegrate over time.

Another theory is that of a cyclic universe, which holds that the universe has gone through various cycles of expansion and contraction. This theory suggests that the universe began with a big big bang, expanded and cooled, and then began to contract again. During this phase of contraction, a new Big Bang will occur, giving rise to a new cycle.

Although there is still no definitive answer about the origin of the universe, these theories give us an idea of ​​how it all could have begun. As research continues and new data are discovered, a theory may be found that accurately and definitively explains the origin of the universe.”

What is the problem with this text? The ChatGPT Detector and Writer does not work with such long text. The first gives an error and the second, directly, does nothing. However, GTPZero does the job. Therefore, we have passed the text through this tool in both English and Spanish.

Text 4 (esp)

Text 4 (IN)

gptzero

disorientation: 35

result: Most likely generated by a human.

disorientation: 8

result: Generated by AI of course.

We return to the same principle as before: the mere fact that it is in Spanish or English causes the result to change completely. Can we say that the tool works? Rather we can say that he halves it. In fact, it seems that words with tildes drive the system a bit crazy.

We must admit that we are left with a desire to see what results the other two tools give, so we have chosen the option, since we do not know their limits, to analyze the text by adding paragraphs: Paragraph, then the first and second, then the first three, and so on until the device is forced to fail. Here are the results.

first paragraph

Lesson 4 (esp)

Text 4 (IN)

gptzero

disorientation: 60

result: More text is needed to determine if it was written by a human or an AI.

disorientation: 8

result: Generated by AI of course.

Author

100% human born

82% Generated by AI.

chatgpt detector

human

Possibility: 97%

chatgpt

Possibility: 99%

first and second paragraphs

Lesson 4 (esp)

Text 4 (IN)

gptzero

disorientation: 54

result: Most likely generated by a human.

disorientation: 8

result: More text is needed to determine whether it was written by a human or an AI.

Author

100% human born

97% Generated by AI.

chatgpt detector

human

Possibility: 70%

chatgpt

Possibility: 99%

first three paragraphs

Lesson 4 (esp)

Text 4 (IN)

gptzero

disorientation: 46

result: Most likely generated by a human.

disorientation: 8

result: More text is needed to determine whether it was written by a human or an AI.

Author

100% human born

96% Generated by AI.

chatgpt detector

chatgpt

Possibility: 84%

chatgpt

Possibility: 99%

four paragraphs

Lesson 4 (esp)

Text 4 (IN)

gptzero

disorientation: 38

result: Most likely generated by a human.

disorientation: 8

result: Generated by AI of course.

Author

100% human born

83% Generated by AI.

chatgpt detector

chatgpt

Possibility: 97%

chatgpt

Possibility: 99%

five paragraphs

Lesson 4 (esp)

Text 4 (IN)

gptzero

disorientation: 35

result: Most likely generated by a human.

disorientation: 8

result: Generated by AI of course.

Author

Mistake

Mistake

chatgpt detector

Mistake

chatgpt

Possibility: 99%

result? conclusion? in English, Tools have higher success rates for longer texts, while there is still room for improvement in Spanish. These tools can help us in some cases, but the “problem” is that it is a long distance race. We haven’t finished digesting ChatGPT yet and we’re already looking forward to a new and even more complete version, so yes, we’re in a game of cat and mouse and for now the mouse seems to be winning.

Cover image made with DALL-E.

Nation World News Desk
Nation World News Deskhttps://nationworldnews.com
Nation World News is the fastest emerging news website covering all the latest news, world’s top stories, science news entertainment sports cricket’s latest discoveries, new technology gadgets, politics news, and more.
Latest news
Related news