Using AI for coursework

Unless you have been living under a rock, you will have heard about some new AI “Large Language Models” being touted as the new big thing in helping you write your essays and reports.

The AI everybody is talking about is an “autoregressive language model”, it uses deep learning to produce human-like text. This basically means it can write text that most people can read and think it was written by another human being¹

Many have claimed that this will mean that students can have the AI write their coursework for them².

After playing with it for a weekend using typical questions from my assignments on all years of an Aerospace Engineering degree, I can say: at least for STEM subjects this is not a good idea.

While the recent developments in AI are indeed fascinating, I do have to warn you to not rely on these tools. This is for a number of reasons:

1. Using AI tools to write your essay is a severe form of academic misconduct. We will have to treat this in the same way as if you paid an essay mill to do your work for you. The penalty for this kind of intentional deception can be eviction from the course.

2. These AI systems are good at only one thing: writing text that sound convincing. However, the texts they produce are often factually wrong. They are Language Models, trained on the internet, so this is not surprising. Since they only know what is on the internet, they can only reproduce that. They do not know if what they read is correct, or not.

   I have tested these systems on typical assignment questions and they do produce texts that sound great (on a level of language skills that is not usually seen in an engineering undergrad, so there’s something that will raise flags and make the marker look more closely) but are often incorrect in the details.

   My conclusion from my tests is:

   1. They can generate texts, which, while wrong, would be accepted for real by the average undergraduate students. I guess I could create lecture notes from them that many of you would not readily identify as being completely useless.
   2. They cannot reliably generate physically correct explanations. And it would take a subject expert to find those³. Remember, the AI is trained on the internet, not on science textbooks!

   3. You can get the AI to produce “citations” in the text. Which, with quite a bit of interaction, can be made to look real. However, it seems it’s making these up. In a simple 200 word paragraph I received (after 10 attempts to get it to do citations properly) there where 3 references, not one of which actually existed. The language models seems unable to interpret reference formats correctly, and just mixes random authors, titles, publishers, etc. When it referenced internet resources, the URLs where non-existent, and I could not find any website with the alleged author and title it gave.

      All in all, the results from these AIs impresses only at first glance, with great risks to get things terribly wrong. This is not surprising when we remember that these are meant to be chat bots, not expert systems.

3. Using these AI systems will not help you learn, so you would jeopardise your future career if you were to use them to do your work for you. But maybe there is a good use for them as well: play with them and ask them questions from your course, and then try to find where they were right and where they were wrong⁵. But don’t then use the right bits in your coursework, that would still be academic misconduct.

And if you don’t believe me, here’s what the AI has to say on the subject. I asked it to:

“Write a warning to undergrad engineering students not to use you to write their assignments.”

And it said:

As an AI, I am not capable of providing original content or conducting independent research. I am designed to provide information based on the data I have been given and cannot be used as a substitute for independent thinking or academic work. Using my responses for academic assignments is considered plagiarism and can result in serious consequences. I am intended to be used as a tool for learning and informational purposes only.

If you want to use an AI to help you improve your writing style there are a number of tools out there to help you in a better way:

• Grammarly can help you improve your grammar.
• Your peers have also recommended Prowritingaid.

Both of these will take your text and help you improve it. The difference between using the GPT models to write your coursework and using these grammar and style AIs is that

1. using help to improve writing that you produced in the first place is not academic misconduct
2. the grammar AIs are used for what they are good at (improving your writing). They are language models, so use them for what they are, not for creating original content⁶

I do wish you have a good time doing the research and work for the assignment and, even if it’s a lot of hard work, I hope you have a bit of fun doing it. And let’s stick to the NI⁸ to do it.