Generative LLMs and the disruption of college paper writing

NOTE: This blog post was written jointly by me (GJ) and ChatGPT using the very process it describes: it started with loosely structured musings and observations on the topic that I dictated to my phone (Google Pixel with a Tensor chip); and then sent to ChatGPT-4-May24 with Plugins with specific instructions to keep the ideas and not add any information but re-write and re-organize the starter “ramblings” into a more coherent prose, aimed at college-level readers. I then re-read and significantly re-wrote (edited) ChatGPT-produced re-write, although I kept its overall flow and structure. I also asked it for a bullet summary (on top) which I have also tweaked significantly. Later, I developed and only partly tested a (not yet final) prompt that takes the improved (re-written) text (based on musings, re-writte, and my final-edits), and re-structures it into an outline of a slide presentation, suggesting slide titles and key idea bullet points for each slide, speaker notes for improvising / presenting, and suggestions of images that would best illustrate the presentation, including the descriptions of images, and keywords for each to find it using online searches. This, last part still requires some tweaking (the plug-in I used found such images, but many of the links it provided were broken), but here is what I got at this point in time…

Key Points:

• Disruptive Advent of Generative AI // Learners and Technological Adoption // Role of Academic Technology Teams // The Imperfections of AI Language Models // Practical Application Example // The Art of Prompt-Crafting // AI for Multi-format Content Creation // Looking to the Future

The advent of generative large language models like ChatGPT, BingChat, or Google Bard (what are they? YouTube IBM Video 8 mins.) has brought about a radical shift in the education landscape, particularly in the area of term-paper writing. Despite this obvious and very radical disruption, and the context in which assigning term papers as if nothing happened no longer makes any sense, a vast majority of instructors have yet to fully grasp the practical, vast implications of these seismic changes and figure out how to take-in and re-think this new, suddenly alien landscape, and integrate the technology into their teaching practice. This is not necessarily a reflection of the instructors’ shortcomings, but rather a result of the rapid emergence of these tools, that found most educators whose main domain of expertise is NOT technology or AI, completely unprepared. And it is not going to be easy.

Students, as is often the case with technological advancements, are generally ahead of their instructors in adopting and utilizing these tools but their way of using it is often a simple shortcut, intended to minimize effort with no harm to the grade, and not crafted thoughtfully to develop better understanding or new learning skills.

Academic technology teams, such as the one I lead, are working diligently to rapidly bridge this gap and provide at least some initial practical, immediately useable responses to this challenge. We aim to make faculty aware of the nature of the disruption that took place, and come up with useful, pedagogically sound use-cases for these new tools, and to develop specific processes to integrate these amazing new tools into their teaching practices in a way that benefits students, strengthens the learning process, and better equips students for the AI-saturated future world. Our goal is to enhance teaching and learning by leveraging the power of these barely emerging technologies, and to increase the awareness of the need to develop new skills how to use this new toolset.

It’s important to note that while AI language models are already very powerful, and are quickly improving with each new iteration, they are not without flaws. They are prone to errors and biases, often stemming from the data they were trained on and the programming tweaks made during their development. These biases and inaccuracies, often referred to as “AI hallucinations,” (IBM YouTube overview 9:37) can significantly impact the quality of the generated content.

One example of a practical application of these models that I have found beneficial (and used to write this blog post) is using them to improve an early rough draft, dictated somewhat casually into my phone (and transcribed by it into editable text), into more polished prose somewhere between a “blog” or popular magazine article, and “academic-style prose”: grammatically correct, but friendly in tone, and not overly formal. This process involves dictating a loosely structured flow of ideas on a specific topic, as they occur to you, and then using a Generative LLM to rewrite the text in a much better organized, logically-flowing, and more formal style. This approach not only saves enormous amounts of time (and, at least for me, the part of writing that is the least enjoyable), but also leverages the AI’s ability to consider a vast amount of parameters simultaneously, something the human brain – despite being able to do so much – is structurally not very well equipped to do. And yet the text that is the result of this transformation only includes my original ideas with no new added information (which I specifically asked it to do this way). In other words, it’s a text I could eventually produce from my initial draft, but it would require quite a bit of effort and a non-trivial amount of time. The technology cuts this process down, to about 20 seconds.

Experimenting with this use case (which I implemented fully to produce this blog post you are reading) lead me to understand that writing and then refining the instructions given to the LLM to generate a text that matches desired outcome (also known as prompting or prompt engineering – although I prefer prompt-crafting) is both a methodical process and a new skill that needs to be honed. This involves being precise with instructions and understanding that the AI may interpret instructions differently than intended. For instance, asking the AI to rewrite a text in a “formal” style may yield different results than asking for an “informal” style. It’s understanding what constitutes “formal”, at least initially was very different from mine, resulting in pompous, stiff re-writes that included my ideas and thoughts, but sounded nothing like me, when I write. It took about 10 iterations with major and minor tweaks to get the result I was happy with (At least as a starting point good enough for my own fine-point edits). Then I re-ran this prompt with 4 more samples of dictated “musings”, and with a few final, minor tweaks I had a prompt I could re-use with somewhat predictably consistent outcome: it only used my ideas, and with some manual post-editing produced something that read like me. Only much, much, very much faster. The process also made me realize – to my absolute, complete surprise – that crafting a prompt you are happy with, is a subjective process, and once you are happy with the outcome you see it a little bit like a password or toothbrush: you are not totally inclined to share it, because it produces something that is very much (conceptually and verbally) YOU. And yet there is something incredibly attractive and exhilarating in the idea that you can go for a relaxing walk in the park (or wherever you like to walk), dictate relaxed musings to your phone (a process that still needs improvement), and then come back to your office and have a well-written version of your musings available to you in a form ready for a final edit in less time it takes to get coffee.

Furthermore, AI can be used to transform this text, and your ideas ideas to be shared with your audiences in different formats: in addition to a narrative “re-write” I crafted a prompt to generate a bullet-point “preview” summary, that gets automatically placed before the text. I am also now experimenting with using the final human>GPT>human textedit to be a basis for automatically generating a slide presentation, with slide titles, key idea bullet points, speaker notes, and even the description and keyword sets for online searches for the images that would match each slide. The initial result is not-quite there, yet, but with a little patience and tweaking I am confident that it will eventually produce useful results that will save time.

This is just the beginning of a long exciting, and slightly scary journey of exploring the uses of generative AI in higher education. As we continue to navigate this new, emotionally tumultuous seascape, we will undoubtedly uncover more ways to harness the power of these tools to enhance teaching and learning.
Typewriter Image by Florian Krauer on Unsplash – thank you!

APPENDIX: Originally the bullet-point summary was too long: I asked ChatGPT-4 to generate a short intro phrase to serve as each points title, and then just used those shorter phrases for the summary I included on top of the post. Here is the complete original version it generated:

• Disruptive Advent of Generative AI: Generative large language models (LLMs) such as ChatGPT, BingChat, and Google Bard have introduced a radical shift in education, particularly in term-paper writing. However, many educators, especially those not specialized in technology or AI, have struggled to grasp the implications and integrate this technology into their teaching.
• Learners and Technological Adoption: Students are typically faster than their teachers in adopting and exploiting these tools, though their use is often geared towards minimizing effort and maintaining grades rather than enhancing understanding or learning skills.
• Role of Academic Technology Teams: Academic technology teams, like the one led by the author, strive to bridge the knowledge gap and provide practical responses to these challenges. Their goals are to raise faculty awareness of the disruption, develop pedagogically sound use-cases, and devise processes to integrate these tools into teaching practices beneficially.
• The Imperfections of AI Language Models: Despite their power and constant improvements, AI language models have shortcomings, including biases and inaccuracies, commonly referred to as “AI hallucinations.” These flaws can significantly impact the quality of the content generated.
• Practical Application Example: One beneficial application involves using AI models to transform an early rough draft dictated into a phone into grammatically correct, more formal, yet friendly prose. This approach capitalizes on the AI’s ability to handle numerous parameters simultaneously, saving a significant amount of time and effort.
• The Art of Prompt-Crafting: Prompt-crafting, a process of writing and refining instructions for the AI model, is a new skill that requires precision and understanding of how the AI interprets instructions. Crafting a suitable prompt is a subjective process, producing results that reflect the user’s personal style.
• AI for Multi-format Content Creation: Beyond transforming text, AI can generate content in various formats such as bullet-point summaries, slide presentations with speaker notes, and keyword sets for online image searches. While the initial results require patience and tweaking, the potential time-saving benefits are substantial.
• Looking to the Future: This exploration marks the beginning of a journey to discover the potential uses of generative AI in higher education, seeking ways to harness these tools to enhance teaching and learning processes.