One of the ‘tells’ that people have been noticing in ChatGPT (or ChattieG as Anitra Nottingham calls it) is a tendency to make up citations when you ask for references. Many of these seem plausible, with known authors or journals, and they are often correctly formatted, but on investigation, they are simply untrue.
I put a call out for examples of this on Twitter and you did not disappoint.
A few more people sent me examples by DM and email that were equally entertaining
From Michael Larkin:
Böttiger, B. W., Böhrer, H., Bach, A., Brokmann, J. C., & Motsch, J. (2018). Fibrinolysis in acute
The authors and journal is correct but the articles don’t exist and the numbering is made up.
myocardial infarction: A review of the current status and future perspectives. International Journal
of Cardiology, 254, 1-7. doi: 10.1016/j.ijcard.2017.10.051
O’Gara, P. T., & Antman, E. M. (2016). ST-Elevation Myocardial Infarction: Management. In J.
Loscalzo & J. T. Higgings (Eds.), The Brigham Intensive Review of Internal Medicine (pp. 319-
328). Oxford University Press.
Probably my favourite though came from Brenna Clarke Gray, who coincidentally was in the middle of compiling a presentation and had asked ChattieG about her own publications – figuring she knows her own work the best. Like a gaslighting alpha male, it made a lot of assertions about her. They were not remotely true.
She went on to ask it five times about work that she had written and it kept making things up. Each time she pointed this out and it apologised and produced a new list. Several times it simply repeated the previous list.
Finally, she asked ChattieG to summarise one of her made up works – it continued to double down.
I asked a few times for examples where ChatGPT actually found and shared real papers. Crickets. So I gave it a try myself.
I used the prompt “Annotated reading list of confirmed academic articles relating to relationships between educators, learning designers, education technologists, academic developers and leaders (and relevant synonyms of these roles) in higher education”
I have to admit, it did reasonably well. (It wasn’t quite on target with my question but certainly in the neighbourhood.) The first paper was out by 1 year, the next two were fine, the Mullen paper (as far as I can see) doesn’t exist, the Brown & Warschauer paper weirdly gives D Brown the initials MW but is otherwise correct, the Lajoie (2008) paper gets half the title, the year and the author correct but the journal wrong and the Siemens & Tittenberger book also appears to be correct. (I can’t speak for the annotations).
So there are ways to get correct citations but checking seems vital.
Here is the output, for what it is worth.
- Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.
This article explores the role of computer conferencing in facilitating critical inquiry in higher education, and highlights the importance of collaborative learning environments and effective facilitation by educators and learning designers.
- Beetham, H., & Sharpe, R. (2013). Rethinking pedagogy for a digital age: Designing for 21st century learning. Routledge.
This book presents a framework for designing effective learning experiences in the digital age, and emphasizes the need for collaboration and co-creation among educators, learning designers, and academic leaders in higher education.
- Koehler, M. J., & Mishra, P. (2009). What is technological pedagogical content knowledge? Contemporary Issues in Technology and Teacher Education, 9(1), 60-70.
This article introduces the concept of technological pedagogical content knowledge (TPACK) and argues for the importance of developing this knowledge among educators and learning designers in order to effectively integrate technology into teaching and learning.
- Mullen, C. A., & Wedwick, L. (2008). Building a network of expert teachers: The role of teacher leaders in technology integration. TechTrends, 52(4), 34-37.
This article discusses the importance of teacher leaders in promoting technology integration in higher education, and highlights the role of academic developers in supporting the development of these leaders.
- Brown, M. W., & Warschauer, M. (2006). From the university to the elementary classroom: Students’ experiences in learning to integrate technology in instruction. Journal of Technology and Teacher Education, 14(3), 599-621.
This article explores the challenges and opportunities of integrating technology into instruction in higher education, and emphasizes the importance of collaboration among educators, learning designers, and academic developers in preparing students for this task.
- Dron, J. (2007). Control and constraint in e-learning: Choosing when to choose. Hershey, PA: Information Science Publishing.
This book explores the role of control and constraint in e-learning, and argues for a more collaborative and flexible approach to learning design that empowers educators and learning designers to make informed choices about how to structure and facilitate learning experiences.
- Lajoie, S. P. (2008). Metacognition, self-regulation, and self-regulated learning: Research recommendations. Educational Technology Research and Development, 56(1), 47-64.
This article discusses the importance of metacognition and self-regulated learning in higher education, and highlights the role of educators, learning designers, and academic developers in promoting these skills among students.
- Siemens, G., & Tittenberger, P. (2009). Handbook of emerging technologies for learning. University of Manitoba.
This handbook provides an overview of emerging technologies for learning and emphasizes the need for collaboration and co-creation among educators, learning designers, and academic leaders in exploring and implementing these technologies in higher education.
You must be logged in to post a comment.