Information Evaluation Skills in the Era of ChatGPT: Implications for Academic Writing and Language Education

Muhamad Abdul Muid; Adi Sucipto; Moh. Azharul Kholili

doi:10.67245/locana.v1i1.21

Authors

Muhamad Abdul Muid Universitas Islam Internasional Darullughah Wadda’wah, Pasuruan
Adi Sucipto Politeknik Pembangunan Pertanian Bogor, Bogor
Moh. Azharul Kholili Sekolah Tinggi Ilmu Tarbiyah Maskumambang, Gresik

DOI:

https://doi.org/10.67245/locana.v1i1.21

Keywords:

Information Evaluation Skills, ChatGPT, Academic Writing, Language Education, AI Literacy

Abstract

Purpose – This study aims to examine the role of information evaluation skills in the use of ChatGPT and to explore their implications for academic writing and language education. The study addresses growing concerns regarding the credibility of AI-generated information, including hallucination, misinformation, and fabricated references, which may affect students’ academic writing quality and learning outcomes.

Design/methods/approach – This study employs a qualitative approach using a library research method. Data were obtained from scholarly literature related to ChatGPT, information literacy, information evaluation skills, AI literacy, academic writing, and language education. The data were analyzed through content analysis and thematic analysis to identify key patterns, relationships, and conceptual connections among the reviewed themes.

Findings – The findings indicate that information evaluation skills are essential competencies that enable students to assess the credibility, accuracy, relevance, and reliability of AI-generated information before using it in academic work. These skills also support the development of critical thinking, academic integrity, AI literacy, and critical digital literacy in language learning. The study proposes a conceptual framework that positions information evaluation skills as a mediating competency linking ChatGPT usage with academic writing quality and language learning outcomes.

Research implications/limitations – This study is limited to literature-based analysis and does not involve empirical data from students, lecturers, or classroom practices. Future studies are recommended to test the proposed conceptual framework through empirical research.

Originality/value – This study contributes to AI-assisted language education by emphasizing information evaluation skills as a key competency for promoting the critical, responsible, and effective use of ChatGPT in academic contexts.

References

Adel, A., & Alani, N. (2025). Can generative AI reliably synthesise literature? Exploring hallucination issues in ChatGPT. AI & SOCIETY, 40(8), 6799–6812. https://doi.org/10.1007/s00146-025-02406-7

Alkamel, M. A. A., & Alwagieh, N. A. S. (2024). Utilizing an adaptable artificial intelligence writing tool (ChatGPT) to enhance academic writing skills among Yemeni university EFL students. Social Sciences & Humanities Open, 10, 101095. https://doi.org/10.1016/j.ssaho.2024.101095

Al-Sofi, B. B. M. A. (2024). Artificial intelligence-powered tools and academic writing: To use or not to use ChatGPT. Saudi Journal of Language Studies, 4(3), 145–161. https://doi.org/10.1108/SJLS-06-2024-0029

Athaluri, S. A., Manthena, S. V., Kesapragada, V. S. R. K. M., Yarlagadda, V., Dave, T., & Duddumpudi, R. T. S. (2023). Exploring the Boundaries of Reality: Investigating the Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References. Cureus. https://doi.org/10.7759/cureus.37432

Azimi, N., Talejerdi, M. R., Ghadaki, M., & Nouri, Z. (2026). Algorithmic Bias Awareness and Ethical Reasoning: Moderating the Impact of Systemic Biases in Generative Artificial Intelligence on Critical Thinking and Information Literacy in Higher Education. Assessment and Practice in Educational Sciences, 4(2), 1–12.

Bobkowski, P. S., & Younger, K. (2020). News Credibility: Adapting and Testing a Source Evaluation Assessment in Journalism. College & Research Libraries, 81(5), 822. https://doi.org/10.5860/crl.81.5.822

Boretti, A. (2026). Hallucinations in generative AI: A threat to scholarly integrity and the urgent need for publisher-led academically supervised verification. Energy Research & Social Science, 136, 104720. https://doi.org/10.1016/j.erss.2026.104720

Chaudhuri, J., & Terrones, L. (2025). Reshaping Academic Library Information Literacy Programs in the Advent of ChatGPT and Other Generative AI Technologies. Internet Reference Services Quarterly, 29(1), 1–25. https://doi.org/10.1080/10875301.2024.2400132

Chelli, M., Descamps, J., Lavoué, V., Trojani, C., Azar, M., Deckert, M., Raynier, J.-L., Clowez, G., Boileau, P., & Ruetsch-Chelli, C. (2024). Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis. Journal of Medical Internet Research, 26(1), e53164. https://doi.org/10.2196/53164

Cheung, L., & Crosthwaite, P. (2025). CorpusChat: Integrating corpus linguistics and generative AI for academic writing development. Computer Assisted Language Learning, 0(0), 1–27. https://doi.org/10.1080/09588221.2025.2506480

Chyne, R. C., Khongtim, J., & Wann, T. (2023). Evaluation of social media information among college students: An information literacy approach using CCOW. The Journal of Academic Librarianship, 49(5), 102771. https://doi.org/10.1016/j.acalib.2023.102771

Dalal, H. A., Taylor, A., & Whitfield, S. (2025). Assessing Students’ Information Literacy: Attitudes and Perceptions of College Students Across Generations. College & Research Libraries, 86(2), 255. https://doi.org/10.5860/crl.86.2.255

Darvin, R. (2025). The need for critical digital literacies in generative AI-mediated L2 writing. Journal of Second Language Writing, 67, 101186. https://doi.org/10.1016/j.jslw.2025.101186

de Araujo, G. C. L., & Schneider, M. A. F. (2025). Critical Information Literacy as a Compass: Using Generative AI in Academic Research and Writing. Library Trends, 73(4), 493–515.

Deep, P. D., & Chen, Y. (2025). The Role of AI in Academic Writing: Impacts on Writing Skills, Critical Thinking, and Integrity in Higher Education. Societies, 15(9). https://doi.org/10.3390/soc15090247

Dergaa, I., Chamari, K., Zmijewski, P., & Saad, H. B. (2023). From human writing to artificial intelligence generated text: Examining the prospects and potential threats of ChatGPT in academic writing. Biology of Sport, 40(2), 615–622. https://doi.org/10.5114/biolsport.2023.125623

Georgopoulou, M. S., Troussas, C., Krouska, A., & Sgouropoulou, C. (2025). Digital Literacy in Higher Education: Examining University Students’ Competence in Online Information Practices. Computers, 14(12). https://doi.org/10.3390/computers14120528

Habib, S., Shah, D. M. A., & Hanif, D. S. (2026). Enhancing Information Literacy in Higher Education: An Analysis of Postgraduate Students’ Everyday Information Practices and Use of Academic Sources. Pakistan Journal of Social Science Review, 5(1), 1091–1102.

Hirsh, S. (2025). Critical Thinking and Academic Writing in the Age of Generative AI: A Call to LIS Educators and Librarians. Pakistan Journal of Information Management and Libraries, 27, i–iv.

Huang, C.-W., Coleman, M., Gachago, D., & Van Belle, J.-P. (2024). Using ChatGPT to Encourage Critical AI Literacy Skills and for Assessment in Higher Education. In H. E. Van Rensburg, D. P. Snyman, L. Drevin, & G. R. Drevin (Eds.), ICT Education (pp. 105–118). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-48536-7_8

Hussain, B. A., Li, S., & Alsanad, A. (2022). Assessment of Information Literacy Abilities: A Case Study of Pakistan. Sustainability, 14(7). https://doi.org/10.3390/su14073807

Jacob, C., Kerrigan, P., & Bastos, M. (2025). The chat-chamber effect: Trusting the AI hallucination. Big Data & Society, 12(1), 20539517241306345. https://doi.org/10.1177/20539517241306345

Karaduman, C. (2025). Pre-Service EFL Teachers’ Perceived AI Literacy and Competency: The Integration of ChatGPT Into English Language Teacher Education. Sage Open, 15(3), 21582440251379712. https://doi.org/10.1177/21582440251379712

Kim, J., Yu, S., Detrick, R., & Li, N. (2025). Exploring students’ perspectives on Generative AI-assisted academic writing. Education and Information Technologies, 30(1), 1265–1300. https://doi.org/10.1007/s10639-024-12878-7

Lee, C., Kim, J., Lim, J. S., & Shin, D. (2025). Generative AI risks and resilience: How users adapt to hallucination and privacy challenges. Telematics and Informatics Reports, 19, 100221. https://doi.org/10.1016/j.teler.2025.100221

Liu, W. (2025). Language teacher AI literacy: Insights from collaborations with ChatGPT. Journal of China Computer-Assisted Language Learning, 5(2), 287–316. https://doi.org/10.1515/jccall-2024-0030

Liu, Y., Park, J., & McMinn, S. (2024). Using generative artificial intelligence/ChatGPT for academic communication: Students’ perspectives. International Journal of Applied Linguistics, 34(4), 1437–1461. https://doi.org/10.1111/ijal.12574

Maphoto, K. B., Sevnarayan, K., Mohale, N. E., Suliman, Z., Ntsopi, T. J., & Mokoena, D. (2024). Advancing students’ academic excellence in distance education: Exploring the potential of generative AI integration to improve academic writing skills. Open Praxis, 16(2), 142–159. https://doi.org/10.3316/informit.T2024041000014190886861950

Meng, X.-P., Guo, X.-G., Fang, J.-W., Chen, J., & Huang, L.-F. (2025). Fostering pre-service teachers’ generative AI literacy and critical thinking: An RSCQA approach. Educational Technology & Society, 28(3), 202–225.

Metze, K., Morandin-Reis, R. C., Lorand-Metze, I., & Florindo, J. B. (2024). Bibliographic Research with ChatGPT may be Misleading: The Problem of Hallucination. Journal of Pediatric Surgery, 59(1), 158. https://doi.org/10.1016/j.jpedsurg.2023.08.018

Mo, Z., & Crosthwaite, P. (2025). Exploring the affordances of generative AI large language models for stance and engagement in academic writing. Journal of English for Academic Purposes, 75, 101499. https://doi.org/10.1016/j.jeap.2025.101499

Monteith, S., Glenn, T., Geddes, J. R., Whybrow, P. C., Achtyes, E., & Bauer, M. (2024). Artificial intelligence and increasing misinformation. The British Journal of Psychiatry, 224(2), 33–35. https://doi.org/10.1192/bjp.2023.136

Naznin, K., Mahmud, A. A., Nguyen, M. T., & Chua, C. (2025). ChatGPT Integration in Higher Education for Personalized Learning, Academic Writing, and Coding Tasks: A Systematic Review. Computers, 14(2). https://doi.org/10.3390/computers14020053

Oladokun, B. D., Ajani, Y. A., Enakrire, R. T., & Adefila, E. K. (2026). AI literacy and ethical use of generative AI in academic writing among LIS scholars in Nigeria. Journal of Information, Communication and Ethics in Society, 1–21. https://doi.org/10.1108/JICES-11-2025-0345

Park, S., & Nan, X. (2026). Generative AI and misinformation: A scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation. AI & SOCIETY, 41(2), 1501–1515. https://doi.org/10.1007/s00146-025-02620-3

Rieh, S. Y., Bradley, D. R., Genova, G., Le Roy, R., Maxwell, J., Oehrli, J. A., & Sartorius, E. (2022). Assessing college students’ information literacy competencies using a librarian role-playing method. Library & Information Science Research, 44(1), 101143. https://doi.org/10.1016/j.lisr.2022.101143

Sharun, K., Banu, S. A., Pawde, A. M., Kumar, R., Akash, S., Dhama, K., & Pal, A. (2023). ChatGPT and artificial hallucinations in stem cell research: Assessing the accuracy of generated references – a preliminary study. Annals of Medicine and Surgery, 85(10), 5275. https://doi.org/10.1097/MS9.0000000000001228

Shin, D., Koerber, A., & Lim, J. S. (2025). Impact of misinformation from generative AI on user information processing: How people understand misinformation from generative AI. New Media & Society, 27(7), 4017–4047. https://doi.org/10.1177/14614448241234040

Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Kennedy School Misinformation Review, 4(5). https://ora.ox.ac.uk/objects/uuid:bfa56657-6e42-4839-876f-26eabd9807b3

Steinerová, J. (2026). AI Literacy in Support of Information Creativity of Doctoral Students. In S. Kurbanoğlu, J. Boustany, S. Špiranec, Y. Ünal, İ. Şencan, D. Kos, E. Grassian, L. Roy, & D. Mizrachi (Eds.), Information Literacy in an AI-Driven World (pp. 167–179). Springer Nature Switzerland. https://doi.org/10.1007/978-3-032-17272-3_14

Tacelosky, K., Sue Kasun, G., Shapiro, B. R., Liao, Y.-C., & Harris, K. (2025). Exploring critical AI literacy in language education: A case study. Foreign Language Annals, 58(4), 966–994. https://doi.org/10.1111/flan.70029

Trixa, J., & Kaspar, K. (2024). Information literacy in the digital age: Information sources, evaluation strategies, and perceived teaching competences of pre-service teachers. Frontiers in Psychology, 15. https://doi.org/10.3389/fpsyg.2024.1336436

van Niekerk, J., Delport, P. M. J., & Sutherland, I. (2025). Addressing the use of generative AI in academic writing. Computers and Education: Artificial Intelligence, 8, 100342. https://doi.org/10.1016/j.caeai.2024.100342

Wang, L., & Ren, B. (2024). Enhancing Academic Writing in a Linguistics Course with Generative AI: An Empirical Study in a Higher Education Institution in Hong Kong. Education Sciences, 14(12). https://doi.org/10.3390/educsci14121329

Xiao, F., Zhu, S., & Xin, W. (2026). Exploring the Landscape of Generative AI (ChatGPT)-Powered Writing Instruction in English as a Foreign Language Education: A Scoping Review. ECNU Review of Education, 9(1), 20965311241310881. https://doi.org/10.1177/20965311241310881

Zhao, P., & Liao, X. (2024). Advancing to the academics: How did first-year Chinese undergraduates evaluate academic literature? Thinking Skills and Creativity, 53, 101600. https://doi.org/10.1016/j.tsc.2024.101600