THE EFFECT OF GENERATIVE ARTIFICIAL INTELLIGENCE USAGE INTENSITY ON STUDENTS’ CRITICAL THINKING AND LEARNING OUTCOMES
DOI:
https://doi.org/10.54124/jlmp.v23i1.192Keywords:
Generative AI, critical thinking, learning outcome, AI literacy, higher educationAbstract
This study examines the effect of Generative Artificial Intelligence (Gen AI) usage on the critical thinking skills and learning outcomes of university students in Indonesia. The novelty of this research lies in the integration of AI literacy as a moderating variable, which has not been studied simultaneously in the context of Indonesian higher education. The study employed a one-shot case study pre-experimental method with 50 students as samples, divided into three groups based on Gen AI usage intensity: low (n=16), moderate (n=17), and high (n=17). Data were analyzed using one-way Analysis of Variance (ANOVA). Results revealed significant differences in critical thinking skills (F(2,47) = 18.43, p < .001) and learning outcomes (F(2,47) = 22.17, p < .001) among the three groups. The high-intensity Gen AI group obtained the highest mean critical thinking score (M = 84.71, SD = 5.34) and the best learning outcomes (M = 87.12, SD = 4.91). This study recommends structured integration of Gen AI into higher education curricula accompanied by enhanced AI literacy for students.
Downloads
References
Acar, O. A. (2023). AI Prompt Engineering Isn’t the Future. Harvard Business Review. https://hbr.org/2023/06/ai-prompt-engineering-isnt-the-future
Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Longman.
Asosiasi Penyelenggara Jasa Internet Indonesia. (2023). Laporan Survei Penetrasi Internet Indonesia 2023. APJII Press.
Badan Pusat Statistik. (2023). Statistik Pendidikan Tinggi Indonesia 2023. BPS RI.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., & Amodei, D. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://doi.org/10.5555/3495724.3495883
Campbell, D. T., & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research. Rand McNally.
Chen, L., Chen, P., & Lin, Z. (2023). Artificial Intelligence in Education: A Review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron & R. J. Sternberg (Eds.), Teaching Thinking Skills: Theory and Practice (pp. 9–26). W. H. Freeman.
Frieder, Simon Pinchetti, Luca Griffiths, Ryan R. Salvatori, Tommaso Lukasiewicz, Thomas Petersen, Philipp C. Chevalier, Alexis Berner, & Julius. (2023). Mathematical capabilities of ChatGPT. https://doi.org/10.48550/arXiv.2301.13379
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States Medical Licensing Examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education, 9(1), e45312. https://doi.org/10.2196/45312
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
Holstein, K., McLaren, B. M., & Aleven, V. (2018). Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms. In C. Penstein Rose (Ed.), Artificial Intelligence in Education (pp. 154–168). Springer. https://doi.org/10.1007/978-3-319-93843-1_12
Indonesian Digital Literacy Survey. (2023). AI Literacy Among Indonesian Higher Education Students 2023. IDLS Institute.
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274
Kemdikbudristek, R. I. (2023). Laporan Survei Penggunaan Kecerdasan Buatan dalam Pembelajaran Mahasiswa Indonesia. Kemendikbudristek Press.
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), 75–86. https://doi.org/10.1207/s15326985ep4102_1
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198
Lembaga Layanan Pendidikan Tinggi. (2023). Survei Kesiapan Dosen dalam Menghadapi Era AI di Perguruan Tinggi. LL-Dikti Press.
Long, D., & Magerko, B. (2020a). What is AI Literacy? Competencies and Design Considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10.1145/3313831.3376727
Long, D., & Magerko, B. (2020b). What is AI literacy? Competencies and design considerations. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16. https://doi.org/10.1145/3313831.3376727
Luckin, R. (2017). Towards artificial intelligence-based assessment systems. Nature Human Behaviour, 1(3), 0028. https://doi.org/10.1038/s41562-016-0028
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63(2), 81–97. https://doi.org/10.1037/h0043158
Mollick, E. R., & Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4475995
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041
Nugroho, A., Syarif, M., & Prasetyo, D. (2024). Pola penggunaan kecerdasan buatan generatif di kalangan mahasiswa Indonesia: Sebuah studi nasional. Jurnal Teknologi Pendidikan Indonesia, 14(1), 45–62. https://doi.org/10.26737/jtpi.v14i1.4401
Paas, F., Renkl, A., & Sweller, J. (2003). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38(1), 1–4. https://doi.org/10.1207/S15326985EP3801_1
Rahmawati, D., & Kurniawan, B. (2023). Dampak penggunaan ChatGPT terhadap academic dishonesty dan kemampuan menulis kritis mahasiswa di universitas Indonesia. Jurnal Pendidikan Dan Kebudayaan, 8(2), 112–128. https://doi.org/10.24832/jpnk.v8i2.3712
Santoso, P. Y., Hidayat, R., & Wulandari, T. (2023). Persepsi mahasiswa terhadap penggunaan ChatGPT sebagai alat bantu pembelajaran: Studi multi-kampus di Jakarta. Jurnal Penelitian Pendidikan, 23(3), 287–303. https://doi.org/10.17509/jpp.v23i3.59124
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257–285. https://doi.org/10.1207/s15516709cog1202_4
Sweller, J., van Merrienboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5
Tamkin, A., Brundage, M., Clark, J., & Ganguli, D. (2021). Understanding the capabilities, limitations, and societal impact of large language models.
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(15), 1–24. https://doi.org/10.1186/s40561-023-00237-x
UNESCO. (2023). Guidance for Generative AI in Education and Research. UNESCO Publishing. https://doi.org/10.54675/HMBB1350
Vygotsky, L. S. (1978). Mind in Society: The Development of Higher Psychological Processes. Harvard University Press.
Wardat, Y., Tashtoush, M. A., AlAli, R., & Jarrah, A. M. (2023). ChatGPT: A revolutionary tool for teaching and learning mathematics. EURASIA Journal of Mathematics, Science and Technology Education, 19(7), em2286. https://doi.org/10.29333/ejmste/13272
Watson, G., & Glaser, E. M. (1980). Watson-Glaser Critical Thinking Appraisal Manual. The Psychological Corporation.
Zawacki-Richter, O., Marin, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education—where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Jurnal Lingkar Mutu Pendidikan
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
-
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
-
NonCommercial — You may not use the material for commercial purposes.
-
NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
