College of Education and Human Development

Secondary Education Program Faculty Collaborates on an NSF-Funded Project to Construct a Digital Platform that Helps Students Fact Check AI-Generated Content

Dr. Erdogan Kaya

Dr. Erdogan Kaya, Assistant Professor in the Secondary Education 6-12 program in the School of Education at George Mason University, is part of a research team working on the development of a new digital platform that will guide K-12 students in critically evaluating and ethically using content produced by Artificial Intelligence (AI) in composing and writing science-focused text. In this endeavor, Dr. Kaya is collaborating with Drs. Amy Hutchison and Lori Bruner from the University of Alabama. All three researchers possess expertise in digital literacies, STEM, and Computer Science education. The initiative, which is titled “RAPID: DRL AI: The Development of a Digital Platform for Evaluating and Using AI-Generated Content for Academic Purposes,” is supported by a grant from the National Science Foundation (NSF). In announcing the funding opportunity through which this grant was awarded, the NSF Directorate for STEM Education highlighted the urgent need for research on the use of AI in K-12 education due to the rapid advancements in machine learning, generative AI, and Large Language Models (LLMs) now taking place.

The web-based platform that is being constructed is called “Compose with AI” and it is intended to help students identify, gather, and critically evaluate content produced by AI tools such as Gemini, Claude, and ChatGPT. The features of the platform will guide students on the ethical use of AI and help them understand the importance of verifying information accuracy.

In explaining how this project came to be, Dr. Kaya commented, “We started this project and initiated the development of ‘Compose with AI’ in response to a crucial observation. Specifically, we noticed that even with access to AI tools, students frequently grapple with utilizing them efficiently, and typically exhibit shortcomings in verifying facts, usually accepting the veracity of AI-generated content, and often falling short in fact-checking. They tend to take AI-generated content at face value.”

To address this situation, Dr. Kaya and his team members are designing the ‘Compose with AI’ platform to empower students with enhanced skills in prompt engineering and critical thinking. “We are committed to teaching students the importance of using these tools effectively, to critically evaluate technology, and to engage with AI ethically” he stated. “This will prepare them to be responsible digital citizens and help them understand the limitations, risks, and affordances of generative AI.”

The structure of the “Compose with AI” platform will include features such as embedded videos, self-regulated learning prompts, and scaffolds based on universal design principles. Once the initial design of the web application is finished, it will be field tested with students in grades 4-8 and their teachers to evaluate its usability. The research team will assess feedback to identify:

• Approaches and strategies that inhibit and enhance students’ abilities to use and critically evaluate content generated with AI;
• Uses of AI that inhibit and enhance students’ abilities to integrate AI-generated content into their writing in ethical ways; and
• The ways and extent to which teachers and students find “Compose with AI” to be useful and usable for integrating AI-generated content into their science-focused writing.

Based on their findings, the research team will revise the web application accordingly to enhance its effectiveness and utility.

Dr. Kaya, who has a Master of Science in Computer Science and is currently pursuing his second master’s in Machine Learning Engineering at George Mason’s College of Engineering and Computing, discussed the impact of AI on teaching and learning. He stated, “As AI becomes increasingly embedded in our society, an understanding of AI will empower learners to make informed decisions, evaluate AI-generated content, recognize inherent biases, and engage with AI ethically and responsibly. The RAPID project addresses the urgent educational challenges posed by the accelerated integration of Large Language Models into classrooms by helping students develop critical evaluation skills to ensure they can assess the reliability and accuracy of AI-generated content.”

An increasing number of educational institutions now recognize the many benefits of incorporating AI into the classroom. Among these is the potential to personalize learning by adapting content and pacing specific to a student’s individual needs and supporting more specialized educational activities for students. In addition, AI can expand creative possibilities by helping students generate music, videos, and images. AI can also create a more inclusive general education classroom by enhancing accessibility for students with disabilities and providing translation support for multilingual learners. Finally, AI can support teachers by assisting them with tasks such as composing parent emails, lesson planning, and creating differentiated activities for students.

While the deployment of advanced chatbots powered by LLMs has generated enthusiasm among K-12 teachers because of the potential of these tools to enhance teaching and learning, their use has also generated concern among these same educators. Issues such as inherent biases in AI algorithms and the risks associated with inaccurate content produced by AI can be problematic. One example referenced by Dr. Kaya is ChatGPT, an AI tool known to produce content that may not always be accurate, and which often struggles with complex quantitative reasoning, including math problems.

Other concerns relate to the need for comprehensive safeguards that will ensure the safe and responsible use of AI by young students. Dr. Kaya cites privacy laws, such as the Children’s Online Privacy Protection Act (COPPA), which prohibits children under the age of 13 from using the services of ChatGPT and which requires children between the ages of 13 and 18 to obtain parental consent before using the app. “This underscores the need to develop new tools specifically designed for students that comply with COPPA regulations,” Dr. Kaya stated.

In his closing remarks, Dr. Kaya emphasized that teachers must be involved in the development and implementation of AI so that it aligns with their expertise and serves as an enhancement to existing teaching practices, rather than as a burden. “The knowledge and perspective of teachers are invaluable,” he stated, “and their guidance is essential for the effective adoption of AI in education...By focusing on these areas, we can maximize the benefits of AI in education while minimizing potential risks, thus making AI a valuable ally. We aim to leverage this NSF RAPID project as a foundation to secure additional funding in the future, which will enable us to extend the reach of the ‘Compose with AI’ tool to benefit a more diverse range of educational settings.”

To learn more about the Computer Science concentration and other degree offerings in the Secondary Education 6-12 program in the School of Education at George Mason University, please visit the program website.