A Symmetric Metamorphic Relations Approach Supporting LLM for Education Technology
Abstract
Question-Answering (Q&A) educational websites are widely used as self-learning platforms, and pre-trained large language models (LLMs) play a crucial role in maintaining content quality. Despite their usefulness, LLMs still fall short of human performance. To tackle this issue, we propose leveraging symmetric Metamorphic Relations (MRs) to enhance LLMs' performance by improving their machine common sense. The goal is to ensure that learners receive more relevant content. This work presents an empirical experiment using one specific symmetric MR, three LLMs, and a publicly available dataset of labelled Stack Overflow data. We employ the symmetric MR to generate training data that augments the machine common sense of LLMs. Additionally, we prepare a separate set of training data consisting of labelled Stack Overflow data for comparison purposes. By comparing the results of a common ability test and the predictions made by LLMs trained with different training datasets, we can assess the potential practicality of our proposed approach. Our experimental results demonstrate that a Bert-based LLM trained with MR-generated data outperforms a Bert-based LLM trained solely with regular labelled data. This outcome highlights the effectiveness of symmetric MRs in enhancing LLMs' performance by improving their machine common sense. Subsequent studies can extend our approach to other domains related to education technology and explore additional MRs to further enhance the study experience of students. different datasets to create distinct groups for comparison. These LLMs then make predictions on various testing datasets to evaluate their levels of machine common sense and performance. The results demonstrate that training LLMs with symmetric MR-generated data can significantly improve the machine common sense level and performance of Bert-based LLMs in the content quality classification of Stack Overflow questions compared to regular training data solely. Keywords- Content quality prediction, large language model, question-answering (Q&A) website, metamorphic relations, machine common sense, natural language processing data augmentation. II. BACKGROUND Additionally, the