Utilization of Generative Pre-trained Transformer Model for Automatic Evaluation and Feedback on Scientific Manuscripts

Abstract

This study aims to explore the use of the Generative Pre-trained Transformer (GPT) model in the automatic evaluation of scientific papers, with a focus on the conformity of the feedback provided with the applicable academic writing guidelines. In this study, GPT was used to analyze manuscripts published in accredited scientific journals and to provide feedback on errors in grammar, spelling, citation format, use of academic terms, content organization, and quality of argument. The results showed that the GPT was highly effective in detecting and correcting technical errors in the manuscripts, with high correction rates for spelling and grammar errors (95% and 93%, respectively). In addition, the GPT also provided relevant suggestions for correcting formatting errors, such as citation and bibliography formats, with an improvement rate of 90%. The model also successfully provided suggestions to improve content organization and argument strengthening in scientific papers.  Although GPT is effective in correcting technical errors and providing structural feedback, human editing is still required to improve substantial and in-depth aspects of scientific papers. This study concludes that GPT can be used as an effective tool in the process of automatic evaluation of scientific papers, but the role of human editors is still needed for optimal results. This study also suggests further development in fine-tuning GPT to improve substantial analysis and strengthening of argument quality in scientific writing.