Effectiveness And Efficiency Of LLM Models Vs Traditional Machine Learning In Sentiment Analysis Of Indonesian Language Product Reviews

Abstract

This research aims to conduct a comparative analysis of the performance and efficiency of several machine learning models in the task of sentiment analysis on Indonesian language customer reviews. In the digital business era, a quick and accurate understanding of customer opinions is a strategic asset for making decisions, from product development to marketing strategy. Four models were evaluated: two Transformer-based models (agufsamudra/indo-sentiment-analysis and ayameRushia/bert-base-indonesian-1.5G-sentiment-analysis-smsa), Naive Bayes, and K-Nearest Neighbors (KNN) on a dataset of 5,400 product reviews. The evaluation metrics used are Accuracy, Precision, Recall, and F1-Score. The results show that the Naive Bayes model and the Transformer model 'agufsamudra/indo-sentiment-analysis' achieve the highest performance with an F1-Score and accuracy of around 95%, significantly outperforming other Transformer models (90%) and KNN (47%). The crucial finding of this research is that the performance of the classical Naive Bayes model is equivalent to the state-of-the-art Transformer model. From an accounting and business perspective, this implies that solutions with much higher computational efficiency (Naive Bayes) can provide a more optimal Return on Investment (ROI) for large-scale implementation of customer sentiment monitoring systems.