Perbandingan Kinerja Tiga Metode LSTM BiLSTM, dan CNN dengan Multi-Head Self-Attention pada Analisis Ujaran Kebencian

• IRVAN YUNIAR M.
• 14210175

---

ABSTRAK

ABSTRAK

Nama              : Irvan Yuniar Marconus

NIM                 : 14210175

Program Studi : Ilmu Komputer

Fakultas           : Teknologi Informasi (S2)

Jenjang            : Strata Dua (S2)

Konsentrasi      : Data Mining

Judul Tesis       : “Perbandingan Kinerja Tiga Metode LSTM BiLSTM, dan CNN dengan Multi-Head Self-Attention pada Analisis Ujaran Kebencian”

Komunikasi telah mengalami perubahan besar dari masa lalu karena perkembangan pesat teknologi saat ini. Media sosial seperti Twitter menjadi yang banyak digunakan orang untuk mengungkapkan perasaan dan pendapat mereka dengan bebas. Komunikasi memainkan peran penting dalam interaksi sosial. Pengumpulan data opini-opini pengguna tentang suatu masalah dapat dilakukan untuk proses analisis sentimen dengan memasukan dalam kategori positif, negatif, atau netral. Pendekatan linguistik diuji menggunakan berbagai jenis fitur dan model untuk analisis sentimen yang akurat, pengujian model LSTM, BiLSTM, dan CNN+MHSA dengan word embedding GloVe untuk mengetahui analisis sentimen ujaran kebencian. Penelitian dilakukan dengan melakukan pengumpulan dataset dari Twitter, labelling data, balance data, text processing, word embedding, modelling dan hasil pengujian. Pengujian model menggunakan model LSTM, BiLSTM dan CNN dengan Multi-Head Self Attention serta word embedding GloVe dengan jumlah dataset sebanyak 6923 data yang terdiri dari 6465 data sentimen bukan ujaran kebencian dan 467 data sentimen ujaran kebencian. Hasil pengujian didapatkan model CNN dengan Attention menghasilkan accuracy 93.32%, lebih tinggi dibandingkan model LSTM dan BiLSTM yang menghasilkan accuracy 93.07%. Hyperparamer learning rate yang digunakan pada pengujian ini adalah 0.001 dan 0.005. Hasil pengujian menunjukkan pada data latih dan uji menunjukkan ada data yang overfit, namun pada hasil pengujian akhir data yang diuji menghasilkan model yang optimal untuk semua metode, hal ini menunjukkan bahwa ketiga metode yang dilakukan sama baiknya dalam analisis sentimen ujaran kebencian menggunakan dataset yang dikumpulkan sendiri, dengan hasil akurasi yang cukup tinggi diatas 90%.

KATA KUNCI

Multi-Head Self Attention,Metode LSTM,BiLSTM,CNN,Analisis Sentimen

---

DAFTAR PUSTAKA

DAFTAR PUSTAKA

[1] A. Karami, M. Lundy, F. Webb, and Y. K. Dwivedi, “Twitter and Research: A Systematic Literature Review through Text Mining,” IEEE Access, vol. 8, pp. 67698–67717, 2020, doi: 10.1109/ACCESS.2020.2983656.

[2] M. U. Salur and I. Aydin, “A Novel Hybrid Deep Learning Model for Sentiment Classification,” IEEE Access, vol. 8, 2020, doi: 10.1109/ACCESS.2020.2982538.

[3] S. Kami? and D. Goularas, “Evaluation of Deep Learning Techniques in Sentiment Analysis from Twitter Data,” in Proceedings - 2019 International Conference on Deep Learning and Machine Learning in Emerging Applications, Deep-ML 2019, Institute of Electrical and Electronics Engineers Inc., Aug. 2019, pp. 12–17. doi: 10.1109/Deep-ML.2019.00011.

[4] K. L. Tan, C. P. Lee, K. M. Lim, and K. S. M. Anbananthen, “Sentiment Analysis With Ensemble Hybrid Deep Learning Model,” IEEE Access, vol. 10, pp. 103694– 103704, 2022, doi: 10.1109/ACCESS.2022.3210182.

[5] A. A. Nagra, K. Alissa, T. M. Ghazal, S. Saigeeta, M. M. Asif, and M. Fawad, “Deep Sentiments Analysis for Roman Urdu Dataset Using Faster Recurrent Convolutional Neural Network Model,” Applied Artificial Intelligence, vol. 36, no. 1, 2022, doi: 10.1080/08839514.2022.2123094.

[6] A. Mohammadi and A. Shaverizade, “Ensemble deep learning for aspect-based sentiment analysis,” International Journal of Nonlinear Analysis and Applications, vol. 12, no. Special Issue, pp. 29–38, 2021, doi: 10.22075/IJNAA.2021.4769.

[7] J. Soni and K. Mathur, “Sentiment analysis based on aspect and context fusion using attention encoder with LSTM,” International Journal of Information Technology (Singapore), vol. 14, no. 7, pp. 3611–3618, Dec. 2022, doi: 10.1007/s41870-022- 00966-1.

[8] C. H. Lin and U. Nuha, “Sentiment analysis of Indonesian datasets based on a hybrid deep-learning strategy,” J Big Data, vol. 10, no. 1, Dec. 2023, doi: 10.1186/s40537-023-00782-9.

[9] A. Sriram, D. Sinha, and S. Gupta, “Performance Comparison of Deep Learning Algorithms for Sentiment Analysis,” International Conference on IoT based Control Networks and Intelligent Systems (ICICNIS 2020) , 2020, [Online]. Available: https://ssrn.com/abstract=3896958

[10] H. A. Shehu et al., “Deep Sentiment Analysis: A Case Study on Stemmed Turkish Twitter Data,” IEEE Access, vol. 9, pp. 56836–56854, 2021, doi: 10.1109/ACCESS.2021.3071393. 54 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri

[11] L. Khan, A. Amjad, K. M. Afaq, and H. T. Chang, “Deep Sentiment Analysis Using CNN-LSTM Architecture of English and Roman Urdu Text Shared in Social Media,” Applied Sciences (Switzerland), vol. 12, no. 5, Mar. 2022, doi: 10.3390/app12052694.

[12] A. K. Sharma, S. Chaurasia, and D. K. Srivastava, “Sentimental Short Sentences Classification by Using CNN Deep Learning Model with Fine Tuned Word2Vec,” in Procedia Computer Science, Elsevier B.V., 2020, pp. 1139–1147. doi: 10.1016/j.procs.2020.03.416.

[13] B. Alkouz, Z. Al Aghbari, and J. H. Abawajy, “Tweetluenza: Predicting flu trends from twitter data,” Big Data Mining and Analytics, vol. 2, no. 4, pp. 273–287, Dec. 2019, doi: 10.26599/BDMA.2019.9020012.

[14] A. R. Isnain, A. Sihabuddin, and Y. Suyanto, “Bidirectional Long Short Term Memory Method and Word2vec Extraction Approach for Hate Speech Detection,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 14, no. 2, p. 169, Apr. 2020, doi: 10.22146/ijccs.51743.

[15] A. Kumar, V. T. Narapareddy, V. A. Srikanth, A. Malapati, and L. B. M. Neti, “Sarcasm Detection Using Multi-Head Attention Based Bidirectional LSTM,” IEEE Access, vol. 8, pp. 6388–6397, 2020, doi: 10.1109/ACCESS.2019.2963630.

[16] Z. Mansur, N. Omar, and S. Tiun, “Twitter Hate Speech Detection: A Systematic Review of Methods, Taxonomy Analysis, Challenges, and Opportunities,” IEEE Access, vol. 11. Institute of Electrical and Electronics Engineers Inc., pp. 16226– 16249, 2023. doi: 10.1109/ACCESS.2023.3239375.

[17] H. A. Shehu et al., “Deep Sentiment Analysis: A Case Study on Stemmed Turkish Twitter Data,” IEEE Access, vol. 9, pp. 56836–56854, 2021, doi: 10.1109/ACCESS.2021.3071393.

[18] L. Khan, A. Amjad, K. M. Afaq, and H. T. Chang, “Deep Sentiment Analysis Using CNN-LSTM Architecture of English and Roman Urdu Text Shared in Social Media,” Applied Sciences (Switzerland), vol. 12, no. 5, Mar. 2022, doi: 10.3390/app12052694.

[19] C. Schröer, F. Kruse, and J. M. Gómez, “A systematic literature review on applying CRISP-DM process model,” in Procedia Computer Science, Elsevier B.V., 2021, pp. 526–534. doi: 10.1016/j.procs.2021.01.199.

[20] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of NAACL-HLT, 2019, pp. 4171–4186. [Online]. Available: https://github.com/tensorflow/tensor2tensor 55 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri

[21] S. E. Saad and J. Yang, “Twitter Sentiment Analysis Based on Ordinal Regression,” IEEE Access, vol. 7, pp. 163677–163685, 2019, doi: 10.1109/ACCESS.2019.2952127.

[22] IBM, “Natural Language Processing.” https://www.ibm.com/topics/naturallanguage-processing (accessed Jul. 05, 2023).

[23] Jiarui Zhang?Yingxiang Li?Juan Tian?Tongyan Li and Institute of Electrical and Electronics Engineers, LSTM-CNN Hybrid Model for Text Classification. 2018.

[24] S. Tabinda Kokab, S. Asghar, and S. Naz, “Transformer-based deep learning models for the sentiment analysis of social media data,” Array, vol. 14, Jul. 2022, doi: 10.1016/j.array.2022.100157.

[25] Prof. Florin Gorunescu, Data Mining: Concepts, Models and Techniques. 2011.

[26] P. Kumar, R. Bhatnagar, K. Gaur, and A. Bhatnagar, “Classification of Imbalanced Data:Review of Methods and Applications,” IOP Conf Ser Mater Sci Eng, vol. 1099, no. 1, p. 012077, Mar. 2021, doi: 10.1088/1757-899x/1099/1/012077.

[27] T. Wongvorachan, S. He, and O. Bulut, “A Comparison of Undersampling, Oversampling, and SMOTE Methods for Dealing with Imbalanced Classification in Educational Data Mining,” Information (Switzerland), vol. 14, no. 1, Jan. 2023, doi: 10.3390/info14010054.

[28] M. U. Salur and I. Aydin, “A Novel Hybrid Deep Learning Model for Sentiment Classification,” IEEE Access, vol. 8, pp. 58080–58093, 2020, doi: 10.1109/ACCESS.2020.2982538.

[29] J. Zhang, Y. Li, J. Tian, and T. Li, “LSTM-CNN Hybrid Model for Text Classification,” 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference(IAEAC 2018), pp. 1675–1680, 2018.

[30] R. Kumar, C. M. Sharma, V. M. Chariar, S. Hooda, and R. Beri, “Emotion Analysis of News and Social Media Text for Stock Price Prediction using SVM-LSTM-GRU Composite Model,” in Proceedings of International Conference on Computational Intelligence and Sustainable Engineering Solution, CISES 2022, Institute of Electrical and Electronics Engineers Inc., 2022, pp. 329–333. doi: 10.1109/CISES54857.2022.9844375.

[31] B. S. Ainapure et al., “Sentiment Analysis of COVID-19 Tweets Using Deep Learning and Lexicon-Based Approaches,” Sustainability (Switzerland), vol. 15, no. 3, Feb. 2023, doi: 10.3390/su15032573.

[32] A. Glenn, P. LaCasse, and B. Cox, “Emotion classification of Indonesian Tweets using Bidirectional LSTM,” Neural Comput Appl, May 2023, doi: 10.1007/s00521- 022-08186-1. 56 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri

[33] S. Sachin, A. Tripathi, N. Mahajan, S. Aggarwal, and P. Nagrath, “Sentiment Analysis Using Gated Recurrent Neural Networks,” SN Computer Science, vol. 1, no. 2. Springer, Mar. 01, 2020. doi: 10.1007/s42979-020-0076-y.

[34] S. Aziz, M. F. Mushtaq, S. Ullah, F. Mushtaq, B. Mughal, and S. Zahra, “Roman Urdu sentiment analysis using Machine Learning with best parameters and comparative study of Machine Learning algorithms,” PakJET Multidisciplinary | Peer Reviewed | Open Access, p. 2020, 2020, doi: 10.51846/vol3iss2pp172-177.

[35] A. Al Munzir, M. Lutfor Rahman, S. Abujar, and S. Akhter Hossain, “Text analysis for Bengali Text Summarization using Deep Learning,” IEEE, vol. 10th ICCCNT, 2019.

[36] B. Chandio et al., “Sentiment Analysis of Roman Urdu on E-Commerce Reviews UsingMachine Learning,” CMES - Computer Modeling in Engineering and Sciences, vol. 131, no. 3, pp. 1263–1287, 2022, doi: 10.32604/cmes.2022.019535.

[37] N. Aslam, F. Rustam, E. Lee, P. B. Washington, and I. Ashraf, “Sentiment Analysis and Emotion Detection on Cryptocurrency Related Tweets Using Ensemble LSTMGRU Model,” IEEE Access, vol. 10, pp. 39313–39324, 2022, doi: 10.1109/ACCESS.2022.3165621.

[38] M. F. Mubaraq and W. Maharani, “Sentiment Analysis on Twitter Social Media towards Climate Change on Indonesia Using IndoBERT Model,” JURNAL MEDIA INFORMATIKA BUDIDARMA, vol. 6, no. 4, p. 2426, Oct. 2022, doi: 10.30865/mib.v6i4.4570.

[39] K. Setyo Nugroho, A. Yullian Sukmadewa, H. D. Wuswilahaken, F. Abdurrachman Bachtiar, and N. Yudistira, “BERT Fine-Tuning for Sentiment Analysis on Indonesian Mobile Apps Reviews,” Arvix, 2021, [Online]. Available: https://research.google/teams/brain.

[40] Statistica, “Pengguna Twitter di Indonesia,” 2022, Accessed: Nov. 16, 2022. [Online]. Available: https://www.statista.com/statistics/490548/twitter-usersindonesia/