Penggunaan Kamus Teks Normalisasi dan Hyperparameter Tuning dalam Pengklasifikasian Tiket Help Desk

• DEDIK ERWANTO
• 14210198

---

ABSTRAK

ABSTRAK

Nama              : Dedik Erwanto

NIM                 : 14210198

Program Studi : Ilmu Komputer

Fakultas          : Teknologi Informasi

Jenjang           : Strata Dua (S2)

Konsentrasi     : Data Mining

Judul               : “Penggunaan Kamus Teks Normalisasi dan Hyperparameter Tuning dalam Pengklasifikasian Tiket Help Desk”

Organisasi saat ini sangat bergantung pada sumber daya dan layanan TI untuk efisiensi internal dan operasi eksternal sehingga menjadikan layanan Helpdesk sebagai faktor penting karena fokus pada perekaman tiket pengguna dan memberikan solusi dengan cepat dan tepat. Jika tiket tidak dapat diselesaikan dengan solusi Layer pertama, tiket tersebut akan diteruskan ke Layer kedua yang merupakan tim ahli untuk menjawab pertanyaan. Proses penetapan dan eskalasi tiket yang dilakukan secara manual oleh organisasi terkadang menghasilkan kesalahan manusia dalam bentuk pemberian tiket ke tim yang salah yang mengakibatkan konsumsi sumber daya yang tinggi, waktu resolusi permasalahan yang lebih tinggi pada akhirnya memperburuk layanan dan kepuasan pengguna. Terkait proses penetapan dan eskalasi tiket Heldesk, diperlukan metode klasifikasi yang tepat untuk mengantisipasi kesalahan dan mempercepat waktu penyelesaian permasalahan. Eksperimen yang dilakukan menggunakan motode teks mining dengan algoritma KNN, XG-Boost dan LSTM dapat secara efektif mengklasifikasikan tiket Helpdesk ke dalam 4 kategori. Penggunaan kamus teks normalisasi lokal dapat meningkatkan kinerja model algoritma dengan peningkatan Accuracy sebesar 0,52% pada model algoritma KNN, 0,40% pada model algoritma XG-Boost dan penurunan 0,63% pada algoritma LSTM. Penerapan Hyperparameter Tuning menggunakan RandomSearchCV dapat meningkatkan Accuracy sebesar 1,17% pada model algortima KNN, 0,35% pada model algoritma XG-Boost dan 0,96% pada algoritma LSTM. Hasil eksperimen menunjukan model algoritma XG-Boost yang menggunakan kamus teks normalisasi lokal dan Hyperparameter Tuning RandomSearchCV dapat mengklasifikasikan tiket Helpdesk dengan nilai Accuracy 89,87%, Precision 89,66%, Recall 89,75% dan F1-Score 89,79%. Model yang dihasilkan dapat digunakan dalam pengklasifikasian tiket Helpdesk.

KATA KUNCI

Helpdesk,machine learning,Kamus Teks Normalisasi,Hyperparameter Tuning

---

DAFTAR PUSTAKA

DAFTAR PUSTAKA

[1] A. Sirin, I. Indarto, and W. Saddewisasi, “Determinan Penyerapan Anggaran Dengan Komitmen Organisasi Sebagai Moderating Variable,” J. Ris. Ekon. dan Bisnis, vol. 13, no. 2, p. 147, 2020, doi: 10.26623/jreb.v13i2.2690.

[2] I. Farida and A. Lestari, “Implementation of E-Government as a Public Service Innovation in Indonesia,” Rudn J. Public Adm., vol. 8, no. 1, pp. 72– 79, 2021, doi: 10.22363/2312-8313-2021-8-1-72-79.

[3] P. H. Mazhisham et al., “Identification of Training Needs Assessment in Organizational Context,” Int. J. Mod. Trends Soc. Sci., vol. 1, no. 5, pp. 20– 30, 2018.

[4] D. S. Zahara Phonna, Fatmayanti, “ANALISIS MANFAAT PELAYANAN HAI-CSO PADA KANTOR PELAYANAN PERBENDAHARAAN NEGARA MEULABOH,” J. Sains Ris., vol. 2, p. https://news.ge/anakliisporti-aris-qveynis-momava, 2022.

[5] R. M. R. At-tamimi and T. E. Siregar, “Measurement of successful implementation of institution level financial application system (SAKTI) web full module with DeLone and McLean information system …,” Budapest Int. Res. …, no. 2017, pp. 10098–10107, 2021, [Online]. Available: https://www.bircu-journal.com/index.php/birci/article/view/3032

[6] Menteri Keuangan Republik Indonesia, “Peraturan Menteri Keuangan Republik Indonesia Nomor 159/PMK/05/2018 Tentang Pelaksanaan Piloting Sistem Aplikasi Keuangan Tingkat Instansi,” 2018.

[7] Menteri Keuangan Republik Indonesia, “Peraturan Menteri Keuangan Republik Indonesia Nomor 171/PMK.05/2021 Tentang Pelaksanaan Sistem SAKTI,” no. 3, pp. 1–72, 2021.

[8] G. G. Wirakanda and I. S. Putri, “Analisis Penanganan Keluhan Pelanggan (Studi Kasus Di Kantor Pos Bandung 40000),” J. Bisnis dan Pemasar., vol. 10, no. 2, pp. 1–11, 2020, [Online]. Available: 74 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri https://ejurnal.poltekpos.ac.id/index.php/promark/article/view/1020/698

[9] S. Shah, “Automatic Ticket Assignment using Machine Learning and Deep Learning Techniques,” 2020.

[10] F. Al-Hawari and H. Barham, “A machine learning based help desk system for IT service management,” J. King Saud Univ. - Comput. Inf. Sci., vol. 33, no. 6, pp. 702–718, 2021, doi: 10.1016/j.jksuci.2019.04.001.

[11] S. P. Paramesh, C. Ramya, and K. S. Shreedhara, “Classifying the Unstructured IT Service Desk Tickets Using Ensemble of Classifiers,” Proc. 2018 3rd Int. Conf. Comput. Syst. Inf. Technol. Sustain. Solut. CSITSS 2018, pp. 221–227, 2018, doi: 10.1109/CSITSS.2018.8768734.

[12] R. Riyaddulloh and A. Romadhony, “Normalisasi Teks Bahasa Indonesia Berbasis Kamus Slang Studi Kasus: Tweet Produk Gadget Pada Twitter,” eProceedings Eng., vol. 8, no. 4, pp. 4216–4228, 2021, [Online]. Available: https://openlibrarypublications.telkomuniversity.ac.id/index.php/engineerin g/article/view/15246/14969

[13] G. S. K. Ranjan, A. Kumar Verma, and S. Radhika, “K-Nearest Neighbors and Grid Search CV Based Real Time Fault Monitoring System for Industries,” 2019 IEEE 5th Int. Conf. Converg. Technol. I2CT 2019, pp. 9– 13, 2019, doi: 10.1109/I2CT45611.2019.9033691.

[14] W. Pannakkong, K. Thiwa-Anont, K. Singthong, P. Parthanadee, and J. Buddhakulsomsiri, “Hyperparameter Tuning of Machine Learning Algorithms Using Response Surface Methodology: A Case Study of ANN, SVM, and DBN,” Math. Probl. Eng., vol. 2022, 2022, doi: 10.1155/2022/8513719.

[15] D. J. Perbendaharaan, “Perdirjen Perbendaharaan nomor PER-8/PB/2020 tentang Layanan Help, Answer, Improve Direktorat Jenderal Perbendaharaan (HAI DJPb),” vol. 2018, pp. 1–21, 2020.

[16] G. Schuh et al., “Data mining definitions and applications for the management of production complexity,” Procedia CIRP, vol. 81, pp. 874– 75 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri 879, 2019, doi: 10.1016/j.procir.2019.03.217.

[17] E. E. Pratama and R. L. Atmi, “A Text Mining Implementation Based on Twitter Data to Analyse Information Regarding Corona Virus in Indonesia,” J. Comput. Soc., vol. 1, no. 1, pp. 91–100, 2020, [Online]. Available:https://ejournal.upi.edu/index.php/JCS/article/view/25502

[18] F. A. Muttaqin and A. M. Bachtiar, “Implementasi Teks Mining Pada Aplikasi Pengawasan Penggunaan Internet Anak ‘Dodo Kids Browser,’” J. Ilm. Komput. dan Inform., pp. 1–8, 2016.

[19] M. Y. Cheng, D. Kusoemo, and R. A. Gosno, “Text mining-based construction site accident classification using hybrid supervised machine learning,” Autom. Constr., vol. 118, no. May, p. 103265, 2020, doi: 10.1016/j.autcon.2020.103265.

[20] L. A. Mullen, K. Benoit, O. Keyes, D. Selivanov, and J. Arnold, “Fast, Consistent Tokenization of Natural Language Text,” J. Open Source Softw., vol. 3, no. 23, p. 655, 2018, doi: 10.21105/joss.00655.

[21] U. N. Tantyoko. Adiwijaya. & Wisesty, “Perbandingan Pembobotan untuk Klasifikasi Topik Berita Menggunakan Decision Tree.” Jurnal Teknologia Vol 2 No 1, pp. 97–113, 2019.

[22] A. H. Nababan, R. Mahendra, and I. Budi, “Twitter stance detection towards Job Creation Bill,” Procedia Comput. Sci., vol. 197, no. 2021, pp. 76–81, 2021, doi: 10.1016/j.procs.2021.12.120.

[23] T. M. Lai, Y. Zhang, E. Bakhturina, B. Ginsburg, and H. Ji, “A Unified Transformer-based Framework for Duplex Text Normalization,” pp. 1–5, 2021, [Online]. Available: http://arxiv.org/abs/2108.09889

[24] N. Aliyah Salsabila, Y. Ardhito Winatmoko, A. Akbar Septiandri, and A. Jamal, “Colloquial Indonesian Lexicon,” Proc. 2018 Int. Conf. Asian Lang. Process. IALP 2018, pp. 226–229, 2019, doi: 10.1109/IALP.2018.8629151.

[25] J. Wang and N. Wang, “Text Features Extraction based on TF-IDF Associating Semantic,” 2018 IEEE 4th Int. Conf. Comput. Commun., pp. 76 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri 2338–2343, 2018.

[26] Y. Z. B, Y. Zhou, and J. Yao, Feature Extraction with TF-IDF and GameTheoretic Shadowed Sets, vol. 1. Springer International Publishing, 2020. doi: 10.1007/978-3-030-50146-4.

[27] S. Qaiser and R. Ali, “Text Mining: Use of TF-IDF to Examine the Relevance of Words to Documents,” Int. J. Comput. Appl., vol. 181, no. 1, pp. 25–29, 2018, doi: 10.5120/ijca2018917395.

[28] S. Andhalkar and B. F. Momin, “Multiclass IFROWNN Classification Algorithm Using OVA and OVO Strategy,” 2018 9th Int. Conf. Comput. Commun. Netw. Technol. ICCCNT 2018, pp. 1–7, 2018, doi: 10.1109/ICCCNT.2018.8493865.

[29] A. Le Glaz et al., “Machine learning and natural language processing in mental health: Systematic review,” J. Med. Internet Res., vol. 23, no. 5, 2021, doi: 10.2196/15708.

[30] A. Jha, M. Dave, and S. Madan, “Comparison of Binary Class and MultiClass Classifier Using Different Data Mining Classification Techniques,” SSRN Electron. J., pp. 894–903, 2019, doi: 10.2139/ssrn.3464211.

[31] T. Phillips and W. Abdulla, “Developing a new ensemble approach with multi-class SVMs for Manuka honey quality classification,” Appl. Soft Comput., vol. 111, p. 107710, 2021, doi: 10.1016/j.asoc.2021.107710.

[32] L. Sunitha and M. B. Raju, “Multi-class classification for large datasets with optimized SVM by non-linear kernel function,” J. Phys. Conf. Ser., vol. 2089, no. 1, pp. 0–5, 2021, doi: 10.1088/1742-6596/2089/1/012015.

[33] L. Xiong and Y. Yao, “Study on an adaptive thermal comfort model with Knearest-neighbors (KNN) algorithm,” Build. Environ., vol. 202, no. December 2020, p. 108026, 2021, doi: 10.1016/j.buildenv.2021.108026.

[34] R. Santhanam, N. Uzir, S. Raman, and S. Banerjee, “Experimenting XGBoost Algorithm for Prediction and Classification of Different Ramraj S , Nishant Uzir , Sunil R and Shatadeep Banerjee Experimenting XGBoost 77 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri Algorithm for Prediction and Classi fi cation of Different Datasets,” Int. J. Control Theory Appl., vol. 9, no. March, pp. 651–662, 2017.

[35] T. Iskandar Zulkarnain Maulana Putra, A. Farhan Bukhori, dan Ilmu Pengetahuan Alam, and U. Gadjah Mada, “Model Klasifikasi Berbasis Multiclass Classification dengan Kombinasi Indobert Embedding dan Long Short-Term Memory untuk Tweet Berbahasa Indonesia (Classification Model Based on Multiclass Classification with a Combination of Indobert Embedding and Long ,” J. Ilmu Siber dan Teknol. Digit., vol. 1, no. 1, pp. 1– 28, 2022, [Online]. Available: https://doi.org/10.35912/jisted.v1i1.1509

[36] C. Schröer, F. Kruse, and J. M. Gómez, “A systematic literature review on applying CRISP-DM process model,” Procedia Comput. Sci., vol. 181, no. 2019, pp. 526–534, 2021, doi: 10.1016/j.procs.2021.01.199.

[37] J. A. Bunge and D. H. Judson, “Implementasi Data Mining Menggunakan CRISP,” Encycl. Soc. Meas., pp. 617–624, 2004, doi: 10.1016/B0-12- 369398-5/00159-6.

[38] C. Ramadhani, I. M. B. Suksmadana, I. G. P. S. Wijaya, and S. Ariessaputra, “Multiclass Classification of Sound Healing With K-Nearest Neighbor Algorithm,” Dielektrika, vol. 8, no. 2, pp. 156–163, 2021.

[39] D. Afreyna Fauziah, A. Maududie, and I. Nuritha, “Klasifikasi Berita Politik Menggunakan Algoritma K-nearst Neighbor (Classification of Political News Content using K-Nearest Neighbor),” Berk. Sainstek, vol. 6, pp. 106– 114, 2018, [Online]. Available:https://jurnal.unej.ac.id/index.php/BST/article/view/9256

[40] A. Revina, K. Buza, and V. G. Meister, “IT Ticket Classification: The Simpler, the Better,” IEEE Access, vol. 8, pp. 193380–193395, 2020, doi: 10.1109/ACCESS.2020.3032840.

[41] Paramesh, “Learning Long-Term Dependencies For Prediction Of It Incident Category Using LSTM Recurrent Neural Networks,” JCST, vol. 38, no. 1,2, pp. 149–200, 2023, doi: 10.5281/zenodo.776695. 78 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri

[42] Ahmada Nuryanto, “Penerapan Text Mining Untuk Klasifikasi Kategori Masalah Pada Tiket It Helpdesk Menggunakan Metode Ensemble Tesis,” Tesis Univ. Nusa Mandiri, 2022.

[43] M. A. Suprayogi and R. A. Supono, “Klasifikasi Helpdesk Menggunakan Metode K-Nearest Neighbor dan TF-ABS,” Techno.Com, vol. 20, no. 4, pp. 508–517, 2021, doi: 10.33633/tc.v20i4.5094.

[44] J. Schad, R. Sambasivan, and C. Woodward, “Predicting help desk ticket reassignments with graph convolutional networks,” Mach. Learn. with Appl., vol. 7, no. August 2020, p. 100237, 2022, doi: 10.1016/j.mlwa.2021.100237.

[45] L. Montgomery, D. Damian, T. Bulmer, and S. Quader, “Customer support ticket escalation prediction using feature engineering,” Requir. Eng., vol. 23, no. 3, pp. 333–355, 2018, doi: 10.1007/s00766-018-0292-3.

[46] C. Zeng, W. Zhou, T. Li, L. Shwartz, and G. Y. Grabarnik, “Knowledge Guided Hierarchical Multi-Label Classification over Ticket Data,” IEEE Trans. Netw. Serv. Manag., vol. 14, no. 2, pp. 246–260, 2017, doi: 10.1109/TNSM.2017.2668363.

[47] A. Mandal, N. Malhotra, S. Agarwal, A. Ray, and G. Sridhara, “Cognitive system to achieve human-level accuracy in automated assignment of helpdesk email tickets,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11236 LNCS, pp. 332– 341, 2018, doi: 10.1007/978-3-030-03596-9_23.

[48] D. Khramov, “Robotic and machine learning: How to help support to process customer tickets more effec- tively,” no. April, 2019, [Online]. Available: https://www.theseus.fi/handle/10024/143200

[49] J. Xu, H. Zhang, W. Zhou, R. He, and T. Li, Signature based trouble ticket classification, vol. 78. Elsevier B.V., 2018. doi: 10.1016/j.future.2017.07.054.

[50] A. Mandal, N. Malhotra, S. Agarwal, A. Ray, and G. Sridhara, “Automated dispatch of helpdesk email tickets: Pushing the limits with ai,” 33rd AAAI 79 Program Studi Ilmu Komputer (S2) Universitas Nusa Mandiri Conf. Artif. Intell. AAAI 2019, 31st Innov. Appl. Artif. Intell. Conf. IAAI 2019 9th AAAI Symp. Educ. Adv. Artif. Intell. EAAI 2019, pp. 9381–9388, 2019, doi: 10.1609/aaai.v33i01.33019381.

[51] P. Zicari, G. Folino, M. Guarascio, and L. Pontieri, “Discovering accurate deep learning based predictive models for automatic customer support ticket classification,” Proc. ACM Symp. Appl. Comput., pp. 1098–1101, 2021, doi: 10.1145/3412841.3442109.

[52] S. Hilda Kusumahadi, H. Junaedi, and J. Santoso, “Klasifikasi Helpdesk Menggunakan Metode Support Vector Machine,” J. Inform. J. Pengemb. IT, vol. 4, no. 1, pp. 54–60, 2019, doi: 10.30591/jpit.v4i1.1125.

[53] A. D. Wibawa and H. Effendi, “The Role of Complaint Application Evaluation in Knowledge Management Implementation for Organization Improvement: Text Processing Case Study in Directorate General of Treasury,” Proc. - 2021 IEEE 7th Inf. Technol. Int. Semin. ITIS 2021, 2021, doi: 10.1109/ITIS53497.2021.9791589.

[54] Menteri Keuangan RI, “Peraturan Menteri Keuangan No. 118 Tahun 2021 Tentang Organisasi dan Tata Kerja Kementerian,” 2021, [Online]. Available: www.jdih.kemenkeu.go.id