Monserrat Vázquez-Hernández, Luis Alberto Morales-Rosales, Ignacio Algredo-Badillo, Luis Villaseñor-Pineda
pp. 310-334, download
(https://doi.org/10.55612/s-5002-069-014)
Abstract
Analyzing students’ opinions about classes, instructors, and the overall learning environment enables institutions to understand students’ needs and propose improvements to enhance the learning process. Furthermore, analyzing opinions at aspect-level allows a detailed analysis and helps to identify the relevant aspects that require attention to enrich the students’ experience. Currently, opinion analysis systems implement deep learning models to increase precision. However, these models can be misled by strategically modified inputs, known as adversarial examples, producing incorrect classifications. In this work, we analyze adversarial examples and their behavior in aspect-level sentiment analysis models to design aspect-aware modifications. Specifically, we define a model for generating aspect-level adversarial examples that modify inputs by targeting aspect–term relationships. The proposed model reduces accuracy by 20.90%, highlighting vulnerabilities in aspect-level models, where even small and targeted changes can degrade performance, providing a foundation for the development of defense mechanisms to ensure reliable predictions.
keywords: Adversarial attacks, Vulnerabilities, Aspect-based, Sentiment analysis..
References
1. Shaik T., Tao X., Li Y., Dann C., McDonald J., Redmond P., Galligan L.: A Review of the Trends and Challenges in Adopting Natural Language Processing Methods for Education Feedback Analysis IEEE Access, 10, (2022).
https://doi.org/10.1109/ACCESS.2022.3177752
2. Altrabsheh N., Gaber M.M., Cocea M.: SA-E: Sentiment analysis for education Frontiers in Artificial Intelligence and Applications. vol. 255 (2013).
https://doi.org/10.3233/978-1-61499-264-6-353
3. Jimmy, Prasetyo V.R.: Sentiment analysis on feedback of higher education teaching conduct: An empirical evaluation of methods AIP Conference Proceedings. vol. 2470 (2022). https://doi.org/10.1063/5.0080182
4. Bal S., Choudhary A.K., Majumdar S., Pal D., Mandal L.: Sentiment Analysis of Online Reviews for Educational Institutions Applications of Machine Intelligence in Engineering (2022). https://doi.org/10.1201/9781003269793-22
5. OSMANOĞLU U.Ö., ATAK O.N., ÇAĞLAR K., KAYHAN H., CAN T.: Sentiment Analysis for Distance Education Course Materials: A Machine Learning Approach Journal of Educational Technology and Online Learning, 3, (2020).
https://doi.org/10.31681/jetol.663733
6. J A., K C., N H., P R., T T., Rokde C.: Sentiment Connect: Opinion Mining on Education System In India International Journal of Progressive Research in Science and Engineering, 3, (2022)
7. Fakhraie N., Hewitt J.: What’s in a Note? Sentiment Analysis in Online Educational Forums Curriculum, Teaching and Learning, MA, (2011)
https://utoronto.scholaris.ca/bitstreams/9ab6e236-1f7a-4d2d-ac68-a57eb3f8747a/download
8. Weiss S.M., Indurkhya N., Zhang T., Damerau F.J.: Text mining: Predictive methods for analyzing unstructured information, (2005). https://doi.org/10.1007/978-0-387-34555-0
9. Pang B., Lee L., Vaithyanathan S.: Thumbs up? Sentiment Classification using Machine Learning Techniques Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, EMNLP 2002 (2002).
https://doi.org/10.3115/1118693.1118704
10. Riloff E., Wiebe J.: Learning Extraction Patterns for Subjective Expressions Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, EMNLP 2003 (2003). https://doi.org/10.3115/1119355.1119369
11. Poria S., Hazarika D., Majumder N., Mihalcea R.: Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research IEEE Trans Affect Comput, 14, (2023). https://doi.org/10.1109/TAFFC.2020.3038167
12. Liu B., Zhang L.: A survey of opinion mining and sentiment analysis. In Mining text data Springer, (2012). https://doi.org/10.1007/978-1-4614-3223-4_13
13. Szegedy C., Zaremba W., Sutskever I., Bruna J., Erhan D., Goodfellow I., Fergus R.: Intriguing properties of neural networks 2nd International Conference on Learning Representations, ICLR 2014 – Conference Track Proceedings (2014).
https://doi.org/10.48550/arXiv.1312.6199
14. Goodfellow I.J., Shlens J., Szegedy C.: Explaining and harnessing adversarial examples 3rd International Conference on Learning Representations, ICLR 2015 – Conference Track Proceedings (2015). https://doi.org/10.48550/arXiv.1412.6572
15. Jia R., Liang P.: Adversarial examples for evaluating reading comprehension systems EMNLP 2017 – Conference on Empirical Methods in Natural Language Processing, Proceedings (2017). https://doi.org/10.18653/v1/D17-1215
16. Tsai Y.-T., Yang M.-C., Chen H.-Y.: Adversarial Attack on Sentiment Classification Presented at the (2019). https://doi.org/10.18653/v1/W19-4824
17. Li J., Ji S., Du T., Li B., Wang T.: TEXTBUGGER: Generating Adversarial Text Against Real-world Applications 26th Annual Network and Distributed System Security Symposium, NDSS 2019 (2019). https://doi.org/10.14722/ndss.2019.23138
18. Alzantot M., Sharma Y., Elgohary A., Ho B.J., Srivastava M.B., Chang K.W.: Generating natural language adversarial examples Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 (2018). https://doi.org/10.18653/v1/D18-1316
19. Xu Y., Zhong X., Yepes A.J., Lau J.H.: Grey-box Adversarial Attack And Defence For Sentiment Classification NAACL-HLT 2021 – 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference (2021).
https://doi.org/10.18653/v1/2021.naacl-main.321
20. Zhang Y., Shao K., Yang J., Liu H.: Universal adversarial attack via conditional sampling for text classification Applied Sciences (Switzerland), 11, (2021).
https://doi.org/10.3390/app11209539
21. Mamta, Ekbal A.: Adversarial Sample Generation for Aspect based Sentiment Classification 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing – Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022 (2022). https://doi.org/10.18653/v1/2022.findings-aacl.44
22. Hu M., Liu B.: Mining and summarizing customer reviews KDD-2004 – Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2004). https://doi.org/10.1145/1014052.1014073
23. Bing L., Minqing H., Junsheng C.: Opinion observer: Analyzing and Comparing Opinions on the Web. In Proceedings of the 14th international conference on World Wide Web (WWW ’05). Association for Computing Machinery, New York, NY, USA, 342–351. https://doi.org/10.1145/1060745.1060797
24. Zhu L., Xu M., Bao Y., Xu Y., Kong X.: Deep learning for aspect-based sentiment analysis: a review PeerJ Comput Sci, 8, (2022). https://doi.org/10.7717/peerj-cs.1044
25. Gonzalez A.: “Conceptos básicos de Machine learning.” Clever Data Big Data Prediction. https://cleverdata. io/conceptos-basicosmachine-learning (2014).
26. Ramos D.L., García L.A.: Aprendizaje profundo para la extracción de aspectos en opiniones textuales. Revista Cubana de Ciencias Informáticas, 13(2), 105-145. http://scielo.sld.cu/scielo.php?script=sci_arttext&pid=S2227-18992019000200105&lng=es&tlng=.
27. Bengio Y., Courville A., Vincent P.: Representation learning: A review and new perspectives IEEE Trans Pattern Anal Mach Intell, 35, (2013).
https://doi.org/10.1109/TPAMI.2013.50
28. Zhang W.E., Sheng Q.Z., Alhazmi A., Li C.: Adversarial Attacks on Deep learning Models in Natural Language Processing: A ACM Trans. Intell. Syst. Technol. 11, 3, Article 24 (June 2020), 41 pages. https://doi.org/10.1145/3374217
29. Dolianiti F.S., Iakovakis D., Dias S.B., Hadjileontiadou S., Diniz J.A., Hadjileontiadis L.: Sentiment analysis techniques and applications in education: A survey Communications in Computer and Information Science. vol. 993 (2019). https://doi.org/10.1007/978-3-030-20954-4_31
30. Van Houdt G., Mosquera C., Nápoles G.: A review on the long short-term memory model Artif Intell Rev, 53, (2020). https://doi.org/10.1007/s10462-020-09838-1
31. Clarizia F., Colace F., De Santo M., Lombardi M., Pascale F., Pietrosanto A.: E-learning and sentiment analysis: A case study ACM International Conference Proceeding Series (2018). https://doi.org/10.1145/3178158.3178181
32. Misuraca M., Scepi G., Spano M.: Using Opinion Mining as an educational analytic: An integrated strategy for the analysis of students’ feedback Studies in Educational Evaluation, 68, (2021). https://doi.org/10.1016/j.stueduc.2021.100979
33. Zhou J., Ye J. min: Sentiment analysis in education research: a review of journal publications, (2020). https://doi.org/10.1080/10494820.2020.1826985
34. Shapiro H.B., Lee C.H., Wyman Roth N.E., Li K., Çetinkaya-Rundel M., Canelas D.A.: Understanding the massive open online course (MOOC) student experience: An examination of attitudes, motivations, and barriers Comput Educ, 110, (2017). https://doi.org/10.1016/j.compedu.2017.03.003
35. Binali H.H., Wu C., Potdar V.: A new significant area: Emotion detection in E-learning using opinion mining techniques 2009 3rd IEEE International Conference on Digital Ecosystems and Technologies, DEST ’09 (2009).
https://doi.org/10.1109/DEST.2009.5276726
36. Ortigosa-Hernández J., Rodríguez J.D., Alzate L., Lucania M., Inza I., Lozano J.A.: Approaching Sentiment Analysis by using semi-supervised learning of multi-dimensional classifiers Neurocomputing, 92, (2012).
https://doi.org/10.1016/j.neucom.2012.01.030
37. Chauhan G.S., Agrawal P., Meena Y.K.: Aspect-based sentiment analysis of students’ feedback to improve teaching–learning process Smart Innovation, Systems and Technologies. vol. 107 (2019). https://doi.org/10.1007/978-981-13-1747-7_25
38. Alrajhi L., Alamri A., Pereira F.D., Cristea A.I.: Urgency Analysis of Learners’ Comments: An Automated Intervention Priority Model for MOOC Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 12677 LNCS (2021). https://doi.org/10.1007/978-3-030-80421-3_18
39. Chen L., Chen P., Lin Z.: Artificial Intelligence in Education: A Review IEEE Access, 8, (2020). https://doi.org/10.1109/ACCESS.2020.2988510
40. Leong C.K., Lee Y.H., Mak W.K.: Mining sentiments in SMS texts for teaching evaluation Expert Syst Appl, 39, (2012). https://doi.org/10.1016/j.eswa.2011.08.113
41. Wen M., Yang D., Rosé C.: Sentiment Analysis in MOOC Discussion Forums: What does it tell us? Proceedings of Educational Data Mining (2014).
https://api.semanticscholar.org/CorpusID:3243907
42. Moreno-Marcos P.M., Alario-Hoyos C., Munoz-Merino P.J., Estevez-Ayres I., Kloos C.D.: Sentiment analysis in MOOCs: A case study IEEE Global Engineering Education Conference, EDUCON. vol. 2018- April (2018).
https://doi.org/10.1109/EDUCON.2018.8363409
43. de Oliveira A.B., Alves A.L.F., de Souza Baptista C.: Using opinion mining in student assessments to improve teaching quality in universities Advances in Intelligent Systems and Computing. vol. 1181 AISC (2021). https://doi.org/10.1007/978-3-030-49342-4_22
44. Ortigosa A., Martín J.M., Carro R.M.: Sentiment analysis in Facebook and its application to e-learning Comput Human Behav, 31, (2014).
https://doi.org/10.1016/j.chb.2013.05.024
45. Toçoğlu M.A., Onan A.: Sentiment Analysis on Students’ Evaluation of Higher Educational Institutions Advances in Intelligent Systems and Computing. vol. 1197 AISC (2021). https://doi.org/10.1007/978-3-030-51156-2_197
46. Zhang W.E., Sheng Q.Z., Alhazmi A., Li C.: Adversarial Attacks on Deep-learning Models in Natural Language Processing, (2020). https://doi.org/10.1145/3374217
47. Yuan X., He P., Zhu Q., Li X.: Adversarial Examples: Attacks and Defenses for Deep Learning IEEE Trans Neural Netw Learn Syst, 30, (2019).
https://doi.org/10.1109/TNNLS.2018.2886017
48. Zhang J., Li C.: Adversarial Examples: Opportunities and Challenges IEEE Trans Neural Netw Learn Syst, 31, (2020). https://doi.org/10.1109/TNNLS.2019.2933524
49. Liang B., Li H., Su M., Bian P., Li X., Shi W.: Deep text classification can be fooled IJCAI International Joint Conference on Artificial Intelligence. vol. 2018- July (2018). https://doi.org/10.24963/ijcai.2018/585
50. Ribeiro M.T., Singh S., Guestrin C.: Semantically equivalent adversarial rules for debugging NLP models ACL 2018 – 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). vol. 1 (2018). https://doi.org/10.18653/v1/P18-1079
51. Gao J., Lanchantin J., Soffa M. Lou, Qi Y.: Black-box generation of adversarial text sequences to evade deep learning classifiers Proceedings – 2018 IEEE Symposium on Security and Privacy Workshops, SPW 2018 (2018).
https://doi.org/10.1109/SPW.2018.00016
52. Jin D., Jin Z., Zhou J.T., Szolovits P.: Is BERT really robust? A strong baseline for natural language attack on text classification and entailment AAAI 2020 – 34th AAAI Conference on Artificial Intelligence (2020). https://doi.org/10.1609/aaai.v34i05.6311
53. Ma Y., Peng H., Cambria E.: Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM 32nd AAAI Conference on Artificial Intelligence, AAAI 2018 (2018). https://doi.org/10.1609/aaai.v32i1.12048
54. Xiao Y., Zhou G.: Syntactic edge-enhanced graph convolutional networks for aspect-level sentiment classification with interactive attention IEEE Access, 8, (2020). https://doi.org/10.1109/ACCESS.2020.3019277
55. Wang W., Wang R., Ke J., Wang L.: TextFirewall: Omni-Defending against Adversarial Texts in Sentiment Classification IEEE Access, 9, (2021).
https://doi.org/10.1109/ACCESS.2021.3058278
56. Vázquez-Hernández M., Villaseñor-Pineda L., Montes-Y-Gómez M.: A Semantic-Proximity Term-Weighting Scheme for Aspect Category Detection Procesamiento del Lenguaje Natural, 69, (2022). https://api.semanticscholar.org/CorpusID:252848418