Sentiment analysis has come out as an important activity in natural language processing (NLP) applications whose data analysis is in high demand at present in the modern world. The BERT (Bidirectional Encoder Representations from Transformers) algorithm has proved to be extremely efficient when it comes to sentiment analysis tasks, and its potential is far exceeding that of conventional algorithms, unlocking their potential however would require fine tuning of their hyperparameters. It is quite a feat to optimise the BERT’s various hyperparameters due to the complicated interaction between them (e.g. the learning rate, batch size, dropout rate, attention heads). In this paper, the Salp Swarm Algorithm (SSA) is used as a bio-inspired metaheuristic optimization technique to optimize the fine-tuning process. Through SSA’s exceptionally efficient search capabilities in modelling multidimensional search space, BERT hyperparameters are optimized systematically to the sentiment classification tasks. A benchmark dataset for sentiment analysis (Sentiment140) is used to evaluate the proposed model. The novelty of the presented model is the fact that it dynamically adjusts its search behaviour in response to performance signals, thus it identifies better-performing parameter sets than conventional methods, leading to successful exploitation of the BERT algorithm that has produced high performing configurations. Extensive evaluations against 3 state-of-the-art search algorithms, namely manual tuning, grid search, and random search are conducted on the Sentiment140 benchmark dataset, demonstrating the superiority of the proposed SSA BERT optimization technique over state-of-the-art methods. The SSA-BERT model achieved a maximum accuracy of 96.4 percent, which is far better than manual tuning, grid search, and random search (65.0 percent, 69.5 percent and 72.0 percent respectively). It also performed better than other existing BERT models used in related literature, which showed accuracy levels between 46.4 and 75.7 percent in accordance with different benchmarks.
Liu B. Sentiment analysis and opinion mining. Springer Nature. 2022;May 31.
2.
Medhat W, Hassan A, Korashy H. Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal. 2014;5(4):1093–113.
3.
Devlin J, Chang MW, Lee K, Toutanova K, Bert. Pre-training of deep bidirectional transformers for language understanding. . In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019;Jun:4171–86.
4.
Rogers A, Kovaleva O, Rumshisky A. A Primer in BERTology: What We Know About How BERT Works. Transactions of the Association for Computational Linguistics. 2020;8:842–66.
5.
Sun C, Qiu X, Xu Y, Huang X. How to fine-tune BERT for text classification? In: China National Conference on Chinese Computational Linguistics; . 2019 Oct 13;194–206.
6.
Mao Y, Liu Q, Zhang Y. Sentiment analysis methods, applications, and challenges: A systematic literature review. Journal of King Saud University - Computer and Information Sciences. 2024;Apr 1;36(4):102048.
7.
Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res. 2012;Feb 1;13(1):281–305.
8.
Bergstra J, Bardenet R, Bengio Y, Kégl B. Algorithms for hyper-parameter optimization. Advances in neural information processing systems. 2011;24.
9.
Hutter F, Hoos HH, Leyton-Brown K. Sequential model-based optimization for general algorithm configuration. Berlin, Heidelberg: Springer Berlin Heidelberg. 2011 Jan 17;507–23.
10.
Talbi EG. Metaheuristics: from design to implementation. John Wiley & Sons. 2009 May 27.
11.
Huang H, Zavareh AA, Mustafa MB. Sentiment analysis in e-commerce platforms: A review of current techniques and future directions. Ieee Access. 2023 Aug 21;11:90367–82.
12.
Mirjalili S, Gandomi AH, Mirjalili SZ, Saremi S, Faris H, Mirjalili SM. Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems. Advances in Engineering Software. 2017;Dec 1;114:163–91.
13.
Yang Z, Jiang Y, Yeh WC. Self-learning salp swarm algorithm for global optimization and its application in multi-layer perceptron model training. Scientific Reports. 2024;Nov 9;14(1):27401.
14.
Xue J, Shen B. A novel swarm intelligence optimization approach: sparrow search algorithm. Systems Science & Control Engineering. 2020;Jan 1;8(1):22–34.
15.
Agrawal UK, Panda N, Tejani GG, Mousarvirad SJ. Improved salp swarm algorithm-driven deep CNN for brain tumor analysis. Scientific Reports. 2025;Jul 9;15(1):24645.
16.
Go A, Bhayani R, Huang L. Twitter sentiment classification using distant supervision. CS224N project report, Stanford. 2009 Dec;1(12).
17.
Li G, Liu F. Application of a clustering method on sentiment analysis. Journal of Information Science. 2012;38(2):127–39.
18.
Raul SK, Rout RR, Somayajulu DVLN. Topic classification using regularized variable-size CNN and dynamic BPSO in online social network. Arabian Journal for Science and Engineering. 2024;49(3):3347–69.
19.
Zhang Y, Wallace BC. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing. 2017;Nov (Volume 1: Long Papers):253–63.
20.
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine learning in Python. J Mach Learn Res. 2011;Nov 1;12:2825–30.
21.
Goodfellow I, Bengio Y, Courville A. Deep learning. MIT Press. 2016;
22.
Kaveh M, Mesgari MS. Application of meta-heuristic algorithms for training neural networks and deep learning architectures: A comprehensive review. Neural Processing Letters. 2023;Aug;55(4):4519–622.
23.
Qaraad M, Amjad S, Hussein NK, Elhosseini MA. An innovative quadratic interpolation Salp Swarm-based local escape operator for large-scale global optimization problems and feature selection. Neural Comput Appl. 2022;Oct;34(20):17663–721.
24.
Abdulsaed E, Alabbas M, Khudeyer R. Hyperparameter Optimization for Convolutional Neural Networks using the Salp Swarm Algorithm. Informatica. 2023;Dec 9;47(9).
25.
Abed-Alguni BH, Paul D, Hammad R. Improved Salp swarm algorithm for solving single-objective continuous optimization problems. Applied Intelligence. 2022;Dec;52(15):17217–36.
26.
Abualigah L, Shehab M, Alshinwan M, Alabool H. Salp swarm algorithm: a comprehensive survey. Neural Computing and Applications. 2020;32(15):11195–215.
27.
Liu Q, Li N, Jia H, Qi Q, Abualigah L, Liu Y. A Hybrid Arithmetic Optimization and Golden Sine Algorithm for Solving Industrial Engineering Design Problems. Mathematics. 2022;May 6;10(9):1567.
28.
Afuan L, Hidayat N, Hamdani H, Ismanto H, Purnama BC, Ramdhani DI. Optimizing BERT Models with Fine-Tuning for Indonesian Twitter Sentiment Analysis. Journal of Wireless Mobile Networks, Ubiquitous Computing, and Dependable Applications. 2025;16(2):248–67.
29.
Zhang H, Cai Z, Ye X, Wang M, Kuang F, Chen H, et al. A multi-strategy enhanced salp swarm algorithm for global optimization. Engineering with Computers. 2022;38(2):1177–203.
30.
Taye MM, Abulail R, Al-Ifan B, Alsuhimat F. Enhanced Sentiment Classification through Ontology-Based Sentiment Analysis with BERT. Journal of Internet Services and Information Security. 2025;15(1):236–56.
31.
Salehin I, Kang DK. A Review on Dropout Regularization Approaches for Deep Neural Networks within the Scholarly Domain. Electronics. 2023;Jul 17;12(14):3106.
32.
Anand S, Saravanasankar S, Subbaraj P. A multiobjective optimization tool for Very Large Scale Integrated nonslicing floorplanning. International Journal of Circuit Theory and Applications. 2013;41(9):904–23.
33.
Zheng Z, Wang Y, Huang Y, Song S, Yang M, Tang B, et al. Attention heads of large language models. Patterns. 2025;Feb 14;6(2):101176.
34.
Ramezani F, Lu J, Hussain F. Task-based system load balancing in cloud computing using particle swarm optimization. International Journal of Parallel Programming. 2014;42(5):739–54.
35.
Ahmedshaeva M, Pulatova N, Khayitov S, Sufiyeva D, Nizomova M. The Role of Natural Language Processing (NLP) in Translating Legal Documents with High Accuracy. Indian Journal of Information Sources and Services. 2025;15(2):83–90.
36.
Prabhu V, Sujai S. Sentimental Analysis of Product Rating. International Academic Journal of Innovative Research. 2022;9(2):18–21.
37.
Ramamohan V, Singhal S, Raj Gupta A, Bolia NB. Discrete simulation optimization for tuning machine learning method hyperparameters. Journal of Simulation. 2023;Sep 2;18(5):745–65.
38.
Gal Y, Ghahramani Z. A theoretically grounded application of dropout in recurrent neural networks. Advances in neural information processing systems. 2016;29.
39.
Shim K, Choi I, Sung W, Choi J. Layer-wise pruning of transformer attention heads for efficient language modeling. In: 2021 18th International SoC Design Conference (ISOCC). 2021;Oct 6;357–8.
40.
Zhang H, Nan H, Yue X, Gao X. A feature selection method based on salp swarm algorithm with a multiround voting mechanism. The Journal of Supercomputing. 2025;81(12):1–28.
The statements, opinions and data contained in the journal are solely those of the individual authors and contributors and not of the publisher and the editor(s). We stay neutral with regard to jurisdictional claims in published maps and institutional affiliations.
,
