Table 4 Average classification results for the evaluated methods. Results are presented in the format of mean percentage value ± SD at 95% confidence interval
Method | Precision | Recall | F-measure |
|---|---|---|---|
Naïve Bayes | 66.4 ± 1.7 | 70.5 ± 2.0 | 68.4 ± 1.6 |
Logistic regression | 69.0 ± 1.8 | 59.6 ± 1.9 | 64.0 ± 1.5 |
Decision tree | 64.7 ± 1.8 | 61.2 ± 2.4 | 62.9 ± 1.7 |
Random forest | 62.8 ± 2.0 | 54.8 ± 2.0 | 58.5 ± 1.7 |
kNN | 53.5 ± 3.2 | 29.1 ± 2.1 | 37.7 ± 2.3 |
SVM | 66.7 ± 2.0 | 53.7 ± 2.0 | 59.5 ± 1.6 |
MLP | 62.1 ± 2.1 | 61.9 ± 2.3 | 62.0 ± 1.6 |
StarSpace | 62.4 ± 1.9 | 63.8 ± 2.2 | 63.5 ± 2.0 |
LSTM | 63.9 ± 1.1 | 64.3 ± 1.3 | 64.1 ± 1.0 |
1D-CNN | 69.3 ± 1.8 | 67.9 ± 1.6 | 68.6 ± 1.7 |
LSTM CNN | 62.3 ± 1.9 | 62.7 ± 1.4 | 62.3 ± 1.7 |
Bi-LSTM | 65.9 ± 1.5 | 65.2 ± 1.5 | 65.6 ± 1.5 |
Bi-LSTM CNN | 64.1 ± 1.4 | 64.0 ± 1.4 | 64.1 ± 1.4 |
Bi-LSTM Attention | 66.5 ± 2.1 | 67.2 ± 1.5 | 66.7 ± 1.9 |
GRU | 65.5 ± 1.5 | 65.1 ± 1.5 | 65.3 ± 1.5 |
CNN 1D + fastText Skip-Gram 100 dims | 60.7 ± 1.7 | 59.1 ± 1.3 | 59.9 ± 1.5 |
CNN 1D + fastText Skip-Gram 300 dims | 63.3 ± 1.3 | 63.1 ± 1.4 | 63.2 ± 1.3 |
CNN 1D + fastText CBOW 100 dims | 66.1 ± 1.3 | 63.9 ± 1.3 | 65.7 ± 1.2 |
CNN 1D + fastText CBOW 300 dims | 74.2 ± 1.5 | 74.1 ± 1.5 | 74.1 ± 1.5 |
CNN 1D + Word2Vec Skip-Gram 100 dims | 63.5 ± 1.9 | 61.5 ± 1.4 | 62.4 ± 1.7 |
CNN 1D + Word2Vec Skip-Gram 300 dims | 70.7 ± 1.5 | 69.5 ± 1.4 | 70.1 ± 1.3 |
CNN 1D + Word2Vec CBOW 100 dims | 61.7 ± 2.2 | 57.5 ± 1.0 | 59.2 ± 1.5 |
CNN 1D + Word2Vec CBOW 300 dims | 66.1 ± 1.7 | 64.1 ± 1.3 | 64.5 ± 1.5 |
CNN 1D + Glove 100 dims | 67.2 ± 1.3 | 67.2 ± 1.2 | 67.2 ± 1.2 |
CNN 1D + Glove 300 dims | 71.3 ± 1.1 | 69.3 ± 1.3 | 70.3 ± 1.2 |
CNN 1D + StarSpace 100 dims | 64.6 ± 1.5 | 63.0 ± 1.5 | 63.7 ± 1.4 |
CNN 1D + StarSpace 300 dims | 70 ± 1.6 | 68.1 ± 1.6 | 68.9 ± 1.6 |
Our combined method | 92.6 ± 4.2 | 97.9 ± 2.4 | 95.1 ± 2.5 |