Skip to main content

Table 2 Optimisation metrics for models created by resampling training data and applying feature reduction. Abbreviations: RFE = recursive feature elimination, kBF = k-best features

From: Development and validation of a case definition for problematic menopause in primary care electronic medical records

Resampling &

Feature Reduction Method

Sensitivity %

(95% CI)

Specificity %

(95% CI)

PPV %

(95% CI)

NPV %

(95% CI)

F1-Score

random over- & under-sampling;

RFE

90.3

(78.7–99.1)

85.7

(77.5–93.4)

59.8

(49.5–74.8)

97.6

(94.7–99.8)

0.720

random over-sampling;

RFE

87.5

(74.8–96.0)

85.9

(79.6–94.3)

59.9

(50.2–76.6)

96.9

(93.9–98.9)

0.711

random under-sampling;

RFE

93.0

(85.4–96.2)

83.5

(80.1–89.8)

56.3

(51.1–66.2)

98.2

(96.4–99.0)

0.701

RFE;

random over- & under-sampling

80.1

(66.2–88.4)

92.0

(85.6–96.1)

70.1

(57.0-82.9)

95.4

(92.2–97.2)

0.748

RFE;

random over-sampling

79.3

(66.2–88.4)

93.5

(88.2–98.5)

74.0

(60.1–93.0)

95.2

(92.2–97.4)

0.766

RFE;

random under-sampling

79.3

(66.2–88.4)

92.5

(88.0-96.1)

70.9

(60.1–82.9)

95.2

(92.2–97.2)

0.749

RFE;

None

81.2

(70.1–91.3)

95.0

(92.9–97.9)

78.6

(69.5–89.9)

95.7

(93.1–98.0)

0.799

random over- & under-sampling;

kBF

86.0

(73.1–95.2)

84.9

(77.8–91.7)

57.1

(47.0-69.2)

96.5

(93.3–98.7)

0.686

random over-sampling;

kBF

87.6

(73.1–99.1)

85.1

(76.3–93.7)

58.4

(47.2–76.1)

96.9

(93.4–99.8)

0.701

random under-sampling;

kBF

87.9

(77.8–99.1)

86.6

(77.3–94.1)

60.9

(49.5–76.2)

97.0

(94.5–99.8)

0.720

kBF;

random over- & under-sampling

80.4

(59.8–96.1)

88.2

(80.6–97.0)

63.1

(52.4–81.6)

95.4

(91.6–98.9)

0.707

kBF;

random over-sampling

81.6

(59.8–99.1)

87.8

(78.4–96.4)

62.4

(49.9–80.1)

95.7

(91.6–99.8)

0.707

kBF;

random under-sampling

84.4

(67.8–99.1)

87.3

(79.1–98.1)

62.7

(50.7–91.3)

96.3

(92.5–99.8)

0.719

kBF;

None

61.2

(39.3–79.9)

97.4

(92.5–99.1)

85.4

(69.3–94.0)

91.9

(87.5–95.4)

0.713