Exploring the performance and explainability of fine-tuned BERT models for neuroradiology protocol assignment

Table 3 Performance results of the biobert model compared with neuroradiologists. Human outperform the model in all but 2 categories

	Number of entries	Number of BioBERT errors	Accuracy %	Number of Human errors	Accuracy %
MR brain demyelinating	395	19	0.95	13	0.97
MR brain mass/mets/infection	457	46	0.90	9	0.98
MR moya-moya with Diamox	184	4	0.98	4	0.98
MR nasopharynx oropharynx	384	11	0.97	5	0.99
MR orbit sinus face	426	60	0.86	15	0.96
MR seizure	355	4	0.99	4	0.99
MR sella	521	13	0.98	1	0.99
MR skull base	443	44	0.90	20	0.95
MR stroke	497	55	0.89	17	0.97
MR vascular malformation/hemorrhage/trauma	433	47	0.88	25	0.94
Weighted Average			0.93		0.97

ISSN: 1472-6947