A qualitative research framework for the design of user-centered displays of explanations for machine learning model predictions in healthcare

Table 2 Perceptions of focus group participants on context of use and perceived influences on model perceptions

User goal (why)	User characteristic (who)		Desired information	Positive (+) and negative (−) influences on perceptions
Verification	Predictive modeling knowledge	Detailed	Predictive performance Alignment with domain knowledge Comparison with existing models Modeling processes	Credibility + high predictive performance + predictions that aligned with clinical knowledge - influential outliers or data errors - counterintuitive risk factors - model limitations
Verification		Basic	Predictive performance Alignment with domain knowledge
Learning	Clinical role	Physician	Obtain patient insights: Prioritization Assessment of status Highlight patients/info of concern	Utility + training on use/interpretation - clinically irrelevant information
Learning		Nurse	Actionable information Alerts to changes Information to intervene or justify consult	Usability + appropriate alerts - high cognitive effort or attention - large time investments

ISSN: 1472-6947