Abstract

Artificial intelligence (AI) has made impressive progress over the past few years, including many applications in medical imaging. Numerous commercial solutions based on AI techniques are now available for sale, forcing radiology practices to learn how to properly assess these tools.

While several guidelines describing good practices for conducting and reporting AI-based research in medicine and radiology have been published, fewer efforts have focused on recommendations addressing the key questions to consider when critically assessing AI solutions before purchase. Commercial AI solutions are typically complex software products whose evaluation requires consideration of many factors.

In this work, authors from academia and industry have joined efforts to propose a practical framework that will help stakeholders evaluate commercial AI solutions in radiology (the ECLAIR guidelines) and reach an informed decision. Topics to consider in the evaluation include the relevance of the solution from the point of view of each stakeholder, issues regarding performance and validation, usability and integration, regulatory and legal aspects, and financial and support services.

Key Points

• Numerous commercial solutions based on artificial intelligence techniques are now available for sale, and radiology practices must learn how to properly assess these tools.
• A practical framework is proposed to guide stakeholders when assessing AI solutions in medical imaging, enabling informed discussions with manufacturers and supporting purchasing decisions.
• Key evaluation topics include:
  • Relevance of the solution for each stakeholder
  • Performance and validation
  • Usability and integration
  • Regulatory and legal considerations
  • Financial aspects and support services