Developing a metrological framework for
assessment of image-based Artificial Intelligence systems
for disease detection
Image-based AI systems for disease detection are increasingly being developed, making necessary their effectiveness and trustworthiness in heterogeneous clinical settings, as well as their evaluation by approved guidelines. To address these points, MAIBAI aims at developing a standardised and impartial framework for performance, generalisability and suitability assessment of AI tools, to enable a more efficient, reliable and reproducible validation of image-based AI systems for disease detection. Using breast screening as an exemplar, AI tools will be benchmarked on a large real-world database of mammographic images, with the final goal of designing a metrological framework for AI assessment and explainability in diagnostic imaging.
The needs
The exponential increase in healthcare data over the last decade, as well as the fast-paced technology developments, have resulted in promising novel AI approaches for diagnostic applications and risk prediction. However, the adoption of AI in clinical settings remains limited, mostly due to i) limited data quality and interoperability across heterogeneous clinical centres and electronic health records, ii) absence of robust validation procedures, iii) distrust of predictions and decisions generated by AI systems, and iv) lack of harmonised government proposals and consensus guidelines on steps for their adoption.
To enable the implementation of image-based AI systems for disease detection, MAIBAI addresses the following specific needs:
- Test of AI tools on large and high-quality medical imaging databases, with data categorised and integrated based on clinically relevant subgroups and image acquisition key factors;
- Provision of a clear methodology to benchmark the quality of predictive AI models, with relevant associated metrics, and interpretation methods for explainable and traceable AI tools;
- Design of a global, standardised, and impartial AI assessment framework.
