Mind the gap: performance metric evaluation in brain-age prediction
Mind the gap: performance metric evaluation in brain-age prediction
Abstract Estimating age based on neuroimaging-derived data has become a popular approach to developing markers for brain integrity and health. While a variety of machine-learning algorithms can provide accurate predictions of age based on brain characteristics, there is significant variation in model accuracy reported across studies. We predicted age based on neuroimaging data in two population-based datasets, and assessed the effects of age range, sample size, and age-bias correction on the model performance metrics r, R2, Root Mean Squared Error (RMSE), and Mean Absolute Error (MAE). The results showed that these metrics vary considerably depending on cohort age range; r and R2 values are lower when measured in samples with a narrower age range. RMSE and MAE are also lower in samples with a narrower age range due to smaller errors/brain age delta values when predictions are closer to the mean age of the group. Across subsets with different age ranges, performance metrics improve with increasing sample size. Performance metrics further vary depending on prediction variance as well as mean age difference between training and test sets, and age-bias corrected metrics indicate high accuracy - also for models showing poor initial performance. In conclusion, performance metrics used for evaluating age prediction models depend on cohort and study-specific data characteristics, and cannot be directly compared across different studies. Since age-bias corrected metrics in general indicate high accuracy, even for poorly performing models, inspection of uncorrected model results provides important information about underlying model attributes such as prediction variance.
Draganski Bogdan、Rokicki Jaroslav、Han Laura K.M.、Ebmeier Klaus P.、Hahn Tim、Cole James H.、Kaufmann Tobias、de Lange Ann-Marie G.、Anat¨1rk Melis、Franke Katja、Westlye Lars T.、Aln?s Dag
LREN, Centre for Research in Neurosciences- Dept. of Clinical Neurosciences, CHUV and University of Lausanne||Dept. of Neurology, Max Planck Institute for Human Cognitive and Brain SciencesNORMENT, Institute of Clinical Medicine, University of Oslo, & Division of Mental Health and Addiction, Oslo University Hospital||Centre of Research and Education in Forensic Psychiatry, Oslo University HospitalDept. of Psychiatry, Amsterdam University Medical Centers, Vrije Universiteit and GGZ inGeest, Amsterdam NeuroscienceDept. of Psychiatry, University of OxfordInstitute of Translational Psychiatry, University of M¨1nsterCentre for Medical Image Computing, Dept. of Computer Science, University College London||Dementia Research Centre, Institute of Neurology, University College LondonNORMENT, Institute of Clinical Medicine, University of Oslo, & Division of Mental Health and Addiction, Oslo University Hospital||T¨1bingen Center for Mental Health, Dept. of Psychiatry and Psychotherapy, University of T¨1bingenLREN, Centre for Research in Neurosciences- Dept. of Clinical Neurosciences, CHUV and University of Lausanne||NORMENT, Institute of Clinical Medicine, University of Oslo, & Division of Mental Health and Addiction, Oslo University Hospital||Dept. of Psychiatry, University of OxfordDept. of Psychiatry, University of Oxford||Centre for Medical Image Computing, Dept. of Computer Science, University College LondonStructural Brain Mapping Group, Dept. of Neurology, Jena University HospitalNORMENT, Institute of Clinical Medicine, University of Oslo, & Division of Mental Health and Addiction, Oslo University Hospital||Dept. of Psychology, University of Oslo||KG Jebsen Centre for Neurodevelopmental Disorders, University of OsloNORMENT, Institute of Clinical Medicine, University of Oslo, & Division of Mental Health and Addiction, Oslo University Hospital
医学研究方法基础医学神经病学、精神病学
Brain-age predictionNeuroimagingMachine learningStatistics
Draganski Bogdan,Rokicki Jaroslav,Han Laura K.M.,Ebmeier Klaus P.,Hahn Tim,Cole James H.,Kaufmann Tobias,de Lange Ann-Marie G.,Anat¨1rk Melis,Franke Katja,Westlye Lars T.,Aln?s Dag.Mind the gap: performance metric evaluation in brain-age prediction[EB/OL].(2025-03-28)[2025-06-04].https://www.biorxiv.org/content/10.1101/2021.05.16.444349.点此复制
评论