Deep Learning Model Produces a Highly Accurate Prediction of Outcomes from Time of Implant to Ten-Years Post Pediatric Heart Transplantation

H. F.. Ahmed¹, H. Shih², M. Hossain³, D. Wu², R. Moore⁴, F. Zafar¹, C. Chin⁴, D. Morales^5
1Division of Cardiothoracic Surgery, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio ²Department of Biomedical Informatics, University of Cincinnati, Cincinnati, OH, USA, Cincinnati, Ohio ³Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA, Cincinnati, Ohio ⁴Division of Cardiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH, USA, Cincinnati, Ohio ⁵Division of Cardiothoracic Surgery, Cincinnati Children's Hospital Medical CenterCincinnati Children's Hospital Medical Center, Cincinnati, Ohio

Purpose: The ability to accurately predict pediatric heart transplant survival beyond the first post-surgical year is elusive, possibly due to relying solely on pre-transplant data. We aimed to use contemporary deep learning methods and incorporate data from all transplantation phases (pre-operative,operative,and post-operative) to achieve a survival prediction model with outstanding accuracy.

Methods: Organ Procurement and Transplantation Network data was extracted from pediatric patients who underwent cardiac transplantation as of March 2020. Data from all phases of transplantation were collected, 250 variables were selected, the model was trained and evaluated, and a deep neural network consisting of four fully connected layers was implemented. Prediction time in this study consists of one month, six months, one year after transplant, and every subsequent year up to ten years. The area under the receiver operating characteristic curve (AUROC) was used to assess model performance for accuracy and Shapley Additive Explanations (SHAP) were used to interpret the model’s decision-making process. The feature importance was calculated by the mean absolute value of the SHAP values of each feature. The feature importance in each time point was standardized to compare across multiple time points.

Results: 10,213 patients met the inclusion criteria. Three models were tested and the one that included follow-up data throughout all 3 phases of care achieved the highest AUROC at all timepoints from 1 month to 10 years after transplant (Figure-1). Predictive accuracy was excellent at 6 months to 1 year and outstanding by 2 years after transplant, defined by an AUROC > 0.8 and 0.9 respectively. An explanation of the prediction model is demonstrated in Figure-2A, which identifies standardized feature importance across prediction years. The top 5 impactful factors affecting 1-month survival included recipient primary diagnosis, ischemic time, distance from the donor and recipient hospitals, donor age, and serum bilirubin. From 2-10 years, renal function, where the recipient receives their transplant care, functional status, and the presence of transplant coronary artery disease became the top impactful factors in mortality prediction. At 10-years, the top 5 factors were post-transplant coronary artery disease, post-transplant renal dysfunction/chronic dialysis, recipient height, permanent pacemaker, and need for renal transplant. The recipient’s primary diagnosis remained a highly impactful factor throughout time and ranked as the 6th most important feature in the model even 10-years post-transplantation. Figure-2B shows the most impactful features affecting survival with ranking over time.

Conclusion: We have developed a highly accurate deep learning model that identifies risk factors influencing survival from time of implant to 10 years after pediatric heart transplantation. Compared to traditional models, the inclusion of transplant follow-up data significantly increases the accuracy of predicting survival. Deep learning models can provide realistic expectations to patients and providers and, with further development, can identify individualized modifiable risk factors to improve outcomes. Future work involves developing a patient-facing interface to provide prediction results to patients, support shared decision-making, and collect more follow-up data within the first year to improve model performance.

Identify the source of the funding for this research project: NIH Grant/ R01HL147957 - “Novel Methods to Grow the Impact of Pediatric Thoracic Transplantation”