Photoacoustic Histology of Human Liver Cancers: Virtual Staining, Segmentation, and Classification via Interconnected Deep Learning

Label-free photoacoustic histology (PAH) has been developed to simplify the conventional laborious and costly process of obtaining histological images in pathology diagnosis. However, poor histological visualization still hinders the clinical use of PAH. In this study, we introduce and interconnected deep learning system for virtual staining, segmentation, and classification of PAH from human liver cancers. The framework is composed of three components that share their results for better analysis: Virtual staining step, which converts photoacoustic images into H&E staining style with explainable contrastive unpaired translation (Explainable CUT); Cell segmentation step, which segments and analyzes the cell nuclear information of the tissue with U-Net; and finally, Cancer classification step, which determines whether the tissue contains cancer with stepwise feature fusion (StepFF) method. Virtual staining uses Explainable CUT, an explainable AI model that allows users to understand the model and generates high-quality staining results similar to real H&E. In addition, for advanced analysis, cell features can be obtained and analyzed by segmentation. Cancer classification using StepFF, which utilizes features from PAH, virtual staining results, and segmentation results together, achieved 98.00% accuracy compared to 94.8% using PAH alone. This interconnected deep learning system for PAH overcomes the histological visual limitations of conventional PAH and provides additional advanced analysis results, showing potential to improve pathology diagnosis.