A Unified Approach to Multi Disease Prediction using Deep Learning Models and Medical Language Intelligence

Abstract

Retinal diseases such as Diabetic Retinopathy, Glaucoma, and other vision-threatening abnormalities are major causes of preventable blindness
when not detected at an early stage. Manual retinal screening requires expert ophthalmologists and is often limited by time, cost, and
availability, especially in rural and underserved regions. To address these challenges, this project titled “Retinal Multi-Disease Prediction”
presents an intelligent screening and awareness support system that combines Convolutional Neural Networks (CNNs) with Large Language
Models (LLMs), including Med-PaLM2, to provide automated retinal disease detection along with patient-friendly explanations. The proposed
system accepts a retinal fundus image as input and performs automated analysis using a deep learning–based CNN model built on advanced
architectures such as EfficientNet. The CNN extracts high-level visual features from the fundus image and classifies it as Normal, Abnormal,
or Dangerous, and can further identify specific retinal conditions such as Diabetic Retinopathy. The image analysis and disease classification
are handled exclusively by the CNN model to ensure reliable and focused visual interpretation. Following the image-based prediction, the
classification result is passed as textual input only to the LLM and Med-PaLM2, which do not analyze the image directly. Their role is to
transform the medical prediction into a clear, patient-friendly explanation using simple language. The generated explanation includes the
detected disease, possible causes, recommended next steps, and guidance on when the patient should consult an ophthalmologist. The system
also supports bilingual explanations in English and Tamil, improving accessibility and patient understanding. The application features a
professional and user-friendly interface, including secure login and sign-up modules, prediction history tracking, dark and light mode themes,
and visually appealing background design to enhance usability. This system is not intended to replace medical professionals or provide
treatment decisions. Instead, it serves as a screening and awareness support tool aimed at promoting early detection, improving health literacy,
and encouraging timely consultation with eye care specialists. By integrating CNN-based medical image analysis with LLM-driven natural
language explanation, the proposed system bridges the gap between technical predictions and human understanding, making retinal health
screening more accessible, interpretable, and impactful.