Visual Foundation Models with Adapter Learning for Parkinson’s Disease Recognition based on OCT Images

Research overview

The text below is the source article’s original abstract under a verified reuse license. cc by

Abstract Parkinson’s disease (PD) is the leading neurodegenerative disorder in the world. Clinical studies have verified that the pathology changes in the retina structure can be potential biomarkers for PD diagnosis based on optical coherence tomography (OCT) images. Scholars have gradually applied deep neural networks (DNNs) to automatic PD recognition and have achieved good results. However, building an accurate DNN-based PD recognition model requires a large number of OCT images and expensive computational resources. Moreover, recent pre-trained visual foundation models (VFMs) have achieved promising success in a variety of visual recognition tasks with the aid of parameter-efficient finetuning techniques by tuning only a few trainable parameters, rather than traditional full finetuning techniques by updating all pre-trained parameters. Therefore, we transfer the pre-trained VFM with adapter learning paradigm (VFM-Adapter) techniques to automatic PD recognition based on OCT images. To be specific, by freezing pretrained parameters of VFMs, we introduce three lightweight adapters: global adapter, local adapter, and token-wise adapter, to effectively leverage the transferability of feature representations from VFMs to OCT images for equipping VFMs with strong PD recognition capability. The results on the clinical OCT-PD dataset show that our proposed VFM-Adapter can achieve competitive or even better PD recognition performance through comparisons to state-of-the-art DNNs with substantially fewer tunable parameters.

Related topics