Wuyang Li^1,3,*, Wentao Pan^1,*, Xiaoyuan Liu^2,3,*, Zhendong Luo², Chenxin Li¹,
Hengyu Liu¹, Din Ping Tsai^2,†, Mu Ku Chen^2,†, Yixuan Yuan^1,†

¹CUHK , ²CityU , ³EPFL

^*Equal Contribution  ^†Corresponding author

🌟 ICCV 2025 Highlight 🌟 (Final Rating: 6, 6, 6)

TL;DR

Unlike conventional endoscopes limited by millimeter-scale thickness, metalenses operate at the micron scale, serving as a promising solution for ultra-miniaturized endoscopy. However, metalenses suffer from intensity decay and chromatic aberration. To address this, we developed MetaScope, an optics-driven neural network for metalens-based endoscopy, offering a promising pathway for next-generation ultra-miniaturized medical imaging devices.

We pioneer a new direction in advancing in-vivo intelligence by integrating OPTICAL SCIENCE, BIOLOGICAL SCIENCE, and COMPUTER SCIENCE.

Motivation

Existing Challenges in Metalens Image

• Color and Position-Dependent Intensity Loss. Color- and position-dependent intensity loss is caused by wavelength-sensitive meta-atom responses leading to RGB imbalance, and off-axis efficiency drop with cos⁴θ vignetting causing peripheral darkening and structured noise.
• Color-Dependent Blur and Misalignment. Color-dependent blur and misalignment arise from wavelength-dependent PSF variations in metalenses, causing defocus and chromatic fringing due to imperfect phase compensation and focal length mismatch across RGB channels.

Our Solution

• To fully leverage observed optical priors, we propose to encode the efficiency and radial priors to adjust the feature learning with restoration awareness
• We propose a novel solution by estimating the full-color PSF distribution via learning inherent Gaussian mixtures adaptively, thereby spatially guiding correction of chromatic aberrations

Framework of MetaScope

• Optics-informed Intensity Adjustment. Metalens images experience severe attenuation in adhering to specific priors. We design an optics-informed intensity adjustment module to effectively correct channel- and spatial-wise attenuation in metalens images by embedding color- and position-dependent optical priors.
• Optics-informed Chromatic Correction Metalens images suffer from chromatic artifacts due to wavelength-specific PSFs. Considering the difficulty in estimating explicit PSFs. we model the implicit PFS distribution via Gaussian Mixture Models (GMM) in an adaptive manner for chromatic correction

Benchmark

Dataset

We construct five benchmark datasets focused on two key clinical tasks: diagnosis and surgery. By projecting public datasets onto a screen and capturing them with a metalens system, we generate corresponding image triplets at high resolution. This supports two research directions—segmentation and restoration using metalens images

Metrics

1. Clinical assistance: Metalens segmentation adopts mIoU and mDICE (IoU/DICE for single-class datasets).
2. Clinical visualization: Metalens restoration uses PSNR and SSIM.

Experimental Results

MetaScope Leader Board

Comparison with state-of-the-art methods on five benchmarks in terms of metalens segmentation (top) and restoration (bottom)

Comparison between conventional systems using high quality ground-truth images without Meta, and our metalens system using the degraded images in Meta version dataset.

Visual Results

Qualitative comparison in terms of state-of-the-art segmentation methods and image restoration methods.

Qualitative comparison of state-of-the-art segmentation methods on Meta-CVC-Colon, Meta-Kvasir-Seg and Meta-EndoVis-18.

Qualitative comparison with state-of-the-art restoration methods on Meta-CVC-Colon, Meta-Kvasir-Seg and Meta-EndoVis-18.

Futher Analysis

Detailed ablation study of the proposed modules on the Meta-CVC-Clinic benchmark

Heatmaps of the feature before and after each module.

Sensitivity analysis of the simulated optical prior and hyper-parameters on the Meta-EndoVis-17 benchmark

📄 BibTeX Citation

📋 Copy Citation

@article{li2025metascope,
  title={MetaScope: Optics-Driven Neural Network for Ultra-Micro Metalens Endoscopy},
  author={Li, Wuyang and Pan, Wentao and Liu, Xiaoyuan and Li, Chenxin and Liu, Hengyu and Tsai, Din Ping and Chen, Mu Ku and Yuan, Yixuan},
  journal={arXiv preprint},
  year={2025}
}