First Endoscopic VBTS with Co-Located Vision and Tactile Sensing for RMIS

MVP-Tac

A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor
for Robot-Assisted Minimally Invasive Surgery

Md Rakibul Islam Prince Jaeeun Kim Yuhao Zhou Mason Vrshek Shivani Reddy Sama Adyaa Khera Sheeraz Athar Zijie Xu Jiabin Liu Shaoting Lin Wei Li Yu She*

Purdue University  |  Stony Brook University  |  Michigan State University

Paper Hardware Coming Soon Code

MVP-Tac integrates co-located vision and tactile sensing in a compact endoscope-compatible form factor.

Abstract

Robot-assisted minimally invasive surgery offers major benefits over open and conventional laparoscopic procedures, yet it still lacks tactile feedback for palpation while operating under strict requirements to preserve reliable vision. MVP-Tac is a compact vision-based tactile sensor that provides co-located vision and tactile sensing using reflective photoelastic imaging. A thin photoelastic elastomer generates stress-dependent interferograms under contact, while a semi-transparent membrane and controllable illumination enable switching between visual and tactile modes. MVP-Tac is validated through force calibration in the 0–2 N range, video-based hardness classification on exposed and subdermal tumor phantoms, and a simulated colonoscopy experiment with vision-guided 3D photomapping and in situ hardness classification.

Paper page 1 Paper page 2 Paper page 3 Paper page 4 Paper page 5 Paper page 6 Paper page 7 Paper page 8

Key Highlights

7 mm

Miniaturized form factor designed for endoscopic and RMIS-related deployment.

Dual Modal

Switches between visual mode and tactile mode using controllable illumination and a semi-transparent membrane.

0–2 N

Force calibration range targeting light palpation and fine contact-rich surgical interaction.

97.37% / 92.11%

Video-based hardness classification accuracy for exposed and subdermal tumor phantoms.

Sensor Design

MVP-Tac combines a dual-mode elastomer module, a reflective circular-polariscope optical module, and a compact perception module. The tactile mode uses stress-induced birefringence in a photoelastic elastomer, while the vision mode allows the camera to see through the semi-transparent membrane when external illumination dominates.

This design avoids structured lighting and supports a compact configuration suitable for constrained surgical environments where visual feedback and tactile palpation must coexist.

MVP-Tac fabrication and calibration

Dual Modality

Vision Mode

Sensor view under ambient room lighting only.

Vision Mode

Sensor view with external illumination enabled.

Tactile Mode

Photoelastic response to a spherical indenter pressed on the sensor pad.

Sphere indenter

Tactile Mode

Photoelastic response to a PH2 screwdriver bit pressed on the sensor pad.

PH2 screwdriver bit

Experimental Results

MVP-Tac force and tumor classification results

Force Calibration and Hardness Classification

MVP-Tac uses image features together with a photoelastic gradient metric, G², for force estimation. The fusion model achieved MAE = 0.0511 N, RMSE = 0.0685 N, and R² = 0.9874 on the test set in the 0–2 N range.

A ResNet-18 + GRU video classifier predicts tumor hardness from 16-frame palpation clips, achieving 97.37% accuracy for exposed tumor phantoms and 92.11% accuracy for subdermal tumor phantoms.

Simulated Colonoscopy Demonstration

MVP-Tac was mounted on a UR5e robot and inserted into an intestine-mimicking phantom to demonstrate a vision-guided search-then-palpate workflow. The system visually scanned the lumen, reconstructed a 3D photomap of the luminal wall, localized nodules, and then switched to tactile mode for local hardness classification.

Video

BibTeX

@misc{prince2026mvptacminiaturizeddualmodalvision,
      title={MVP-Tac: A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor for Robot-Assisted Minimally Invasive Surgery}, 
      author={Md Rakibul Islam Prince and Jaeeun Kim and Yuhao Zhou and Mason Vrshek and Shivani Reddy Sama and Adyaa Khera and Sheeraz Athar and Zijie Xu and Jiabin Liu and Shaoting Lin and Wei Li and Yu She},
      year={2026},
      eprint={2607.18660},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2607.18660}, 
    }