Fast motion feedback is crucial in computer-aided surgery (CAS) on moving
tissue. Image-assistance in safety-critical vision applications requires a
dense tracking of tissue motion. This can be done using optical flow (OF).
Accurate motion predictions at high processing rates lead to higher patient
safety. Current deep learning OF models show the common speed vs. accuracy
trade-off. To achieve high accuracy at high processing rates, we propose
patient-specific fine-tuning of a fast model. This minimizes the domain gap
between training and application data, while reducing the target domain to the
capability of the lower complex, fast model. We propose to obtain training
sequences pre-operatively in the operation room. We handle missing ground
truth, by employing teacher-student learning. Using flow estimations from
teacher model FlowNet2 we specialize a fast student model FlowNet2S on the
patient-specific domain. Evaluation is performed on sequences from the Hamlyn
dataset. Our student model shows very good performance after fine-tuning.
Tracking accuracy is comparable to the teacher model at a speed up of factor
six. Fine-tuning can be performed within minutes, making it feasible for the
operation room. Our method allows to use a real-time capable model that was
previously not suited for this task. This method is laying the path for
improved patient-specific motion estimation in CAS.
Sontje Ihler, Max-Heinrich Laves, Tobias Ortmaier