Thesis Description
“Time is brain” is a frequently used term associated with the fast diagnosis and treatment of cerebrovascular
disease, especially with stroke. In fact, cerebral cell death in the affected areas starts within only a few
minutes after the impairment of the vessel [vB¨ud22]. A distinction is made between ischemic (occlusion of
vessels that results in blood shortage) and hemorrhagic strokes (injury or rupture of a vessel that results in a
bleeding) [Kha+23]. The relevant anatomical structures for this include the circle of Willis (CoW) and the main
surrounding arteries. For the diagnostic process, the tracking of all of these vessels in the volumetric CT data
can be time consuming. In order to improve this process, it would be beneficial to unfold these vessels from the
3D CT data onto a 2D image plane. However, the complex geometry of the CoW and the almost perpendicular
orientation of some vessels to each other make it infeasible for the whole structure to be unfolded properly by
common visualization techniques like the curved planar reformation [Kan+02]. A heuristic mesh-based approach
to solve this problem has been presented with CeVasMap [Ris+23], but the unfolding and merging of all major
vessels creates strong distortions in some areas, especially for the internal carotid artery and the basilar artery.
Instead of a heuristic method, the transformation itself can be seen as an optimization problem, which allows
for a more flexible transformation and a better incorporation of constraints through the loss function.
Neural fields are currently gaining more and more popularity in computer vision due to their ability to
provide a continuous approximation of a field function, which enables sampling at arbitrary points and resolutions
[Ram+22]. The core element is a simple multi-layer perceptron which typically gets coordinates as
input and outputs the respective field value at those coordinates [Xie+22]. Since these values are lying in the
reconstruction domain, a mapping to the sensor domain is needed in order to evaluate the results and compute
the loss [Xie+22]. A few examples for fields that can be described by this are 2D images, 3D shapes or even
full 3D scenes [Xie+22; Mil+20]. Outside of the field of computer vision, neural fields have also gained popularity
in robotics, audio processing, physics and medical imaging [Xie+22]. In the latter, many applications
have emerged, including CT and MRI image reconstruction [Zan+21; Sun+21] and medical image segmentation
[Kha+22]. In addition to that, the implicit deformable image registration model proposed by Wolterink et
al. [Wol+22] fits a 3D deformation vector field for the registration of CT images, which has similarities to the
transformation, that this thesis aims to find. However, in this work the neural field is trained to produce the
3D coordinates that need to be sampled into the respective 2D image coordinates, such that the resulting image
contains the unfolded vessels. The goal is to find a suitable transformation for each sample in the dataset, such
that the neural field is trained to overfit on only one sample, unlike a typical neural network which needs a
large amount of data for training and testing. In order to achieve a visually pleasing training result, different
quality criteria have to be evaluated and incorporated into the loss function. These include multiple constraints
for the transformation, such as diffeomorphism and isometry. Due to the high degrees of freedom for the transformation,
a good initialization has to be found as well. The mentioned constraints and defined loss terms are
iteratively improved to further enhance the quality of the unfolded image. First, the neural field is trained to
unfold one singular vessel, to reduce the complexity of the geometric structure that has to be unfolded. Once
this yields satisfying results, the neural field can be extended to unfold multiple vessels at the same time.
Summary:
1. Implement a basic neural field architecture to fit a 2D to 3D transformation inspired by Wolterink et
al. [Wol+22]
2. Investigate and tune different loss terms and quality measures for the neural field to unfold phantom
geometries and individual vascular structures
3. Assess and discuss the quality of the resulting images
4. If possible, extend the approach to unfold multiple vessels
References
[Kan+02] A. Kanitsar, D. Fleischmann, R. Wegenkittl, P. Felkel, and E. Groller. CPR – curved planar reformation.
In IEEE Visualization, 2002. VIS 2002. Pages 37–44, 2002. doi: 10.1109/VISUAL.2002.
1183754.
[Kha+22] M. O. Khan and Y. Fang. Implicit Neural Representations for Medical Imaging Segmentation. In L.
Wang, Q. Dou, P. T. Fletcher, S. Speidel, and S. Li, editors, Medical Image Computing and Computer
Assisted Intervention – MICCAI 2022, pages 433–443, Cham. Springer Nature Switzerland, 2022.
isbn: 978-3-031-16443-9.
[Kha+23] A. S. Khaku and P. Tadi. Cerebrovascular Disease. https://www.ncbi.nlm.nih.gov/books/
NBK430927/, January 2023.
[Mil+20] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. NeRF:
Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV, 2020.
[Ram+22] S. Ramasinghe and S. Lucey. Beyond Periodicity: Towards a Unifying Framework for Activations
in Coordinate-MLPs. In S. Avidan, G. Brostow, M. Ciss´e, G. M. Farinella, and T. Hassner, editors,
Computer Vision – ECCV 2022, pages 142–158, Cham. Springer Nature Switzerland, 2022. isbn:
978-3-031-19827-4.
[Ris+23] L. Rist, O. Taubmann, H. Ditt, M. S¨uhling, and A. Maier. Flexible Unfolding of Circular Structures
for Rendering Textbook-Style Cerebrovascular Maps. In H. Greenspan, A. Madabhushi, P.
Mousavi, S. Salcudean, J. Duncan, T. Syeda-Mahmood, and R. Taylor, editors, Medical Image
Computing and Computer Assisted Intervention – MICCAI 2023, pages 737–746, Cham. Springer
Nature Switzerland, 2023. isbn: 978-3-031-43904-9.
[Sun+21] Y. Sun, J. Liu, M. Xie, B. Wohlberg, and U. S. Kamilov. CoIL: Coordinate-Based Internal Learning
for Tomographic Imaging. IEEE Transactions on Computational Imaging, 7:1400–1412, 2021. doi:
10.1109/TCI.2021.3125564.
[vB¨ud22] H. J. von B¨udingen. Was Bedeutet ”Zeit ist Hirn”? https : / / schlaganfallbegleitung . de /
wissen/zeit-ist-hirn, July 2022.
[Wol+22] J. M. Wolterink, J. C. Zwienenberg, and C. Brune. Implicit Neural Representations for Deformable
Image Registration. In E. Konukoglu, B. Menze, A. Venkataraman, C. Baumgartner, Q. Dou, and
S. Albarqouni, editors, Proceedings of The 5th International Conference on Medical Imaging with
Deep Learning, volume 172 of Proceedings of Machine Learning Research, pages 1349–1359. PMLR,
June 2022. url: https://proceedings.mlr.press/v172/wolterink22a.html.
[Xie+22] Y. Xie, T. Takikawa, S. Saito, O. Litany, S. Yan, N. Khan, F. Tombari, J. Tompkin, V. Sitzmann,
and S. Sridhar. Neural Fields in Visual Computing and Beyond. Computer Graphics Forum, 2022.
issn: 1467-8659. doi: 10.1111/cgf.14505.
[Zan+21] G. Zang, R. Idoughi, R. Li, P. Wonka, and W. Heidrich. IntraTomo: Self-supervised Learning-based
Tomography via Sinogram Synthesis and Prediction. In 2021 IEEE/CVF International Conference
on Computer Vision (ICCV), pages 1940–1950, 2021. doi: 10.1109/ICCV48922.2021.00197.