Analysis of Learning Dynamics in HORN-based Convolutional Neural Networks for Computer Vision

Type: MA thesis

Status: open

Supervisors: Michael S. Simo, M. Sc.

Analysis of Learning Dynamics in HORN-based Convolutional Neural Networks for Computer Vision

Background

Dynamic neural networks are gaining increasing attention as they explicitly model temporal processes and may enable more efficient and biologically plausible learning mechanisms. A promising approach in this context is the HORN model (Harmonic Oscillator Recurrent Network), which is based on continuous dynamics and has already demonstrated interesting properties such as fast learning behavior and high computational efficiency.

In this thesis, the HORN model will be extended to convolutional neural networks (HORN-CNN) and systematically investigated. Preliminary results indicate that HORN-based models are competitive with classical CNNs at comparable parameter counts, while in some cases learning faster and requiring fewer computational operations.

Objective

The objective of this thesis is the systematic analysis of the learning behavior and internal dynamics of HORN-based convolutional neural networks, as well as their comparison to classical CNN architectures.

A particular focus lies on understanding the underlying dynamics of the model, especially with respect to observed phenomena such as fast initial learning.

The central research question is:

Do HORN-based models really learn faster and more efficiently?

Tasks

The thesis will include the following components:

  • Literature review

    • Dynamic neural networks
    • Efficient CNN architectures
    • HORN model and related approaches

  • Reproduction and extension of existing experiments

    • Implementation and training of HORN-CNN models
    • Comparison with classical CNN baselines
    • Experiments on datasets such as MNIST, CIFAR-10, and potentially CIFAR-100

  • Systematic analysis of model parameters

    • Influence of parameters such as timesteps, num_nodes, and heterogeneity
    • Investigation of stability and convergence behavior

  • Analysis of learning and network dynamics

    • Examination of internal states over time
    • Analysis of activations, dynamics, and potential saturation effects
    • Identification of mechanisms behind the observed “fast learning” behavior

  • Efficiency analysis

    • Comparison of FLOPs, parameter count, and performance
    • Positioning of results in the context of efficient deep learning models

  • Optional extensions

    • Architectural or implementation improvements
    • Exploration of additional datasets or model variants

Requirements

  • Ability to work independently and in a structured manner
  • Solid knowledge of machine learning / deep learning
  • Experience with Python and Pytorch
  • Interest in mathematical understanding of models and dynamical systems
  • Ability to work independently and in a structured manner

Apply

Please send me an Email to michael.simo.seyaze@fau.de with the subject:

Application | Thesis | HORN-based CNN | <Your Full Name>

Please include a short motivation and your earliest possible start date.

Attach as one single PDF:

  • CV
  • Transcript of records (dated)
  • Optional: code links (GitHub, etc.)