Seminar Humans in the Loop: The Design of Interactive AI Systems


  • Bernhard Kainz


Time and place:

  • Time and place on appointment

Fields of study

  • WPF MT-MA from SEM 2
  • WPF AI-MA from SEM 2
  • WPF INF-MA from SEM 2

Prerequisites / Organizational information

recommended prerequisites:
Deep Learning ML Prof. Dr. Andreas Maier 2+2 5 x E
Pattern Recognition ML Prof. Dr. Andreas Maier 3+1+2 5 x E
Maschinelles Lernen für Zeitreihen ML Prof. Eskofier, Prof. Oliver Amft, Dr. Ch. Mutschler 2+2+2 7.5 x E


Human-in-the-Loop Machine Learning describes processes in which humans and Machine Learning algorithms interact to solve one or more of the following:
Making Machine Learning more accurate
Getting Machine Learning to the desired accuracy faster
Making humans more accurate
Making humans more efficient
Aim of this seminar is to give students insights about state-of-the-art Active Learning and interactive data analysis methods. Students will work independently on specific topics including implementation and analytical components alongside lectures delivered by the course lead, guest lectures and flipped classroom sessions, where students explore a topic independently, which is then discussed in class. Several potential topics will be provided but students are also encouraged to propose their own topics (after discussion with course lead).
Topics covered will include but are not limited to:
Introduction to Human-in-the-Loop Machine Learning

  • Active Learning Strategies:

  • Uncertainty Sampling

  • Diversity Sampling

  • Other Strategies

Annotating Data for Machine Learning
  • Who are the right people to annotate your data?

  • Quality control for data annotation

  • User interfaces for data annotation

Transfer Learning and Pre-Trained Models
  • What are Embeddings?

  • What is Transfer Learning?

Adaptive Learning
  • Machine-Learning for aiding human annotation

  • Advanced Human-in-the-Loop Machine Learning

Recommended Literature

17 Bibliography A specific reading list will be established at the beginning of each term, general literature is listed below: Quinn J, McEachen J, Fullan M, Gardner M, Drummy M. Dive into deep learning: Tools for engagement. Corwin Press; 2019 Jul 15. Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning. Cambridge: MIT press; 2016 Nov 18. Budd S, Robinson EC, Kainz B. A survey on active learning and human-in-the-loop deep learning for medical image analysis. arXiv preprint arXiv:1910.02923. 2019 Oct 7.

Additional information

Expected participants: 10