Visual Analytics and Computer Vision meet Cultural Heritage

Wider research context/theoretical framework

In the last decades, archives and museums made extensive efforts to foster the digital preservation of cultural heritage (CH) artifacts such as historical photographs and amateur films. These collections of digitized artifacts offer immense potential to increase the knowledge of our heritage by investigating patterns and relationships. The systematic analysis and presentation of these large amounts of visual media data are, however, still strongly limited due to the lack of metadata and suitable analysis approaches. This impedes historians and film archivists but also lay users in analyzing, interpreting, and thereby preserving the memories of human cultural history.

Hypotheses/research questions/objectives

Our cooperative doctoral programme aims to close basic and applied research gaps by combining computer vision (CV) and visual analytics (VA). We jointly work on the advancement of interactive analysis, exploration, and presentation of historical visual media collections. The central hypothesis behind this is that combining CV and VA is necessary to unlock large cultural heritage collections and open them to a wide range of users. 

Approach/methods

We follow a problem-driven research paradigm determined by the cultural heritage domain. We leverage human-in-the-loop machine learning (ML) and integrate CV and VA methods for the analysis of cultural objects and their relationships across space and time. This includes ML, information retrieval and CV for analysis as well as situated visualization and storytelling for exploration and presentation. We quantitatively evaluate the results against ground truth annotations and through user studies.

Faculty

The project is a collaboration of researchers from St. Pölten University of Applied Sciences (FHSTP) and TU Wien. Seven faculty members contribute complementary research expertise on CV, VA, and CH across basic and applied research. The project shall elevate the existing scholarly ties between FHSTP and TU Wien to a new level of inter-institutional cooperation. 

Doctoral programme

The doctoral programme foresees 5 thematically related PhDs. The topics focus on the interactive analysis of visual media in cultural heritage collections and complement each other in terms of methods, use cases, and addressed users. Each topic is supervised by a team of 2-3 researchers (at least one from each institution), and a comprehensive education and training programme tailored to research is provided.

Added value

Added value is generated at multiple levels: 1) embedding doctoral students in a team of high profile visual computing researchers; 2) increasing junior faculty members’ experience in supervision skills; 3) leveraging the complementary strengths of applied and basic research; 4) sharing of resources between institutions; 5) opening up large cultural heritage collections to a wide range of users by combining CV and VA.