Collaborative Video and Image Annotation

The Heidelberg Research Architecture (HRA)¹ is the Digital Humanities Section of Heidelberg University’s Cluster of Excellence Asia and Europe in a Global Context – Shifting Asymmetries in Cultural Flows.² It brings together a team of IT scientists, software developers, and database architects with international researchers and students. In collaborative projects, the HRA aims to develop an integrated digital humanities environment for interdisciplinary studies of transcultural dynamics.

The core of the HRA research environment is Tamboti, the central metadata framework³ which is used to store standards-based metadata information (e.g. MODS, VRA core 4, TEI, MADS) in an XML database⁴. Researchers can access their resources using a platform-independent browser interface to share and collaboratively work on metadata. In addition, an Open Annotation Collaboration (OAC)⁵ module for comments and research annotations is under development. Flexible webservices and user-friendly interfaces for data input (using XForms) will help to further accumulate resources for research.

This poster presentation provides two examples of research projects focussing on visual resource material.

First, the Video Annotation Database⁶ allows to annotate a film as a whole and to comment on individual scenes by using the Public Access Digital Media Archive (pad.ma)⁷ video annotation software. It offers different types of annotation: transcripts to record the narration, descriptions to annotate scenes, and keywords to add subjects, identify locations, etc. All textual input can be searched. In addition, keywords may be geo-referenced and browsed or searched in a map-interface.

Second, for digital still images a separate module is being developed using the VRA core 4 XML schema⁸. Besides storing images with metadata its aim is to provide researchers with a set of useful tools to directly link the visual material with their research. Important features for its users will be the opportunity of collaborative metadata editing, a flexible system for sharing of resources, as well as tools for annotation and comments based on OAC. In addition, the module will allow to automatically extract embedded metadata on ingest via exif-tool. It will also be possible to mark, annotate and link parts of images, to group images on virtual lighttables, and to save and export image sequences (‘visual itineraries’).

Two ongoing research projects will exemplify the use of the databases:

Example 1: Global Politics on Screen

As an example for empirical historical research concentrating on visual source material, the project Global Politics on Screen – A Japanese Film on the Lytton Commission in 1932⁹ tests the potential of video annotation tools by combining pad.ma with models of historical film interpretation. In this research based training project, a team of history students and researchers centres on analysing and commenting a silent Japanese propaganda film about a crucial moment of world history – the Japanese invasion of Manchuria in 1931 which led the way to the Second World War. While the League of Nations commission aimed at preventing further military disputes in the ‘Far East’ the focus of the film producers was to propagate their own interpretation of international politics. The analysis of the film is a starting point to approach the complex international diplomatic and economic relations of the early 1930s. The project builds up a platform to explore the global contexts and public representations of these critical historical incidents from a new perspective.

Example 2: Priya Paul Collection

The Priya Paul Collection of Popular Art¹⁰contains more than 4.200 illustrations from late 19th and 20th century India. It is one of the finest collections of ephemera such as old posters, calendars, postcards, commercial advertisements, textile labels and cinema posters. In a collaborative endeavor with Tasveerghar – A Digital Network of South Asian Popular Visual Culture¹¹the collection was digitised and is now being annotated by experts from history, art history, visual and media anthropology, and ethnology. The subststantial research metadata of the collection is a showcase for  the implementation of VRA Core 4 XML in the metadata framework and a user-friendly VRA metadata editor is under development. Content analysis is closely connected with the intense study of the material which already resulted in fourteen research essays and extensive work of annotation. With the new digital still image module, Tamboti will serve as a platform to integrate archive and metadata storage with scientific discussion and analysis within a single framework.

Screenshot 1: Video Annotation Database – Editor

Screenshot 2: Image from the Priya Paul Collection – Image Viewer (HeidICON ¹²)