Sound & Vision co-organised the second International Workshop on Linked Media (LIME). It was held at the European Semantic Web Conference on May 25th. Its aim is to promote the principles of Linked Media on the Web by exchanging knowledge on applications and services that make it easier to share, find and re-use online media.
Sound & Vision presented the case study of Na de bevrijding XL (After the liberation XL). In this interactive version of a television programme on the post-WOII period, archival sources were linked to specific segments of the programme.
Linked Media case studies @ LIME
The LIME workshop [1] keynote was given by Sebastian Schaffert, CTO of RedLink. Schaffert presented the case study of the Red Bull Content Pool, a vast online archive of videos and images related to extreme sports events organised by Red Bull. The content contains rich metadata, such as interview transcripts, scene descriptions and music cue sheets. This metadata was however not really exploited for searching, and it was really difficult for re-users to find specific clips within the videos that were of interest to them.
Thus, a project was set up, in which Linked Media principles were used. For example: Media Fragment URIs were used to uniquely identify either the videos themselves or specific parts within a video, both temporal (e.g. from 3-10 seconds) and spatial (a specific object in a (sequence of) frame(s)). This made it possible to add metadata at a very fine-grained level, which in turn makes it easier for customers to find what they need. Furthermore, a common metadata model was devised to represent the video metadata, such as information about the creator, genre, keywords, language and technical properties. This model was mapped to popular standards and ontologies, such as MPEG-7 and TV-Anytime. Using all of these Linked Media principles created an archive that enables users to find very specific fragments much easier.
Although the case study presented by Schaffert was very exciting, the term Linked Media was then critically analysed by Lyndon Nixon, Assistant Professor of the Department of New Media Technology at MODUL University Vienna. He examined what the term actually means and entails. For example, if you want to link information to a ZDF documentary on a self-portrait by Rembrandt uploaded to YouTube, what is the actual media object you want to annotate, and what are its properties? Is it the documentary, the frame in which the image of Rembrandt’s self-portrait appears? What is the date: is it the upload date or the date the portrait was made? And how do you uniquely identify all these things using URIs?
The LinkedTV project served as a case study. In it, Sound & Vision develops scenarios related to the antiques roadshow Tussen Kunst & Kitsch, of Dutch broadcaster AVRO. The various art objects are annotated and enriched with data such as e.g. their creators, art styles and related content, using Linked Data and Linked Media principles.
Na de bevrijding XL: Contextualising Broadcasts, Tablet-First
The LIME workshop was closed with Sound & Vision’s presentation of Na de bevrijding XL. Technically speaking, it doesn’t make use of Linked Media standards quite yet, but the principles of Media Fragments and linking information to these fragments are firmly embedded.
In January this year, public service broadcaster NTR aired the first episode of Na de bevrijding, a seven-part television series about the first five years after World War II in The Netherlands. The series paints a pervasive picture about this lesser known period in Dutch history, in which the recently liberated nation slowly began to recover from the chaos of war. In each episode, 5 to 6 people are interviewed about the post-WOII period (1945-1950) between the liberation and the reconstruction of the parts of the country destroyed by the war. All episodes contain a lot of archival footage, information from personal diaries, newspapers, photos and radio shows, both from archives and personal collections. [2] Sound & Vision cooperated with NTR to realise a tablet-first online ‘XL’ edition in expansion of each episode. This interactive version lets visitors browse through the original archive sources by providing full access to the films, soundtracks, photos and newspaper articles used in the series. The basis for Na de bevrijding XL were the separate fragments of each episode, to which specific concepts (e.g. topics (“Farmers fight for the future”) and persons (“Jan Drenthe”)) were assigned. Each fragment contains a related archival source -- images, videos and audio -- either shown in the programme, or that formed the basis for the research, thus further linking and enriching the viewing experience.
Besides contextualising and enriching Na de bevrijding, a second goal of the project is to technically explore the possibilities for Sound & Vision to develop other interactive publications. In order to accommodate a broad group of internet users and since viewing television on tablets is increasing more and more (50% increase from 2012-2013) [3], a tablet-first approach was chosen for the development. This means that although Na de bevrijding XL works on any modern browser, it was designed with tablet users in mind.
Timeline view of Na de bevrijding XL. The source timeline with the programme fragments (bottom) shows thumbnails from the archive materials related to the programme (right). The highlighted thumbnail corresponds to a specific archival source (left).
Technical Realisation
The project’s technical partner, Videodock, built a HTLM5 web application that works on all major contemporary browsersand devices. The interface design focuses on ease of use for novice and older users. When users watch the programme, the timeline (see above) highlights the specific item that the segment was selected from. In the sidebar, the element with minimal description and a provenance link comes up, allowing the user to switch back and forth to the archive layer.
The back-end made use of an editor-friendly, minimal set-up. This allows web editors to quickly make changes to the desired length and visibility of the archive clips and multiple checks in collaboration with the editors of the series. Time cues, file locations and descriptions were edited in a shared Google Spreadsheet, which was used by the technical partners to link the programme fragments to the archival sources.
Future Work
As described above, all the archival sources in Na de bevrijding XL were added manually, to a Google Spreadsheet. In order to scale the process of enriching other programmes or larger parts of the Sound & Vision archive with related sources, all elements in the enrichment chain would need to become heavily automated, because the process would otherwise be much too labour-intensive. This would entail using tools and techniques developed in research projects like LinkedTV and AXES, both of which Sound & Vision is involved in. This could be using techniques such as automatically diving a programme in distinct chapters (scene segmentation) and detecting relevant visual or textual concepts to link to. This can for instance be done by looking for specific terms like ‘Westerbork’ in the subtitles and automatically linking to relevant content in archives.
These are just some examples, and a simplification of the effort and complexity of techniques and steps needed to facilitate the enrichment of archival content. However, with the growing group of television viewers using second screens and the desire for audiovisual archives like Sound & Vision to contextualise and enrich their holdings, these automated processes are crucial for fulfilling these needs. It is vital that the work being done in this area continues in collaborative environments in which content producers, memory institutions and technical research organisations participate.
More information
- For more information about Na de bevrijding XL, see the R&D blogs about the project (In Dutch).
- Na de bevrijding XL was realised with the support of the Images for the Future programme.
- For the full LIME paper on Na de Bevrijding XL, click here.
Footnotes
[1] For all LIME workshop presentations, see http://www.linkedtv.eu/event/lime2014.
[2] For the full list of collections, see http://www.geschiedenis24.nl/na-de-bevrijding/colofon.html.
[3] Eds, Trends in Digitale Media. Intomarkt GfK (2013).