|
Seamless Image Archiving for Museums and Enhanced Sharing Experiences | |
The Royal Museums of Fine Arts of Belgium are setting up a digitization lab which will envision new techniques related to the life cycle of the images, especially regarding their acquisition, storage, and secure sharing including enhanced user experiences, such as content-based retrieval.
In a digital society strongly relying on images, advanced processing techniques will offer the consumer as well as the digital curator enhanced experiences and tools giving them easy, seamless access to art collections.
This research takes part in the dynamic instilled by the Victoria & Albert Museum (London), the world’s largest museum of applied and decorative arts and design (including sculpture) through its ReACH (Reproduction of Art and Cultural Heritage) initiative, launched in 2017. The ReACH declaration, aimed at museum and private sector contractors, enumerates points to look out for and best practices regarding the preservation and sharing of art and cultural heritage.
The aim of this project is to address the three main axes highlighted by this declaration, namely, reproduction, storage and sharing.
Numerous advances have been made in acquisition techniques going beyond regular Bayer linear sensors (such as non-Bayer interpolated RGB sensors, sparse image sensors, light field sensors, multi-shot capture, etc.) and in dynamic range (HDR) imaging as well as the use of non-linear opto-electrical transfer functions in place of the usual gamma correction curves.
These techniques and their suitability for either short-term or long-term acquisition processes will be investigated.
To allow for the multiplicity of possible usages of the distributed images, two approaches will be considered: either storing each image with a specific resolution and a specific format to match every foreseen usage for this image and then, sending the most adequate version to the final user or storing each image in its native, maximal resolution, using a quality-preserving compression algorithm, and then re-compressing it on-the-fly at the best suited resolution and format for each user request. In the case of wavelet-based coding systems (such as JPEG-2000), a flexible bit stream is produced and re-compression can be alleviated by adequate selection of the information in the compressed stream.
Quality assessment is currently conducted by performing a full-chain calibration of the acquisition, compression, decompression and reproduction of the artwork. This type of workflow is adequate when a given digital image of an artwork is intended to be reproduced through specific devices but does not account for today’s variety of usages of the digital copies of artworks: display on an HD, 4K, or 8K television or computer screen, mobile device display, projection on a white screen, printing on a variety of media, etc.
Calibration and quality models (involving various metrics, such as PSNR, SSIM, curvelets, etc.) will be investigated and adjusted in order to offer to the viewer the best visual experience possible. The relevance of no-reference image quality assessment models, traditionally aimed at natural scenes, will be investigated in the case of digitized artworks.
For every digital image of an artwork being archived a mechanism to display the difference with each version of this specific artwork will be developed. Those differences could be caused by an improvements in the acquisition techniques and processes (image sensors, lighting techniques, etc.) or by deterioration of the original artwork over time. To assess the nature of this difference, a semantic matching algorithmic approach will be developed on the basis of key features.
New compression techniques based on machine learning could considerably increase the efficiency of the compression for an identical compressed bit stream size. Moreover, ongoing work on video coding for machines enables efficient compression not only for humans but also to optimize feature detection by further automated procedures. Including feature maps in the encoded bit stream, or even proceeding to automated feature extraction and identification during the compression process, could make it possible to enrich the metadata associated with an artwork automatically. Such automatically augmented metadata could support advanced queries from a user, e.g., based on the semantic content of artworks.
The museum community is faced with the need for Open Access images. Through the introduction of artificial intelligence, the search for works within a collection as we know it today could gain spontaneity. One or more images could be provided to the user after a conversation that targets their request and expectations very precisely. This would be a new way of accessing images, based on a question-and-answer session between man and machine, with the aim of bringing art and the public together.
In order to provide the virtual visitor with an enhanced experience, a both visually and semantically attractive online interface should offer access to the Museum’s online collections. The visual experience should give the user natural and comfortable access to each picture, enabling the user to visualize it under the best conditions, or maybe even changing the type of illumination. We could combine such seamless and natural access to the artwork (front-end) with optimized extraction and distribution of content (through intelligent selection and compression mechanisms).
Immersion is part of museum exhibitions. Given the multiplication of immersive experiences, public’s confidence could be increased by providing the visitors with tools to identify the origins of the digital images used and to assess whether the visual quality of the original artworks has been compromised by improper processing of the images received from the museum.
Robust watermarking techniques, aimed at embedding a mark invisible to the naked eye but designed to survive severe alteration of the digital document, could enable a visitor of a digital or immersive exhibition to check the origin of the digitized artworks simply by using a watermark detection application able to identify the museum (if any) from which a given image comes.
Fragile watermarking techniques could be exploited to ensure that a digital image of an artwork has not been inadequately handled, meaning that its visual quality has not been compromised. A visitor could for instance use an application to detect the robust watermark on a digital document (identifying its origin) as well as to assess the preservation of the fragile watermark, which would confirm that the image has undergone no harmful or improper processing.