Lossless JPEG-2000 Archival Format Deployment and Space Savings
"Explore the benefits of deploying lossless JPEG-2000 as an archival format, including significant space savings and the technical processes involved. Learn about the impact on digital preservation systems and collections, highlighting the challenges and solutions encountered during the conversion process."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Saving Space Deploying Lossless JPEG-2000 as an Archival Format Dean Smith, Digital Preservation Lead, The Frick Art Reference Library smith@frick.org Slides available at https://tinyurl.com/SavingSpace2020 For additional detail, please visit my repository at https://tinyurl.com/SavingSpaceGitLab
What Photoarchive mounts look like Reproduction only Full mount front Full mount back Mount flap Francisco de Goya, The Forge, courtesy of The Frick Collection
Calculating the space savings Average TIFF in 75MB (minus) average JP2 is 30MB (equals) 45MB per image file saved (times) 3 million image files (equals) 135TB saved for each instance of the collection (times) Four copies Access versions on Xinet DAMS Preservation versions in triplicate: On-prem server, Arkivum s Private Cloud, and LTO tape (equals) 540TB saved in aggregate 540TB = Wow!
Highlights of the conversion process Dot-gain 20% grayscale profile is incompatible with JPEG-2000 Need Photoshop to update the profile Kakadu performs the primary conversion Can t transfer metadata due to format incompatibility ExifTool transfers file metadata from original TIFF to resultant JPEG-2000 ImageMagick checks for image fidelity Jpylyzer checks for file validity
Impact on Xinet DAMS and Digital Collections public portal Xinet needed to add support for JPEG-2000 in order to render images Digital Collections struggles to keep up with the more complex format
Impact on Perpetua Struggles to maintain ingest throughput at scale: Normalization defaults needed to be changed Keep JPEG-2000 as is rather than convert to TIFF and save both Drop functionality to store image derivative in the repository database Archivematica struggled to convert JPEG-2000 to low-res JPG at scale
Final Thoughts Converting to JPEG-2000 is worth the trouble! 540TB of space savings is profound, and will compound over time From the inverse perspective, adding 540TB of storage is prohibitive Technical debt will abate as we work through the backlog Presumably, handling of JPEG-2000 will improve (Famous last words?)
Thank you for your time and attention! There are too many people to properly thank within a two-minute video. Extra special acknowledgement needs to be given to: The Digital Lab Team at The Frick Art Reference Library, especially Luciano Johnson and Kylie Schmitt All the development teams whose tireless efforts make tools such as ExifTool, Jpylyzer, and ImageMagick possible The NDSA Digital Preservation 2020 program committee for making this event a reality and for giving me the opportunity to share this work Questions or comments? Please send feedback to me: smith@frick.org The slides can be viewed with less lightning at: https://tinyurl.com/SavingSpace2020 More detailed information is also available here: https://tinyurl.com/SavingSpaceGitLab
