Wednesday, June 7, 2017

Librarians and Triple-Eye-Eff to the rescue of scientific imaging

Somewhat unrecognized by us life-scientists, when it comes to delivering impressive images and associated annotations in an interoperable manner, across diverse media types and document repositories, librarians and cultural collection curators are running circles around us!

Welcome to the International Image Interoperability Framework or IIIF (Triple-Eye-Eff) for short. When I started exploring image servers for biological imaging (specifically around high content screening applications) it was not long before I started coming across references to IIIF and its associated technologies of image servers, viewers, annotation servers and compatible APIs. The technologies are open source, well-accepted, firmly established, and supported by a talented group of technical experts that call museums and libraries home to their daily work. Importantly, IIIF practitioners meet at some great places. The 2017 IIF Conference is taking place at the Vatican this week (June 5-9, 2017).

After reviewing some of the IIIF community goals, I see that the they align almost perfectly with those of biologists and life scientists working with scientific image collections.
  • A scientific study creates a collection of high resolution images similarly to a collection of images created by scanning the pages of a manuscript in a library collection.
  • Scientist require delivery of images at various resolutions (thumbnails for quick inspection and high resolution for detailed inspection), for dynamic viewing similarly to how a curator would examine the scanned pages of an ancient manuscript. The IIIF-Image API specifies the web service endpoints that return an image in response to a standard HTTP or HTTPS request.
  • The sequence and order of the images is important. A book is an ordered collection of pages, starting with a cover page and followed by an ordered  sequence of pages. Similarly, images from a HCS assay are organized as "plate collections" each with a specified order of columns and rows. The IIF Presentation API defines the structure and layout of a complex image-based object using JSON-LD (linked data) notation.
  • Scientific images are best understood in context. The ability to add annotation and metadata to any collection or individual image is critical. IIIF accomplishes this through the IIIF Annotation API which is an extension of the Open Annotation (now Web Annotation) data model. I IIF web viewer/client tools (like the Mirador viewer, developed by Stanford University) support direct user highlighting and annotation of an image area of interest. 
  • Scientist need to accurately communicate experimental results. The ability to draw attention to specific features in the image by adding graphical annotations is a necessity. Many IIIF based tools support this functionality, and allow users to add labels and markings (geometric shapes, arrows, hand drawn markings etc) as overlays to the images.
  • Scientific images collected in different repositories and by different technologies frequently need to be shared and integrated with other types of information. A central role of IIIF technologies is to provide the APIs and framework to share these image collections across multiple sites.

I should mention that IIIF has not gone completely unnoticed outside the realm of cultural collection curators. There are already a number of life-science and earth-science projects and research applications that use at least part of the IIIF technologies.

Here are a few examples:

From Biology (Histology Examples)
Annotation
From Other Sciences

Despite this almost perfect alignment of goals between IIIF and scientific imaging, there are still some important functional gaps. In my preliminary evaluation, I've noticed that although there have been a few discussions on this topic, multi-spectral images are not a focus of the current IIIF specification,  Multi-spectral imaging is a core principle in biological imaging and the need to combine multiple channel images into a composite image is ubiquitous.

There are also some areas where due to my limited experience with the IIIF presentation API I have unanswered questions. For example, how does IIIF handle Z-stacks comprising a 3D image, or video frames comprising time series (time lapse imaging)? Would the framework be able to  present these time or space-related images appropriately?

 My excitement for combining IIIF with my work on HCS imaging is such that I'm attending the 2017 Vatican meeting!! More notes after the meeting will come. 
Posted by at 1 comment:
Labels: ,
Subscribe to: Posts (Atom)