Jenkins Active Choices & Dynamic JavaScript Generation

Motivation

Many Jenkins job build forms using Active Choices parameters also include dynamically generated JavaScript. JavaScript is included into the build form in either of two ways:

1. Directly from an Active Choice Reactive Reference parameter, returning an HTML <script> element or
2. From an Active Choice Reactive Reference parameter reading a JavaScript file from the Jenkins server (either in JENKIND_HOME/userContent or JOB_NAME/BuildScipts folder of the job) and returning HTML with the file contents in the <script> element.

In addition, many job builds publish an interactive HTML build report that imports and uses JavaScipt libraries and scripts. To preserve the long term functionality for these reports (even as the custom JavaScript code may evolve) I also archive the custom JavaScript used in these forms, so each report uses a version of the script with  which it was intended to work with

As an example, I use the Openseadragon.js library (for web-viewing high resolution zoomable images) in a number of build forms. A Jenkins Active Choices Reactive Reference parameter rendered as an HTML <script> element that uses the OpenSeadragon.js library to render biological images on a Jenkins Build Form is shown below:

Initially, specific versions of this library were hard-coded into the Active Choice parameter groovy scripts and the text templates.

I revised the design strategy so that we can have a single, dynamic reference to the latest OpenSeadragon library that can then be referenced and used by Active Choice parameters, scripts and HTML templates

Overall Strategy

Implementation requirements

Global Jenkins Property

The library version is maintained as a global Jenkins parameter. We will use this as the single reference that can update all other occurrences of the library. So in the Jenkins Configuration we setup a Global Variable like:

OPENSEADRAGON_JS=openseadragon-bin-2.4.0

Active Choices Reference Parameter (on build form)

This dynamic parameter (code gist here) performs the following functions:

• Parametrizes the JavaScript template to create the code used in the build form
• Loads the JavaScript into the build form
• Writes this dynamic JavaScript to a custom session specific WORKSPACE as the Report JavaScript  (if it will be included in the build generated HTML report)

Scriptlet Build Step: Generates HTML from Template

• Parametrizes the HTML  Report template
• Writes the generated dynamic HTML to an HTML Report File. (Note that in some cases the HTML Report loads and uses the JavaScript file generated by the Active Choices parameter above)
• Code gist here

Scriptlet Build Step: Copy Session JavaScipt to Job Workspace 

• Copies the session specific JavaScript file to the job workspace so that it can be archived and used in the build HTML report
• Code gist here

Librarians and Triple-Eye-Eff to the rescue of scientific imaging

Somewhat unrecognized by us life-scientists, when it comes to delivering impressive images and associated annotations in an interoperable manner, across diverse media types and document repositories, librarians and cultural collection curators are running circles around us!

Welcome to the International Image Interoperability Framework or IIIF (Triple-Eye-Eff) for short. When I started exploring image servers for biological imaging (specifically around high content screening applications) it was not long before I started coming across references to IIIF and its associated technologies of image servers, viewers, annotation servers and compatible APIs. The technologies are open source, well-accepted, firmly established, and supported by a talented group of technical experts that call museums and libraries home to their daily work. Importantly, IIIF practitioners meet at some great places. The 2017 IIF Conference is taking place at the Vatican this week (June 5-9, 2017).

After reviewing some of the IIIF community goals, I see that the they align almost perfectly with those of biologists and life scientists working with scientific image collections.

• A scientific study creates a collection of high resolution images similarly to a collection of images created by scanning the pages of a manuscript in a library collection.
• Scientist require delivery of images at various resolutions (thumbnails for quick inspection and high resolution for detailed inspection), for dynamic viewing similarly to how a curator would examine the scanned pages of an ancient manuscript. The IIIF-Image API specifies the web service endpoints that return an image in response to a standard HTTP or HTTPS request.
• The sequence and order of the images is important. A book is an ordered collection of pages, starting with a cover page and followed by an ordered  sequence of pages. Similarly, images from a HCS assay are organized as "plate collections" each with a specified order of columns and rows. The IIF Presentation API defines the structure and layout of a complex image-based object using JSON-LD (linked data) notation.
• Scientific images are best understood in context. The ability to add annotation and metadata to any collection or individual image is critical. IIIF accomplishes this through the IIIF Annotation API which is an extension of the Open Annotation (now Web Annotation) data model. I IIF web viewer/client tools (like the Mirador viewer, developed by Stanford University) support direct user highlighting and annotation of an image area of interest. 
• Scientist need to accurately communicate experimental results. The ability to draw attention to specific features in the image by adding graphical annotations is a necessity. Many IIIF based tools support this functionality, and allow users to add labels and markings (geometric shapes, arrows, hand drawn markings etc) as overlays to the images.
• Scientific images collected in different repositories and by different technologies frequently need to be shared and integrated with other types of information. A central role of IIIF technologies is to provide the APIs and framework to share these image collections across multiple sites.

I should mention that IIIF has not gone completely unnoticed outside the realm of cultural collection curators. There are already a number of life-science and earth-science projects and research applications that use at least part of the IIIF technologies.

Here are a few examples:

From Biology (Histology Examples)

• example from kidney section (Microscopes International)
• Mouse Brain example (Protein Atlas)
• Cancer Digital Slide ArchiveThe Human Protein Atlas

e.g. Mouse brain tissue

http://bivi.co/blog/ehistology-online-annotated-histology-mouse-embryo

Annotation

• MicroDraw: Collaborative Annotation for Histology
• e.g. BigBrain human brain slice

From Other Sciences

• Feisty Photon - Astrophotography by Darryn Lavery
• e.g. A Tour Around the Carina Nebula
• Mindat: Mineralogy Database
• e.g. Gold: Au
• NIST Computational Science in Metrology
• e.g. Material Science: Moche
• ROSPHOTO Spectrozonal photography usage for attribution and analysis in archival photography
• ZooSphere

Despite this almost perfect alignment of goals between IIIF and scientific imaging, there are still some important functional gaps. In my preliminary evaluation, I've noticed that although there have been a few discussions on this topic, multi-spectral images are not a focus of the current IIIF specification,  Multi-spectral imaging is a core principle in biological imaging and the need to combine multiple channel images into a composite image is ubiquitous.

There are also some areas where due to my limited experience with the IIIF presentation API I have unanswered questions. For example, how does IIIF handle Z-stacks comprising a 3D image, or video frames comprising time series (time lapse imaging)? Would the framework be able to  present these time or space-related images appropriately?

My excitement for combining IIIF with my work on HCS imaging is such that I'm attending the 2017 Vatican meeting!! More notes after the meeting will come.