Wednesday, October 10, 2018

Jenkins Active Choices & Dynamic JavaScript Generation

Motivation

Many Jenkins job build forms using Active Choices parameters also include dynamically generated JavaScript. JavaScript is included into the build form in either of two ways:

  1. Directly from an Active Choice Reactive Reference parameter, returning an HTML <script> element or
  2. From an Active Choice Reactive Reference parameter reading a JavaScript file from the Jenkins server (either in JENKIND_HOME/userContent or JOB_NAME/BuildScipts folder of the job) and returning HTML with the file contents in the <script> element.

In addition, many job builds publish an interactive HTML build report that imports and uses JavaScipt libraries and scripts. To preserve the long term functionality for these reports (even as the custom JavaScript code may evolve) I also archive the custom JavaScript used in these forms, so each report uses a version of the script with  which it was intended to work with

As an example, I use the Openseadragon.js library (for web-viewing high resolution zoomable images) in a number of build forms. A Jenkins Active Choices Reactive Reference parameter rendered as an HTML <script> element that uses the OpenSeadragon.js library to render biological images on a Jenkins Build Form is shown below:

Initially, specific versions of this library were hard-coded into the Active Choice parameter groovy scripts and the text templates.

I revised the design strategy so that we can have a single, dynamic reference to the latest OpenSeadragon library that can then be referenced and used by Active Choice parameters, scripts and HTML templates

Overall Strategy


Implementation requirements

Global Jenkins Property

The library version is maintained as a global Jenkins parameter. We will use this as the single reference that can update all other occurrences of the library. So in the Jenkins Configuration we setup a Global Variable like:
OPENSEADRAGON_JS=openseadragon-bin-2.4.0

Active Choices Reference Parameter (on build form)

This dynamic parameter (code gist here) performs the following functions:
  • Parametrizes the JavaScript template to create the code used in the build form
  • Loads the JavaScript into the build form
  • Writes this dynamic JavaScript to a custom session specific WORKSPACE as the Report JavaScript  (if it will be included in the build generated HTML report)

Scriptlet Build Step: Generates HTML from Template

  • Parametrizes the HTML  Report template
  • Writes the generated dynamic HTML to an HTML Report File. (Note that in some cases the HTML Report loads and uses the JavaScript file generated by the Active Choices parameter above)
  • Code gist here

Scriptlet Build Step: Copy Session JavaScipt to Job Workspace 

  • Copies the session specific JavaScript file to the job workspace so that it can be archived and used in the build HTML report
  • Code gist here



Monday, September 17, 2018

RStudio Data Structure Helpers

Frequently an R function outputs a data object that has internal structure and perhaps other nested objects.The internal structure of an R object can be reviewed in more detail with the structure command

str(object)

Furthermore, RStudio provides multiple helpers for accessing nested structures in the generated objects. Let's quickly review what can be done.

Start with the Global Environment viewer

The right pane of an RStudio R session displays the session Global Environment.



The icons on the right of the list of session objects (indicated in the screeenshot by blue and yellow arows) allow you to easily review and drill down to the structure of these objects.

For example, clicking on the table-like icon opens a tab in RStudio with a tabular display of the object (usually a data frame)

Complex (non-Tabular) objects


Complex objects are identified by their type.

For example, the tsneT1 object is of type '2Dreduction', and we can reveal its complex structure by clicking on the magnifying glass icon to the right of the list. This opens a new RStudio editor tab and displays the following:

Reviewing elements of complex objects 

We can still further review the elements of the tsne complex object.
Hovering over the elements of the complex tsneT1, object reveals a 'console code' symbol to the right.
Clicking on the 'embedding' list element enters in the console the R command for exploring this element's structure.

Exporting elements of complex objects


Finally, we may need to export one of the elements of the complex object for further analysis. In my case, I was interested in exporting the t-SNE embeddings (dimenstions) so I used the following command

## Export TSNE and labels
write.csv(tsneT1[["embedding"]], file.choose(), row.names=LabelsT1)

This exported the tsneT1 embeddings to a file on the file  system

In Summary

These are some quick tips on how to use the RStudio structure viewing tools for exploring R object data structures for debugging and analysis purposes. I hope you find them useful