This session provided a wide range of talks but nevertheless there were some common themes.   Adam’s talk provided some provoking thoughts about the use of containers immortalised in the use of a poo emoji!!  The well made point that containers don’t provide a silver bullet: you can wrap an application in a container but it won’t fix any shortcomings in the quality of the application itself.  The need for good engineering remains.   The benefit comes in the ability to orchestrate containers together.  In this there are encouraging signs with  technologies like Shifter and Singularity which enables the use of containers on traditional batch compute environments.

Jade presented by Jacob Tomlinson from the the MetOffice gives another good example of how the use of orchestration and use of off-the-shelf applications can be used as prefabricated units to build overall system in a short space of time.  This has been underpinned by the use of cloud, removing the inertia around set up associated in more traditional IT environments.   Jade makes use of JupyterHub and Dask to build an interactive SaaS development environment for researchers to analyse datasets.  Jupyter Notebooks provide the user interface: web-based access to a Python programming environment together with the ability to embed inline documentation and save code and documentation together in notebook which can be re-used and shared.  Dask provides a means of task parallelism largely abstracting this behind the Python developer so that they do not need to be concerned with how to re-engineer their code to work in this way.

Other researchers are also using Jupyter including Joni Pelham from Cranfield who like the MetOffice has been experimenting with hosting on public cloud providers.  Azure and other public cloud providers have SaaS ready-made Jupyter environments available.  Lee Stott from Microsoft presented on how these can be used alongside tools for machine learning available through Azure.

Finally, the OpenStack foundation gave a short presentation highlighting a different kind of orchestration, that between cloud providers.   OpenStack seems to have reached its “young adulthood” with it being adopted for public clouds as well as private clouds and is being used in many sectors (not just scientific research). As more are choosing OpenStack it is provide the potential for federation between them by virtue of a common API.