David Yuan, EMBL-EBI

We have developed the EMBL-EBI Cloud Portal (ECP) to provide a common set of user interfaces and programming interfaces to support major cloud providers in a consistent, agnostic manner. ECP reduces the learning curve and skill requirements to port and deploy pipelines in the clouds.

We lifted and shifted a legacy pipeline for RNA families analysis (Rfam) from a bare-metal LSF-based stack to OpenStack via ECP. With minimal changes to the pipeline itself, but by using cloud features intelligently we have made major improvements changing the Rfam pipeline from a single-user local application to a shared multi-user application accessible over the Internet.

While lift-and-shift is a good starting point to help port legacy pipelines from on-premises to a cloud platform, many cloud features can be used to realise significant benefits with little extra effort. However, pipelines need to evolve to make the best use of cloud-native technologies.

This talk presents the methods by which we ported and enhanced the Rfam pipeline, and the lessons learned during the exercise.