Self-serve web hosting for research applications

(a.k.a. CloudLabs)

Raquel Alegre, Jonathan Cooper, Anastasis Georgoulas

Research Software Development Group

University College London

Outline

• ​Project motivation
• Objectives
• Progress so far
  • ... and a demo!
• Plans for the future

Project Motivation

• Researchers often invest their team's budget on buying powerful machines only used for a short period of time.

• ​​Researchers currently moving to the cloud, find it very hard to choose provider and resources and often choose a solution that is not suitable.

• Research applications like data processing or analysis tend to have high specs, but don't need high uptime.

Project Objectives

• Ease provisioning of infrastructure
  • ​Abstract cloud provider requirements and jargon
• Streamline the publishing of research applications and data​
• Integrate with UCL authentication, networking, and domain registration capabilities
  • UCL SSO​
  • DNS mapping from provider's URL to ucl.ac.uk domain

Project Objectives

​To provide a platform that enables rapid deployment of web-based applications and services for research outputs, algorithms and data.

• Users can automatically deploy web applications directly from source control in minutes
  • Integrated with GitHub and Docker​
• Cater for non-technical users​
  • Documentation and training
  • Different options to deploy depending on experience
    • Option for guided step-by-step provisioning
• Lower costs compared to non-cloud alternatives
  • ​Free-tier for UCL researchers

Progress so far

• ​RSDG@UCL are working on a web portal to be tested by a small number of researchers by end of October 2017.
  • Feedback and usage data will inform the next months of work until summer 2018.

• Tech stack:
  • Web development: Flask (Python) 
  • Database: PostgreSQL
  • Automatic deployment: Terraform
  • Cloud provision: Microsoft Azure

Progress so far

• The web portal currently allows for creation, destruction and provision of VMs on the cloud, including:
  • access to code in GitHub repositories and ssh configuration
  • custom launch of a lightweight webapp including provision of its dependencies
  • admin account setup to access VM after provision
  • display status, log and information about own hosts

• Staging server available at staging.cloudlabs.rc.ucl.ac.uk.

Demo!

Future plans

• Web portal beta release with a controlled number of users to inform next steps.​
• Explore integration with MyFinance.
• Explore support for data storage and data processing.
• Implemente automatic DNS server mapping from UCL to Azure URLs.
• Cater for more options when creating a new host: operating system, type of VM, static/dynamic IP, etc.