April 28th 2016

From West-Life
Jump to: navigation, search

Present: Francois Matthieu Tomas Chris

Current status, Luna. A basic level of connection between the portal and data services, including B2DROP (through WebDAV) and DropBox (through their API).

Current status, STFC: a VM with a dropbox folder mounted as a folder. Also provides web server with UI to manage files - the user enters credentials here. (In the proof of concept, the CCP4 suite is installed.) The virtual folder should be standalone component. The VRE portal will access it in the same was as it accesses the e-Science portal. So STFC will translate e.g. iCAT access to an interface like B2DROP.

Luna will build VRE on top of STFC outputs (not STFC working on top of Luna products). STFC output should expose an RESTful API for use by the VRE. Is WebDAV sufficient? Luna has already defined a RESTful API. Current VRE version uses custom API layer to connect to Dropbox, Google Drive, and Amazon. This code will be transfered to STFC so that they can include it in their virtual folder implementation. Then Luna will update the VRE so that it uses virtual folder appliance as storage backend. This avoids duplication of work.

Followup work will require domain knowledge, and access to scientific repositories with special interfaces. STFC will do this, e.g. translating iCAT to a B2DROP like WebDAV API. Luna will continue to provide a web UI to the virtual folder; STFC will cease to do this.

The VRE cannot run large jobs locally, it must submit to EGI. So there is a base VM, and task VMs are built on it. Note: job submit will be decided later. This meeting focused on implementing the data part of WestLife.

Should there be a central file server? WebDAV permits 302 Redirect responses, so it need not be a bottleneck. User can select file from virtual folder, then the work portal needs the file. Does the portal fetch the file, and POSTs the file to the work portal. Where possible, do something more efficient: work portal may sometime be able to access the file directly, e.g. data collected at Diamond is available to the CCP4 portal locally. NOTE: the privileged option on the short term will be that e-science portals embed, like the VRE, the WP6 virtual folder appliance, so that users instead of uploading files can select files from their virtual folder. Then files fetched by the virtual folder appliance are treated like files uploaded via the e-science portal web interfaces. This allows portal operators to minimize effort to achieve data portability.

On a longer term it could be interesting to have VMs running on EGI have a local virtual folder appliance so that they can fetch directly files from the exterior. ACTION: Tomas to discover whether EGI VMs can connect to files in B2DROP.

ACTION: a global West-Life github portal so we can share the code. Will also use it for portal content. Francois will do.

VRE: can it be deployable with the appliance? Each person has own VRE? We also want a central version, i.e. a public portal. This would have to handle credentials for multiple users.

Useful if small analytics can be run on server, without overhead of EGI submission.

Ales Krenek will report soon on AAI and job submission.

Hackathon in July (Francois is available first 3 weeks), using CMS to add content to portal. Requires web development skills to implement, and also scientific knowledge to write content. ACTION: Chris to do Doodle poll, then liaise with Francois. Note: for CMS, Luna will deploy Github CMS Jekyll. Content will be stored in WestLife github organization repositories, and people will just need a github account to edit content for their own portal.

Useful to have early tests with weNMR and Scipion web tools to embed virtual folder in their web interface. May mean adding an extra form for EUDAT credentials. Maybe a second day for the Hackathon to take initial practical steps on this? The advantage of doing so is that to achieve data portability between portals and storage services, each e-science portal web server would use WP6 virtual folder appliance as a storage backend, just like the VRE. End users would be able, instead of uploading files, to select them from their virtual folder. Those files are then treated by e-science portals just like uploaded files.

We discussed changes to the report for D5.3.

ACTION Logins to portal, Chris to send list of people to Francois.