From Proof of Concept to Production Services at Scale: MMG Updates and Future Needs
Explore the journey of MMG from proof of concept to production services at scale, including updates on current progress, short-term and long-term needs, and plans for integration into cloud platforms post-Excelerate. Learn about evaluating performance, cost optimization, resource requirements, and establishing sustainable business models in the cloud environment.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
MMG: from proof-of-concept to production services at scale (part II) Lars Ailo Bongo (ELIXIR-NO, WP6) WP4 F2F, 19-20 June 2017, Prague, Czech ELIXIR-EXCELERATE is funded by the European Commission within the Research Infrastructures programme of Horizon 2020, grant agreement number 676559. www.elixir-europe.org/excelerate
Outline MMG updates Short term needs (= EOSC pilot) Long term needs Demo Discussion about business models 2
MMG on Elixir Compute Platform Done: In progress: META-pipe on AWS Open META-pipe web app for ELIXIR users EMG in CWL Documentation of best practices TODO: Test META-pipe and EMG at scale ELIXIR compute cloud business model Deploy META-pipe and EMG production service on cloud Incorporate other MMG pipelines such as BioMaS META-pipe on cPouta, CESNET and EGI Federated Clouds META-pipe integrated with ELIXIR AAI EMG on Embassy cloud and GCP Marine metagenomics portal Metagenomics workshops using cPouta 3
MMG EOSC Pilot 1. MMG, Compute Platform, EGI Elixir CC 2. Aims: 1. Evaluate the performance of META-pipe and EMG at scale using EOSC resources. 2. Cost-optimizethe analyses on EOSC. 3. Evaluate the use of elasticityin EOSC for execution of job queues. 4. Develop a full-service delivery model and potential business model between the stakeholders and entities involved. 3. Not funded 4. Next step? https://docs.google.com/document/d/124x5ygyE5xIUVHJOq94TwoqLxHgABxGhmra wEmXdN5w/edit# 4
MMG needs (next two years) Cloud Business model / resources needed to analyze end-user data Resources needed to generate reference databases Pipeline optimizations Storage and data transfer Object store Cache for pipeline reference databases and VMs Optimize META-pipe deployment (reference DBs >> user data) Optimize EMG data transfer to/from cloud 5
MMG needs (next two years) AAI Integrated with object store Service registry/ help desk Help end-users to get resources 6
MMG needs (after Excelerate) Cloud Sustainable business model Maintain cloud setup tools Easy deployment of new pipelines, databases, and services Storage and data transfer Access data in ENA on behalf of users (as in EBI) AAI Established service Service-registry / help-desk ? 7
Business model alternatives 1. Users gets resources 1. Project or national funding/ resource 2. Pan-European resources (EOSC?) 3. Direct payment (industry) 4. Elixir pool of funding 2. Service provider gets resources 1. Elixir node(s) 2. Elixir funded 3. Pan-European resources 3. Resource provider gets funding 1. Elixir nodes 2. Elixir funded 8
Business model alternatives 1. User gets resources 1. Many small allocations 2. Compute platform problem 2. Service provider gets resources 1. Few big allocations 2. MMG problem 3. Resource provider gets funding 1. Few big grants 2. Resource provider problem 9
Summary How to get resources at scale? How to establish compute platform business model? What happens to MMG use case and compute platform after Excelerate? How can others take advantage of MMG experiences? 10
Demo Alternative 1 11
