Exploring LSST JupyterHub Deployment for Astronomy Data Analysis
LSST JupyterHub, designed by Gareth Francis at EPCC, is a project aimed at prototyping a JupyterHub environment for the LSST UK Data Access Centre. This deployment involves using Kubernetes on OpenStack, with additional services like a reverse proxy, local Docker repository, and ELK for logging. By using Helm, JupyterHub is deployed to enable users to connect and spawn Jupyter servers on demand, facilitating easy data analysis. The project also plans to integrate external services like EGI and GitHub, with future phases focusing on user integration, data sharing, and security improvements.
Uploaded on | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
LSST JUPYTERHUB Gareth Francis g.francis@epcc.ed.ac.uk
Overview Jupyter notebooks are becoming popular amongst astronomers as a convenient way to explore and analyse data The Jupyter notebook is part of the LSST Science Platform Design Objective of this project is to prototype a JupyterHub environment that will ultimately be deployed as part of the LSST UK Data Access Centre
Deployment - Kubernetes Targets an OpenStack cloud currently STFC OpenStack Use Rancher to deploy a K8s cluster Have also tested Magnum (on AlaSKA) Rancher Kubernetes OpenStack
Deployment Additional services Reverse proxy Local Docker repository ELK for logging Rancher Kubernetes Proxy ELK Docker repo OpenStack
Deployment JupyterHub Use Helm to deploy JupyterHub JHub Rancher Kubernetes Proxy ELK Docker repo OpenStack
Usage Users connect to JupyterHub Hub spawns Jupyter servers on demand Jupyter Jupyter JHub Jupyter Rancher Kubernetes Proxy ELK Docker repo OpenStack
External services EGI Check-in for AAI GitHub for example notebooks etc. GitHub Check-in Jupyter Jupyter JHub Jupyter Rancher Kubernetes Proxy ELK Docker repo OpenStack
Next phase Get some users! Usage accounting Database integration Data sharing Integration with Spark/external job systems Service robustness Security and user/group isolation
Issues for IRIS Can we get Magnum as standard? How can we scale up/down to cope with demand peaks? How do we handle running on multiple sites?
Questions? GitHub: https://github.com/lsst-uk/jhub-test
