Overview of Distributed Storage System with dCache in High-Performance Computing Environment

distributed storage system with dcache n.w
1 / 23
Embed
Share

Explore a distributed storage system utilizing dCache middleware in the high-performance computing environment of UB RAS. Learn about the project participants, current implementation, access protocols, middleware selection, and future plans for this cutting-edge technology.

  • Storage system
  • dCache
  • Distributed Computing
  • Middleware
  • High-Performance Computing
rayaanacharya
auer914 Follow

Uploaded on | 0 Views

Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.

Download Presentation

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.

E N D

Presentation Transcript

  1. Distributed storage system with dCache E.Y.Kuklin, A.V.Sozykin, A.Y.Bersenev Institute of Mathematics and Mechanics UB RAS, Yekaterinburg G.F.Masich Institute of ontinuous edia echanics UB RAS, Perm Distributed Computing and Grid-technologies in Science and Education, Dubna, 2014

  2. Introduction High-performance distributed computing environment of UB RAS: 40 Research Institutes of UB RAS in 7 regions (Yekaterinburg, Arkhangelsk, Syktyvkar, Orenburg, Perm, Izhevsk, Chelyabinsk). Universities. Industrial enterprises. Distributed data storage system. 2

  3. Introduction Project participants: Institute of Mathematics and Mechanics UB RAS (IMM UB RAS) computational recourses and storage system. Institute of ontinuous edia echanics UB RAS (ICMM UB RAS) backbone networks. Supported by grant of UB RAS 12-P-1-2012 and by the RFBR grant 14-07-96001r_ural_a. 3

  4. Overview About dCache. Middleware selection. Current implementation. Some problems. Future plans. 4

  5. About dCache Distributed storage system. Experimental data. Commodity hardware. Facilities in hundreds of terabytes. Single file system tree. Simple storage extension. 5

  6. About dCache Access protocols: FTP WebDAV NFS 4.1 SRM gridFTP dCap 6

  7. Middleware selection StoRM dCache DPM European Middleware Initiative (EMI) 7

  8. Middleware selection Support for both types of protocols. High quality documentation. Wide community. Easy to install and administer. 8

  9. Middleware selection JINR NDGF 9

  10. Current implementation 10

  11. Current implementation Plan Triton cluster URAN supercomputer NFS v4.1 NFS v4.1 DSS UB RAS dCache broker host, pool1a, pool1b dCache pool4 dCache pool2a, pool2b dCache pool3a, pool3b Experimental setups IMM UB RAS, Yekaterinburg ICMM UB RAS, Perm 456 km 10 GE

  12. Current implementation Hardware Supermicro servers: 4 storage nodes. Scientific Linux 6.5. Intel Xeon CPU E5607. 24 HDD Seagate SATA3 3Tb. 210 Tb usable capacity. 12

  13. Current implementation File system ? XFS EXT4 Choice between XFS and EXT4. We need good I/O performance. The file system should be natively supported by Linux kernel. 13

  14. Current implementation File system XFS EXT4 14

  15. Current implementation Backbone network 456 km distance. DWDM optical fiber. Operated by ECI-Telecom. 10GE encapsulated in 2x -channels. NFS v4.1. 15 Laboratory of Telecommunication and Information Systems ICMM

  16. Current implementation Problems Long distance. Network stack configuration. Intel network cards drawbacks. Custom-built kernel. 16

  17. Current implementation Computational resources IMM UB RAS: the URAN supercomputer. 7th position in Top50 (CIS). ICMM UB RAS: the Triton cluster. 17 URAN

  18. Current implementation Network benchmark 387 32

  19. Current implementation Network benchmark

  20. Current implementation Possible replication IMM UB RAS [DWDM channel] ICMM UB RAS Data1 Data2 Data2 Data1 Replication NFS 4.1 NFS 4.1 Experimental setups URAN 20

  21. Future plans Near future: Performance and fault tolerance benchmarks. More concern on security and monitoring. Long-term plans: Increasing of the storage system volume. Integration with experimental setups in ICMM UB RAS and UrFU. 21

  22. Future plans Joining GRID projects: National Nanotechnology Network (GridNNN). Worldwide Large Hadron Collider Computing Grid (WLCG). 22

  23. Thank you for your attention. Evgheniy Kuklin IMM UB RAS key@imm.uran.ru

Related


No related presentations.

More Related Content