Dcache Lustre Configuration and File Management
Explore the setup and management of Dcache and Lustre systems including backend mounts, file transfers, storage pool configurations, and file deletion procedures. Learn about Lustre pools, HSM settings, and pool manager configurations for efficient file handling on Lustre file system.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Dcache lustre Lustre backend mount /beslustre Dcache-lustre : srm://bes-srm.ihep.ac.cn/dcache/bes/lustre dcap://bes-srm.ihep.ac.cn/dcache/bes/lustre gsiftp://bes-srm.ihep.ac.cn/dcache/bes/lustre /beslustre/dcache/bes/lustre
Srmcp -2 file:////tmp/testfile01 srm://bes-srm.ihep.ac.cn/dcache/bes/lustre/ file:////tmp/testfile01 testfile disk pool (/storage/bes-d01/pool/data/<pnfsid>) Lustre (/beslustre/dcache/bes/lustre/testfile01) delete from pool srmcp srm://bes-srm.ihep.ac.cn/dcache/bes/lustre/testfile01 file:////tmp/testfile01 Testfile01 (/beslustre/dcache/bes/lustre/testfile01) disk pool (/storage/bes- d01/pool/data/<pnfsid>) local disk lustre /beslustre/dcache/bes/lustre/testfile01 srmrm srm://bes-srm.ihep.ac.cn/dcache/bes/lustre/testfile01 Pool Lustre
Dcache Main configuration Dcache root DefaultRetentionPolicy=REPLICA # DefaultAccessLatency=ONLINE # disk DefaultRetentionPolicy=CUSTODIAL # DefaultAccessLatency=NEARLINE # HSM HSM pool
PoolManager pools pool group lustre dcache pool group pool group dcache
Pool configuration lustre dcache dcache lustre hsm set osm -hsmBase=/beslustre hsm set osm -pnfs=/dcache hsm set osm -command=/root/hsm.sh
hsm.sh put put transfer files from pool to lustre get get transfer files from lustre to pool R Remove emove delete files on lustre and pool
admin > rep ls 00002324D90A776C4F38998D8B3E2BFEC9C0 <C-------X--L(0)[0]> 798334 si={jinrwan:STATIC} 0000912829CBA3034A0C91B01CA2444CD4C0 <C----------L(0)[0]> 798334 si={jinrwan:STATIC} 00002324D90A776C4F38998D8B3E2BFEC9C0 <-P---------L(0)[0]> 798334 si={jinrwan:STATIC} 0000B173405A3DCC4A7B897613CC4982C62A <C----------L(0)[0]> 798334 si={jinrwan:STATIC} CPCScsRDXEL ||||||||||| ||||||||||+-- (L) File is locked (currently in use) |||||||||+--- (E) File is in error state ||||||||+---- (X) File is pinned (aka "sticky") |||||||+----- (D) File is in process of being destroyed ||||||+------ (R) File is in process of being removed |||||+------- (s) File sends data to back end store ||||+-------- (c) File sends data to client (dCap,FTP...) |||+--------- (S) File receives data from back end store ||+---------- (C) File receives data from client (dCap,FTP) |+----------- (P) File is precious, i.e., it is only on disk +------------ (C) File is on tape and only cached on disk.
