??????? ???????? ? DMS gLite (???????????? ?????)

1
??????? ???????? ? DMS gLite (???????????? ?????)

• ?????? ?.?.
• ????????????? ???????? ??????? ??????
• ?.???????

2
????????

• ???? ??????? ???????? ??????? ??????? ?????? ?
  ??????? ? gLite.
• ???? 2 ?????? ??????, ??????? ?????
  ?????????????? ? ???????? ????? ???????
• lfc-... ??????? ??? ?????? ? ????????? LFC,
  ??????? ?????????? ???????????? ????? LFN ? SURL
  (Site URL)
• lcg-... ???????, ??????? ????????? ??????????
  ????? ?/?? SE (Storage Element) ? ?????????
  ??????? ??????. ????? ????, ??????? lcg-... ?????
  ???????????? ?????????????? ????? SE ? ?????????.

3
?????????? ?????????

• ???????? ?????????? ?????????

printenv grep LCG LCG_CATALOG_TYPElfc LCG_LOC
ATION/opt/lcg LCG_TMP/tmp LCG_GFAL_INFOSYSgt5.p
npi.nw.ru2170 printenv grep
LFC LFC_HOSTcluster.pnpi.nw.ru LFC_PORT5010

• ????????? ?????????? ?????????

export LCG_GFAL_INFOSYSgt5.pnpi.nw.ru2170 export
LCG_CATALOG_TYPElfc export LFC_HOSTcluster.pnpi
.nw.ru
4
??????? ?????????? ???????

• ??? ??????? ????? ?? ????? ???????????? ????
  login ??? ???? ??????, ?? ?????? ?????
  ???????????? ???? ??????????? userid,
  ???????????? ??????? ?????????? user1, user2,...
• ????? ????????? ????????? ??????, ?????? ??????
  ??????? ?????????? ????????? MY_DIR
• export MY_DIRltuseridgt

export MY_DIRuser0 printenv grep
MY_DIR MY_DIRuser0
5
??????? lfc-...
lfc-chmod ???????? ????? ??????? ? ?????/?????????? LFC
lfc-chown ???????? ????????? ? ?????? ??? ?????/?????????? LFC
lfc-delcomment ??????? ???????????, ????????? ? ??????/???????????
lfc-getacl ???????? ACL ??? ?????/??????????
lfc-ln ??????? ????????????? ?????? ?? ????/??????????
lfc-ls ??????? ?????? ?????? ? ??????????
lfc-mkdir ??????? ??????????
lfc-rename ????????????? ????/??????????
lfc-rm ??????? ????/??????????
lfc-setacl ??????????/???????? ACL ??? ?????/??????????
lfc-setcomment ????????/???????? ???????????
6
lfc-ls

• ????? ?????? ?????????? ? ??????
• lfc-ls -cdiLlRTu --class --comment
  --deleted --display_side --ds path
• ??? path ?????????? LFN ??????????
  (????????????)
• -l - ????????? ?????
• -R - ??????????? ????? ???????????
  ?????????? ?? ????????????!!
• LFC ????? ??????????? ?????????
• /grid/ltVO_namegt/ltdir_namegt
• ??? ????? ?? ????? ????? ??????/?????? ?
  ??????????????? ??????????
• ????? ?????????? ?????????? ????????? LFC_HOME,
  ????? ?????????? ????????????? ????

???????????? ?????????????
???????????? ???? LFC
7
lfc-ls (??????)
lfc-ls -l /grid drwxrwxr-x 1 root 103
0 Oct 28 0043 alice drwxrwxr-x
10 root 102 0 Dec 15
0455 atlas drwxrwxr-x 1 root 108
0 Jun 16 2006 biomed drwxrwxr-x 0
root 104 0 Mar 22 2006
cms drwxrwxr-x 3 root 101
0 Dec 19 1433 dteam drwxrwxr-x 0 root
109 0 May 17 2006
fusion drwxrwxr-x 0 root 105
0 Mar 22 2006 lhcb drwxrwxr-x 2 root
107 0 Jan 19 1537
nw_ru drwxrwxr-x 1 root 112
0 Feb 12 1215 ops lfc-ls
/grid/nw_ru/tutorial data
8
lfc-mkdir

• ???????? ?????????? ? LFC
• lfc-mkdir -m mode -p path...
• ??? path - ?????????? LFN ??????????
  (????????????),
• -m mode ?????? ????? ??????? (??
  ????????? 777)
• -p ???? ???????, ?? ????? ??????? ???
  ?????????????? ???????????? ?????????????
• ??? ?????? ????? ? ???? (? ??????????? ?
  ????????) ??????????, ? ??????? ????????????
  ????, ??? ?????? ????????????
• ??????
• lfc-mkdir /grid/nw_ru/tutorial/data/ltuseridgt,
• ??? ?????? ltuseridgt ?????????? userlt???? ?????gt,
  ???????? user1,
• ??? ????????? ????????????? ?????????? ?????????
  MY_DIR
• lfc-mkdir /grid/nw_ru/tutorial/data/MY_DIR

lfc-mkdir /grid/nw_ru/tutorial/data/MY_DIR
lfc-ls -l /grid/nw_ru/tutorial/data drwxrwxr-x
0 133 107 0 Feb 12
1439 user0
9
?????????? ????????? LFC_HOME

• ??????? ?????????? ?????????? ????????? LFC_HOME
• export LFC_HOME/grid/nw_ru/tutorial/data/MY_DIR

export LFC_HOME/grid/nw_ru/tutorial/data/MY_DI
R lfc-ls lfc-ls ../ user0

• ? ?????????? ??????? lfc-... ????? ????????????
  ??? ??????????, ??? ????????? (???? ???? ??
  ?????????? ? ??????? /)

10
?????????? ?????? ?? UI

• ??????? ??????? ?????????? ?? UI ? ???????
  ????????? ???? ? ???

cd tutorial ls data wms cd data ls
mkdir MY_DIR ls user0 cd MY_DIR echo
"Text example file" gt text_file.txt
ls text_file.txt
11
????? SE

• ????? ???, ??? ?????????? ???? ?? SE, ?????
  ?????????, ????? SE ???????? ? ?????? ?????? ???
  ????? ??.

lcg-infosites --vo nw_ru se Avail Space(Kb)
Used Space(Kb) Type SEs ----------------------
------------------------------------ 5140000000
20000000 n.a cluster.pnpi.nw.ru 3168
70 162631 n.a
grid.scc.ioffe.ru
12
lcg-utils
Replica Management
lcg-cp ?????????? ???? ?? ???? ?? ????????? ????????? (UI)
lcg-cr ?????????? ???? ?? SE ? ???????????????? ??? ? ????????
lcg-del ??????? ???? ???? (??? ???????, ??? ??? ???????)
lcg-rep ?????????? ????? SE ? ??????????? ???????
lcg-gt ???????? TURL ??? ?????? SURL ? ????????? ????????
lcg-sd ?????????? ?????? Done ??? ??????? SURL ? SRM ???????
File Catalog Interaction
lcg-aa ???????? ??????? ? LFC ??? ??????? GUID
lcg-ra ??????? ??????? ? LFC ??? ??????? GUID
lcg-rf ???????????????? ? LFC ????, ??????????? ?? SE
lcg-uf ??????? ??????????? ? LFC ?????, ???????????? ?? SE
lcg-la ?????? ???? ????????? ??? ??????? SURL, GUID ??? LFN
lcg-lg ???????? GUID ??? ??????? LFN ??? SURL
lcg-lr ?????? ???? ?????? ??? ??????? GUID, SURL ??? LFN
13
lcg-cr

• ????????? ???? ?? SE ? ???????????????? ??? ?
  ????????
• lcg-cr -d dest_file dest_host -g guid -l
  lfn -v --verbose --vo vo_name src_file
• ???,
• dest_host ?????? ??? SE ??????????
• dest_file ?????????? SURL ????? (?????????
  ??????? sfn// ??? srm//)
• guid ???????? GUID ?????. ???? ???????????, ??
  GUID ????? ????????????
• lfn ?????????? LFN ??? ?????
• vo ??? ??????????? ???????????
• src_file ?????????? ????-???????? ????????
  ???????????? ????????? file/// ??? gsiftp///

lcg-cr --vo nw_ru file///PWD/text_file.txt
-l lfnfile1.txt -d cluster.pnpi.nw.ru guiddf836c
86-3c8a-4af6-9d6d-25828d5d535a lfc-ls
-l -rw-rw-r-- 1 133 107 18
Feb 12 1652 file1.txt
14
lcg-cr (????????? ?????)

• ????? ???? ?????? ????? v

lcg-cr -v --vo nw_ru file///PWD/text_file.txt
-l lfnfile1.txt -d cluster.pnpi.nw.ru Using
grid catalog type lfc Using grid catalog
cluster.pnpi.nw.ru Source URL file////home/elm/t
utorial/data/user0/text_file.txt File size 18 VO
name nw_ru Destination specified
cluster.pnpi.nw.ru Destination URL for copy
gsiftp//cluster.pnpi.nw.ru/cluster.pnpi.nw.ru/ms
s/2/dpm/nw_ru/2007-02-14/file6ed633c2-a432-4c6a-89
0e-1c8dce872110.44759.0 streams 1 set
timeout to 0 seconds Alias registered in Catalog
lfn/grid/nw_ru/tutorial/data/user0/file1.txt
18 bytes 0.03 KB/sec avg 0.03
KB/sec inst Transfer took 1120 ms Destination URL
registered in Catalog srm//cluster.pnpi.nw.ru/dp
m/pnpi.nw.ru/home/nw_ru/generated/2007-02-14/file6
ed633c2-a432-4c6a-890e-1c8dce872110 guide9cbc50c-
dee7-4a7b-adab-5c49235af626
15
lfc-ln

• ??????? ? ???????? ????????????? ?????? ?? ????
• ??????? lcg-ln ??????? ????????????? ?????? ??
  ???? ??? ?????????? ? ???????????? ? linkname ???
  ? ????????? directory
• lfc-ln -s target linkname
• lfc-ln -s target directory

lfc-ln -s file1.txt link1 lfc-ls
-l -rw-rw-r-- 1 133 107
18 Feb 14 1831 file1.txt lrwxrwxrwx 1 133
107 0 Feb 14 1832 link1 -gt
file1.txt
16
lcg-lr

• ???????? SURL ???? ?????? ?????
• ?????? ?????????? ????? SURL ??? ???????????
  ????? (???????). ??????? lcg-lr ??????? ??????
  SURL ??? ???? ?????? ?????????? ?????
• lcg-lr --vo vo_name file
• ???,
• vo ??? ??????????? ???????????
• file ?????????? ??? ????? ? ??????? GUID, LFN
  ??? SURL (????? SURL ????? ???? sfn ??? classic
  SE ??? srm)

lcg-lr --vo nw_ru lfnfile1.txt srm//cluster.p
npi.nw.ru/dpm/pnpi.nw.ru/home/nw_ru/generated/2007
-02-14/file6ed633c2-a432-4c6a-890e-1c8dce872110
17
lcg-lg

• ???????? GUID ?????
• ??????????? ??????? ??? ????????? GUID ???
  ??????? ?????, ??????? GUID ??? ??????? LFN ???
  SURL.
• lcg-lg --vo vo_name file
• ???,
• vo ??? ??????????? ???????????
• file ?????????? ??? ????? ? ??????? LFN ???
  SURL (????? SURL ????? ???? sfn ??? classic SE
  ??? srm)

lcg-lg --vo nw_ru lfnfile1.txt guide9cbc50c-d
ee7-4a7b-adab-5c49235af626 lcg-lg --vo nw_ru
srm//cluster.pnpi.nw.ru/dpm/pnpi.nw.ru/home/nw_ru
/generated/2007-02-14/file6ed633c2-a432-4c6a-890e-
1c8dce872110 guide9cbc50c-dee7-4a7b-adab-5c49235a
f626
18
lcg-gt

• ???????? TURL ??? ??????? (?????)
• ??????? ??? ????????? TURL ??? ??????? SURL. ???
  ????? ???????????? ??? ??????????, ??????? ?????
  ???????????? FTS ??? ???????? ??????
• lcg-gt surl protocol
• ???,
• surl SURL ?????????? ??????? ?????
• protocol ?????????? ??? ????????? ??? ???????
  ????? ??? ??????? SE (????? SURL ????? ???? sfn
  ??? classic SE ??? srm)

lcg-gt srm//cluster.pnpi.nw.ru/dpm/pnpi.nw.ru/h
ome/nw_ru/generated/2007-02-14/file6ed633c2-a432-4
c6a-890e-1c8dce872110 rfio rfio//cluster.pnpi.nw.
ru//mss/2/dpm/nw_ru/2007-02-14/file6ed633c2-a432-4
c6a-890e-1c8dce872110.44759.0 44761 0 lcg-gt
srm//cluster.pnpi.nw.ru/dpm/pnpi.nw.ru/home/nw_ru
/generated/2007-02-14/file6ed633c2-a432-4c6a-890e-
1c8dce872110 gsiftp gsiftp//cluster.pnpi.nw.ru/cl
uster.pnpi.nw.ru/mss/2/dpm/nw_ru/2007-02-14/file6
ed633c2-a432-4c6a-890e-1c8dce872110.44759.0 44762
0
19
lcg-rep

• ??????????? ???? ?? ?????? SE ? ????????????????
  ? ????????
• gLite ???????????? ?????????? ??????, ?.?. ???
  ?????? GUID ?? ?????? SE ????? ????????????
  ????????? ?????????? ????? ?????. ??? ????????
  ?????? ???????????? ??????? lcg-rep
• lcg-rep -d dest_file dest_host -v
  --verbose --vo vo_name src_file
• ???,
• dest_host ?????? ??? SE ??????????
• dest_file ?????????? SURL ????? (?????????
  ??????? sfn// ??? srm//)
• vo ??? ??????????? ???????????
• src_file ?????????? ????-???????? ????????
  ???????????? ????? LFN, GUID ??? SURL

lcg-rep -d grid.scc.ioffe.ru --vo nw_ru
srm//cluster.pnpi.nw.ru/dpm/pnpi.nw.ru/home/nw_ru
/generated/2007-02-14/file6ed633c2-a432-4c6a-890e-
1c8dce872110 lcg-lr --vo nw_ru
lfnfile1.txt sfn//grid.scc.ioffe.ru/storage/gene
rated/2007-02-14/file6c045c1f-ff4e-4542-911f-7e726
a293a10 srm//cluster.pnpi.nw.ru/dpm/pnpi.nw.ru/ho
me/nw_ru/generated/2007-02-14/file6ed633c2-a432-4c
6a-890e-1c8dce872110
20
lcg-cp

• ??????????? ???? c SE (?? ????) ?? ????????? UI
• lcg-cp -v --verbose --vo vo_name src_file
  dest_file ???,
• vo ??? ??????????? ???????????
• src_file ??? ?????-????????? (????????? ?????
  LFN, GUID,SURL ??? ????????? ????). ??? SURL -
  sfn// ??? srm//)
• dest_file ??? ?????-????? (????????? ?????????
  file/// ??? gsiftp//)

21
lcg-cp(???????????)
lcg-cp -v --vo nw_ru srm//cluster.pnpi.nw.ru/dp
m/pnpi.nw.ru/home/nw_ru/generated/2007-02-14/file6
ed633c2-a432-4c6a-890e-1c8dce872110
file///PWD/text_file_copy.txt Using grid
catalog type lfc Using grid catalog
cluster.pnpi.nw.ru Source URL srm//cluster.pnpi.
nw.ru/dpm/pnpi.nw.ru/home/nw_ru/generated/2007-02-
14/file6ed633c2-a432-4c6a-890e-1c8dce872110 File
size 18 VO name nw_ru Source URL for copy
gsiftp//cluster.pnpi.nw.ru/cluster.pnpi.nw.ru/ms
s/2/dpm/nw_ru/2007-02-14/file6ed633c2-a432-4c6a-89
0e-1c8dce872110.44759.0 Destination URL
file////home/elm/tutorial/data/user0/text_file_co
py.txt streams 1 set timeout to 0
(seconds) 0 bytes 0.00 KB/sec
avg 0.00 KB/sec inst Transfer took 3050 ms
ll total 8 -rw-r--r-- 1 elm users
18 Feb 14 1840 text_file_copy.txt -rw-r--r--
1 elm users 18 Feb 12 1541
text_file.txt
22
lcg-del

• ??????? ??????? (??? ???????)
• lcg-del -v --verbose -a -s se --vo
  vo_name file
• ???,
• a ???? ???????, ?? ????????? ??? ???????
• se ?????????? SE, ? ???????? ????????? ???????
• vo ??? ??????????? ???????????
• file ??? ????? (????????? ????? LFN, GUID,SURL
  ??? ????????? ????). ??? SURL - sfn// ??? srm//)

lcg-del -v -a --vo nw_ru lfnfile1.txt VO name
nw_ru set timeout to 0 seconds lfc-ls
-l lrwxrwxrwx 1 133 107 0 Feb
14 1832 link1 -gt file1.txt
23
lfc-rm

• ??????? ????(??????????) ?? ????????
• lfc-rm -f -i filename
• lfc-rm -f -i r dirname
• ???,
• f ?? ???????? ????????? ?? ??????, ???? ?????
  ?? ??????????
• i ???????????? ???????? ??????? ????? ???
  ??????????
• r ??????????? ????????

lfc-ls -l lrwxrwxrwx 1 133 107
0 Feb 14 1832 link1 -gt file1.txt lfc-rm
link1 lfc-ls -l