Getting data home

1
Getting data home
?
2
Getting data home
?
FTP
3
Getting data home
?
FTP
tapes
4
Getting data home
?
FTP not secure
tapes incompatible
5
Getting data home
?
FTP not secure
tapes incompatible
distracting
6
Getting data home
?
http//bl831.als.lbl.gov/jamesh/beamline/moving_d
ata.html
7
Getting data home
?
http//bl831.als.lbl.gov/jamesh/beamline/moving_d
ata.html http//bcsb-web.als.lbl.gov/beamline/Sec
tor8/DataBackup.htm
8
Getting data home
?
http//bl831.als.lbl.gov/jamesh/beamline/moving_d
ata.html http//bcsb-web.als.lbl.gov/beamline/Sec
tor8/DataBackup.htm http//ucxray.berkeley.edu/j
amesh/elves/manual/tricks.html
9
Can you login?

• ssh user_at_home

10
Universal directory-transfer

• tar cf - .
• ssh user_at_home
• cd /dir tar xvf -

11
Universal directory-transfer

• tar cf - .
• ssh user_at_home
• cd /dir tar xvf -

12
Universal directory-transfer

• tar cf - .
• ssh user_at_home
• cd /dir tar xvf -
• Password
• Ctrl-z
• bg

13
Making it faster

• tar cf - .
• compress
• ssh user_at_home
• cd /dir uncompress tar xvf -

14
The easy way

• http//bl831.als.lbl.gov/jamesh/elves/scripts/sen
  dhome
• sendhome . user_at_home/dir

15
Real Time

• http//bl831.als.lbl.gov/jamesh/elves/scripts/sen
  dhome
• sendhome first_001.img user_at_home/dir

16
Through a firewall

• tar cf - .
• ssh user_at_firewall
• ssh fileserver
• cd /dir tar xvf -

17
The other way

• ssh user_at_home
• cd /dir
• ssh user_at_beamline
• cd /data tar cf - .
• tar xvf

18
The other way, faster

• ssh user_at_beamline
• cd /dir tar cf - . compress
• uncompress
• tar xvf

19
The easy other way

• ssh user_at_beamline
• download yourname
• tar xvf

20
What if it stops?
21
What if it stops?

• ssh user_at_beamline
• download yourname frame_046.img
• tar xvf

22
What if it stops?

• http//bl831.als.lbl.gov/jamesh/elves/scripts/sen
  dhome
• sendhome frame_046.img user_at_home/dir

23
What if Im confused?
24
What if Im confused?
4 steps
25
1) Generate a filename list

• find . -name .img
• gt! here.log

26
2) Generate a done list

• ssh user_at_home
• cd /dir find .
• gt! there.log

27
3) Make the todo list

• diff there.log here.log
• awk /gt/print 2
• gt! 2bsent.log

28
4) Feed still left list to tar

• cat 2bsent.log
• tar cTf - -
• ssh user_at_home
• cd /dir tar xvf -

29
What if I cant login?

• ssh user_at_home
• warning Connecting to home failed Connection
  Refused

30
SneakerNet
31
SneakerNet

• Tape
• Firewire
• DVD

32
FireWire
33
Realtime Firewire

• 1)plug in

34
Realtime Firewire

• plug in
• share it

35
Realtime Firewire

• plug in
• share it
• smbclient

36
Realtime Firewire

• tar cf - .
• smbclient
• //laptop/share U userpassword" -N -Tx -

37
Realtime Firewire

• find .
• tar cTf - -
• smbclient
• //laptop/share U userpassword" -N -Tx -

38
The easy way

• http//bl831.als.lbl.gov/jamesh/beamline/backup_f
  irewire.com.txt
• backup_firewire.com .

39
DVD
40
DVD
41
HOWTO make a DVD

• mkisofs -r -J .
• gt! image.iso

42
HOWTO make a DVD

• mkisofs -r -J .
• gt! image.iso
• dvdrecord dev0,0,0
• data image.iso

43
The easy way

• http//bl831.als.lbl.gov/jamesh/beamline/backup_l
  ocal.com.txt
• backup_local.com .

44
The easy way

• http//bl831.als.lbl.gov/jamesh/beamline/backup_l
  ocal.com.txt
• backup_local.com .
• http//bl831.als.lbl.gov/jamesh/beamline/autolabe
  l.com.txt
• autolabel.com

45
(No Transcript)
46
DVD data archive
47
DVD data archive
48
DVD data archive
49
DVD data archive
50
DVD data archive
51
DVD data archive
52
DVD data archive
53
Acknowledgments
Beamline 8.3.1 staff

• George Meigs
• Jane Tanamachi