Storage Resource Broker

1
Storage Resource Broker

• USER INTERFACE
• George Kremenek
• kremenek_at_sdsc.edu

2
Types of Interface

• C language API
• S-commands Unix, Win NT, W2000
• Java GUI
• Win32 GUI
• WEB

3
SRB API

• Programmatic API
• High-level API
• Low-level API
• SRB Manager API
• Command Level Interface - Scommands
• Graphical User Interface - srbBrowser
• Web Utilities

4
SRB API Interface
Application
MCAT
SRB Master
5
High Low-level API

• Low-level API
• talks to resource drivers
• no registration of data sets in MCAT
• no authentication through MCAT
• User provides all information
• High-level API
• Uses low-level API to access resources
• Registers data management information in MCAT
• Uses MCAT for authentication and meta information
• Uses MCAT for resource and data discovery
• Access/store data in remote SRB

6
Low-level API

• srbFileOpen(conn, storType, host, fileName, mode)
• srbFileCreate(conn, storType, host, fileName,
  mode)
• srbFileClose(conn, fd)
• srbFileUnlink(conn, storType, host, fileName)
• srbFileRead(conn, fd, buffer, length)
• srbFileWrite(conn, fd, buffer, length)
• srbFileSeek(conn, fd, offset, whence)
• srbFileSync(conn, fd)
• srbFileStat(conn, storType, host, fileName,
  statBuf)
• srbFileMkdir(conn, storType, host, dirName, mode)
• srbFileRmdir(conn, storType, host, dirName, mode)
• srbFileChmod(conn, storType, host, fileName, mode)

7
Low-Level API (contd )

• srbDbLobjOpen(conn, storType, resourceLoc,
  positionName, mode)
• srbDbLobjCreate(conn, storType, resourceLoc,
  positionName, mode)
• srbDbLobjClose(conn, dd)
• srbDbLobjUnlink(conn, storType, host, fileName)
• srbDbLobjRead(conn, dd, buffer, length)
• srbDbLobjWrite(conn, dd, buffer, length)
• srbDbLobjSeek(conn, dd, offset, whence)

8
srbBrowser - A SRB Graphical Interface

• A java GUI
• Interface with SRB servers using the client
  API library.
• Performs most SRB operations - cp, replicate,
  import, export, metadata query, etc.

USER
Java GUI
Obtain users metadata information via SRB.
Invoke SRB operations
SRB Agent
MCAT
Proxy operation
9
SRB Command Line Interface
Environment File
USER
SRB shell commands Sls, Scp, Scat, Sput, Sget,
...
MCAT
SRB Agent
Proxy operation
10
High-level API

• srbObjOpen(conn, objChar, mode, collectionName)
• srbObjCreate(conn, objName, objType,
  resourceName, collectionName,
  pathName, size)
• srbObjClose(conn, od)
• srbObjUnlink(conn, objChar, collectionName)
• srbObjRead(conn, od, buffer, length)
• srbObjWrite(conn, od, buffer, length)
• srbObjSeek(conn, od, offset, whence)
• srbObjMove(conn, objChar, collectionName,
  newResourceName, newPathName)
• srbObjReplicate(conn, objChar, collectionName,
  newResourceName, newPathName)
• srbObjProxyOpr(conn, Operation, sourceDesc,
  targetDesc)

11
High-Level API (contd )

• srbGetDatasetInfo(conn, objChar, collectionName,
  resultStruct, requiredNumber)
• srbGetMoreInfo(resDesc, resultStruct,
  requiredNumber)
• srbGetDataDirInfo(conn, conditionList,
  selectList, resultStruct)
• srbModifyDataset(conn, objId, collectionName,
  newValue1, newValue2, modifyType, resourceName,
  pathName)
• srbCreateCollect(conn, parentCollectionName,
  childCollectionName)
• srbListCollect(conn, CollectionName, flag,
  resultStruct)
• srbModifyCollect(conn, CollectionName, newValue1,
  newValue2, newValue2, modifyType)
• srbModifyUser(conn, newValue1, newValue2,
  modifyType)
• srbSetAuditTrail(conn, setValue)

12
System Manager API

• srbChkMdasAuth(conn, userName, userAuth, domain)
• srbChkMdasSysAuth(conn, userName, userAuth,
  domain)
• srbRegisterUser(conn, userName, domain, password,
  userType, userAddress, userPhone, userEmail)
• srbRegisterUserGrp(conn, userGrpName,
  userGrpPassword, userGrpType,
  userGrpAddress, userGrpPhone, userGrpEmail)

13
Sinit -v, Sexit

• Sinit initializes client environment
• Sexit ends a SRB session started with Sinit
• A client may communication with servers with
  different "release version", but they must have
  the same "API version
• See /.srb/.MdasEnv and environment variables

14
/.srb/.MdasEnv file

• mdasCollectionHome '/home/kremenek.sdsc'
• mdasDomainHome 'sdsc'
• srbUser 'kremenek'
• srbHost srb.sdsc.edu'

15
.MdasEnv, .MdasAuth

• setenv srbPort '5566'
• valid schemes are 'PASSWD_AUTH', 'SEA_AUTH',
  'SEA_ENCRYPT',
• 'GSI_AUTH', 'GSI_SECURE_COMM'
• AUTH_SCHEME 'ENCRYPT1'
• defaultResource 'dl1-unix-sdsc

16
Scd collection

• changes the working SRB collection
• without a collection the mdasCollectionHome value
  in the
• /.srb/.MdasEnv file will become the new working
  collection.

17
Spwd

• displays current working SRB collection

18
Sannotate switches

• -w position annotation dataName
• -u timestamp newAnnotation dataName
• -R -t timestamp -p position -U
  userName_at_domainName -Y n -L n -T dataType
  dataNamecollectionName
• facility for annotations on data objects

19
Sappend switches

• localFileName srbTarget
• -i srbTarget
• -s srbObj srbTarget
• appends a local file, stdin, SRB object
• to an existing SRB object

20
Sattrs

• lists the queriable MCAT attributes used in
  conditions for choosing SRB objects.

21
Scat switches list

• -C n -T ticketFile -t ticket -A
  condition srbObj
• reads each srbObj from SRB to stdout
• -A option, only srbObj which conform to the
  condition are chosen
• If using a ticket, one need not give a srbObj name

22
Schmod switcheslist

• -c -a -p -r -dc warn newUserName
  domainName collection srbObj
• grants/changes access permits for the operand
  collection or srbObj ... for newUserName in
  domainName
• granted new permission for all replica

23
Scp switches list

• -n n -fpra -c container -S
  newResourceName -P newPathName
• srcObj destObj
• srcObj ... target collection
• -r source collection... target collection
• -p prompts, -f force, -a force all replica,
  -r copy recursively, -n replica number

24
Serror number

• describes SRB errors
• takes an error number generated by SRB/MCAT
  system and displays a text human readable message
  relating to the error

25
Sget switcheslist

• -n n -pfr -A condition srbObjCollection
  ... localFilelocalDirectory
• exports one or more objects from SRB space into
  local file system
• -A option, only srbObj which conform to the
  condition are chosen.

26
SgetColl switcheslist

• -ahc -I -W -U userName -Y number -L
  number -A condition collName
• display information about SRB data objects
• -a display permissions, -W all users,
  -c container, -U for usr/group,-I in all
  collections, -Y output format, -A condition
  option "ltAttrgt ltCompOpgt ltValuegt"

27
SgetD switcheslist

• -phPrReasdDc -I -W -U userName -Y
  number -L number -P dataType -A condition
  dataName
• display information about SRB data objects
• -p basic parameters, -r storage information, -a
  permissions, -d audit info., -c collection info.,
  -W for all users, -Y number format

28
SgetR switcheslist

• -lhdDp -L number -Y number -T resourcTy
  pe resourceName
• display information about SRB resource(s)
• -l display comprehensive list, -d list objects,
  -D with details, -p for physical resources
  only,-T resource type list for the given type, -Y
  number controls display format

29
SgetT switcheslist

• -h -u -v -L n -Y n -F fileName -T
  ticket -D dataName -C collection -U -c
  userName domainName
• display information about SRB tickets for a given
  ticket, dataName or collection
• -u ticket-users perspective,-F file for the
  ticket, -T ticket, -D dataName, -C collection, -U
  userName

30
SgetU switcheslist

• -pPhdatg -L number -Y number -T
  userType userName_at_domainName
• displays information about a group or user
  userName_at_domainName
• -p user/group name, -a access permissions, -d
  domain(s), -t audit info. -g group(s), -c
  collection access, -T info. for user type

31
Sgetappend sl

• -C n -p -A condition srbObj ...
  localFile
• exports object(s) into local file system
• -p prompts before operation, -C replica number,
  -A condition list ( separated) the form
  "ltAttrgt ltCompOpgt ltValuegt

32
Sls sl

• -hl -L number -r-f -A condition
  collectionsrbObj ...
• display and properties of objects
• -r recursively for sub-collections, -f force
  argument to be interpreted as a collection.-l
  long format, giving owner, replica , physical
  resource, siz, time of creation

33
Slscont sl

• -a -l or container
• display metadata of SRB containers
• Slscont displays all containers
• Slscont XYZ all inContainer objects will be
  listed
• -l metadata in long format,-a accessible by the
  user rather than owned by the user

34
Smeta sl

• modifies metadata information about SRB data
  objects
• -i -I metaAttrNameEqValue -I
  metaAttrNameEqValue ... dataName
• -u metadataIndex metaAttrNameEqValue dataName
• -d metadataIndex dataName
• -c -i -I metaAttrNameEqValue -I
  metaAttrNameEqValue ... collectionName
• -c -u metadataIndex metaAttrNameEqValue
  collectionName

35
Smeta cont.

• -c -d metadataIndex collectionName
• -R -I metaAttrNameOrCondition -I
  metaAttrNameOrCondition ... -Y n -L n -T
  dataType dataNamecollectionName
• -c -R -I metaAttrNameOrCondition -I
  metaAttrNameOrCondition ... -Y n -L n -T
  dataType collectionName

36
Smeta cont.

• Smeta provides facility for inserting,deleting,
  updating and accessing meta-data on data object
  dataName or collection collectionName
• Currently, we support 10 string attributes and
  two integer attributes
• 'all permission for modify, 'read for view

37
Smkcont sl

• -S resourceName -D dataType -s
  containerSize container
• creates a new SRB container
• container" may be an absolute path or a relative
  path (will be created in the user's container
  collection path - /container/userName.domainName)

38
Smkdir sl

• -N collection
• creates a new SRB collection is a format
  ltpath_namegt/ltnew_collection_namegt.
• -N option overrides the default association with
  container of its parent collection

39
SmodColl sl

• -dh -c value collName
• modifies information about collections in
  collName
• -h help, -d delete, -c container_name is updated

40
SmodD sl

• -s-t-c newValue dataName
• modifies metadata information about SRB data
  objects
• -s change size
• -t change type
• -c insert comments

41
Smv sl

• srbObj targetObj
• collection newcollection
• srbObj ... Collection

42
Spasswd

• changes password of current user
• changes the current user's password both in the
  Meta Catalog as well as in the Client
  Authorization Environment file
• password change persists across sessions with SRB

43
Spcommand sl

• -h -H hostAddr command
• proxy command operation. Request a remote SRB
  server to execute arbitrary commands on behalf of
  client on the hostAddr (or srbHost in the
  .MdasEnv)
• the proxy commands should be installed in the
  /usr/local/srb/bin/commands directory

44
Sphymove sl

• -C n -p -P newpathName srbObj ...
  newresourceName
• moves one or more SRB objects to the
  newresourceName at new path newpathName (if given
  ). The old copy is deleted and the MCAT catalog
  is also updated

45
Sput sl

• -fpra -c container -D dataType -S
  resourceName -P pathName localFileNamelocalDir
  ectory ... TargetName
• imports one or more local files and/or
  directories
• -p prompts, -f force, -a force all replica,-r
  recursively, -c container

46
Sregister sl

• -p -D dataType -S size -R resourceName
  RegisteringObjectPath ... TargetName
• -c -p -D dataType -S size -R
  resourceName RegisteringObjectPath
  srbObjectName
• registers one or more files into SRB space

47
Sreplcont sl

• -S resource container
• replicate a container copy to a specific resource
• For containers that have multiple "permanent" and
  "cache" copies, this is a way to put a copy of
  the container on a specific resource

48
Sreplicate sl

• -n replicaNum -pr -S resourceName -P
  pathName srbObjcollection
• makes one more copy of srbObj or collection
• -p prompts, -r recursively, -n replicaNum, -P
  full or relative newpathName to move the object,
  -S new resourcename

49
Srm sl

• -p -A condition srbObj
• -p -A condition -r srbObjcollection
• remove files from SRB space
• -p prompts, -r recursively (the collection will
  be emptied of datasets and removed), -A condition
  options("ltAttrgtltCompOpgt ltValuegt)

50
Srmcont sl

• -f container
• remove an empty existing SRB container
• -f Force the removal of all inContainer objects
  stored in this containe before removing the
  container

51
Srmdir collection

• deletes an existing SRB collection

52
Srmticket sl

• -F fileName ticket
• removes a previously issued ticket. One has to
  own the ticket to remove it
• -F fileName specifies the file in which the
  ticket is stored

53
Ssyncont sl

• -d -p container
• synchronize the "permanent" copies of the
  container with the "cache" copy.
• when an inContainer object is created or opened
  for I/O, all I/O are done only to the "cache"
  copy
• -d delete cache copy, -p to primary only

54
Stcat sl

• -T ticketFile -t ticket -A condition
  hostName srbObj
• display files read from SRB space for a
  ticketuser
• -T option to give a filename containing a ticket,
  -t option for giving a ticket directly, -A
  condition "ltAttrgt ltCompOpgt ltValuegt"

55
Sticket sl

• -F fileName -B beginTime -E endTime -N
  AccessCount -D dataName -C collName -R
  collName user_at_domain
• issue tickets for SRB objects and collections
• -D option for asingle data object, -C option for
  SRB collection, -R option recursively

56
Stls sl

• -v -L number -Y number -F fileName
  ticket -A condition hostName
• display objects and sub-collections in SRB
  collection for a given ticket
• -v verbose, -F fileName specifies the file in
  which the ticket is stored

57
Stoken sl

• -L number -Y number typeName
• isplays information about metadata type typeName
• typeName can be one of ResourceType, DataType
  (default), UserType, Domain, Action,
  AccessConstraint

58
SRBIO
59
SRBIO

• Open
• creat
• read
• write
• close
• lseek
• fopen
• fread
• fwrite
• fclose
• fseek

• fflush
• fgetc
• fgets
• fputc
• fputs
• getc
• putc
• ungetc
• vfprintf
• fprintf
• fscanf

60
Web Utilities

• Sgetw - copies a SRBobject into server site
• Sputw - copies local file in SRBspace
• Scatw - displays SRBobject on browser
  (handles types)
• Slsw - displays information of SRBobjects