Using Expect to automate repetitive tasks in the Innopac

1
Using Expect to automate repetitive tasks in the
Innopac

• Mark Dahl
• Lewis Clark CollegeWatzek Library

2
The problem repetitive tasks

• Updating featured lists
• Running lists forstatistics, collection
  development reports, etc.
• Initializing and verifying backups
• Moving order information to item records
• Maintaining external databases (journal title
  database, acquisitions database, etc.)

3
The solution Expect (or)

• Expect an extension to the TCL programming
  language for controlling interactive programs
• OCLC Macro Language (an extension of Visual
  Basic)
• Macro Express (a powerful Windows-based
  automation tool)
• Other???

4
What is Expect?

• A little robot that can go into III and do
  whatever you want it to whenever you want it to.

5
Features of Expect

• Powerful programming language features if/then
  logic, functions, regular expressions, variables,
  etc.
• Runs nicely in a Linux/Unix environment
• Expect scripts can be set to run at a certain
  time using cron
• Already many scripts available

6
What an Expect Script Looks Like
get books from gen send -h "b" send -h
"" expect "P BARCODE " send -h "\n" send -h
"O" expect -- "OR" send -h "u" expect --
"ID" send -h "" expect "ID " send "\r" send
-h "s" name the review file expect -- "What
name would you like to give this file of
records?" send -- "Patbcs\r" may need to add
some sleep/timeout security here if search is
lengthy! expect " to continue" send -- "
" list completed, now print out
7
A simple use of Expect and III update a list
8
expect "Choose one (S,D,C,O,M,A,X)" send -- "m"
9
expect "Choose one (I,A,G,P,R,F,V,C,L,S,Q) send
-- "l"
10
send -- initials
send -- password
11
Expect BOOLEAN Send 03
12
Expect LOCATION Send Expect LOCATION
Send vid\r Expect Enter action Send S
13
A Patron File

• A comma delimited file of current library
  patrons, their id numbers, and addresses is
  needed for various applications mounted on a
  Linux server ereserve authentication, proxy
  server authentication, interlibrary loan
  requesting, etc.

14
Patron File Script

• Logs into III
• Enters create lists, finds a free list
• Creates list of patrons based on given criteria
• Creates output file in comma-delimited format
• Puts file on server using FTS
• Runs 7X a day via cron

15
OPAC Usage Statistics

• OPAC search stats on III are only available for
  the last 10 days
• This script captures these statistics once a
  month and emails them so that you have a record
  of search statistics

16
OPAC Usage Statistics Script

• Logs in
• Goes to the Public Catalog Search Statistics
  pages in III
• Captures values
• Emails search values

17
OPAC statistics results

• To mdahl_at_cocc.edu
• Subject Catalog Usage
• Fri Mar 16 142724 2001
• Number of USER keyed searches......... 3345
• Number of SYSTEM suggested searches... 444
• Number of RECORDS retrieved........... 205,438
• Number of searches LIMITED............ 0
• Number of EXPORTED records............ 0
• Number of DISPLAYS invoked............ 7807

18
Tech Services Statistics

• There was a need to monitor acquisitions and
  cataloging
• volume on a monthly basis
• There were categories of cataloged items, such as
• government documents and gift books that could be
• distinguished from one another in create lists.
• The expect script would run once a month and send
  an
• email with the statistical data

19
Tech Services Statistics Script

• Log into III
• Pick a free list
• Run several searches using date-specific criteria
• Email the results of those searches

20
Tech Services Statistics Results

• To mdahl_at_cocc.edu
• Subject Tech Services Stats for previous month
• Sun Apr 1 050002 2001
• New order records created
• Orders received
• Total number of books cataloged
• Total number of books copy cataloged
• Total number of books PromptCat cataloged
• Total number of gift books cataloged
• Total number govt. docs. cataloged

21
III Backup Initialize and Verify

• With Expect around, why go through the tedious
  process of initializing and verifying tapes?
• Backups need only involve only the changing of
  tapes and entry on a paper log
• A full backup can be run every day even without
  staff in attendance

22
Initialize Script

• Log in, enter backup area
• Initialize tape
• Include date in tape name
• Log out
• Run at 11pm right before backup

23
Verify Script

• Log in, go to backup area
• Verify current tape
• Log out
• Run early in the morning after backup finished

24
Backup results

• Unattended Backup Log
• Date Time ID Type Tape Name
  Status
• 01 M 10-14 0313 1737 Full backup1014
  COMPLETED, VERIFIED
• 02 Su 10-13 0313 1736 Full backup1013
  COMPLETED, VERIFIED
• 03 Sa 10-12 0313 1735 Full backup1012
  COMPLETED, VERIFIED
• 04 F 10-11 0313 1734 Full backup1011
  COMPLETED, VERIFIED
• 05 Th 10-10 0313 1733 Full backup1010
  COMPLETED, VERIFIED
• 06 W 10-09 0313 1732 Full backup1009
  COMPLETED, VERIFIED
• 07 Tu 10-08 0313 1731 Full backup1008
  COMPLETED, VERIFIED
• 08 M 10-07 0313 1730 Full backup1007
  COMPLETED, VERIFIED
• 09 Su 10-06 0313 1729 Full backup1006
  COMPLETED, VERIFIED
• 10 Sa 10-05 0313 1728 Full backup1005
  COMPLETED, VERIFIED
• 11 F 10-04 0313 1727 Full backup1004
  COMPLETED, VERIFIED
• 12 Th 10-03 0313 1726 Full backup1003
  COMPLETED, VERIFIED
• 13 W 10-02 0313 1725 Full backup1002
  COMPLETED, VERIFIED
• 14 Tu 10-01 0313 1724 Full backup1001
  COMPLETED, VERIFIED
• 15 M 09-30 0313 1723 Full backup0930
  COMPLETED, VERIFIED
• 16 Su 09-29 0313 1722 Full backup0929
  COMPLETED, VERIFIED

25
Moving order data to item record

• III feature that allows you to archive order
  record data to item records
• Useful because III charges lots for order records
• Script came from Old Dominion U

26
Moving order data script

• Script takes a create-list of bibliographic
  records
• Records can only have one order record and one
  item record
• Script goes into Update Existing Records-Review
  File Records and parses through the file

27
Moving Order Data, cont.

• Goes into each item record and imports order data
• I modified script to add the selector name to a
  note field in the item record
• Script produces a results file with errors, etc.

28
Moving Order Data results

• B1000824x Last updated 04-04-02
  Created 11-09-94 Revision 5
• LANG eng LOCATION watzk BIB LVL m
  BCODE3 n
• SKIP 0 CAT DATE 09-13-94 MAT TYPE
  a COUNTRY cau
• 090 PS3552.U4bB8
• 100 1 Bukowski, Charles
• 245 10 Burning in water, drowning in flame
• 260 Los Angeles,bBlack Sparrow Press,c1974
• I10008706 ITEM Upd 04-04-02
  Crtd 11-09-94 Rec. 1 of 4383
• COPY 1 PATRON 0 ODUE DATE
  - - LOANRULE 0
• ICODE1 3 LPATRON 1006775 IUSE3 0
  STATUS w
• ICODE2 n LCHKIN 11-09-95 RECAL
  DATE - - INTL USE 1
• I TYPE 100 INVDA - - TOT
  CHKOUT 1 COPY USE 0
• PRICE 0.00 RENEWALS 0 TOT RENEW
  1 IMESSAGE
• OUT DATE - - OVERDUE 0 LOCATION
  wmain OPACMSG
• DUE DATE - -
• 979 o126136b35961cc05-25-99d5448e06-04-99
  f21.00h06-04-99kwbna
• lwrep mfnbp-q-r-s-
• BARCODE 35209004065500
• NOTE SELECTOR BA Smith

29
Moving Order Data results, cont

• Results summary (update items)
• 4850 done
• 152 had too many records
• 0 had no item records
• 0 had orders with copies 0
• 0 had busy records
• 3 had orders with no paid fields
• 35 had a 979 already
• (mark orders for deletion)
• 4850 done

30
Journal Title Database data

• Goal to make a comprehensive print and
  electronic journal database
• Serials Solutions provides the electronic data
• III provides the local print/microform holdings
  data
• Need to output both bibliographic and checkin
  record data on a regular (weekly) basis

31
Journal Title Database script 1

• Logs in, enters create lists, finds empty list
• Searches for bibliographic records for all
  journals (everything with a 222 or 229)
• Outputs comma delimited file of journal title,
  alternate titles, .b, publisher, etc.

32
Journal Title Database Script 2

• Log in, go to create lists, find free list
• Script searches for all checkin records with
  location codes that apply to serials
• Outputs file with lib has data and parent
  bibliographic record numbers

33
Expect script(running as a cron job)logs into
ILSand instructs systemto produce text
filelist and ftp it to external server
ILS
Web form sendsquery to PHP page
Text list ftped to serverin delimited text
format
PHP page queriesSQL database
PHP pageproduces htmlwith results of query
Text file opened upby Perl script and
loadedinto external SQL database
SQL databaseon external server
34
(No Transcript)
35
Strategies to reduce coding

• Beware, with each release of III, menus change,
  and Expect scripts may break
• Have your scripts share a file of procedures that
  achieve common tasks in III (logging in, finding
  an empty list, searching for a particular value
  in a list, etc.)

36
Example of procedure

• PROCEDURE to get into the featured lists area
  of III
• proc getin
• global initials expectinitials
• expect "M,A,X)"
• send -- "m"
• expect "L,S,Q)"
• send -- "l"
• expect "Please key your initials "
• send -- initials
• expect "Please key your password"
• send -- "expectinitials"

37
Example of script using procedures

• Login to III and get into create lists
• login
• getin
• Now do the list
• startlist b
• marcfield "222\r" "" "\r"
• boolean "a"
• enterfield "7" "BCODE3" "" "-"
• boolean "o"
• marcfield "229\r" "" "\r"
• boolean "a"
• enterfield "7" "BCODE3" "" "-"
• finishlist "serials"

38
References

• Expect web site
• http//expect.nist.gov/
• A good expect/III tutorial from Central
  Washington University
• http//www.lib.cwu.edu/systems/expect/
• A good book on Expect
• Don Libes. Exploring expect a tcl-based toolkit
  for automating interactive programs. Sebastopol,
  CA O'Reilly Associates, Inc., 1995.

39
Contact Info

• Mark Dahl
• dahl_at_lclark.edu
• 503-768-7339
• http//www.lclark.edu/dahl/