Cluster Software Preparation, Distribution

1
Cluster Software Preparation, Distribution
Maintenance

• Applications Software Group
• Activity Review
• December 2004

2
The Clusters

• Computers and Peripherals in Clusters

3
Hardware Software

• Linux 106 stations 22 applications
• Macs 124 stations 50 applications
• Windows 193 stations 65 applications
• includes third party applications that AppSoft
  prepares and titles that others prepare, e.g.
  Pine - ? (formerly Rob Siemborski), Mozilla -
  Chaskiel Grundman
• includes 13 Instructional Technology Windows
  computers in
• Baker Adamson Wing, Baker A51 A53,
  Baker/Porter 100, Doherty 1112, 1212, 2210
  2315, Hamerschlag B103 B131, Margaret Morrison
  A14 103, Scaife 125

4
Goals

• Provide consistent good user experience and
  satisfy most user expectations
• Satisfy teaching requirements for undergrad
  courses
• Prevent accidental/malicious changes to software
• Secure machines from security threats
  (local/remote)
• Maintain high availability/uptime (most clusters
  24/7 completely or predominantly unstaffed)
• Ability to delegate first level troubleshooting
  and downloads

5
Staff Day to Day Maintenance

• Cluster Managers/CCons MacOS, Linux Windows
  first responders
• Tony Farah IT auditoria first responder
• Dave Hawranko CFA on site support first
  responder
• Debbie Cherry MacOS sysadmin, software
  preparation
• Jim Foraker/Stephen Hathorne Andrew Linux
  software preparation, Linux sysadmin
• Mike Kelleher Linux on site support
• Deanna Pasternak (w Collin Barnett) Windows
  sysadmin, software preparation

6
MacOS StaffPrepare, Deploy, Maintain

• Debbie Cherry 100
• Project manager, sysadmin, application
  preparation
• Jim Foraker 65
• MacOS programmer, infrastructure set up
• Joe Jackson 10
• Architect
• Bryan Ferguson (student) 75 summer, 10 acad
  year
• Scripting, application preparation
• Karen Lukas 5
• Oversight

7
Windows StaffPrepare, Deploy, Maintain

• Deanna Pasternak 100
• System maintenance, application preparation
• Collin Barnett 10
• RIS/DIST server sysadmin, liaison to AppSoft
• John Orefice 20
• SUS MOM server administration for clusters
  others
• 1 FTE from among all Windows staff
• Windows AD administration for clusters others
• Karen Lukas 5
• Oversight

8
MacOS Preparation

• Steps in preparation
• Use Radmind to set a test Mac to a known state
• Install an application
• Use Radmind to create a loadset transcript
  directory of changed files
• Iterative process (the meat of the work)
• Test and correct problems with functionality and
  file permissions, add serial number, customize
  preferences, key
• Create additional loadset(s) overloads
• Test for proper function and repeat as necessary
• Aggregate loadsets into command files for each
  cluster

9
MacOS Maintenance

• Nightly update during which
• Radmind restores existing files to known good
  state
• Newly released software installed
• Radmind deposits a timestamp in a log for update
  status check
• Delete outdated user files
• Reboot as necessary

10
Windows Preparation

• Customize or build MSIs
• MSI relational db of instructions data to
  install an app
• 10 of apps come as MSI to use as is or customize
• 90 of apps need to have MSIs built from scratch
• Little change in last 4 years
• MSIs built specifically for cluster computer
  configuration
• Define sets of MSIs for each unique cluster
  collection
• Boot each computer from CD to initiate Netboot
• Install OS and applications from MSIs stored on
  server

11
Windows Maintenance

• Nightly reboot restore to known state, release,
  update, remove applications
• SUS distribute updates and patches (push)
• Symantec obtain updates to virus definitions
  check daily
• MOM monitor performance
• Terminal Services remote repair, rebuild
• Cluster managers, AppSoft staff rebuild/refomat
  if necessary
• Time consuming
• May require hands on intervention if reformat
  needed

12
Dependencies

• Openafs.org
• Server installation, maintenance, repair
• Server functionality
• Functional network w helper addresses identified
  to routers
• LDAP/Kerberos for authentication function,
  schema
• Backups Amanda for MacOS, Arcserve for Windows

13
Dependencies

• Clusters Group hardware installation, cluster
  staffing
• Instructional Technology auditorium decisions
  and first responder function
• Software Licensing, acquisition, retention
  decisions
• Others in UES developing new plans
• Computers in classrooms

14
Challenges

• Making everything work together properly
• Being able to say no to sw requests
• New, odd security, short notice, alpha, beta
• Having adequate time and skills to test
  thoroughly
• Persuading faculty to test function prior to
  first class
• Rebuilding Windows pcs after being restored to
  net
• One offs

15
Challenges

• Outside groups that meet in clusters during the
  summer
• Special events (workshops, conferences) scheduled
  in clusters requiring special software or support
  for peripherals
• No requirement to seek approval prior to
  commitments that involve clusters and software
• Usually in summer when we are busiest drop
  everything and change schedule to accommodate them

16
Resources

• Exposure
• No backup for essential staff Debbie, Deanna,
  Jim, Joe
• Freeze

17
Resources

• How close to saturation are we?
• Could distribute existing software to more
  computers of same type with additional first
  responders
• Cannot handle additional unique software
  collections
• Cannot handle unique hardware that requires
  customized procedures
• Cannot handle CFA 323-like set up for Windows

18
Resources

• Solutions
• Hire additional staff
• Reduce of applications in all clusters
• Require standardization on 1 of replicated
  applications, e.g. CAD, 3D animation, statistics,
  mathematics
• Be ruthless about trimming little used
  applications
• Do fewer updates each year
• End support for specialty clusters and functions
• Turn CFA Multimedia Studio over to CFA

19
Resources

• Solutions
• Require requesting departments to make initial
  software purchases
• Reduce variety of supported equipment
• Change distribution technology
• Disk imaging evaluation next 6 months
• Terminal Server pilot next 6 months

20
What ifAppSoft cut 25

• Had 12.5 fewer resources in 2004 -- result
• Stopped updating Andrew Solaris software
• Mike Kelleher (Clusters Group) started doing some
  on site maintenance on Andrew workstations
• Fall semester 2004
• Relied much more heavily on cluster managers to
  rebuild problem Windows computers

21
What ifAppSoft cut additional 25

• Retain AppSoft
• Drop MacOS or Window from clusters
• Drop support for Linux to desktops
• Disband AppSoft
• Reassign Windows and MacOS staff to other
  managers
• Other managers assume faculty liaison duties of
  AppSoft manager

22
What ifAppSoft adds 25 1 FTE

• Add to AppSoft
• Provide MacOS and/or Windows staffing backup
• Do additional testing on existing software and
  machine types to improve reliability and
  performance
• Reassume responsibility for Andrew Linux
  applications packaging and deployment
• Reassume responsibility for Andrew Linux cluster
  computers on site maintenance
• Better document environment and procedures