Enterprise CRM Solution

1
Observing Process Astronomer's Integrated
Desktop Scheduling Block Based Observing GBT
e2e Software Review May 3, 2005 Amy Shelton
(ashelton_at_nrao.edu) Nicole Radziwill
(nradziwi_at_nrao.edu)
2
Ease of Use Project Description from August 2004
Quick Look Data Display
SB Executor (turtle.d)
GBT OT
Status Screen
uses
BUILD
Configuration API
EDIT
Observing API
Console Windows
RUN
Balancing API
MONITOR
Easy Access to Documentation, Help

• Scheduling Blocks
• Annotated Index of Observation
• Execution Log File

Astronomers Integrated Desktop (Astrid)
Observation Management Database (e2e Project
Database)
Note that APIs are very telescope-specific, yet
are used to abstract the observation into terms
not specific to the telescope
3
GBT Observing Process

• Why do we want an Integrated Desktop?
• Astronomers dont have to learn multiple
  applications to accomplish the observing task
• Remote observers only launch one application,
  manages on-screen real estate well
• Error reports do not require knowledge of what
  application error was reported in
• Why do we want to transition to a Scheduling
  Block based system?
• Observing Motivations
• Enable observation data mining we can track
  what was observed more effectively
• Encourage up front preparation maximize
  throughput of science per observing session
• Enable automated dynamic scheduling
• Facilitate proposal review process we can use
  data from past proposals to review current
• Balance interactivity needs with need to optimize
  high frequency weather
• Simplify routine operations such as regression
  tests and all sky pointing
• Characterize observations better so that pipeline
  processing can be enabled in the future
• Technical Motivations
• Enable more efficient troubleshooting we can
  track what people did, and what errors occurred,
  much more easily
• Code is written to accommodate a discrete number
  of well-defined levels of abstraction
• Distinct categories of usage (astronomers use
  apps and application components, experts use
  HLAPIs, programmers use LLAPIs)

4
Observation Process Preparation Execution
Currently, Observation Process focuses on
single SB execution. The Science Program level
is currently unimplemented.
Adopted on the GBT as defined by ALMAbut
slightly customized for the GBT (e.g.
details of ObsProcedure).
NRAO e2e terminology used throughout
presentation.
5
Observing Process Data Model

• Ties observation to proposal
• Allows data mining on GBT observations

Customizedfor usewith GBT

• Add security table
• Do we need tables for
• Observing cases
• (representative SBs)?
• What have we learned
• About this data model
• From last year?
• Add security
• Handling of FIFO queue
• What about obstarget?

6
High-Level Architecture
GUIs
Application
Application Component
0..n
uses
HLAPIs
Expert User
Programmer
uses
uses
LLAPIs
Control System
Observed Data (in an EDF)
Real-Time Data (streaming)
7
Astronomers Integrated Desktop (Astrid)

• What is Astrid?
• Astrid is a container from which you use various
  GBT applications, e.g. Observation Management and
  Data Display.
• Multiple application instances are supported,
  e.g. two Data Displays.
• Why is Astrid so important?
• Simplifies observation startup. Users simply
  type astrid at any Linux prompt.
• Reduces the number of applications that observers
  have to launch to begin observing to ONE most
  useful for observing remotely!
• Observers do not have to know the difference
  between those applications to report issues.

8
Astrid Architecture

• Application Component Tabs Allows the user to
  launch multiple applications within a single
  container. They provide a GUI interface to the
  HLAPIs for all users.
• HLAPIs are always available to the expert user
  to bridge the gap between non-interactive SB
  based observing and the desired for interactive
  observing. Useful for special purpose tasks
  (balancing mid-scan) and commissioning.

9
Astrid Screen Shot
Drop-Down Menus
Tool Bar
Application Components
Python Command Console Expert User access to
HLAPIs, e.g. Balancing and Configuration
Application Component Log Window
10
Available Astrid Application Components

• Astrid Launches a separate Astrid session
• DEAP Data Extraction Analysis Program
  provides general data display and manipulation
  capabilities.
• Python Editor A text editor that provides
  Python syntax highlighting.
• Text Editor A text editor for general use.
• Data Display Provides real time display of
  data plus offline data perusal.
• Logview Used to examine engineering FITS
  files.
• Observation Management Edit/Submit/Run
  Scheduling Blocks on the telescope.

What about other app Components (gfm
data Display, gbtstatus??) Blank screen not
too Exciting here ?
11
Astrid User Help
12

• Describe here the process
• Writing an SB
• Upload to DB/validate
• Submit to DB
• Monitor Progress
• Monitor telescope status w/gbtstatus
• Show how you do this in CONTEXT of astrid
• Also explain how in the unified environment, you
  can edit SB, submit SB, monitor its progress, and
  check GBT status (even though final issues being
  worked out with GBT status over next couple
  months)

13
Summarize Scan Types Observing Directives

• List out the currently supported Scan Types Obs
  Directives, maybe also specify wiki pages they
  are listed on
• Are there any new ones that we didnt know we
  needed last year, but working with the staff
  scientists, we realized were important?
• Any adjustment of requirements? Things that
  didnt work out so well in observing, but you
  changed them, and now they are OK?
• I would spend some time talking about Autopeak
  Autofocus too, since those are unique to us, but
  process of putting them in there could be useful
  for other nrao projects

14
Lessons Learned Deploying Software on Working
Telescope

• Technology Transfer
• Astrid deployment delayed because technology
  transfer between Software Development staff and
  Observing Support staff is taking much longer
  than originally planned for
• Staged Deployment
• Program Layer not yet implemented
• Observers expectations already influenced by
  previous observing experiences at the GBT
• Abuse at SB layer
• Long SBs
• for loops
• Training documentation provided to encourage
  best practices

15
Lessons Learned Deploying Software on Working
Telescope

• Up-front preparations
• Major paradigm shift, especially for support
  staff
• Observing strategies can be developed minutes
  before observing, or even on the fly, but
• Constructing SBs requires forethought
• Authorized projects must be entered into database
  ahead of time
• Observer name must be entered into database ahead
  of time, and associated with their projects
• SBs need to be validated for observational
  integrity
• Ultimately will make most efficient use of
  telescope time
• Dedicated Forum for Software Development Response
• Captures valuable user feedback
• Visible indicator to Support Staff of Software
  Developments response to user-initiated
  suggestions/issues.

16
Lessons Learned Aligning Development with
Organizational Goals

• Remote observing needs require simplified
  application startup
• Make most efficient use of bandwidth
• Reduce number of applications needed to observe
• One of the major issues driving the development
  of Astrid
• Early tests using VNC very promising
• Importance of validation a priori
• To reduce lost telescope time, SB must be created
  before scheduled observation.
• Validation currently catches syntactic errors
  ahead of time
• Plans to expand Validator to further reduce time
  lost to construction of
• Interactive observing is still a requirement
• Support commissioning activities, e.g. new
  receivers
• Support pulsar observations
• Astrids Python console provides access to HLAPIs
  that supplement observing intent captured in SBs

17
Lessons Learned Additional Requirements

• Project Data Directory
• Missing mechanism for specifying project data
  directory
• One project, many sessions
• Sessions kept in separate directories
• Sessions may have overlapping Subscan numbers
• Need for Online/Online (monitor only)/Offline
  Modes
• One user should have control of telescope at a
  time
• One or more users would like to see what is going
  on, e.g. support staff, operators, collaborators
• Operators have ultimate control
• Users with upcoming observations need mechanism
  for submitting SB to database
• Additional Control Mechanisms
• e.g. device manager on/off
• How much direct interaction with the control
  system should be enabled in the technology?
• Usability Issues and Bugs
• Accumulating user feedback, which will be
  prioritized and implemented as resource
  allocations permit

18
Enabling Full Functionality in 2005

• Described at http//wiki.gb.nrao.edu/bin/view/Soft
  ware/ObservingToolsv2
• Institute Online/Offline Modes C3
• Improve Responses to Stops, Aborts Control
  System Failures C4
• Moving Sources Source Catalogs C5, C6
• Improved SB Submission FIFO Queueing Model
  C6, C7
• Enable Offline Validation of SBs using
  full-telescope software simulator bound to
  production control system software C8

• Maybe want to illustrate some of these problems.
• Like the zillion-click process to get an SB to
  run on
• The telescope, how its not instantiated at
  submit
• Time, how SBs disappear from job queue and
• This does not reflect natural palette that
  astron
• Expect to select their SBs from, etc.

19
Remaining Issues

• 1. Implementing a GUI to build Scheduling Blocks
• Observing Issues
• Bla Bla
• Technical Issues
• We have a Java prototype build last year that
  looks very similar to ALMA OT, but builds single
  SBs and does not manage Science Programs
• Should we
• 2. Enabling Science Program Management

20
Conclusions

• Because of cooperative work between SDD and
  scientists this year, we can expect complete
  transition of GBT observing for all standard
  observing modes to Scheduling Block based system
  by end of 2005
• Up front planning required by observers at that
  point will significantly reduce burden on
  scientific staff
• Next year, can turn focus to usability issues,
  Science Program level, and managing the
  dynamically scheduled queue (as planned in 2004)