Condor Gotchas II

1
Condor Gotchas II
(this time it's cynical!)

• John Kewley
• Daresbury Laboratory e-Science Centre

2
What is a "gotcha"

• This presentation was originally called "Hints
  and Tips", but I think the term "Gotchas" (aka
  pitfalls) is more appropriate.
• From Wikipedia "A 'gotcha' is a feature of a
  system, a program or a programming language that
  works in the way it is documented, but is
  counter-intuitive and almost invites mistakes
  because it is both enticingly easy to invoke and
  completely unexpected and/or unreasonable in its
  outcome."

3
Outline

• Some "gotchas", mostly related to submit files
• Custom attributes
• Environment variables (and whitespace)
• The new syntax is it a panacea?
• If time, also a tip for formatting with
  condor_status

4
Custom attributes

• How do I declare my own attributes so my jobs can
  distinguish between Red Hat 9 and other Linuxes?
• On Red Hat 9 machines add the following to its
  configuration
• OPSYS_FLAVOUR RH9
• OPSYS_FLAVOUR "RH9"
• STARTD_EXPRS OPSYS_FLAVOUR
• STARTD_ATTRS OPSYS_FLAVOUR
• It can then be matched with a REQUIREMENTS
  statement
• REQUIREMENTS (OPSYS_FLAVOUR RH9)
• REQUIREMENTS (OPSYS_FLAVOUR RH9)
• REQUIREMENTS (OPSYS_FLAVOUR "RH9")
• REQUIREMENTS (OPSYS_FLAVOUR ? "RH9")

v
5
Custom Attributes

• Declare the attribute and match against it
• Always quote non-numerics
• Export the attribute by adding to STARTD_ATTRS,
  rather than STARTD_EXPRS
• Remember the use of the meta operators ? and
  !

6
Environment variables

• How do I pass ROOTC\ to Windows?

ENVIRONMENT ROOTC\ ARGUMENTS first_param
ENVIRONMENT ROOTC\\ ARGUMENTS first_param
ROOTCARGUMENTS
ROOTC\ARGUMENTS
ENVIRONMENT ROOTC\\\ ARGUMENTS first_param
ENVIRONMENT ROOTC\? ARGUMENTS first_param
ROOTC\\ARGUMENTS
ROOTCARGUMENTS
v
ENVIRONMENT ROOTC\\ ARGUMENTS first_param
ENVIRONMENT ROOTC\DUMMY"" ARGUMENTS
first_param
v
ENVIRONMENT ROOT"C\" ARGUMENTS first_param
?
7
Environment variables
CondorVersion 6.6

• How about passing more than one environment
  variable on Windows and/or Linux?
• E1AB E1AB
• ?E2DE ?E2DE
• E1AB E1AB
• E2DE E2DE

ENVIRONMENT E1AB?E2DE
ENVIRONMENT E1AB??E2DE
ENVIRONMENT E1AB?E2DE
ENVIRONMENT E1ABE2DE
v
v
8
Environment Variables

• What about using newlines for better formatting
  when passing lots of environment variables?
• E1AB
• ?E2DE
• E1AB
  
• E2DE

v
ENVIRONMENT E1AB?\ E2DE
ENVIRONMENT E1AB?\ E2DE
ENVIRONMENT E1AB\ E2DE
ENVIRONMENT E1AB\ E2DE
9
Environment variables

• Remember a trailing backslash will escape the
  newline, even if you add a trailing space or
  escape it with another backslash, or both. You
  must escape it and have a blank line afterwards.
• Be careful about blanks and tabs in the middle of
  the environment line, they may render your
  declarations impotent. Probably better to avoid
  whitespace.

10
Whitespace, or not

• We have already seen that \ escapes a newline,
  even with whitespace after it. What about the
  comment char? How about

ARGUMENTS one two \ Above 2 Args are
important \ three
ARGUMENTS one two \ three
ARGUMENTS A B 2 Args ?
11
The "new" syntax

• Put double quotes around the entire string.
• Use whitespace to separate arguments. To include
  whitespace, surround by single quotes.
• To insert a double quote, repeat it.
• To insert a single quote, repeat it inside a
  single-quoted section.

12
How are previous gotchas handled?

• The following all work as expected for passing
  environment variables
• ENVIRONMENT "ROOTC\"
• ENVIRONMENT "E1AB E2AC"
• ENVIRONMENT "E1AB \
• E2AC"

?
13
What? No Gotchas?

• How do I setup an environment variables for a
  single quote or a double quote?
• SQUOTE"'"
• DQUOTE"\"" DQUOTE'"'
• in old Syntax
• ENVIRONMENT SQUOTE"'"
• ENVIRONMENT DQUOTE"\"" ENVIRONMENT
  DQUOTE'"'

14
SQUOTE

• New Syntax
• Trying to produce SQUOTE"'"
• ENVIRONMENT "SQUOTE""''''"""

15
DQUOTE

• New Syntax
• Trying to produce DQUOTE'"'
• ENVIRONMENT "DQUOTE''''""''''"
• Trying to produce DQUOTE"\""
• ENVIRONMENT "DQUOTE""\"""""

16
In Summary

• Good writing is all about spelling, grammar and
  punctuation. The same is true with condor
  configuration and submit files.
• Use the new arguments/environment syntax!

17
Tip for condor_status

• condor_status -f "-6s" Arch -f "-7s" OpSys \
• -f " -12s\n" OPSYS_FLAVOUR sort uniq -c
• 1
• 1 INTEL LINUX Gentoo
• 1 INTEL LINUX Mandrake10
• 2 INTEL LINUX RH80
• 3 INTEL LINUX RH9
• 2 INTEL LINUX SUSE80
• 6 INTEL LINUX SUSE90
• 5 INTEL LINUX WBL
• 1 INTEL WINNT40INTEL WINNT50INTEL WINNT51

18
Tip for condor_status

• condor_status -f "-6s" Arch -f "-7s" OpSys \
• -f " -12s" OPSYS_FLAVOUR -f "\n" OpSys sort
  uniq -c
• 1
• 1 INTEL LINUX Gentoo
• 1 INTEL LINUX Mandrake10
• 2 INTEL LINUX RH80
• 3 INTEL LINUX RH9
• 2 INTEL LINUX SUSE80
• 6 INTEL LINUX SUSE90
• 5 INTEL LINUX WBL
• 1 INTEL WINNT40
• 1 INTEL WINNT50
• 1 INTEL WINNT51