Tutorial: Condor on Windows

1
TutorialCondor on Windows
2
Overview

• Simple job submission
• Accessing network shares
• Handling DLL dependencies
• Checkpointing in Vanilla universe
• Submitting interpreted scripts

3
Overview

• Simple job submission
• Accessing network shares
• Handling DLL dependencies
• Checkpointing in Vanilla universe
• Submitting interpreted scripts

4
Simple Job Submission
5
condor_store_cred

• Contacts local SchedD and asks it to securely
  store a users password
• Password is placed encrypted in a registry
  location

myp4sswd
y0urs

• C\gtcondor_store_cred add
• Account gquinn_at_CROW
• Enter password
• Operation succeeded.

6
Job Execution Submit Side
SchedD
Secure Password Store
Submit
myp4sswd
y0urs
Submit
Shadow
7
Job Execution Execute Side
Starter
Jobs run using a Condor-specific account with
minimal privileges.
Job
condor-reuse-slot1
8
Overview

• Simple job submission
• Accessing network shares
• Handling DLL dependencies
• Checkpointing in Vanilla universe
• Submitting interpreted scripts

9
Accessing Files on a Network Share

• The condor-reuse-slotX accounts are local
  accounts without network privileges
• To access files on a network share, the job needs
  to run using an account that has access to the
  shared network resources
• Using a CredD, its possible to have jobs run with
  the credentials of the submitting user

10
The Windows CredD

• A centralized repository for user passwords

myp4sswd
y0urs
store password

• C\gtcondor_store_cred add
• Account gquinn_at_CROW
• Enter password
• Operation succeeded.

CredD
ltpasswordgt
11
The Windows CredD
SchedD
myp4sswd
fetch password
y0urs
ltpasswordgt
Shadow
Submit machines can use the CredD to impersonate
the user in the shadow
12
The Windows CredD
Starter
fetch password
myp4sswd
y0urs
ltpasswordgt
Job
Execute machines can use the CredD to run jobs as
the desired user
13
Securing the CredD

• NTSSPI can be used to authenticate to CredD and
  send the password encrypted over the network

myp4sswd
y0urs

• C\gtcondor_store_cred add
• Account gquinn_at_CROW
• Enter password
• Operation succeeded.

CredD
store password
14
NTSSPI Authentication

• Uses Windows built-in authentication capability
• Used for user-to-daemon authentication
• Always works when client and server are on the
  same machine
• Can also be used over the network if the user
  account exists on both the client and server and
  the passwords match

15
Securing the CredD
Starter
myp4sswd
fetch password
y0urs
Job
Condor normally runs as SYSTEM, and therefore
cant use NTSSPI
16
Securing the CredD

• Options for securing password fetch operations

• Kerberos
• SSL
• Password

17
Password Authentication

• Daemon-to-daemon communication is secured using a
  shared secret (the pool password)
• Pool password must be distributed to all machines
  in the pool before enabling
• Pool password is stored using the -c option to
  condor_store_cred
• Password can be set remotely using -n option
  (make sure connection is authenticated!)

18
Setting the Pool Password

• Enable CONFIG access from the machine that you
  will run condor_store_cred
• If setting the password remotely, ensure the user
  account that runs the condor_store_cred command
  exists on every machine and has a common password
• Set the pool password on every machine

HOSTALLOW_CONFIG (HOSTALLOW_ADMINISTRATOR)
C\gt condor_store_cred add c n
node01.cs.wisc.edu
19
Configuring the CredD Host

• Once pool password has been distributed, the
  CredD can be configured

Set security settings so that full security to
the credd is required CREDD.SEC_DEFAULT_AUTHENTICA
TION REQUIRED CREDD.SEC_DEFAULT_ENCRYPTION
REQUIRED CREDD.SEC_DEFAULT_INTEGRITY
REQUIRED CREDD.SEC_DEFAULT_NEGOTIATION
REQUIRED Require NTSSPI auth for password
storing CREDD.SEC_WRITE_AUTHENTICATION_METHODS
NTSSPI Require PASSWORD auth for password
fetching CREDD.SEC_DAEMON_AUTHENTICATION_METHODS
PASSWORD
20
Configuring the CredD Host

• Use the condor_config.local.credd example file
• Include this file as one of the local
  configuration files

C\Condorgt copy etc\condor_config.local.credd
.\
LOCAL_CONFIG_FILE (LOCAL_DIR)/condor_config.loc
al \ (LOCAL_DIR)/condor_config.local.credd
21
Running Jobs as Submitting User

• Submit and execute machines need to be configured
  to use the CredD

CREDD_HOST vault.cs.wisc.edu STARTER_ALLOW_RUNA
S_OWNER True SEC_CLIENT_AUTHENTICATION_METHODS
\ NTSSPI, PASSWORD CREDD_CACHE_LOCALLY True
22
Running Jobs as Submitting User

• Example submit file

universe vanilla executable whoami.exe log
whoami.log output whoami.out run_as_owner
true queue
23
Overview

• Simple job submission
• Accessing network shares
• Handling DLL dependencies
• Checkpointing in Vanilla universe
• Submitting interpreted scripts

24
Jobs with DLL Dependencies
25
Jobs with DLL Dependencies

• Two ways to deal with DLL dependencies
• Ship the DLLs to the execute machine with the job
• Make sure the job runs on an execute machines
  with the needed DLLs, and knows where to find them

26
Shipping DLLs with the Job
universe vanilla initial_dir
c\cygwin executable sleep.exe log
sleep.log transfer_input_files
cygwin1.dll,cygintl-8.dll,cygiconv-2.dll queue
27
Using DLLs on the Execute Side

• Windows searches directories in the PATH
  environment variable for needed DLLs

C\gt set PATHPATHc\cygwin\bin
universe vanilla executable
c\cygwin\sleep.exe log sleep.log getenv
true queue
28
Overview

• Simple job submission
• Accessing network shares
• Handling DLL dependencies
• Checkpointing in Vanilla universe
• Submitting interpreted scripts

29
Checkpointing in Vanilla Universe

• No Standard universe on Windows
• Some jobs can do their own checkpointing
• Condor will warn your job before eviction by
  sending the WM_CLOSE window message
• For console applications, this translates to a
  CTRL_CLOSE_EVENT

30
Checkpointing in Vanilla Universe

• Catching a eviction notice in a C-based console
  app
• Need to let Condor know that partial results are
  valuable in the submit file

SetConsoleCtrlHandler(ckpt_function, TRUE)
when_to_transfer_output ON_EXIT_OR_EVICT
31
Overview

• Simple job submission
• Accessing network shares
• Handling DLL dependencies
• Checkpointing in Vanilla universe
• Submitting interpreted scripts

32
Submitting Scripts

• Can just submit the interpreter as the executable
  and the script as an input file

executable c\perl\bin\perl.exe arguments
script.pl args for script transfer_executable
false
33
Submitting Scripts

• If UNIX-style ! behavior is desired, use a
  USER_JOB_WRAPPER
• Useful for using a single submit file that works
  on all platforms

executable script.pl arguments args for script
34
USER_JOB_WRAPPER

• open the actual executable so we can look for
  interpreters
• if (open(EXECUTABLE, ARGV0))
• see if the first two bytes are !
• my two_bytes
• if ((read(EXECUTABLE, two_bytes, 2) 2)
  (two_bytes eq "!"))
• look for the /usr/bin/perl interpreter
• if (ltEXECUTABLEgt /\/usr\/bin\/perl/)
• unshift(_at_ARGV, "c\\cygwin\\bin\\perl.exe")
• system(_at_ARGV)

35
Questions?

• Windows BOF
• Thursday, 1100 1200
• Room 221