Security Service Challenge in Asia Pacific

1
Security Service Challenge in Asia Pacific

• Jinny Chien
• Academia Sinica Grid Computing
• OSCT
• Security Workshop on 19th April in Taipei

2
Motivation

• After todays training, site manager could
• Handle the Incident Response Procedure
• Check the security communication channels
• Deal with the sudden security attacks
• Which information we lost
• Overview
• Introduction
• Security Service Challenge in AP
• Conclusion

3
Security Service Challenge (SSC)

• The objective
• The goal of the LCG/EGEE Security Service
  Challenge, is to investigate whether sufficient
  information is available to be able conduct an
  audit trace as part of an incident response, and
  to ensure that appropriate communications
  channels are available.
• The concept
• At first CERN security team submit a testing
  job to the specific sites and site security
  contact must according to the clues and reply the
  answer at the limited time. In general the
  challenge executed once every year.

4
SSC-Objective
5
Stages / Role of SSC

• Stages of the SSC
• Security Challenge targeting the principal site
  of each of the LCG/EGEE Regional Operation
  Centers(ROC)
• Security Challenge targeting the individual sites
  in each ROC
• Roles
• The Test Operator (TOP) who submits the
  challenging job, issues the alert, escalates the
  alert as required and checks the response.
• The Security Contact of the target site, who
  receives and acknowledges the alert, makes the
  necessary investigation and submits the response
  back to TOP

6
SSC

• The challenge is executed by submitting a Grid
  Job from
• a User Interface (UI).
• SSC level 1
• challenges the Workload Management System(WMS) of
  the Grid Resource Broker(RB) and Computing
  Element(CE)
• SSC level 2
• challenges the Storage Elements(SE) on the Grid
• SSC level 3
• challenges the Operational Diligence of the
  LCG/EGEE Grid Sites
• Material for SSC
• The material is available for download from
  https//twiki.cern.ch/twiki/bin/view/LCG/LCGSecuri
  tyChallenge

7
SSC Common Setup

• SSCs were run in two stages
• Stage 1 targeting the principal sites in the
  regions
• Stage 2 targeting the individual sites in each
  ROC
• The jobs were submitted from an User
  Interface(UI) to a chosen Grid Computing
  Element(CE) via a Resource Broker (RB) using
  standard Grid commands
• They consist of a set of small, non-intrusive
  programs.
• Not intrusive, only legal operations are
  executed (job submission), file transfer,)
• No penetration tests, no execution of exploits
  etc.

8
SSC-1 Objective and Setup

• SSC-1 (2005- March 2006) targeted the Workload
  Management System(WMS) Resource Broker (RB) and
  Computing Element (CE)
• It tested whether sufficient information was
  available and whether communication channels were
  sufficiently open.
• Did not address the Security Incident Response
  Procedure
• Used Savannah as the vehicle for communication
  between the Test Operator (TOP) and the Target
  sites.

9
SSC-1 - Task

• Given Time range, IP-address of the target
  computer, UNIX-UID of challenging job on target
• The Sites had to find out
• The DN of grid-credentials/certificate used by
  the job submitter?
• The IP-address of the submitting network device
  (UI)?
• The name of the executable which ran on the
  target computer?
• The data and the precise time when the executable
  ran?

10
SSC-1 Practical

• Date - 2006-03-08
• - and time period of challenge,
• between 082300 -and- 083400 UTC
• Virtual Organization (VO)
• LCG/EGEE siteName
• Resource Broker (RB)
• Regional Operation Center (ROC)
• IP-address of the target computer
• lcg00189.grid.sinica.edu.tw
• UNIX-UID of challenging job on target 18118
• --- Security_Service_Challenge_Description
  ------------

• Subject Security Service Challenge
• Local date and time of request creation
• 2006-03-08 103839 (CET, UTC2)
• Initials of test operator psa
• Dear LCG/EGEE Site Security Officer,
• This e-mail constitutes a security service
  challenge
• alert. You have received this because you have
  opened
• an e-mail destined to this site's security
  officer. In
• case you are not the security officer of this
  site,
• please forward this e-mail to -
• aproc-security_at_list.grid.sinica.edu.tw
• just stating so. This will allow us to improve
  our
• procedures, and we thank you in advance.
• We thank you for your collaboration,

11
SSC-1 in AP

• Executed time 2006/3/5 2006/3/13
• Targeted Sites
• Australia-UNIMELB-LCG2
• GOG-Singapore
• INDIACMS-TIFR
• LCG_KNU
• Taiwan-IPAS-LCG2
• Taiwan-NCUCC-LCG2
• TOKYO-LCG2,
• TW-NCUHEP
• Total sites are 8
• The final report
• https//twiki.cern.ch/twiki/pub/LCG/SSC1/SSC_1_Deb
  rief_2006-04-18.pdf

12
SSC-2 Objective and Setup

• SSC-2 tested the traceability of storage
  operations (2007).
• From the Worker Node (WN) a sequence of seven
  storage operations have been executed.
• lcg_crx, lcg_lgx, lcg_repx, lcg_rx, lcg_cpx,
  lcg_delx
• Did not address the Security Incident Response
  Procedure
• Used the Global Grid User Support (GGUS) as the
  vehicle for communication between the Test
  Operator and the Target Sites.

13
SSC-2 - Task

• Given User DN, Time range and SE
• The Sites had to find out
• 1. For each of the identified storage operation,
  please indicate
• The exact time (UTC).
• The type of operation.
• The URLs, filenames, catalog names and file paths
  involved.
• 2. Please indicate the IP-address of the User
  Interface (UI)
• that was used for the Job Submission

14
SSC-2 in AP

• Executed time 2007/4/20 2007/5/4
• Targeted Sites
• 18 sites, 8 countries
• The procedure is http//lists.grid.sinica.edu.tw/a
  pwiki/Security_Service_Challenge?highlight28secu
  rity29
• The final report could be found
  https//twiki.cern.ch/twiki/pub/LCG/SSC2/SSC_2_Sta
  ge_2_Report_AsiaPacific.pdf

15
The result of SSC2

• Status
• Error could not submit a SSC job
• OK success
• Reply
• Yes Reply the answer
• No Not reply the answer
• Feedback
• Yes provide the feedback
• No Not provide the feedback

16
SSC-3 Objective and Setup

• SSC-3 -a more realistic simulation of an
  incident, it challenges the Operational
  Responsiveness of LCG/EGEE Grid Sites.
• The Job is launched from a User Interface (UI)
• It runs with valid credentials.
• Once running, it will exploit its environment to
  conceal its activities.
• Sign of life will be reported through an
  out-of-band channel.

17
SSC-3 Objective and Setup II

• Alert
• The Alert is sent to the CSIRT e-mail address
  registered in the Grid Operations Center Data
  Base (GOCDB)
• The text clearly identifies the alert as a test.
• The Grid identity of the submitting user is
  indicated.
• The Site is asked to deal with the Alert
  following approved Incident Response Procedures.
• Send alert mails to
• VO managers 4 weeks ago
• Alert-mail to sites
• roc-security-contact to 2 weeks ago

18
SSC-3 Incident Response

• The Incident Response is broken up in three
  activities
• Communication
• Acknowledgment/Heads-up report to the indicated
  e-mail address.
• Alert to the VO manager.
• Verification that the responsible Certification
  Authority (CA) has been notified.
• Filing of the final report.
• Containment
• Identification of the Job and killing of its
  processes.
• Suspension of the offending user at the
  challenged Site.
• Forensics
• Discovery of emitting Site and contact to the
  Sites CSIRT.
• Analysis of network traffic.
• Analysis of the submitted binaries.

19
SSC-3 in AP (in progress)

• Receive a ticket from GGUS
• Include the text THIS IS A TEST in the
  Subject field
• The distinguished name (DN) of the user
• Ticket time

20
SSC-3 in AP (in progress)

• Send a notification to ROC
• After receiving the testing incident ticket,
  please send a short notification to APROC via
  roc_at_lists.grid.sinica.edu.tw
• The incident information should be including
• Site name
• The basic information of security contact
• The description of incident
• Time(s) of main events (including timezone)
• etc.

21
SSC-3 in AP (in progress)

• Initial analysis and classification
• According to the mail, security contact must
  check the local servers.
• Network traffic analysis
• Kill malicious jobs
• Ban malicious users
• etc.
• Contact Certification Authority manager
• Certification Authority information in Asia is
  http//www.apgridpma.org/CA/CertificateAuthorities
  .html
• Contact Virtual Organization manager
• Trace log file to know which VO is mapping by
  this malicious user.
• All VO information could be found
  https//cic.gridops.org/index.php?sectionvo

22
SSC-3 in AP (in progress)

• Post-incident analysis
• Distribute the whole incident and process
• Feedback or comment for the challenge
• Follow-up
• TOP will re-submit a job to completed sites, if
  the defense of security is strengthened, the job
  will be failed
• APROC will collect the activities record and send
  the final report to OSCT before September

23
SSC-3 in AP (in progress)

• Receive a ticket from GGUS
• Send a notification to ROC
• Initial analysis and classification
• Contact Certification Authority manager
• Contact Virtual Organization manager
• Post-incident analysis

24
SSC-3 in AP (in progress)

• ASGC executes SSC3 in 7th April,2009.
• The following sites are already completed in this
  Challenge .
• This challenge is in progress

25
Outcome

• The following question encountered by security
  contacts at SSC3
• Lack the specific incident response procedure
• How to find the contact info of VO manager
• How to kill or trace the job
• The definition of the final report
• How to use GGUS portal to reply the ticket

26
Conclusion

• The challenge is from EGEE Operational Security
  Coordination Team (OSCT)
• The goal of the LCG/EGEE Security challenge is to
  conduct an audit trace as part of an incident
  response to ensure that appropriate communication
  channels with available sufficient information
• SSC executes
• SSC 1
• challenges the Workload Management System(WMS) of
  the Grid
• SSC 2
• challenges the Storage Elements(SE) on the Grid
• SSC 3
• challenges the Operational Diligence of the
  LCG/EGEE Grid Sites
• SSC3 in AP is still in progress

27
Reference

• OSCT public webpage http//osct.web.cern.ch/osct/n
  /
• Security Service Challenge https//twiki.cern.ch/t
  wiki/bin/view/LCG/LCGSecurityChallenge
• Incident Response Procedure https//edms.cern.ch/f
  ile/428035/LAST_RELEASED/Incident_Response_Guide.p
  df
• The SSC toolkit https//twiki.cern.ch/twiki/bin/vi
  ew/LCG/LCGSecurityChallenge

28

• Question