ArcGIS for Server: Administrative Scripting and Automation

1
ArcGIS for Server Administrative Scripting and
Automation

• Shreyas Shinde
• Ranjit Iyer

2
Agenda

• Introduction to server administration
• Command line tools
• ArcGIS Server Manager
• Overview of the administrative API
• Automation
• Popular scripting workflows
• Publishing of services
• Setting up permissions
• Extracting popular extents
• Monitoring services and databases
• Scaling and statistics

3
Introduction to server administration
4
The administrators toolbox
5
Command line tools

• Located under /arcgis/server/tools
• A collection of most popular tools
• Categories
• Administrative credentials
• For resetting administrative accounts
• SSL/certificate management
• Backup and restore
• Managing cache
• Lock the tools down using OS file permissions!

6
Command line tools

• Demo

7
ArcGIS Server Manager

• Browser based interactive application
• Located at http//server.domain.com6080/arcgis/m
  anager
• Simple, easy-to-use
• Ability to manage your entire server site
• Even though you are accessing Manager on a
  specific server node
• Great for common workflows!

8
le Tour de Manager

• Demo

9
ArcGIS Server Manager under the hood
Simple HTTP requests to the administrative API
10
Overview of the administrative API

• Entire server administration is an API
• If ArcGIS Server Manager can use it, so can you!
• RESTful
• GET and POST of JSON data
• Well organized and well documented
• Lots of samples
• Scriptable
• Designed for automation!

11
The administrative API
Admin API
12
Why automate?

• Common repeated workflows
• Plot popular extents for the data
• Scheduled actions
• Restarting services when data is updated
• Build map caches during low demand by moving
  machines to caching cluster
• Reactive actions
• Adding capacity (server nodes) to a cluster when
  demand increases
• Send email notifications when errors are logged
• Works with all programming/scripting languages
  that support HTTP!

13
Where should you begin?
14
Publish Services

• Demo

15
Motivation

• Publishing many services
• Moving services between Dev, Staging, Production
  environments
• Backup/restore is an option but sometimes you
  just want to republish services
• Can publish from Desktop and Manager
• Tedious if you have lots of services
• Automation is your friend!

16
Task Publish a list of services listed in a
file

• Help system has a script that uses ArcPy
• ArcPy is an python library that complements the
  Admin REST API
• Provides many convenience functions

17
Caveats

• Works great, just need a few lines of code
• However, you need ArcPy on the machine
• Script publishes service definitions sequentially
• For every SD
• Upload
• Publish
• With many SDs to publish this can be a very slow
  process
• Publishing 100 services took hours

18
We can do better

• Run the script from any machine
• No ArcPy dependency
• Exploit concurrency
• Parallel uploads
• Parallel publishing
• If your script is amenable to concurrency,
  exploit it by all means

19
Publishing a service with an SD

• Two step process
• Upload the SD to the Server
• QUICK
• Invoke the publishing tool
• INTENSIVE

20
Script design
Upload
Publish
Upload
SD
SD
SD
SD
UpldID
UpldID
UpldID
Publish
Upload
21
Comparison results
Speed up of 3
ArcPy publishing script
DIY multi-threaded publishing script
22
Starting a service

• Demo

23
Options

• Manager
• Admin
• Script

Start Geometry Service
24
Anatomy of an Admin script

• Main function
• Drives the logic
• Internal utility functions
• Get Token
• Make HTTP requests
• Parse JSON responses

25
Apply Permissions

• Demo

26
Task Apply permissions to services

• Permissions are the way to secure your service
• Do it in Manager
• Tedious for lots of services. Must automate!

27
Demo

• Show them previously published services in
  Manager
• Show them roles
• Run the script
• Discuss the script

28
Health check for services

• Demo

29
(No Transcript)
30
Health check for databases

• Demo

31
(No Transcript)
32
Popular extents

• Demo

33
(No Transcript)
34
Elastic scaling

• Demo

35
Task Dynamically add machines to your site

• Add machines to reduce latency and increase
  throughput
• Dynamically add them when a latency threshold is
  breached
• De-allocate machines when traffic is less

36
Demo

• Start with a single node site
• Make service requests and watch latency (well
  use JMeter)
• Run a script that adds a machine to a cluster
• Watch the latency drop

37
Statistics in 10.3 (Sneak peak)
38
Statistics at 10.3

• Server is always collecting statistics
• Avg response
• Number of requests
• Number of timeouts
• . . .
• Report defines the resource, metric and interval
• Previous day, last 7 days, last 30 days
• Custom time window
• Server chooses intelligent data point intervals
  in the reports
• REST API to harvest all this information

39
Python is great!!
40
Short detour

• Most modern languages (like Python) support
  Threads
• Some languages provide concurrency as a language
  feature
• Go
• Erlang
• Clojure
• Scala
• Suited for concurrent programming

41
Go

• Invented at Google
• About concurrency
• Lightweight threads (goroutines)
• Typed thread-safe communication and
• synchronization (channels)
• Readable concurrent code

42
Erlang

• Invented at Ericsson
• Functional language
• Light weight processes
• Can spawn 100s of thousands
• of them
• Concurrency via message passing
• Actors model

43
Python is great!!

• Dont get us in trouble, we dont want to start a
  rebellion ?

44
Thank you

• Please fill out the session survey
• First Offering ID 1146
• Second Offering ID 1344
• Online www.esri.com/ucsessionsurveys
• Paper pick up and put in drop box

45
Questions?
46
Resources
GIS Tutorial for Python Scripting Esri Press, 2014 Paperback and e-book Just released! Offers several hands-on tutorial exercises.
Python Scripting for ArcGIS Esri Press, 2013 Paperback and e-book Good reference text
47
(No Transcript)