Mehdi Adda,

1
Mehdi Adda, PhD Student at University Of
Montreal http//www-etud.iro.umontreal.ca/addameh
d/
mehdi.adda_at_gmail.com
2
Rails deployment Putting the pieces together
mehdi.adda_at_gmail.com
Montr eal On Rails, December 2007
3
Outline

• Startup kit
• From a basic server deployment to an efficient
  solution

4
Startup kit

• A happy machine that will play the role of a
  server
• a 2.4Ghz with 2Giga Ram
• Linux dist.
• Centos 5
• DBMS
• . Mysql
• Ruby kit
• Ruby, ruby gems, Rails
• A Rails application to deploy
• . www.thadsa.com
• An internet connexion

5
Ruby, Ruby gems and rails setup

• Ruby

• Ruby Gems

cd /usr/local/src wget
http//rubyforge.rubyuser.de/rubygems/rubygems-0.9
.5.tgz tar zxvf rubygems-0.9.5.tgz cd
rubygems-0.9.5 ruby setup.rb

• Rails

gem install rails v1.2.6 --include-dependenci
es
6
From a basic server deployment to a viable
solution

• Webrick
• Mongrel
• Mongrel Nginx
• Many Mongrels Nginx
• Mongrel_cluster Nginx
• Seesaw Mongrel_cluster Nginx
• Monit Seesaw Mongrel_cluster Nginx

7
Webrick
8
Webrick

• HTTP server for ruby on rails applications
• Standard ruby library

slow, not scalable Generally used only for
development mode or for very very small and
simple application
9
Mongrel
10
Mongrel

• High performance HTTP application server
• Written in Ragel and C
• Supports Ruby On Rails, OgNitro, Camping, and
  IOWA, Merb

11
Mongrel

• Setup

gem install mongrel

• Startrestartstop

cd myapp
mongrel_rails start -d -e production
--port 80
mongrel_rails restart stop
12
Mongrel

• How to Serve static files ?

13
Mongrel
14
Nginx
15
Nginx

• Serves static files
• Proxy and load balancing dynamic requests
• Faster
• Built-in memcached
• Lightweight
• Easy to setup and configure

16
Nginx setup

• Configure and install

export NGINX0.6.17 cd /usr/local/src
wget http//sysoev.ru/nginx/nginx-NGINX.tar.gz
tar xfz nginx- NGINX.tar.gz cd
nginx- NGINX ./configure
--pid-path/usr/local/nginx/logs/nginx.pid \
gt--sbin-path/usr/local/sbin/nginx
--with-md5/usr/lib \ gt--with-sha1/usr/lib
--with-http_ssl_module --with-http_dav_module
\ gt--with-http_stub_status_module make make
install
17
Nginx setup

• Get and Install daemon scripts

wget http//notrocketsurgery.com/files/nginx
-O /etc/init.d/nginx chmod x
/etc/init.d/nginx

• Start and stop Nginx

/etc/init.d/nginx start stop

• Edit the nginx.conf (/usr/local/nginx/conf)

18
Mongrel Nginx

• nginx.conf

. upstream monapp-mongrel server
192.168.0.1008000 server listen
80 server_name monapp.com root
/home/projects/rails/sites/monapp/current/public
location / if (-f request_filename.h
tml) rewrite (.) 1.html break
if (!-f request_filename)
proxy_pass http//monapp-mongrel break

19
Mongrel Nginx

• nginx.conf

nginx_config_generator
20
Mongrel Nginx

• Generate nginx config file
• Install nginx_config_generator

gem install nginx_config_generator

• Generate an YML config file

generate_nginx_config --example gt config.yml

• Generate nginx config file

generate_nginx_config config.yml nginx.conf
21
Mongrel Nginx

• config.yml

define default root and alternative roots root
capi /home/projects/rails/sites/s/current/pub
lic default /home/projects/rails/sites/s
declare sites here sites monapp-mongrel
one upstream server upstream
- 192.168.0.1008000 server_name
monapp.com root capi
22
MongrelNginx ready
/etc/init.d/nginx start
23
Mongrel Nginx

• Rails isnt thread safe
• Mongrel will serve 1 request at time
• Upgrades, restarts, crashes
• Downtime
• Run multiple instances of mongrel

24
Many Mongrels Nginx
Network
Dynamic requests
Nginx lt80gt

Static files
Mongrel lt8000gt
Mongrel lt8001gt
Mongrel ltNgt
File system
25
Many Mongrels Nginx

• nginx.conf

upstream many-mongrels server
192.168.0.1008000 server 192.168.0.1008001
server 192.168.0.1008002 server
listen 80 sets the domains that
this host server requests for server_name
monapp.com doc root root
/home/projects/rails/sites/monapp/current/public
location / add .html to the end of
the url and check if it exists in fs if
it exists then keep url with .html and serve it
as static file if (-f request_filename.html
) rewrite (.) 1.html break
if the file doesnt exist proxies the
request to the mongrel upstream server if
(!-f request_filename) proxy_pass
http//many-mongrels break

26
Many Mongrels Nginx

• Restart Nginx

/etc/init.d/nginx start

• Many Mongrels gt each instance have to be started
  restarted stopped individually
• Is it possible to manage them all as one cluster ?

27
Mongrel_cluster
28
Mongrel_cluster Nginx
Network
Dynamic requests
Nginx lt80gt
Mongrel_cluster

Static files
Mongrel lt8000gt
Mongrel lt8001gt
Mongrel ltNgt
File system
29
Mongrel_cluster Nginx

• mongrel_cluster setup

gem install mongrel_cluster
--include-dependencies

• Generate the mongrel_cluster.yml

mongrel_rails clusterconfigure -e
production -p 8000 \ gt-N 3 -c /home/projects/rails
/sites/monapp/current\ gt-a 192.168.0.100 --user
mongrel --group mongrel

• mongrel_cluster.yml

user mongrel group mongrel cwd
/home/projects/rails/sites/monapp/current log_file
log/mongrel.log port "8000 environment
production address 192.168.0.100 pid_file
tmp/mongrel.pid servers 3
30
Mongrel_cluster Nginx

• Start stoprestart all mongrels

mongrel_rails clusterstartstoprestart

• Selective start/stop/restart

mongrel_rails clusterrestartstoprestart
--only 8001
31
Mongrel_cluster Nginx

• Mongrel_cluster restart
• Stops all instances before restarting them
• Downtime
• Stop and restart mongrel instances one by one
• Old and new code running at the same time
• Requests wil be proxied to a restarting instance
• More intelligent way to restart of mongrels

32
Seesaw
33
Seesaw Mongrel_cluster Nginx

• Goals
• At least one Mongrel instance available when
  restarting
• Ensure to not mixing old and new code

34
Seesaw Mongrel_cluster Nginx

• Install seesaw

gem install seesaw

• Generate config files

cd monapp mongrel_rails seesawconfigure
--server nginx

• seesaw.yml, http_cluster/config_.yml

35
Seesaw Mongrel_cluster Nginx

• seesaw.yml

--- restart_cmd kill -HUP cat
/usr/local/nginx/logs/nginx.pid config_symlink
cluster.conf mongrel_config_path
config/mongrel_cluster.yml config_path
config/http_cluster config_files all
cluster_all.conf 1 cluster_1.conf 2
cluster_2.conf symlink_cmd ln -sf
36
Seesaw Mongrel_cluster Nginx

• In monapp/config/http_cluster

• cluster_all.conf

upstream many-mongrels server
192.168.0.1008000 server
192.168.0.1008001 server
192.168.0.1008002 server
192.168.0.1008003 server
192.168.0.1008003

• cluster_1.conf

upstream many-mongrels server
192.168.0.1008000 server
192.168.0.1008001 server
192.168.0.1008002

• cluster_2.conf

upstream many-mongrels server
192.168.0.1008003 server
192.168.0.1008004
37
Seesaw Mongrel_cluster Nginx

• Edit nginx.conf

http ... include /path/to/http_cluster/clus
ter.conf server . ..
location / proxy_pass http//many-mongrels
...

• Restart nginx

38
Seesaw Mongrel_cluster Nginx

• Start the new cluster

mongrel_rails seesawstart

• Restart the cluster

mongrel_rails seesawbounce
39
Seesaw Mongrel_cluster Nginx
switch_to_half_cluster 1 shutdown half 2
symlink to 1 webserver restart stop mongrels
2 stopping port 8002 start mongrels 2
starting port 8002 switch_to_half_cluster 2
shutdown half 1 symlink to 2 webserver
restart stop mongrels 1 stopping port 8000
stopping port 8001 start mongrels 1 starting
port 8000 starting port 8001 start cluster
symlink to all webserver restart done
40
Seesaw Mongrel_cluster Nginx
Cluster 1
Cluster 2
Switch to cluster 1 and restart cluster 2
Switch to cluster 2 cluster 1
Switch to cluster 2 and restart cluster 1
Cluster 1
Cluster 2
Cluster 1
Cluster 2
Alive mongrel instance
Restarting mongrel instance
Nginx server
41
Seesaw Mongrel_cluster Nginx

• Nginx is restarting three times

• . without losing any request !

• When nginx recieves the HUP signal
• It tests the config file (new or default)
• Re-open log files
• Listen sockets
• Runs new workers to serve all new coming
  connections
• Send graceful shutdown to old ones
• Old workers close sockets but continue to serve
  current clients
• Old workers shutdown

42
Seesaw Mongrel_cluster Nginx

• What if
• Mysql down
• Some mongrels down,
• Nginx crached,
• One of the stack processes consuming to much
  memory
• Automatic process monitoring

43
Monit
44
Monit

• Managing and monitoring utility
• Automatic maintenance and repair
• Start a process if it does not run
• Stop a process if it does not respond
• Restart a process if it consumes much resources
• and monitor files, directories for changes
  (timestamp, checksum, size)
• Send an email as an event occurs

45
Monit

• Setup

wget http//www.tildeslash.com/monit/dist/monit-
4.10.tar.gz tar zxvf monit-4.10.tar.gz cd
monit-4.10 ./configure make make install

• Copy monit config file

cp monitrc /home/projects/monitrc
46
Monit common config

• Run it as daemon at check services at regular
  interval

set daemon 180

• Set syslog logging with the daemon facility

set logfile syslog facility log_daemon

• Set mail server name for alerts

set mailserver mail.myserver.com

• Set email format

set mail-format from alert_at_monserver.comsubjec
t SERVICE EVENT at DATEmessage Monit
ACTION SERVICE at DATE on HOST
DESCRIPTION.
47
Monit process monitoring

• Monitor one Mongrel instance

mongrel 8000 check process
mongrel_8010 with pidfile /path/to/mongrel.8000.pi
d start program "/usr/bin/mongrel_rails
clusterstart -C
/path/to/mongrel_cluster.yml --clean
--only 8000" stop program
"/usr/bin/mongrel_rails clusterstop -C
/path/to/mongrel_clu
ster.yml --clean --only 8000 if failed
port 8000 protocol http with
timeout 10 seconds then restart
if totalmem is greater than 70.0 MB for 5
cycles then restart if 3
restarts within 5 cycles then
alert if cpu is greater than 80 for 3
cycles then restart
48
Monit process monitoring

• Monitor Nginx

nginx check process nginx with
pidfile /usr/local/nginx/logs/nginx.pid
start program "/etc/init.d/nginx start"
stop program "/etc/init.d/nginx stop"
if failed host 0.0.0.0 port 80
then restart

• Monitor Mysql

mysql check process mysql with
pidfile /var/run/mysqld/mysqld.pid
start program "/usr/local/sbin/nginx c
/usr/local/nginx/conf/nginx.conf etc/init.d/mysql
start" stop program
"/etc/init.d/mysql stop" if failed
port 3306 then restart

• Monit config for Nginx and mongrel_cluster

http//monitr.atmos.org/generators/nginx
49
Monit Seesaw Mongrel_clusterNginx

• Start monit

monit -c /path/to/monitrc

• Who will monitor monit ??

50
Monit Seesaw Mongrel_clusterNginx

• Edit /etc/inittab and add

mo2345respawn/usr/local/bin/monit Ic
/home/projects/monitrc

• Run monit at startup and
• With respawn monit will be automatically restated

• Dont forget to restart the machine
• or just reload inittab

telinit -q
51
Is it perfect ?

• An efficient and scalable solution
• Easy to setup, configure and monitor
• If it meets your needs . its perfect for you
• More fun with virtual machines (xen), event
  driven mongrels (Swiftiply proxy), and lets have
  faith on god to monitor them all

52
Some references

• http//wiki.codemongers.com/Main
• http//brainspl.at/
• http//synaphy.com.au
• http//blog.kovyrin.net/
• http//blog.codahale.com
• http//topfunky.net/
• http//hostingfu.com/
• http//rubyjudo.com/
• http//errtheblog.com
• http//blog.tupleshop.com
• http//www.tildeslash.com/monit/
• http//www.cyberciti.biz/
• http//mongrel.rubyforge.org/
• http//blog.labratz.net
• http//monitr.atmos.org/

53
Thank you merci ?