Activities

January 2010
M T W T F S S
« Dec   Feb »
 123
45678910
11121314151617
18192021222324
25262728293031

Nginx -Apache -Proxy configuration on Centos-Plesk server

Question,
My requirements are,

1. I have few very high traffic websites and I need to get maximum performance to the users on the peak time and my web server should serve the pages as must faster than Apache does.
2. Apache is primary server and heavy loaded and pages are loaded slowly on peak time. It using API extension (dso) to serve the php pages.
3, I want to boost the web server performance even though mod_deflate, zip mode compression are configured on website files.
4. My OS is Centos 5.3.1

Solution,
a. Configure Nginx as reverse proxy for Apache server. Nginx run on 80 and Apache run on 8080.
b. Install spawn-fcgi simple program for spawning FastCGI processes.
c. Identify the domain names which are to be proxied and serve by the Ngnix.
d. Lock the Apache domain configuration files for avoiding overwriting it through Plesk administration url.

Installing Nginx

# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/$(uname -m)/epel-release-5-3.noarch.rpm
# yum install nginx
# yum install spawn-fcgi

Use this scripts to start and stop the spawn-fcgi process easily. Link is .. here..

# wget http://machedo.com/wp-content/uploads/2010/01/php_cgi
# cp php_cgi /etc/init.d/php_cgi
# chmod +x /etc/init.d/php_cgi
# /etc/init.d/php_cgi start

Execute this to verify the process started.

# netstat -tulpn | grep :9000

Add the following lines on “/etc/rc.local” file will help to start this process each boot time.

[root@u15382543 ~]# vi /etc/rc.local
/etc/init.d/php_cgi start

NB: I am really thankful to Vivek (cyberciti.biz) to provide me a quick setup for Nginx installation easily.

2. Next step is little complicated
1. We need to edit Apache main conf file and the Plesk Apache main configuration file, replace the each virtualhost entry listening on port 80 to 8080. If you have mulitple ip’s , Skip the following step. You can easily replace all the 80 to 8080 by check this

< VirtualHost 45.52.144.13:80 > => < VirtualHost 45.52.144.1:8080 >
ServerName mydomain.com:80 => ServerName mydomain.com:8080

 # vi /etc/httpd/conf/httpd.conf

, Modify “ Listen 80 and Namevirtualhost 80 if it is not commented.

 # vi /etc/httpd/conf.d/zz010_psa_httpd.conf
 # chattr +i  /etc/httpd/conf.d/zz010_psa_httpd.conf

(file locked )
Pls find the last entry on the zz010_psa_httpd.conf which tells the configuration file location of the all the domain hosted on the server. We need all those files are locked after changing the port.
Eg: Include /var/www/vhosts/ngnix.com/conf/httpd.include this is the Apache domain conf. entry appended by Plesk on web.
2. Restart the Apache
# service httpd restart
If you get the warning like this, probably some include files are still not configured to listen
[warn] VirtualHost 49.32.113.160:8080 overlaps with VirtualHost 49.32.113.160:8080, the first has precedence, perhaps you need a NameVirtualHost directive.
You can verify it by : http://myheavydomain.com:8080

Note: If you have provided with multiple ips. It’s advise to run Nginx to use unused Plesk ip. I modify the entry on nginx config file to

listen 74.28.17.98:80;

. Pls do the same for all nginx virualhost files used. I removed the unused ip from Plesk and create a virtual lan adapter (etho:2)for using the ip

vi /etc/sysconfig/network-scripts/ifcfg-eth0:2
DEVICE="eth0:2"
IPADDR="74.28.17.98"
NETMASK="255.255.255.255"

So that you can continue to use Plesk for administrating domains and sites.

3. Configuring Nginx

 Config. directory : /etc/nginx
   virtualhosts config. :  /etc/nginx/conf.d/

my sample conf. file can be get from here

I have added/modified certain entries on it in order to serve css, js through Nginx .
user psacln;
worker_processes 4;
gzip on;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;

3. Configuring Virtualhosts on Nginx
Identify the sites which serve through Apache and Nginx. Here I uses mylitehdomain.com –> Apache and myheavydomain.com –> Ngnix.
There are three steps
a. create a proxyconfig file
b. create virtualhost entry for Ngix which serving php pages directly
c. create virtualhost entry for Nginx to proxing all the request to Apache.

a. Create a proxy.conf file and save it on “/etc/nginx/conf.d/”
# vi /etc/nginx/conf.d/proxy.conf, Contents should be like this

proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 16k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;

b. Create a virtualhost entry for nginix domain (myheavydomain.com) and entries should like this,
#vi /etc/nginx/conf.d/myheavydomain.com.conf

server {
listen 74.28.17.98:80;
server_name www.myheavydomain.com myheavydomain.com;
# access_log /var/log/httpd/nginx.access.myheavydomain.com.log;
# error_log /var/log/httpd/nginx.errors.myheavydomain.com.log warn;
large_client_header_buffers 4 8k; # prevent some 400 errors
root /var/www/vhosts/myheavydomain.com/httpdocs;
index index.php;
location / {
expires 30d;
error_page 404 = @joomla;
log_not_found off;
}
location @joomla {
rewrite ^(.*)$ /index.php?q=$1 last;
}
location ~ .php$ {
root html;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /var/www/vhosts/myheavydomain/httpdocs$fastcgi_script_name;
include fastcgi_params;
}
# Select files to be deserved by nginx
# location ~* ^.+.(swf|zip|rar|html|htm|pdf)$ {
location ~* ^.+.(zip|rar|html|htm|pdf)$ {
root /var/www/vhosts/myheavydomain.com/httpdocs/;
expires 7d;
}
}

c. Create a virtualhost entry file for proxying all requests from mylitedomain.com to Apache.

#vi /etc/nginx/conf.d/mylitedomain.com.conf

server {
listen 74.28.17.98:80;
server_name www.mylitedomain.com.com mylitedomain.com.com;
access_log off;
error_log /var/log/httpd/nginx.errors.mylitedomain.com.log warn;
location / {
proxy_pass http://www.mylitedomain.com.com:8080/;
include /etc/nginx/conf.d/proxy.conf; # the proxy.conf file
}
location ~* ^.+.(zip|rar|html|htm|pdf)$ {
root /var/www/vhosts/mylitedomain.com.com/httpdocs/;
expires 7d;
}
}

Again restart nginx service

#nginx -t
#service nginx reload

Pls note : Do not restart the nginx service unless it required. #service nginx reload will do the same.

Thing we need to do is,
Enabling mod_rpaf modules on Apache.

Begin by installing said module:
# wget http://stderr.net/apache/rpaf/download/mod_rpaf-0.6.tar.gz
# tar zxvf mod_rpaf-0.6.tar.gz
# cd mod_rpaf-0.6
# make rpaf-2.0 && make install-2.0
Add the following lines on http.conf file

LoadModule rpaf_module /usr/lib64/httpd/modules/mod_rpaf-2.0.so

If you run apache2, replace “apxs” by “apxs2″ in the command below. If apxs/2 isn’t installed on your machine, run this command first: yum install httpd-devel. Don’t worry it won’t crash down the Plesk. After the installation pls restart the apace service
# service httpd restart
# service nginx reload

Note : Last thing I changed is set 755 permission to all webfiles ( here var/www/vhosts/mylitedomain.com.com/httpdocs) becasue I have created the user “psacln” for Nginx and configured the primary group to “psacln” which would help this user to reach ” plesk web files”. Then next thing , Plesk can’t create or modify Apache config file and can’t add any files in to it. So the new domain/subdomain creation will not work for you. Pls create a vhost template file and add it manually respect to the domain creation. Once it is completed , configure it to listen on 8080 and lock the file.

Eg:
For locking files : # chattr +i /var/www/vhosts/mydomain.com/conf/httpd.include
For unlocking files : # chattr – /var/www/vhosts/mydomain.com/conf/httpd.include.

Pls excuse me because I have no other choice to run the site for a long time without having any issues even though manual creation of domain web config file, adding subdomains are also make headache to you. But there is no any way to get both Plesk and Ngix running on same server. :-)

8 comments to Nginx -Apache -Proxy configuration on Centos-Plesk server

  • ssh

    Thx a lot! The combination of Apache + Plesk is bullshit… but with Nginx as reverse proxy is better :)

    BTW.. syn flood attack`s are now more easy to support them.. :)

  • pleskuser

    Hello really nice tutorial, thank you for adding! I have one question. I have a server with multiple ip adresses. I use 2 for my namesever, and have one remaining. I am planning on using that one for nginx. Can I get some more details on how to do that exactly.

    Both apache and Nginx will be listening on port 80, will this not cause a conflict?

  • admin

    Hello,

    We can’t run both application running on a single port.

    You can use thrird one for Nginx since you have given 3 ip address. Name servers using the port 53 and 80 is still not used. So that you will get these 2 ips on Plesk and it can be used for Apache which running on the port 80 and can run Ngnix using the port 80.

    my recommend combination is,

    You may configure/add only two ips on Plesk Control panel and leave the last ip for Nginx and configure it manually using ssh.

    Thanks,
    Liju

  • pleskuser

    Thank you for the reply. I removed the 3th ip from plesk.

    I created this:

    vi /etc/sysconfig/network-scripts/ifcfg-eth0:2
    DEVICE=”eth0:2″
    IPADDR=”xx.xx.xx.xx”
    NETMASK=”255.255.255.xxx”

    ———————————-
    then in ssh:

    #ifup eth0:2
    # ifconfig

    Now the new Adapter is created
    ———————————

    And in nginx.conf

    #
    # The default server
    #
    server {
    listen xx.xx.xx.xx:80;
    server_name _;

    —————————————-

    Now when I:

    #service start nginx

    I get the following errors:

    Starting nginx: [emerg]: bind() to xx.xx.xx.xx:80 failed (98: Address already in use)

    ———————————-

    Do you know what the problem is?

  • admin

    Hello,

    Don’t worry about this. Keep the backup of each Apache file you are going to edit.

    Edit the ngnix config file and configure Ngnix to listen only on the third ip you given rather than all the network interface connected.

    It should be like this after changing,

    server {
    listen 4.28.17.9:80;
    server_name _;

    -Liju

  • admin

    Also find the entry of ngnix ip is used on on each httpd.include files under the respective domains.

    /var/www/vhosts/include/httpd.include and set the Apache listen only to both Plesk Ips by adding

    Listen x.x.x.x:80
    Listen x.x.x.x:80

    Restart network and apache

  • pleskuser

    Hello it is working! Nice tutorial! This is very good to set up, I got a performance boost of 200%.

    I still have one remaining question. I can enable using fastCGI in plesk for the domain. Should I enable fastCGI in Plesk also, or is using spawn-cgi enough.

    Because when I look in my phpinfo i see that server API with is still Apache. Only when i enable fastCGI on the domain I see that server API is cgi/fastcgi.

  • [...] using the power of cgi script to perform the php compilation speed. Obviously the combination of Nginx+ php-cgi module performs much better for small high speed websites. But this new technology php-fpm would also [...]

Leave a Reply

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>