Activities

February 2015
M T W T F S S
« Jan   Mar »
 1
2345678
9101112131415
16171819202122
232425262728  

Secure your NGNIX hosting along with SEO tips

Now most of the rails applications as well as php sites are moving to Nginx server due to it’s light weight and fast page loading features. Recently I had launched a php application application on Nginx web server.

SEO recommended webserver settings

1. Create appropriate file having the entries for robot.txt and sitemap.xml

2. URL SEO Friendly

To acheve this you may need to add this lines ” try_files $uri $uri/ /index.php?$args;” under virtualhost server segment.

3. HTML Compression/GZIP Test

To enable Gzip compression, you may need to add following lines in the Main configuration file (/opt/nginx/conf/nginx.conf) under “http” segment.

  gzip on;
    gzip_min_length  1100;
    gzip_buffers  4 32k;
    gzip_types    text/plain application/x-javascript text/xml text/css;
    gzip_vary on;

4. Page Cache Test

You need to enable APC caching on your php compiler. In this server, i’m using php-fpm to compile the php script. So I has to restart this service after the APC installation.

#yum install php-pecl-apc.x86_64
      #/etc/init.d/php-fpm restart

Note : You may need to increase the APC cache size to more than 8MB if you are using it for generating big documents (/etc/php.d/apc.ini) and value is apc.max_file_size=6M
.

5. URL Canonicalization

Canonicalization describes how a site can use slightly different URLs for the same page (for example, if http://www.example.com and http://example.com displays the same page but do not resolve to the same URL). If this happens, search engines may be unsure as to which URL is the correct one to index.

6. Directory Browsing

You need to disable directory browsing to deny access to your files over browser directly. Add ” autoindex on;” inside your virtualhost server segement

 autoindex on;

7. Deny Libwww-perl Access : –

Check if your server allows access from User-agent Libwww-perl. Botnet scripts that automatically look for vulnerabilities in your software are sometimes identified as User-Agent libwww-perl. By blocking access from libwww-perl you can eliminate many simpler attacks.
Add the following lines in your virtualhost file, This should be copied under “location /” area

map $http_user_agent $limit_bots {
     default 0;
     ~*(google|bing|yandex|msnbot) 1;
~*(AltaVista|Googlebot|Slurp|BlackWidow|Bot|ChinaClaw|Custo|DISCo|Download|Demon|eCatch|EirGrabber|EmailSiphon|EmailWolf|SuperHTTP|Surfbot|WebWhacker) 1;
     ~*(Express|WebPictures|ExtractorPro|EyeNetIE|FlashGet|GetRight|GetWeb!|Go!Zilla|Go-Ahead-Got-It|GrabNet|Grafula|HMView|Go!Zilla|Go-Ahead-Got-It) 1;
~*(rafula|HMView|HTTrack|Stripper|Sucker|Indy|InterGET|Ninja|JetCar|Spider|larbin|LeechFTP|Downloader|tool|Navroad|NearSite|NetAnts|tAkeOut|WWWOFFLE) 1;
~*(GrabNet|NetSpider|Vampire|NetZIP|Octopus|Offline|PageGrabber|Foto|pavuk|pcBrowser|RealDownload|ReGet|SiteSnagger|SmartDownload|SuperBot|WebSpider) 1;
~*(Teleport|VoidEYE|Collector|WebAuto|WebCopier|WebFetch|WebGo|WebLeacher|WebReaper|WebSauger|eXtractor|Quester|WebStripper|WebZIP|Wget|Widow|Zeus) 1;
~*(Twengabot|htmlparser|libwww|Python|perl|urllib|scan|Curl|email|PycURL|Pyth|PyQ|WebCollector|WebCopy|webcraw) 1;
 }

if ($limit_bots = 1) {
              return 403;
            }

8. Server Signature

Add this line on your main Ngnix configuration file (/opt/nginx/conf/nginx.conf) under “http” area.

server_tokens off;

9.IP Canonicalization

To achieve this goal, you may redirect the site hosted IP to the same domain. ie You site could not be accessible over the IP because it will lead to create duplicate content in other search engines. You may need to add following lines in Ngnix virtual host config. file.

if ($host !~* ^(www))
{
   rewrite ^/(.*)$ $scheme://www.mydomain.com/$1 permanent;
}

The following are showing my custom virtual hosting file (/opt/nginx/conf.d/www.mydomain.conf) which I created for WordPress application.

server {
    listen       80;
    server_name  162.2.3.7  mydomainapp.com www.mydomain.com;

    #charset koi8-r;
    #access_log  /var/log/nginx/log/host.access.log  main;
      autoindex on;
      index index.php;
# enforce www (exclude certain subdomains)
if ($host !~* ^(www))
{
   rewrite ^/(.*)$ $scheme://www.mydomain.com/$1 permanent;
}
    location / {
        root    /home/mydomain-www/public_html;
        autoindex off;
        index  index.html index.php;

map $http_user_agent $limit_bots {
     default 0;
     ~*(google|bing|yandex|msnbot) 1;
~*(AltaVista|Googlebot|Slurp|BlackWidow|Bot|ChinaClaw|Custo|DISCo|Download|Demon|eCatch|EirGrabber|EmailSiphon|EmailWolf|SuperHTTP|Surfbot|WebWhacker) 1;
     ~*(Express|WebPictures|ExtractorPro|EyeNetIE|FlashGet|GetRight|GetWeb!|Go!Zilla|Go-Ahead-Got-It|GrabNet|Grafula|HMView|Go!Zilla|Go-Ahead-Got-It) 1;
~*(rafula|HMView|HTTrack|Stripper|Sucker|Indy|InterGET|Ninja|JetCar|Spider|larbin|LeechFTP|Downloader|tool|Navroad|NearSite|NetAnts|tAkeOut|WWWOFFLE) 1;
~*(GrabNet|NetSpider|Vampire|NetZIP|Octopus|Offline|PageGrabber|Foto|pavuk|pcBrowser|RealDownload|ReGet|SiteSnagger|SmartDownload|SuperBot|WebSpider) 1;
~*(Teleport|VoidEYE|Collector|WebAuto|WebCopier|WebFetch|WebGo|WebLeacher|WebReaper|WebSauger|eXtractor|Quester|WebStripper|WebZIP|Wget|Widow|Zeus) 1;
~*(Twengabot|htmlparser|libwww|Python|perl|urllib|scan|Curl|email|PycURL|Pyth|PyQ|WebCollector|WebCopy|webcraw) 1;
 }
if ($limit_bots = 1) {
              return 403;
            }

       allow all;
       try_files $uri $uri/ /index.php?$args;
    }

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
location ~ \.php$ {
 root            /home/mydomain-www/public_html;
 fastcgi_split_path_info ^(.+\.php)(/.+)$;
 fastcgi_pass    unix:/var/run/php-fpm/php-fpm.sock;
 fastcgi_index   index.php;
 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 include      fastcgi_params;
 }

}

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>