Activities

August 2020
M T W T F S S
« Apr    
 12
3456789
10111213141516
17181920212223
24252627282930
31  

Configure Postfix relay server using Amazon SES

Recently I had a requirement to provide a which sent from a server should be reached in user INBOX. certain emails were not been go through the Google App email list. As far as I can see that, the application uses standalone smtp installed on the hosted server which ip/network was not been added in spf list. So this email should have low IP reputation and hence will not be reached on users INBOX all the time. In this scenario, I has to configure a relay server which will be using authorized network/ip sources.

First step

1. Create SMTP users in Amazon SES account. See the following screen,

SES1

SES-2

SES-3

2. Modify the Postfix main configuration file
Now you need to install certain SASL packages for enabling postfix sasl authentication mechanism.

# yum install mailx cyrus-sasl cyrus-sasl-plain cyrus-sasl-md5
      #yum install postfix

Then add the following lines at the bottom of the postfix main configuration file (/etc/postfix/main.cf) file.

relayhost = email-smtp.us-east-1.amazonaws.com:587
smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_use_tls = yes
smtp_tls_security_level = encrypt
smtp_tls_note_starttls_offer = yes
smtp_sasl_mechanism_filter = PLAIN LOGIN
smtp_generic_maps = hash:/etc/postfix/generic

relayhost : This is the hostname which postfix push the emails. This will be different based on your SES regional choice.
smtp_sasl_password_maps : This is the file where we put the Amazon SES credentails. Pls note this is not the Access Key and Access ID.

Note: You should create a new user within the Amazon SES settings window itself. Otherwise this IAM account will not be honored.

smtp_generic_maps : This is one of the important settings since each of SES account has to verified the from header domain name and email address. So that you can not impersonate the emails. By default if you send an email from console, FROM address will be root@finaconn-web01.localdomain. So that Amazon will not permit to send such emails where FROM field was not verified.

3. Creating SASL password file
Grab your Amazon smtp server name, user and password from the first step and create a file and arrange the data in a below manner
smtphost:587 username:password

#cat /etc/postfix/sasl_passwd
email-smtp.us-east-1.amazonaws.com:587 AKIAIBLAAAAAJCA:Alae0CZMzINNNNNNEEEEEAAmYv/pUa

create postfix maping database

#postmap /etc/postfix/sasl_passwd

4. Rewrite FROM address of local domain name to authorized domain name.

Create a file (/etc/postfix/generic) and put all the possible values to be falls on “FROM email header ” which is attaching to each email.

[root@web04 ~]# cat /etc/postfix/generic | tail -n 6
root@fc-web04   notifications@mydomain.com
*   notifications@mydomain.com
root@web04.localdomain web04-notifications@mydomain.com
root@web03.mydomain.com web03-notifications@mydomain.com
root@web05.localdomain web05-notifications@mydomain.com

Restart postfix service,

[root@web04 ~]# service postfix restart
Shutting down postfix:                                     [  OK  ]
Starting postfix:                                          [  OK  ]
[root@web04 ~]#

Now you are completed the configuration for sending email from your local postfix smtp server. which is relaying emails with Amazon SES account.

Now it’s the time to configure your server to become the hub of sending outbound emails from all your local email servers. You have to do two things on this server

On Master Outbound Host end

a. Set mysqdomain nane to your domain name (mydomain = mydomain.com)
b. Set this value to all (inet_interfaces = all)
c. Set mynetworks to send emails (mynetworks = 192.168.10.0/24, 127.0.0.0/8). Mostly you have to uncoment the line and edit as needed.
d. Restart your postfix server to become a good relay Master Host.

[root@web04 ~]# service postfix restart
Shutting down postfix:                                     [  OK  ]
Starting postfix:                                          [  OK  ]
[root@web04 ~]# netstat -nlp | grep ":25"
tcp        0      0 0.0.0.0:25                  0.0.0.0:*                   LISTEN      3038/master
[root@web04 ~]#
[root@web04 ~]#

Now the postfix is listening to all the network interface.

6. Config. changes on Relay host End
The only thing what you have to do is, to uncomment the line relayhost and add your new server IP in /etc/postfix/main.cf file.

relayhost = 192.168.2.10
 service postfix restart

Moral of the story : It has been observed that Amazon SES is a lazy email server as we can not send bulk mail easily and should have some delay around 30 – 90 sec delay got in email response. Also I found Amazon documentation of the same purpose was a buggy one. Will not work for me 🙁

File exists: mod_lua: Failed to create shared memory segment on file /tmp/httpd_lua_shm

This is the one strange error I’m getting from one the staging server. When I’m trying to restart the web server , it’s always showing, pid exists. stopped. It’s a kind of weird incident. As far I know that there are some part of Apache module libraries /programs are loaded in memory and Apache service restart could help to release those program from memory.

[Mon Feb 01 05:23:38.098868 2016] [:emerg] [pid 30647] AH00020: Configuration Failed, exiting
[Mon Feb 01 05:25:16.270747 2016] [suexec:notice] [pid 30668] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Feb 01 05:25:16.289871 2016] [auth_digest:notice] [pid 30669] AH01757: generating secret for digest authentication ...
[Mon Feb 01 05:25:16.290335 2016] [lua:error] [pid 30669] (17)File exists: mod_lua: Failed to create shared memory segment on file /tmp/httpd_lua_shm.30669

I tried to delete the listed files from /tmp location. Noting gonna work. Some while after Googling, it has come to know that reload command will did the trick. It’s worked like a charm. 🙂

[root@sh-web02 ~]# service httpd reload
Reloading httpd:                                           [FAILED]
[root@sh-web02 ~]# service httpd status
httpd is stopped
[root@sh-web02 ~]# apachectl restart
httpd not running, trying to start
[root@sh-web02 ~]# service httpd status
httpd (pid  30708) is running...

Funny hacks which observed recently

I had two incidents those are reporting some malfunction with their platform. The best one which I found that a Google App account hack.

Incident A

Once customer complaint that he does NOT receiving any emails on his Google account even though he could able to login and send emails from the web interface without any issues. You may pls note he is taking about Google email account. What do you thinking now ??? Are you a Gmail holder and experienced something similar.

As a Google Apps Admin, I had checked the google apps email by using another user account and found no issues. I did also send one email to the customer who complaining the issue. Finally I had reset his account password and try some email to this email address. I did not receive any emails yet. Strange .. very strange !!! I could not blame Google for this issue and do not have any thoughts of Google smtp are bad one. Can you guess what will be the issues ? ….. I’m starting checking his all the inboxes … finally …. I observed that his trash box has recent emails.. Great !! :- this was the gear up of having more thoughts on the issues..

Core issue
1. His email address was compermised and was been used simple password like name123456 as password.
2. His email account has been used for sending bulk emails. But user do not know thing activity because he does not reeving any failure delivery emails.
3. The hacker set a Google filter for deleting any emails which coming towards his INBOX.
4. Everything seems normal when a user accessing the gmail.

It’s a simple hacking but from a genious brain which I called Operator hacking.

hack_one

Incident B

The second thing was about .htaccess

When a user access the site from Google search engine, they got redirected to another porn site link. But when we access the site directly, there will not be any issue. I had asked a Dev team to checkout the application source code for the changes as they are using php scripts. They responded with no code was modified in files. So I had a false thoughts that if the site was compromised, hackers can submit wrong sitemap url and Google using that url for this redirection.

One of the dev team found that issue of redirection. The hacker put some additional mod_rewrite rules for redirecting site only which come any SEO reference and put one additional file.

RewriteCond %{ENV:REDIRECT_STATUS} 200
RewriteRule ^ - [L]
RewriteCond %{HTTP_USER_AGENT} (google|yahoo|msn|aol|bing) [OR]
RewriteCond %{HTTP_REFERER} (google|yahoo|msn|aol|bing)
RewriteRule ^(.*)$ baritone-crossable.php?$1 [L]

He does not crack anything on the existing site and user does not having any issue.

For both two scenarios, We can learn a common fact is neither for the hackers did not make any disturbance on existing platform (Google account and SEO redirection) but they gain the purpose of making marketing by using someone resources.

Git Error : remote HEAD refers to nonexistent ref, unable to checkout.

We are using GitLab application to manage the version for all the project. One of the dev team has got an error while he is trying to clone the existing git repo for the first time.

@erw-165:~/public_html$ git clone git@github.mycompany.com:COMadmin/myproject.git
Cloning into 'myproject'...
remote: Counting objects: 5552, done.
remote: Compressing objects: 100% (4453/4453), done.
remote: Total 5552 (delta 1189), reused 5218 (delta 979)
Receiving objects: 100% (5552/5552), 77.31 MiB | 179.00 KiB/s, done.
Resolving deltas: 100% (1189/1189), done.
Checking connectivity... done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.

I’m also seeing this kind of error for the first time. I had restarted the gitlab service as a false thought of if there is something with GitLab rails application. But that does not helped me much. So I decided to explore it more.

When I was checking the project reposity over GitLab UI and found that there is no master branch has been created there under “FILES” area. This was the culprit of error.

Solution

Create a new branch named master in Git and try to do the checkout again. It will work flawlessly.

Have a great day ::).

Ubuntu 14.04 : Setting up vFTPd server easily

I have an urgent requirement to setup a ftp server on one the Ubuntu desktop for a interim solution. I have installed vsftpd easily using apt command and started it.

root@web-119:/etc/apt# apt-get install vsftpd
root@web-119:/etc/apt# netstat -anlp | grep "21"
tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN                     12362/vsftpd

Everything seems good and I had to setup chroot jail enviorment on this ftp server. I just immediately comment out the line “chroot_local_user=YES” and restart the vsftpd service.

But I’m hitting the error as shown below

Thu Jan 14 11:53:34 2016 [pid 12488] [demouser] FAIL LOGIN: Client "192.168.0.14"
Thu Jan 14 11:53:40 2016 [pid 12511] CONNECT: Client "192.168.0.14"

Finally I found from Google, that this vsftpd daemon using vsftpd pam authentication similar like standalone ftp user credentials. Like what Filezilla server did. So I have to disable that auth type to ftp . The line is become on /etc/vsftp.conf

pam_service_name=ftp

Sometime you would hit by “500 OOPS: vsftpd: refusing to run with writable root inside chroot()”.

reason : VSFTP will not allow users to have write permission on their root folder. So you have to remove “write” permission from that user To achieve this, execute this command

chmod a-w /home/user2/

Before
drwxr-xr-x 2 liju liju 4096 Jan 14 16:43 liju

After
#chmod a-w /home/liju/
dr-xr-xr-x 2 liju liju 4096 Jan 14 16:43 liju

The following values should be updated on “/etc/vsftp.conf” file to complete a perfect ftp server.

a. Uncomment write_enable=YES
b. Set FTP banner name : ftpd_banner=Welcome to MyOnline FTP service
c. Enable chroot jail : chroot_local_user=YES
d. Reset pam authentication to ftp : pam_service_name=ftp
f. Restart vsftpd service.
g. Execute this command on every user creation time : chmod a-w /home/users_home_folder

SSH service fails to restart , Not allowed because not listed in AllowUsers

Recently I have created a new user in one of my linux server and try to verify the credentials over ssh . But I used to get ” Permission denied” message even though verified the credentails from the local login itself. So it is something like weird situation and seeing the first time in my life. After checking the ssh log file (/var/log/secure) it is showing some detailed error report as shown below,

Nov 26 00:22:28  sshd[19434]: User prod-write from 18.72.164.78 not allowed because not listed in AllowUsers
Nov 26 00:22:28  sshd[19435]: input_userauth_request: invalid user prod-write
Nov 26 00:22:36  sshd[19434]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=18.72.164.78  user=prod-write
Nov 26 00:22:37  sshd[19434]: Failed password for invalid user prod-write from 18.72.164.78 port 54078 ssh2

I searched the /etc/ssh/sshd_config file and did not see any “AllowUsers” is set over there. Then I tried to update the SSH package and restart the ssh service. But I could not restart the ssh as it is fails while stopping it.

[root@web13 ~]# /etc/init.d/sshd restart
Stopping sshd:                                             [FAILED]
Starting sshd:                                             [  OK  ]

I was seen that from the log((/var/log/secure)) below entries

Nov 26 23:11:43 sshd[2111]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use.
Nov 26 23:11:43 sshd[2111]: fatal: Cannot bind any address.
Nov 26 23:13:31 [2131]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use.

SSH-iissue

So I fall in trouble since I do not have any option to go since ssh is the only way to connect the server and I can not kill that process.

Solution.

I have set a cronjob to restart ssh service on every 5 minutes and verified that it is working. After that I have killed all the ssh process by “killall -9 sshd”. After 4 minutes later I could able to login to the server either over root and the user which I created successfully.

Moral of the story

You can see from the ssh log file, there were lots of connection refusal requests are coming towards the server. This is the only reason while ssh is not being restarted since it is busy with refusing ssh connection. So I had late to identify it sadly ::( So set ssh firewall rules now itself and manage the access.

Cheers !!