This document covers the following issues:

• How spammers work
• Identifying the Server
• Detecting Spammer on a Mail Server
• Detecting Spammer on a Web Server
• Further Steps

How Spammers Work

There are several ways spammers can use your system to send spam, but these are the most typical:

1. Through mail server
   • Directly from the mail account - usually you can easely find who does it, by checking From: parameter in the mail queue. (You should be doing it while spam is in process).
   • By installing cgi script on your mail box, that relays email through your mail server. Your system is still properly configured. You have to allow other user to send mail from webservers for confirmation emails and formmail scripts.
2. Directly from the webserver. - Some users would create an account and install the script that will be working as SMTP itself. It will be delivering mail directly, without using your mailserver.

Identifying the Server

The easiest way to detect the spammer is while the spammer sends out emails. Usually spammers will be using your server for as long as they can - so once you get a complaint - get to it right away.

There are two signs that your server is used for spamming:

1. Your mail server is unusually slow
2. You get a complaint from users/upstream provider

If you get the spam message, analyze its header. It would usually contain one of your mail or webserver IP address. This way you can tell which server the spammer uses.

Detecting Spammer on a Mail Server

If the spam is sent out from the mailserver, you have to monitor the queue:
# cd ~qmaild/queue/mess -- outgoing message queue
Call this script many times, like in a loop or something (this would look for Received lines in messages). For example, if you know that the spam is sent from from "anonymous@domain.com", use it in the script, to find messages with that text in the message.

# for i in `grep -r "anonymous@domain.com" *|sed
's/^\(.\/.......\):.*$/\1/g'
| sort | uniq`; do echo $i; grep "^Received:" $i; done

It will print out lines that start with Received for all the messages that has line anonymous@domain.com

If you want to get a complete message to the screen, use:

		
# for i in `grep -r "anonymous@domain.com" *|sed
's/^\(.\/.......\):.*$/\1/g'
| sort | uniq`; do echo ''

The Received line usually looks similar to the following:
Received: (qmail 4868 invoked by uid 101); 14 Feb 2002 01:55:27 -0000

uid is user id (from password file) on one of your web servers. Go to web servers, and search for that uid, this way you will find the username of the user, and you can find the account in H-Sphere.

Besides, you may check if any of your customers executes CGI script that connects directly from your webserver to a remote smtp hosts.

Usually if you execute:
netstat -n |grep :25

on that server, you will see a bunch of connections with outgoing port 25.

If this is the case, try executing:
ps auxww|grep -v httpd|grep -v root|grep -v mysql

and you will see a bunch of processes (usually perl) executed by one user. Due to the fact that it takes some time to connect to a remote system, and
send a message, those scripts hang in the memory and are easy to be observed.

If the scripts are php based, they are harder to detect.
Try doing the following:
cd /proc
ls -l */cwd|grep /hsphere/local/home

If you see any users directory displayed large number of times, he is probably the offender.

There are two ways to prevent such spam from re-appearing:

#1 Setup firewall on each web server, so that it would forbid any connection to outgoing port 25, but your mail server. telnet xyz.xyz.com 25 - should be blocked but telnet mail.inethosts.com 25 - should be opened

#2 Close port 25 for all your servers, but mailserver on the level of the switch.

Detecting Spammer on a Web Server

If the spam is sent out from the web server, do netstat -n, you should see bunch of outgoing connections to port 25. You can find who is doing that by ps -auxww. You will usually see a bunch of perl interpreters running, see who the user is, and what are the scripts that he is running. Usually the scripts fork a bunch of processes that are actually used for spamming.

Further Steps

Once you figure out who sends the spam, you should suspend the account. In most cases, spammers will use stolen credit cards, and it should be 100% in any case against your AUP.

To prevent such style of attacks, enable iptables (ipchains) on your server, and prevent any outgoing connections to port 25, to any IP, but your mailserver IP.

You might also want to install SpamGuad on the mail server. The installation takes the following steps:

1. Download SpamGuard
2. Exectue tar xfz spamguard-x.x.tar.gz
3. Go to /root/inst/spamguard-x.x/
4. Read INSTALL, README files
5. Install SpamGuard following the instructions in INSTALL, README files

Warning: For the time being, there is no effective way of combining mailing lists and spamguard protection. You need to configure spamguard manually by setting the maximum allowed number of recipients.