Home           RSS           Search

October 15, 2013





Spamd tarpit and greylisting daemon

anti-spam tutorial for FreeBSD



Spamd is a fake sendmail daemon which rejects mail from spammers or compromised systems. It is designed to be very efficient so it does not impact the receiving machine, but instead puts the burden of proof on the sending machine. This is done by making the sending machine wait a period of time before the delivery of their first piece of mail. This wait period is called the grey listing time and it makes sure the sender is actually a mail server respecting mail RFC's. If the remote server is a known spammer then Spamd will tarpit them and stutter their connection for as long as they are willing to stay connected.

At Calomel.org we are able to go from over 900 spam attempts per day hitting the mail server to less than one actually getting through (that host was subsequently blacklisted by a baysian filter). The spammers are still knocking at the door, but they are talking to spamd and not the real mail server. Spamd is a lot more efficient than a mail server because it does not do DNS lookups or spam blacklist checks. It just expects the remote server to respect RFC protocols and try to deliver their mail again before being white listed. Simple, but powerful.

Setting up Spamd is incredibly easy and will not take more then a few minutes. Before we show the steps of setting up the daemon lets go through the basics of how spamd works, what the database entries look like and what you can expect from spamd.



How does Spamd work?

Spamd works like this: When a remote mail server wants to deliver mail to your server it connects and sends the "From:" and "To:" headers. Spamd takes this information and queues it in a database (/var/db/spamd). The entry in the database is called a "tuple" made up of the three entries of remote ip address, the "From:" and the "To:" email header fields. Spamd does _not_ accept any other part of the email, like the body or headers, thus it reduces bandwidth usage significantly.

During the first connection and every subsequent connection until a host is white listed spamd will stutter sending and accepting data to 1 character per second for the first 10 seconds. After this period, the rest of the connection will go at full speed. The purpose of stuttering is to take up the time of the remote system because spammers get paid by the amount of mail the deliver. If we can slow their systems down then they make less money. Some spammers know about this stuttering and they will simply disconnect and go away. This is our goal.

Once we have the "tuple" in the database spamd sends the remote server the error "451 Temporary failure, please try again later." This means the remote server will need to try to send the email again at a later time. Legitimate mail servers like Sendmail, Postfix, Qmail, and proprietary mailers like Gmail and Hotmail are RFC compliant. They will send the email again and again until a timeout period (between 1 and 4 hours) at which the email is considered undeliverable. There are some exceptions to this rule by big mails like Gmail, but we will cover those later on this page.

Spamd's default config (-G25:4:864) is setup to make the remote server retry delivering the email for at least 25 minutes. After that 25 minute mark and before the grey timeout of 4 hours the remote server must attempt to deliver the email to be white listed. If the remote server does not try again before the timeout of 4 hours the database entry is deleted and they must go through the entire process again. If they try again between 25 minutes and 4 hours of the first connection they are white listed and their connection goes directly to the mail server for any future connections. A white listed connection will stay in the database for 864 hours (36 days) at which time if they have not connected again at least once they will be removed from the database.

When accepting mail spamd considers sending hosts to be of three types:

blacklisted hosts are redirected to spamd and tar pitted. These are also called "trapped" hosts. They are communicated with very slowly at 1 character per second for the entire session to consume the sender's resources. Mail is rejected with either a 451 (default) or 550 error message making the sender machine use up CPU time and memory to queue the message. A blacklisted host will not be allowed to talk to a real mail server and will be stuttered at every time they connect.

greylisted hosts are redirected to spamd, but spamd has not yet decided if they are likely spammers. They are given a temporary failure message by spamd when they try to deliver mail. This means they must wait a period of time before they are allowed in. A real mail server will try to deliver its mail over and over for a period of time, sendmail for example is 5 days before it gives up.

whitelisted hosts do not talk to spamd. White listed hosts are sent to the real mail server and are not delayed at any time. This is because they have already gone through the grey listing method and are now considered a "mostly" trusted server. The remote host must still go through all of the normal mail daemon checks you may have setup on you MTA. Other checks you may have are like Spamhaus real time blacklists (RBL) or hostname verifications.



What do the database entries look like?

To see the entries in the Spamd database you can execute the command "spamdb" without arguments. This will print out all of the entries in the database. You can use sort to help put the entries in order by typing "spamd | sort". All dates and times are in Unix time. Use the command "date -r <time>" to convert into a more human readable format.

A GREY "tuple" entry is keyed on the ip, From: and To: values. If the remote server has the same ip and sends a different From: or To: email address then it is a different "tuple" and will entered as a different database entry. Here is an example of a single GREY listed database entry of spamd:

GREY|87.182.96.240|p57b660f0.dip.t-dialin.net|<tennis5@pctcu.com>|<spamd@your_host.org>|1200326584|1200337384|1200337384|1|0

A GREY listed entry is a "tuple" that has not been whitelisted yet.



A WHITE listed entry is an ip address that has passed the spamd checks and can deliver mail without interruption.

WHITE|192.43.244.163|||1200074938|1200076797|1203439747|2|131

A WHITE entry is only keyed on an ip address. All email is sent to the real mail server bypassing spamd.



If you are using OpenBSD or FreeBSD then you may also want to check out the highly secure mail daemon on our OpenSMTPD "how to" (smtpd.conf) Guide.







Example 1: Protect a real mail server by front ending it with Spamd

The first example is the most useful. We are going to put spamd in front of a real mail server. This will protect the mail server and allow it to deal with the real mail instead of the deluge of spam. The idea is: if the host is whitelisted they will go to the mail server. If the host is grey or black listed they will goto spamd.

First, put the following entries into the /etc/rc.conf.local file.

## Add the following to /etc/rc.conf.local
#
spamd_flags="-4 -G25:4:864 -h your_hostname.org -l127.0.0.1 -n \"Sendmail 8.11.4/8.11.1\" -S10 -s1 -v -w1"
spamd_black=NO
spamlogd_flags="-I -i lo0"

These will start spamd with the following options:



How long should the pass time be? (-G25:4:864 "default 25 minute pass time")

The pass time is the amount of time we will grey list an ip address's "tuple" before they must connect again to be white listed. This pass time is used to keep spammers out by delaying mail delivery for newly seen ip addresses. Spammers are impatient as they get payed per delivered email, not per email delivery attempt. Spamd is used to delay their delivery attempts in order to reduce their profit margins.

According to RFC 2821 a mail server must try to deliver the same piece of mail for no less than 4 to 5 days using a 30 minute interval between retries. It is possible that a remote mail server is down so the sending system should retry again and again to deliver its mail, but not be abusive. This is a good place to start as most spammers will attempt to deliver their mail for much less time (see the following scrollable window). We do not suggest going higher than 60 minutes because some web mail sites like MSN will stop delivering mail after one hour; this is violation of RFC.

We suggest setting the pass time to as high as you are comfortable with. Use a time between 10 and 55 minutes. You are welcome to set it as low as 2 minutes, but it is possible that some spammers might get white listed. After setting up spamd take some time, go through the logs and look for patterns. Adjust the pass time as necessary.

The following scrollable window shows a sample of verified spammer ip addresses which connected to calomel.org, how many times they connected and for how long (duration). After the duration they disconnected and were not seen using the same "tuple" (remember a tuple is an ip, From: address and To: address) for at least 4 hours. For example, the ip 91.147.198.157 made 10 attempts using 10 different "tuples" in 12 minutes. Since they did not try to deliver the same "tuple" past the "pass time" of 25 minutes they were never white listed.

attempts	ip address 	   duration
  10		 91.147.198.157     12 min
  10		 213.203.110.103     8 min
   6		 81.195.17.214       2 min
   5		 91.122.244.158      2 min
   5		 88.231.111.22      14 min
   5		 87.69.100.123      12 min
   5		 86.70.206.253      15 min
   5		 86.49.88.124       11 min
   5		 85.110.92.70       14 min
   5		 84.101.16.108      12 min
   5		 83.27.72.227       17 min
   5		 82.210.155.134      6 min
   5		 81.190.157.126      5 min
   5		 78.37.199.149      16 min
   5		 77.41.85.134        6 min
   5		 70.67.160.112       6 min
   5		 61.19.30.194        6 min
   5		 61.187.188.110      3 min
   5		 60.21.69.179        3 min
   5		 60.14.136.106       7 min
   5		 59.38.200.134       3 min
   5		 58.18.85.138        6 min
   5		 222.91.172.250      5 min
   5		 221.216.37.155      3 min
   5		 220.76.48.226       6 min
   5		 219.159.63.15       6 min
   5		 219.130.66.30       5 min
   5		 218.23.86.11        6 min
   5		 218.201.44.54       5 min
   5		 200.104.24.70       8 min
   5		 190.157.45.66       3 min

Using a "passtime" of 25 minutes should work fine. Test out how people connect to your server and increase or decrease as necessary. We prefer our settings to be on the high side ( more than 25 min) as only one in 1000 spammers might get through. The negative is it will take longer for a new ip to get white listed and longer for that first piece of mail to be delivered. Remember, once the ip is whitelisted they go directly to the real mail server and are not delayed.



What about spamlogd? Using the "-I -i lo0" argument.

Spamlogd is the white list updater. It watches mail from ip addresses using the log facility of pf. Spamlogd updated the spamd-white list table used in the rules of pf.conf. When hosts connect, spamlogd sees the successful delivery and updates the spamdb database for that ip. It advances the last column counting how many emails were delivered and it resets the whiteexp entry to give the ip another 864 hours (36 days) to connect again. In order for spamlogd to work properly you must add the "log" directive onto the pf rules that pass connections to and from your real mail server.

As of OpenBSD v5.1, spamdlogd will white list ip address that connect to your real smtp daemon (sendmail, postfix, etc.) _AND_ hosts that your mail server connects to. The idea being that if you send mail to a mail server then you would want that mail server to reply without having to go through spamd.

The problem is if you are NAT'ing connections from your firewall to randomize outgoing ports like we do in the Calomel.org pf.conf. Lets say a spammer sends us mail. We accept the connection and then we NAT the return TCP connection back to the spammer. By NAT'ing the outgoing return connection spamlogd sees this as if our box is initiating a connection back to them. Spamlogd will then white list the spammer on their very first attempt! When the spammer connects a second time they are already in the spamd-whitelist table and they sail through to the real mail server. Not good. So, in order to be safe execute spamlogd with the "-I" argument, for example "spamlogd -I -i lo0". This specifies spamlogd is only to whitelist _inbound_ SMTP connections.



IMPORTANT: If you need some assistance with Pf then check out our Pf firewall config "how to" which explains all the options of pf and includes fully working examples.


Setting up pf

Next, we need to setup Pf to send mail that has been whitelisted to the real mail server and all other mail to spamd to be greylisted. This will involve setting up a persistent Pf table called <spamd-white> and redirecting mail depending on the hosts listed within.

The following rules will do this; if the host is in the <spamd-white> table the connection goes to real mail server listening on localhost port 25. If the host ip address is not in <spamd-white> then its connection is redirected to spamd. Very simple and straight forward. Note: we will not be using the spamd blacklist table <spamd-greytrap>. The idea is if the remote ip is not in the whitelist table it will never get to the mail server.

## FreeBSD Pf syntax. Add the following to /etc/pf.conf
################ Tables ####################################
 table <spamd-white> persist
#
################ Translation #################################
 rdr on $ExtIf inet proto tcp from !<spamd-white> to ($ExtIf) port smtp -> lo0 port spamd
 rdr on $ExtIf inet proto tcp from  <spamd-white> to ($ExtIf) port smtp -> lo0
#
################ Filtering #################################
 pass in log on $ExtIf inet proto tcp from  <spamd-white< to lo0 port smtp  keep state
 pass in log on $ExtIf inet proto tcp from !<spamd-white< to lo0 port spamd keep state

--OR--

## OpenBSD Pf syntax. Add the following to /etc/pf.conf
################ Tables ####################################
 table <spamd-white> persist
#
################ Filtering #################################
 pass in log on $ExtIf inet proto tcp  from  <spamd-white> to ($ExtIf) port smtp keep state rdr-to lo0
 pass in log on $ExtIf inet proto tcp  from !<spamd-white> to ($ExtIf) port smtp keep state rdr-to lo0 port spamd



Checking /etc/mail/spamd.conf

The spamd.conf file is used to list out black list files. We do _not_ suggest using blacklist because of the unknowns involved in their collection. We do not know what affiliations the people collecting the ips may have or their agenda. To be safe we suggest avoiding blacklists. Use spamd to make your own lists.

The following is all you need in your /etc/mail/spamd.conf as to void out all remotly controlled black lists.

all:\
        ::



HELPFUL HINT: For an added layer of protection again spam you can use a bayesian spam filter. Check out our Bogofilter "how to" Anti-Spam Guide. With a little time and understanding you could easily filter up to 99% of any remaining spam.



Optional: Greytrapping by seeding an email address spammers are supposed to find

Greytrapping is the seeding of an email address so that spammers can find it, but normal users can not. If the email address is used then the sender must be a spammer and they are black listed.

Lets use the email address "greytrap@your_domain.org" as an example. If we took this email and put it in the source HTML of our web site normal humans would not see it. Spammers on the other hand use web page scrapers and bots to harvest email address; so they would find this address.

When the spammer sends mail with the destination address of "greytrap@your_domain.org" spamd knows this is a spammer and SPAMTRAP's them. When a host that is currently greylisted attempts to send mail to a spamtrap address, it is spamtrapped for 24 hours by adding the host to the spamd spamlist <spamd-greytrap>.

You can add as many spamtrapped email address as you want. We even suggest looking through the logs and seeing what invalid addresses spammers are sending mail to. You can then add these addresses too.

To enter a greytrap email into the spamd database use the following format. You can add multiple spamtrap email addresses.

spamdb -T -a 'greytrap@your_domain.org'

To seed the SPAMTRAP email address in HTML use the following format. It looks like a real email address to the spammer's bots, but it will not be visible to any humans looking at the web page.

<a href="mailto:greytrap@your_domain.org"></a>



Optional: Greytrapping all but allowed domains and email addresses

We highly suggest setting up a pre-verified list of email addresses that are known to be valid. Any email server sending mail to an address not on the list must be a spammer; or a confused human. Either way this is a very powerful feature to use and is highly recommended.

The file /etc/mail/spamd.alloweddomains can contain a list of verified domains and addresses which are allowed to go through the grey listing process. They are _not_ whitelisted, just allowed to prove they are good hosts by going through the grey list process. Think of it as a white list of acceptable domains or emails this machine will be willing to receive. If a remote machine is not sending mail to a valid address listed in our spamd.alloweddomains then they should not be contacting us.

spamd.alloweddomains can be used to specify a list of domain name suffixes or full email addresses which must match each destination email address in the grey list. Any destination address which does not match one of the suffixes listed in spamd.alloweddomains will be trapped, exactly as if it were sent to a spamtrap address. When a host that is currently greylisted attempts to send mail to a spamtrap address, it is blacklisted for 24 hours by adding the host to the spamd blacklist <spamd-greytrap>.

In our pf example above we only have two rules, spamd-white and not (!) spamd-white. Since a host in the spamd-greytrap table is not in spamd-white then it will be stuttered at for as long as the remote machine is connected.

This is one of the most powerful options in spamd. If you can keep a list of valid email addresses current in the spamd.alloweddomains file then all other attempts will be grey trapped. At Calomel.org, this grey trapping option traps over 99% of the spammers and leaves our real mail server to handle real mail.

For example, if spamd.alloweddomains contained the following:

@your_domain.org
critical.com
mary@your_domain.org

The following addresses will _NOT_ be spam trapped:

mary@your_domain.org
frank@your_domain.org
bob@critical.com
bobby@nobby.critical.com
robert@whoop.critical.com

The following address _WILL_ be spam trapped:

baker@test.your_domain.org     - invalid sub domain
marge@machine.your_domain.org  - invalid sub domain
bob@wrong_name.com              - not your hostname



Starting Spamd in grey list mode

To get spamd working you can reboot the box and make sure that the spamd processes have started. You should see the user "_spamd" running on ports 8025 and 8026. If you run "ps -aux | grep spamd" you should see the following processes listed:

root@machine: ps -aux | grep spamd
_spamd   22041  0.0  0.1  9804  1160 ??  Is    Mon11AM    0:01.59 spamd: (pf <spamd-white> update) (spamd)
_spamd   22052  0.0  1.0  9748 10276 ??  S     Mon11AM    0:01.99 spamd: [priv] (greylist) (spamd)
_spamd   22063  0.0  0.1  9848  1236 ??  S     Mon11AM    0:00.14 spamd: (/var/db/spamd update) (spamd)
_spamd   22074  0.0  0.1   560  1124 ??  Ss    Mon11AM    0:02.29 /usr/libexec/spamlogd -I -i lo0



Grey list mode activated

Now, all you have to do is wait for remote mail servers to connect and get grey listed. Use the command "spamd" to list out the database and watch as the hosts get trapped. When a valid mail server connects after the grey list time (25 minutes) and before the greylist timeout (4 hours) they will be whitelisted. Those host will then connect directly to the real mail server. All other hosts who do not connect again will have their entries delete from the database and they will have to start the process all over again.



IMPORTANT NOTE: web mail sites that use multiple mail servers to deliver one email

Web mail services like Google, Hotmail and some other sites use multiple email servers. This would not normally be a problem for grey listing except those services randomly attempt to deliver a single email from any of the mail servers in the pool. The first attempts comes from server_1 then the second from server_3 and the third might be attempted from server_1 again. This causes a problem with grey listing because it is possible that none of the mail servers will be used more than a few times to try to deliver the email. Thus, none of the servers will get white listed.

To fix this problem we have a few options:

Normal Volume Site: If your site gets enough emails from services like Google, Hotmail, etc then they will eventually get whitelisted due to the amount of mail being sent. You would really only need 50-100 emails an hour to be sent to your mail server for many of the Gmail servers to eventually be whitelisted within an hour or so. With enough email grey listing is not a problem.

Low Volume Site: If your mail server is a personal server and you do not get a lot of emails then you may have a problem with Google, Hotmail, etc. You also have a few options available.

Option 1: You could manually whitelist the servers according to the MX records registered to the services. You would only need to white list less than 50% of them as the odds are that a single email will hit at least one of the whitelisted servers during attempted delivery. The rest of the servers will be whitelisted automatically over time.

Option 2: You could watch or have a script monitor the logs and send yourself an alert if one of the servers in question connect. This is more hands on, but it should only take a week or so to find all of the ip in question.

Option 3: Once they are white listed you can extend the whitelist time of 864 hours (-G25:4:864) with a script. If you grep for all the white listed entries and just add the ips again using "spamdb -a "ip" it will extend the whitexp time another 36 days (864 hours). Here is a shell script called "spamd_extend_whitelist.sh" that will list out the WHITE ips from spamdb and refresh their white expire time to 36 days. Run it in a cron job on the first of the month to, in effect, have a permanent white list. FYI: every time you use "spamdb -a" on an ip it looks like another new email was delivered according to spamd. In effect, the last column in the spamdb database updates by one.

#!/bin/sh
#
## Calomel.org .:. spamd_extend_whitelist.sh
#
for i in `spamdb | grep WHITE | awk -F "|" '{print $2}'`;
 do
  /usr/sbin/spamdb -a $i
done

Option 4: If the white listed ips are always going to be allowed then setup another table in Pf, put the white listed ips in there and always forward them to the real mail server.

Option 5: You could setup your own account on the free services and send yourself 10 emails at once. Looking at the spamd database you will see all of the ips that connect and they should get white listed normally. If not, just manually white list them.



Grey list unfriendly hosts: Some hosts use unique sender ids when delivering mail. Others will attempt to deliver mail once and never again. If you expect to receive mail from these types of host then they should be whitelisted manually. Use the script "spamd_whitelist.sh" and a pf table to redirect these types of hosts directly to your mail server.


How about a script? White listing using "spamd_whitelist.sh"

Whitelist script: spamd_whitelist.sh :If you wanted to create a whitelist of the most common grey list friendly mail servers that would also be a good option. Here is a script that will dig the host names of the listed domains and make a text file with the results.

#!/bin/sh
#
## Calomel.org  spamd_whitelist.sh
#

FILE=spamd-spf.txt

rm -f $FILE
touch $FILE

### This first list are for domains who have SPF records.
for domain in \
        aol.com \
        apple.com \
        amazon.com \
        s._spf.ebay.com \
        m._spf.ebay.com \
        p._spf.ebay.com \
        p2._spf.ebay.com \
        c._spf.ebay.com \
        gmx.net \
        _spf.google.com \
        spf-a.hotmail.com \
        spf-b.hotmail.com \
        spf-c.hotmail.com \
        spf-d.hotmail.com \
        _spf-a.microsoft.com \
        _spf-b.microsoft.com \
        _spf-c.microsoft.com \
        southwest.com
do
echo \#$domain >> $FILE;
dig $domain TXT +short | tr "\ " "\n" | grep ^ip4: | cut -d: -f2 >> $FILE;
done

### This second list is for domains you specifically want to allow
### that may not be publicly available or do not have SPF records.
echo \#privatelist >> $FILE;
for privatelist in \
         10.0.0.0/8 \
         127.0.0.0/8 \
         172.16.0.0/12 \
         192.168.0.0/16
do
echo $privatelist >> $FILE;
done

Output of spamd_whitelist.sh would look similar to this:

#aol.com
152.163.225.0/24
205.188.139.0/24
205.188.144.0/24
205.188.156.0/23
205.188.159.0/24
64.12.136.0/23
64.12.138.0/24
64.12.143.99/32
64.12.143.100/32
216.34.51.0/24
#_spf-a.microsoft.com
216.99.5.67
216.99.5.68
207.46.50.82
#_spf-b.microsoft.com
131.107.65.22
217.77.141.52
217.77.141.59
#_spf-c.microsoft.com
203.32.4.25
131.107.70.12
131.107.70.16
86.61.88.25

...many more lines...

Now that you have a list of hosts you can add them all as white list entries to a PF table we will call "company-white". We have to make a PF table because we can not add CIDR-formatted network blocks (i.e. 192.168/16 or 10/8) to spamdb. Using the following few lines in pf will populate the table, redirect traffic and pass the packets to the mail server.

################ Tables ####################################
 table <company-white> persist file "/tools/pf_company_whitelist"

################ Filtering #################################
 pass in log on $ExtIf inet proto tcp from <company-white> to ($ExtIf) port smtp flags S/SA synproxy state rdr-to lo0







Example 2: Run Spamd with the sole purpose of annoying spammers

If you do not run a real mail server, but wish to annoy spammers then using spamd in blacklist mode is perfect. It will trap any connection sent to port 25 of your ip/host and stutter the connection. This will waste the time and queue memory of the remote system. The theory being, if you do not have a real mail server then why would a host connect to you unless they are a spammer.

Add the following lines. The line "spamd_black=YES" turns black list only mode on.

## Add the following to /etc/rc.conf.local
#
spamd_flags="-4 -G25:4:864 -h your_hostname.org -l127.0.0.1 -n \"Sendmail 8.11.4/8.11.1\" -S10 -s1 -v -w1"
spamd_black=YES
spamlogd_flags="-I -i lo0"

These will start spamd with the following options:



Setting up pf

Now, add the following to your pf.conf. All these rules do is direct all connections from any remote host to port 25 on your machine directly to spamd. There they will be stuttered at and their time wasted.

## Add the following to /etc/pf.conf
#
# Spamd ( external mail attempts to spamd server )
 pass in log on $ExtIf inet proto tcp from any to ($ExtIf) port smtp flags S/SA synproxy state rdr-to lo0 port spamd
#


Checking /etc/mail/spamd.conf

The spamd.conf file is used to list out black list files. We do _not_ suggest using blacklists because of the unknowns involved in their collection. We do not know what affiliations the people collecting the ips may have or their agenda. To be safe we suggest avoiding blacklists. Spamd in blacklist mode stutters all connections so this is really a moot point.

The following is all you need in your /etc/mail/spamd.conf as to void out all black lists.

all:\
        ::


Starting Spamd in black list mode

To get spamd working you can reboot the box and make sure that the spamd processes have started. You should see he user "_spamd" running on port 8025. If you run "ps -aux | grep spamd" you should see the following processes listed:

root@machine: ps -aux | grep spamd
_spamd   22063  0.0  0.1  9848  1236 ??  S     Mon11AM    0:00.14 spamd: (/var/db/spamd update) (spamd)
_spamd   22074  0.0  0.1   560  1124 ??  Ss    Mon11AM    0:02.29 /usr/libexec/spamlogd -I -i lo0



Black list mode activated

Now, all you have to do is wait for remote mail servers to connect and they will be blacklisted. If you use the program "pftop" or use the command "systat states" you can watch the connections.







What can I expect of spamdb in the real world ?

We have been using spamdb for years. Spamdb is a the gatekeeper you must go through before you get to the real mail server. On average we receive hundreds of smtp connections per day; thousands of attempts per week. These are some of the patterns we have noticed.

The average spammer works week days just like the people they are trying to send mail to. Monday is the day most spammers will start their assault. On average, the most spam is delivered on Monday and Tuesday. The volume will start to drop off on Wednesday and continue as we head towards Friday. Spam on the weekend is lower volume and may be the left over noise from bots still trying to deliver mail.

Here we see a graph of the spamd traffic for one month. We are graphing the number of connections per day. Notice the pattern of high traffic at the beginning of the week and how it drops down towards the weekend.

                    Spamd traffic for a Month
250|                                               *
   |     *             *                           *
200|   * *   *         *           * *           * * *
   |   * * * *       * * * *       * * *   *     * * * * *
150|   * * * *       * * * * *     * * * * *     * * * * * *
   | * * * * * * *   * * * * * *   * * * * * * * * * * * * *
100| * * * * * * * * * * * * * * * * * * * * * * * * * * * *
   | * * * * * * * * * * * * * * * * * * * * * * * * * * * *
50 | * * * * * * * * * * * * * * * * * * * * * * * * * * * *
   |---------------------------------------------------------
     S M T W T F S S M T W T F S S M T W T F S S M T W T F S
         week 1        week 2        week 3        week 4

The reason spammers have almost the same hours as normal people is they need to send out the latest round of scams and have servers accept them before they are blocked by the admins or banned by RBLS. The spammers have a limited window to send out their junk mail and move onto the next scam. If a spammer can send out mail and have a victim accept it right away that gives less time for anti-spam checks to evolve to block them. This is why spam is normally not sent on the weekend. By the time Monday rolled around the anti-spam groups would have a large enough sampling of the spam to block it before the work week started.

On average, 20% of the ip addresses that are trapped return to be stuttered at again and again by spamd. The other 80% either never return or never complete the entire smtp handshake process. One item to note is that many spam scanners machines (especially from Asia) only connect for a few seconds before disconnecting. Real mail servers would never do this. So, if you setup your own real mail server, like postfix, sendmail or qmail, to delay talking to a smtp host for, lets says 60 seconds after they connect, a real mail servers will stay around and spammers will more then likely disconnect.

Spammers get their email address lists from other spammers, scrapper bots looking at web sites and bulk "verified" email databases they can buy. Most of the time the "verified" email address are just made up emails that point to your domain. There is no honor among thieves. These types of smtp connections are easy to deny using a spamd.alloweddomains white list of known emails you will accept. If the email does not match one of the entries in the list the ip is trapped.

Other times the spammer has your actual email address. You can stutter at them, but eventually they may get whitelisted. You may want to increase your greylist time from the default 25 minutes to 30, 45 or 60 minutes. If the spammer gets past spamd they will deliver their spam to you. You then have a small list of ips of the spammer systems and can manually add those ips to a blacklist. You can also use a real-time black list (RBLS) such as Spamhaus.org" on your real mail server. We have an example Postfix mail server that uses quite a few anti-spam measures at our Postfix config "how to" (main.cf).

Blocking the majority of spam is really not too hard. Blocking 99% of it takes a little bit of work and understanding. With spamd you can easily block the zombie spam bot machines and with a competently setup mail server you can deny the rest.







Questions?

Clients connecting to Spamd are overloading my network connection! What can I do?

If spamd is using to much bandwidth by sending too many packets try increasing the stutter time "-S90" and the stutter speed "-s5". For example, 600 connections total is 600 packets per 5 seconds. The network would need to handle 120 packets average per second each way; around 100 kilobytes in upload bandwidth. This might still be too much bandwidth for your connection, but you could increase the values as needed or decrease the amount of connections spamd will accept with maxcon "-c". Maxcon may not exceed kern.maxfiles - 200, and defaults to 800. Reduce maxcon to 200 or 100. Figure out how much bandwidth you can afford to use on spamd. Each packet sent from spamd is no more than 83 bytes; 53 bytes payload and 23 bytes for the tcp header. If you want to spam trap 100 clients at 1 packet per second each then you will need (0.83*100)/1 = 83 kilobytes per second. If you increased the stutter time to 5 seconds then you would need (0.83*100)/5 = 16.6 kilobytes per second. You could also put spamd traffic in a lower priority queue and limit total spamd bandwidth. Take a look at our how to for (HFSC) Hierarchical Fair Service Curve Packet Scheduler for more information.

Do you have a script I can run to tally the results of Spamd?

Yes. We have a script called Spamd Statistics that will make a HTML page of the results. It will go through the logs and tally all of the time spammers wasted, the amount of bandwidth your host used to respond to spammers and more.

How do I list out the Spamd database?

Use the command "spamdb" or sort the list with "spamdb | sort".

How can I change the log file for spamd (/var/log/daemon to /var/log/spamd)?

Spamd sends log messages to syslogd using facility daemon and, with increasing verbosity, level err, warn, info, and debug. The following syslog.conf section can be used to log connection details to a dedicated file:
!spamd
daemon.err;daemon.warn;daemon.info  /var/log/spamd

How about using the Greyscanning script by Bob Beck?

The grey scanning perl script will watch the spamd logs and make decisions on hosts that connect. We highly suggest at least taking a look at the script as we find it an incredibly powerful tool. It will "process spamdb output and look for patterns. Mainly we look at the greylist entries, and make some decisions about them. if they look excessively spammish, we take some action against them, by running spamdb -t -a to add them as a TRAPPED entry to spamd, meaning spamd in greylist mode will tarpit them for the next 24 hours." Bob Beck's greyscanner.pl

How can I extract just the ip addresses from spamdb?

spamdb | grep WHITE | awk -F "|" '{print $2}'

Do you have a script to show my spamdb whitelisted ips and statistics ?

This script will show the a numerically sorted list of white listed ip address and their related host names. Also, there is a count of WHITE, SPAMTRAP and TRAPPED entries. We use this to quickly identify the top spammed email addresses to our domain and top spammer ips. This script simply cats the /var/log/daemon log file. If you setup your server to rotate logs every week for example, then when you run spamdb.sh it will show you stats for the week.
#!/bin/sh
#
## Calomel.org .:. spamdb.sh
#

if [ $# -eq 0 ]
   then
    echo ""
    echo "  Calomel.org    spamdb.sh $arg "
    echo "--------------------------------------"
    echo "show    = all whitelisted ips with hostnames"
    echo "stats   = ips in all tables"
    echo ""
   exit
 fi

if [ $1 = "show" ]
   then
    echo " "
     for i in `spamdb | grep WHITE | awk -F "|" '{print $2}' | sort `;
      do
       echo -n $i; echo -n -e "\t"; host $i | awk '{print $5}'
      done
    echo " "
   exit
 fi

if [ $1 = "stats" ]
   then
     echo "Calomel.org .:. spamdb.sh stats"
     echo " "
     echo "Spamd incoming smtp connections"
     echo -n "  grey list attempts         = ";cat /var/log/daemon | grep disconnected | grep -vc spamd-greytrap
     echo -n "  spamd-greytrap tar pits    = ";cat /var/log/daemon | grep disconnected | grep -c spamd-greytrap
     echo -n "  total connections to spamd = ";cat /var/log/daemon | grep -c disconnected

     echo " "
     echo "Spamd statistics"
     echo -n "  hours spammers have wasted = ";cat /var/log/daemon | grep disconnected | awk '{s += $9} END {print s/3600}'

     echo " "
     echo "Spamdb database classes"
     echo -n "  SPAMTRAP trigger addresses = ";spamdb | grep -c ^SPAMTRAP
     echo -n "  TRAPPED black listed ips   = ";spamdb | grep -c ^TRAPPED
     echo -n "  WHITE listed and clear ips = ";spamdb | grep -c ^WHITE

     echo " "
     echo "Top 10 To: email addresses"
     cat /var/log/daemon | grep " spamd" | grep ": (" | awk '{print $10}' | sort | uniq -c | sort -r | head -10 | tr -d \<\>

     echo " "
     echo "Top 10 remote mail server ips"
     cat /var/log/daemon | grep disconnected | awk '{print $6}' | sort | uniq -c | sort -r | head -10 | tr -d \:
     echo " "
   exit
 fi

What is the maximum white expire (whiteexp) time I can set?

864 hours is the upper limit. You can not set the white expire time (-G25:4:864) above this. Take a look for the script "spamd_extend_whitelist.sh" in the examples above. It will 'touch' all of the white listed spamdb entries and re-expire them in another 36 days. Run this script in a cron job on the first of the month and you will have a permanent white list.

How can I manually WHITE list ips addresses in the database?

spamdb -a 127.0.0.1

How can I manually remove ips addresses in the database?

spamdb -d 127.0.0.1

How can I add a SPAMTRAP email address?

spamdb -T -a 'spamtrap@domain.org' 

How can I manually TRAP (i.e. spamd will ignore) an ip for 24 hours?

spamdb -t -a 127.0.0.1

How can I convert Unix time to standard date format ?

date -r 1200337384

Where can I find more information about Spamd?

man spamd  ...and... man spamdb





Questions, comments, or suggestions? Contact Calomel.org or