Pflogsumm is a log analyzer for Postfix. It can mail out a summery on a daily, weekly or monthly basis and is fully configurable. The generated report can show you patterns in email traffic so you can better determine bandwidth limitations, mail server abusers or if your mail server needs to be upgraded. Statistics can be very helpful in showing, for example, when most of your mail is sent or received or if that one mail user is sending out more than their fair share of mail. Pflogsumm will give you the ability to quickly determine what machines on your network are the top (ab)users and the proof to follow up company policies with numbers.
Pflogsumm is a single perl file and its only dependancies are perl, the perl module Date::Calc (which is probably already installed on your system) and the proper location of a Postfix log file.
Before we get started setting up pflogsumm lets take a look at the output of an example email report. We think it is important to see what you are going to get out of a tool before you spend the time and energy setting it up.
Below is a scrollable window and you will see all of the information gathered from a half day on the calomel.org mail server (hosts are obfuscated in the example). Scroll though the email and notice all the patterns shown. Also notice the email is text only. There are no pictures or "manager-friendly" pie graphs available in pflogsumm. Only the data in an easy to read format.
From: root@your_host.com (Root User)
To: root@your_host.com
Date: Mon, 1 Jan 2020 11:00:00 -0400 (EDT)
Subject: Postal Statistics Mon Jan 1 11:00:00 EDT 2007
Postfix log summaries for Jan 1
Grand Totals
------------
messages
845 received
1208 delivered
0 forwarded
2 deferred (3 deferrals)
12 bounced
822 rejected (40%)
2 reject warnings
0 held
0 discarded (0%)
86983k bytes received
168531k bytes delivered
214 senders
157 sending hosts/domains
176 recipients
77 recipient hosts/domains
message deferral detail
-----------------------
smtp (total: 3)
3 mailbox unavailable (in reply to RCPT TO command
message bounce detail (by relay)
--------------------------------
c.mx.maal.yahoo.com[256.39.55.3]:25 (total: 3)
3 dd This user doesn't have a yahoo.com account (dothingoctam_267...
none (total: 1)
1 spam body
smtp.iccm.rcenaria.es[153.145.82.2]:25 (total: 2)
2 User unknown in local recipient table (in reply to RCPT TO comm...
message reject detail
---------------------
RCPT
Client host rejected: DHCP1 check (total: 271)
98 arcor-ip.net
78 paltel.net
14 t-dialin.net
10 charter.com
9 ctbcnetsuper.com.br
9 i59F51603.versanet.de
6 catv-5062d7f9.catv.broadband.hu
6 tkb.net.pl
5 bezeqint.net
4 brasiltelecom.net.br
4 sbb.co.yu
3 cia.com
3 adsl-81-7-96-45.zebra.lt
3 verizon.net
3 195-240-166-103-static.dsl.ip.tiscali.nl
3 p2d215.traco.pl
3 ppp85-140-244-82.pppoe.mtu-net.ru
2 gibconnect.com
1 fibertel.com.ar
1 bigpond.net.au
1 dslcom3-125.express.oricom.ca
1 203-233-222-201.adsl.terra.cl
1 fdn.com
1 mindspring.com
Client host rejected: DHCP2 check (total: 17)
12 veloxzone.com.br
3 chello089079077128.chello.pl
1 dial050238.pool.invitel.hu
1 cc1206100-a.mp1.dr.home.nl
Helo command rejected: need fully-qualified hostname (total: 2)
1 tm.net.my
1 dyn-85.204.185.47.tm.upcnet.ro
Recipient address rejected: User unknown (total: 13)
2 time@your_host.com
2 toogp@your_host.com
2 restel@your_host.com
1 tewaslio@your_host.com
1 e@your_host.com
1 gfu@your_host.com
1 twased@your_host.com
1 fellow@your_host.com
1 odsnmeifg@your_host.com
1 msdhad@your_host.com
Sender address rejected: Domain not found (total: 10)
9 aw-confirm@email.ebay.com
1 jun-liprashant@amefi.org
cannot find your hostname (total: 463)
145 194.24.251.235
130 64.32.178.103
24 77.73.21.114
24 189.12.229.24
18 41.233.124.21
12 62.182.2.101
11 209.120.212.32
9 59.22.242.31
9 82.223.40.74
9 29.21.131.58
9 121.15.248.102
9 223.13.161.146
9 228.209.159.149
9 222.127.127.249
8 82.149.82.219
6 52.187.55.161
6 72.52.24.23
6 82.214.224.190
6 22.162.254.162
6 82.105.61.171
6 82.245.236.2
6 211.226.145.149
5 64.222.14.254
5 203.62.52.61
5 218.227.20.61
5 222.122.49.9
4 72.9.222.102
4 212.129.197.148
cleanup
body (total: 35)
27 And you will At last your new life! Like a real man with a re...
1 60% of long-term relationship breakups report that sexual pro...
1 And you will Finally your new life! Like a real man with a r...
1 Hello! I am bored this evening. I am nice girl that would lik...
1 Hello! I am tired today. I am nice girl that would like to ch...
1 Hello! I am tired this evening. I am nice girl that would lik...
1 Hello! I am bored this afternoon. I am nice girl that would l...
header (total: 11)
4 Content-Type: application/x-msdownload; name="Attachments001....
3 Content-Type: application/x-msdownload; name="WinZip.BHX"
1 Content-Type: application/x-msdownload; name="SeX.mim"
1 Received: from lyris.networkworld.info (Lyris.networkworld.in...
1 Subject: Emails
1 Subject: ?o??o?o? ????? ? ?????o??o? ???????????? ???o???????...
message reject warning detail
-----------------------------
RCPT
Helo command rejected: Host not found (total: 2)
1 telecam.net.ar
1 ber246.neaplus.adsl.tpnet.pl
message hold detail: none
message discard detail: none
smtp delivery failures
----------------------
connection refused (total: 3)
1 mail.example.com
1 hotmail.ten.org
1 felix.com
operation timed out (total: 3)
2 timetested.com
1 whataboutme.net
Warnings
--------
smtpd (total: 187)
30 64.55.178.153: hostname virginia39.seemeplayme.com verification...
15 189.55.209.54: hostname 18912209024.user.veloxzone.com.br verif...
12 52.181.2.151: address not listed for hostname return.wdc.pl
11 259.190.252.32: address not listed for hostname mail.affinity-n...
9 45.233.154.21: hostname host-41.233.134.21.tedata.net verificat...
9 85.21.151.58: hostname host58-131-21-89.tz.ru verification fail...
8 85.149.52.219: address not listed for hostname hosted.by.mostwo...
6 55.187.55.161: hostname adsl-dynamic-pool-xxx.fpt.vn verificati...
6 85.152.554.162: hostname 143-254-162.dsl.primorye.ru verificati...
6 55.155.61.171: hostname dsl.static8510561171.ttnet.net.tr verif...
6 85.215.224.190: hostname dsl.dynamic81214224190.ttnet.net.tr ve...
5 65.255.14.254: hostname 254-14-251-64.serverpronto.com verifica...
5 255.65.52.61: address not listed for hostname mail.fenying.com.tw
3 85.97.55.163: hostname dsl.dynamic859745163.ttnet.net.tr verifi...
3 253.155.103.185: hostname 203-150-103-185.inter.net.th verifica...
3 256.245.30.24: hostname unknown.hostforweb.com verification fai...
3 85.215.204.3: hostname dsl.static812152043.ttnet.net.tr verific...
3 85.242.76.53: hostname dsl88.242-19509.ttnet.net.tr verificatio...
3 159.165.114.63: hostname dsl-189-165-114-63.prod-infinitum.com....
3 85.251.254.66: address not listed for hostname smtp.dgcsystems.net
Fatal Errors: none
Panics: none
Master daemon messages: none
Per-Hour Traffic Summary
time received delivered deferred bounced rejected
--------------------------------------------------------------------
0000-0100 33 40 0 0 50
0100-0200 51 64 1 1 41
0200-0300 44 63 0 0 43
0300-0400 84 162 0 9 36
0400-0500 89 147 0 0 26
0500-0600 74 95 0 1 112
0600-0700 72 91 0 1 79
0700-0800 41 52 0 0 178
0800-0900 50 87 2 0 105
0900-1000 186 256 0 0 126
1000-1100 120 150 0 0 27
1100-1200 1 1 0 0 1
1200-1300 0 0 0 0 0
1300-1400 0 0 0 0 0
1400-1500 0 0 0 0 0
1500-1600 0 0 0 0 0
1600-1700 0 0 0 0 0
1700-1800 0 0 0 0 0
1800-1900 0 0 0 0 0
1900-2000 0 0 0 0 0
2000-2100 0 0 0 0 0
2100-2200 0 0 0 0 0
2200-2300 0 0 0 0 0
2300-2400 0 0 0 0 0
Host/Domain Summary: Message Delivery (top 5)
sent cnt bytes defers avg dly max dly host/domain
-------- ------- ------- ------- ------- -----------
871 159992k 0 2.8 s 1.6 m your_host.com
62 5632k 0 3.7 s 36.0 s another_host.com
48 96048 0 25.8 s 5.2 m yahoo.com
33 82959 0 17.8 s 2.0 m teldes.ney
17 62542 0 3.5 s 6.6 s gmail.com
Host/Domain Summary: Messages Received (top 5)
msg cnt bytes host/domain
-------- ------- -----------
455 73589k your_host.com
44 120031 another_host.com
34 3052k google.com
24 548k netmail.net
19 589k yahoo.com
top 5 Senders by message count
------------------------------
99 user1@your_host.com
76 dat@your_host.com
66 host@your_host.com
44 root@your_host.com
44 telme@your_host.com
top 5 Recipients by message count
---------------------------------
106 myuser@your_host.com
76 main@your_host.com
72 hello@your_host.com
59 felix@your_host.com
48 foul@your_host.com
top 5 Senders by message size
-----------------------------
45738k twat@your_host.com
23968k halloart@your_host.com
5881k yuer@your_host.com
2673k twotone@your_host.com
2534k whatup@your_host.com
top 5 Recipients by message size
--------------------------------
24666k me@your_host.com
24449k geter@your_host.com
24335k what@your_host.com
24316k whodat@your_host.com
23960k finallydone@your_host.com
If this looks like a tool you could use then lets take a look at the quick three-step setup.
Step 1: To get started you first need to download the pflogsumm.pl perl script. Download pflogsumm.pl here.
Step 2: Extract the files from the tar ball and put _only_ the perl script pflogsumm.pl into /usr/local/bin/ . Make sure the permissions are 700 for security.
Step 3: Setup a cron job to mail out the report every day at 11:59pm (23:59).
#minute (0-59)
#| hour (0-23)
#| | day of the month (1-31)
#| | | month of the year (1-12 or Jan-Dec)
#| | | | day of the week (0-6 with 0=Sun or Sun-Sat)
#| | | | | commands
#| | | | | |
#### pflogsumm mail report
59 23 * * * /usr/local/bin/pflogsumm -u 5 -h 5 --problems_first \
-d today /var/log/maillog | mail -s "pflogsumm report `date`" root
This is the same line used to generate the example email shown in the scrollable table above. The cron job is going to go through the postfix log in /var/log/maillog and report todays stats from 12:00am to 11:59pm. We are going the see the top five(5) senders and receivers of email by volume and size. The report is going to be mailed to root with the subject like "pflogsumm report Mon Jan 1 11:00:00 EDT 2007".
I would suggest running the cron job line at least once to make sure everything works. If you do not get any errors and the email comes through then you are done. If you experience problems, then take a look at the question and answer section at the bottom of this page. Finally, if you want more information on mail tools or postfix itself including and "how to" setup of the Postfix config then checkout the Calomel main page.
Postfix is blocking my pflogsumm emails!!If you are seeing that the postfix body_checks you put in place, which are now being reported by pflogsumm.pl are in turn being blocked by postfix then you need to put an exception into the body_checks file. This is easily done by adding this line to the _top_ of your postfix body_checks file:
### allow pflogsumm reports through postfix (body_checks file) ###
/^ {6,11}[[:digit:]]{1,6}[ km] / OK
I get an error about Date::Calc no found! Whats the problem?This is perl module pflogsumm uses to calculate the date. It needs to be installed on your system. You will find many distributions have rpm's for perl modules. You can also install Date::Calc using CPAN which is just as easy.
Can you show me more information about pflogsumm? I highly suggest checking out the Pflogsumm FAQ for more information. It is a light hearted question and answer session that helps put to rest a lot of the emails the authors has recieved.