Pfstat is a tool to make graphs out of the data from the pf firewall logs. It can graph throughput in or out of an interface, the amount of packets blocked, states seen and many other values. Pfstat graphs are especially useful to spot odd behavior like bandwidth over usage and aggressive scanning or DDOS attacks. With interface graphs you can keep an eye on traffic patterns and better understand your network.
The following is the configuration file you can use with the latest version of pfstat. It will generate every variation of graphs from the logs that we could think of. It will also make those graphs in daily, weekly and monthly formats to show you long term statistics of your connection. You might not need all of them, but this will give you the chance to see what you want and what you can do without.
NOTE: This configuraion is for Pfstat versions 2.2 and above. Previous versions used a completely different format. If you have not already done so, please upgrade as the newest versions have significantly more functionality.
#######################################################
### Calomel.org pfstat.conf BEGIN
#######################################################
#
## Bytes in and out plus states
#
collect 1 = interface "em0" pass bytes in ipv4 diff
collect 2 = interface "em0" pass bytes out ipv4 diff
collect 3 = global states entries
image "/var/www/htdocs/pfstat/pfstat_day.jpg" {
from 1 days to now
width 980 height 200
left
graph 1 "in" "bytes/s" color 0 192 0 filled,
graph 2 "out" "bytes/s" color 0 0 255
right
graph 3 "states" "entries" color 192 192 0
}
image "/var/www/htdocs/pfstat/pfstat_week.jpg" {
from 1 weeks to now
width 980 height 200
left
graph 1 "in" "bytes/s" color 0 192 0 filled,
graph 2 "out" "bytes/s" color 0 0 255
right
graph 3 "states" "entries" color 192 192 0
}
image "/var/www/htdocs/pfstat/pfstat_month.jpg" {
from 1 months to now
width 980 height 200
left
graph 1 "in" "bytes/s" color 0 192 0 filled,
graph 2 "out" "bytes/s" color 0 0 255
right
graph 3 "states" "entries" color 192 192 0
}
#######################################################
#
## Pass/Block packets per second In/Out
#
collect 4 = interface "em0" pass packets in ipv4 diff
collect 5 = interface "em0" pass packets out ipv4 diff
collect 6 = interface "em0" block packets in ipv4 diff
collect 7 = interface "em0" block packets out ipv4 diff
image "/var/www/htdocs/pfstat/pfstat-packets_day.jpg" {
from 1 days to now
width 980 height 200
left
graph 4 "pass in" "packets/s" color 0 192 0 filled,
graph 5 "pass out" "packets/s" color 0 0 255
right
graph 6 "block in" "packets/s" color 255 0 0,
graph 7 "block out" "packets/s" color 192 192 0
}
image "/var/www/htdocs/pfstat/pfstat-packets_week.jpg" {
from 1 weeks to now
width 980 height 200
left
graph 4 "pass in" "packets/s" color 0 192 0 filled,
graph 5 "pass out" "packets/s" color 0 0 255
right
graph 6 "block in" "packets/s" color 255 0 0,
graph 7 "block out" "packets/s" color 192 192 0
}
image "/var/www/htdocs/pfstat/pfstat-packets_month.jpg" {
from 1 months to now
width 980 height 200
left
graph 4 "pass in" "packets/s" color 0 192 0 filled,
graph 5 "pass out" "packets/s" color 0 0 255
right
graph 6 "block in" "packets/s" color 255 0 0,
graph 7 "block out" "packets/s" color 192 192 0
}
#######################################################
#
## Inserts, removals and searches per second
#
collect 8 = global states inserts diff
collect 9 = global states removals diff
collect 10 = global states searches diff
image "/var/www/htdocs/pfstat/pfstat-states_day.jpg" {
from 1 days to now
width 980 height 200
left
graph 8 "inserts" "states/s" color 0 192 0 filled,
graph 9 "removals" "states/s" color 0 0 255
right
graph 10 "searches" "states/s" color 255 0 0
}
image "/var/www/htdocs/pfstat/pfstat-states_week.jpg" {
from 1 weeks to now
width 980 height 200
left
graph 8 "inserts" "states/s" color 0 192 0 filled,
graph 9 "removals" "states/s" color 0 0 255
right
graph 10 "searches" "states/s" color 255 0 0
}
image "/var/www/htdocs/pfstat/pfstat-states_month.jpg" {
from 1 months to now
width 980 height 200
left
graph 8 "inserts" "states/s" color 0 192 0 filled,
graph 9 "removals" "states/s" color 0 0 255
right
graph 10 "searches" "states/s" color 255 0 0
}
#######################################################
#
## Bits per seconds per Queue (for the Calomel.org PF Example)
#
collect 11 = queue "ack" pass bytes diff
collect 12 = queue "dns" pass bytes diff
collect 13 = queue "ssh" pass bytes diff
collect 14 = queue "bulk" pass bytes diff
image "/var/www/htdocs/pfstat/pfstat-queues_day.jpg" {
from 1 days to now
width 980 height 150
left
graph 11 bps "ack" "bits/s" color 0 192 192,
graph 12 bps "dns" "bits/s" color 192 0 192,
graph 13 bps "ssh" "bits/s" color 255 0 0,
graph 14 bps "bulk" "bits/s" color 192 192 0
}
image "/var/www/htdocs/pfstat/pfstat-queues_week.jpg" {
from 1 weeks to now
width 980 height 150
left
graph 11 bps "ack" "bits/s" color 0 192 192,
graph 12 bps "dns" "bits/s" color 192 0 192,
graph 13 bps "ssh" "bits/s" color 255 0 0,
graph 14 bps "bulk" "bits/s" color 192 192 0
}
image "/var/www/htdocs/pfstat/pfstat-queues_month.jpg" {
from 1 months to now
width 980 height 150
left
graph 11 bps "ack" "bits/s" color 0 192 192,
graph 12 bps "dns" "bits/s" color 192 0 192,
graph 13 bps "ssh" "bits/s" color 255 0 0,
graph 14 bps "bulk" "bits/s" color 192 192 0
}
#######################################################
#
## All other Counters
#
collect 15 = global counters match diff
collect 16 = global counters bad-offset diff
collect 17 = global counters fragment diff
collect 18 = global counters short diff
collect 19 = global counters normalize diff
collect 20 = global counters memory diff
collect 21 = global counters bad-timestamp diff
collect 22 = global counters congestion diff
collect 23 = global counters ip-option diff
collect 24 = global counters proto-cksum diff
collect 25 = global counters state-mismatch diff
collect 26 = global counters state-insert diff
collect 27 = global counters state-limit diff
collect 28 = global counters src-limit diff
collect 29 = global counters synproxy diff
image "/var/www/htdocs/pfstat/pfstat-errors_day.jpg" {
from 1 days to now
width 980 height 200
left
graph 17 "frag" "/s" color 192 0 192,
graph 22 "cong" "/s" color 0 192 192,
graph 23 "iopt" "/s" color 0 0 255,
graph 24 "csum" "/s" color 192 192 0,
graph 25 "mism" "/s" color 255 0 0
# others are usually all zero here
right
graph 15 "match" "/s" color 0 192 0
}
image "/var/www/htdocs/pfstat/pfstat-errors_week.jpg" {
from 1 weeks to now
width 980 height 200
left
graph 17 "frag" "/s" color 192 0 192,
graph 22 "cong" "/s" color 0 192 192,
graph 23 "iopt" "/s" color 0 0 255,
graph 24 "csum" "/s" color 192 192 0,
graph 25 "mism" "/s" color 255 0 0
# others are usually all zero here
right
graph 15 "match" "/s" color 0 192 0
}
image "/var/www/htdocs/pfstat/pfstat-errors_month.jpg" {
from 1 months to now
width 980 height 200
left
graph 17 "frag" "/s" color 192 0 192,
graph 22 "cong" "/s" color 0 192 192,
graph 23 "iopt" "/s" color 0 0 255,
graph 24 "csum" "/s" color 192 192 0,
graph 25 "mism" "/s" color 255 0 0
# others are usually all zero here
right
graph 15 "match" "/s" color 0 192 0
}
#######################################################
### Calomel.org pfstat.conf END
#######################################################
Step 1: Install pfstat from package or from source. For the example we are using the package from OpenBSD which is v2.2.
Step 2: Place the pfstat.conf file from above into your /etc/pfstat.conf. You should backup the default pfstat.conf file for future reference if you want to. Make sure to change the interface from "em0" to your card type if you have a different network interface name.
Take some time and look through the config file and familiarize yourself with the options. For example the section "Bits per second per Queue (for the Calomel.org PF Example)" might need to be changed to reflect the names of the queues in your pf.conf.
Step 3: Pfstat will make the graphs in jpg form in the web directory of your choice. For our example we placed all of the files in "/var/www/htdocs/pfstat/" so our web server can access them. Make sure your directory structure exists before executing pfstat for the first time.
Step 4: Note that in order to collect interface statistics, interface logging has to be enabled using 'pfctl -l iface' (3.1-stable and prior) or 'set loginterface iface' if your pf.conf (-current). Make sure you have interface logging enabled otherwise the pfstat database will be empty and your graphs will be void.
Step 5: Pfstat will run a job every minute to collect information about the interface. This data will be put into the file /var/db/pfstat.db and accessed by the graph generator command and the clean up script.
The best way to run pfstat is from a cron job. There are three(3) commands that need to be run.
This is an example of the crontab you can use.
#minute (0-59) #| hour (0-23) #| | day of the month (1-31) #| | | month of the year (1-12) #| | | | day of the week (0-6 with 0=Sun) #| | | | | commands #| | | | | | #### Generate pfstat graphs * * * * * /usr/local/bin/pfstat -q -d /var/db/pfstat.db 10 6-19 * * * /usr/local/bin/pfstat -p -d /var/db/pfstat.db 25 3 * * * /usr/local/bin/pfstat -t 30 -d /var/db/pfstat.db
Now that pfstat is configured and running you will now be able to see your graphs. Given some time you will be able to see patterns on your network interface and how it is used through the month. You may notice that network bandwidth drops during the weekends when employees go home, or you might notice a spike in use at night that might call for further investigation.
You have the choice of looking at the jpg's individually, but it might be a better idea to setup a very simple web server and serve out the graphs this way. Check out the Calomel.org Home Page for instructions on setting up a dead simple web server called tHttpd. It really will take less than five minutes to setup. We also have detailed instructions for Lighttpd and Apache. Once you have a web server up just make a simple html page that points to all of the jpgs.
One of the big problems you may have noticed is the size of the jpg files pfstat uses for graphs. A standard graph which is the width of 980px and a height 200px can be as much as 70 kilobytes. If you have 10 graphs on the page it takes quite a bit of bandwidth and loads slowly especially if you are bandwidth limited.
You can reduce the size of the jpgs significantly by changing the quality of the pictures. If you kept the same amount of graphs and reduced the size of the jpgs from 70KB to 20KB the pages would be more responsive.
Option #1: Using a shell script to reduce the jpg quality
If you installed pfstat in package form then the following script is the easiest way to modify the jpgs. The following script will reduce the jpeg quality of the existing pfstat graphs to 30% of normal and optimize the format. The name of the script is called reduce_pfstat_size.sh . You might want to test out the quality level to suite your needs. 30% is more than clear, but you might prefer another setting. When you are ready to use the script, run it directly after the line "pfstat -p -d /var/db/pfstat.db" which makes the graphs.
#!/bin/csh # ## Calomel.org reduce_pfstat_size.sh # ## Change to pfstat graph directory cd /var/www/htdocs/vault/ # ## Reduce the quaility of pfstat jpgs to 30% foreach file ( pfstat*.jpg ) djpeg $file | cjpeg -optimize -quality 30 > $file:r.temp mv $file:r.temp $file end
Option #2: Editing the pfstat source to reduce the jpg quality
If you build pfstat from source then you can have it generate the graphs in the quaility you specify. Look in the graph.c file for the following:
if (matrix->type == 0)
gdImageJpeg(im, out, 95);
else
gdImagePng(im, out);
The "95" above is the percentage of quality for the jpeg. Just replace 95 with 30 for a 30% quality level. This hint is courtesy of the technical reference site Pantz.org
How log does pfstat take to generate graphs? On a P4 3GHz (1gig ram) running pfstat with 30 days of logs it will generate the graphs in less than 60 seconds. The amount of time pfstat runs depends on the amount of graphs it needs to make and the amount of data in the logs. On a slower system you might want to only generate the necessary graphs and only when you know people are going to access them. For example, during business hours.