Using OpenBGPD to distribute pf table updates to your servers
One of the challenges faced when managing our OpenBSD firewalls is the distribution of IPs to pf tables without manually modifying /etc/pf.conf on each of the firewalls every time.
This task becomes quite tedious, specifically when you want to distribute different types of changes to different systems (eg administrative IPs to a firewall and spammer IPs to a mail server), or if you need to distribute real time blacklists to a large number of systems.
The following post outlines one a method of distributing such lists using OpenBGP to deliver them into your pf tables.
The background
We are responsible for a dozen or so firewalls, spread all over the Internet. We utilize a lot of pf tables in order to ease the configuration of pf. Our tables include IPs for admins, blacklists, spammers etc. Different firewalls utilize different tables, for instance a mail server only needs to block spammers on mail related ports, on the other hand the admins table needs to be distributed to all the systems that perform any type of filtering, so that remote administration is always possible, eliminating any risk of lock-out.
After reading the presentation of "Using BGP for real-time import and export of OpenBSD spamd entries" (links at the end of this post), we came to the realization that its not only spam-related white/black lists that one could distribute; we could also achieve the distribution of IPs to remote OpenBSD firewalls, using the same principles.
The idea is that we will utilize BGP communities, with each community representing a single pf table. So for instance the community 65000:1 corresponds to the pf table <admins>.
For our examples we will assume a central OpenBGP router (responsible for distributing the IP/community pairs), an OpenBSD firewall (fw1) and an OpenBSD mail server (mail1).
The following examples have been tested under OpenBSD 5.6.
The router
The central OpenBGP server that will act as router requires a bit more legwork and preparations, mostly for the management part, however it is still simple enough. Before we start, we have to decide what tables we need to distribute and assign them to their corresponding communities. We will be using a separate private AS number (from 64512 - 65534 inclusive) for each of the two clients (fw1 and mail1).
The AS numbers will be
- 65000: router
- 65001: fw1
- 65002: mail1
The BGP AS community to PF tables mapping, that we will use is
- community 65000:1:<admins> A table that holds the IPs used by us to access the remote servers through ssh.
- community 65000:2:<spammers> A table that holds the IPs of spammers we collect and actively block
First we create the configuration file for openbgpd, located at /etc/bgpd.conf
routerAS="65000"
AS $routerAS
router-id 10.0.0.1
fib-update no
nexthop qualify via default
group RS {
announce all
set nexthop no-modify
enforce neighbor-as no
multihop 64
ttl-security no
holdtime min 60
softreconfig in no
neighbor 0.0.0.0/0 {
passive
}
}
deny from any
allow to any
Activate the service by editing /etc/rc.conf.local
bgpd_flags=""
Start the service and check that its running
/etc/rc.d/bgpd start bgpctl status
Add a sample IP for each table, we will add the IP 1.1.1.1 to the <admins> table community and the IP 2.2.2.2 to the <spammers> table community
bgpctl network add 1.1.1.1/32 community 65000:1 bgpctl network add 2.2.2.2/32 community 65000:2
Verify the entries got added by executing
bgpctl show rib community 65000:1 bgpctl show rib community 65000:2
The output of the command will look something like the following
flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale origin: i = IGP, e = EGP, ? = Incomplete flags destination gateway lpref med aspath origin AI*> 1.1.1.1/32 0.0.0.0 100 0 i
In order to completely remove an IP from all your tables you can execute
bgpctl network delete 1.1.1.1/32
However if the IP you want to delete exists on more than one communities it is safer to use something like the following, assuming the IP 1.1.1.1/32 is already on communities 65000:1 and 65000:2 and we only want to allow the 65000:1 community (eg deleting the 65000:2 entry for the IP) we have to use the network add command
bgpctl network add 1.1.1.1/32 community 65000:1
The clients
Now that you have your main router up and running lets go to the clients and configure the PF and OpenBGP. Since both client configurations are similar we will only outline the mail1 system which better reflects the principles
First prepare the pf configuration by adding something similar to your /etc/pf.conf
table <admins> persist counters table <spammers> persist counters . . . # allow admins to connect to any port pass quick from <admins> # block connections from <spammers> to port 25 block quick inet proto tcp from <spammers> to port 25
Make sure that you have loaded the pf configuration, since bgp requires the presence of the tables
pfctl -nf /etc/pf.conf && pfctl -vf /etc/pf.conf
We prepare the /etc/bgpd.conf for the client
routerAS="65000"
AS 65002 # Different number for each client
fib-update no # Mandatory, to not update the local routing table
group "pftabled" {
remote-as $routerAS
multihop 64
announce none # Do not send Route Server any information
neighbor 10.0.0.1
}
match from group pftabled community $routerAS:1 set pftable "admins"
match from group pftabled community $routerAS:2 set pftable "spammers"
Configure bgpd to run on startup by editing /etc/rc.conf.local
bgpd_flags=""
Start the service and confirm that you get updates
/etc/rc.d/bgpd start
You should see now the IP's being populated
# pfctl -t admins -T show 1.1.1.1 # pfctl -t spammers -T show 2.2.2.2
Keep in mind
There are some things that you need to keep in mind with this method
- When the bgpd router exits all the pf tables on all clients are wiped clean (i think there is a way around this one)
- When bgpd client exits the local pf tables are wiped clean (I think there is a way around this one also)
- In case you need to add an IP to more than one community you would need to combine them into a single command
bgpctl network add 1.1.1.1/32 community 65000:1 community 65000:2
- It might be good, depending on your usage to also perform a dump of the tables locally on each system, eg through crontab
bgpctl show rib community 65000:1 | awk '{print $2}'|sed 's/\/.*$//'|grep '[0-9]' >/etc/pf-admins.conf - BGPd is not meant for this type of use
- We did keep the setups oversimplified for clarity reasons make sure you read the manual pages before copy/pasting
Final words
The benefits of this method become more visible when you have to distribute "real-time" updates, either with regards to security incidents or for syncing purposes (eg authpf tables).
I hope you enjoyed the reading, feel free to contact me with questions/corrections/suggestions through Twitter @PantelisRoditis.