Using OpenBGPD to distribute pf table updates to your servers

OpenBGP+PFOne of the challenges faced when managing our OpenBSD firewalls is the distribution of IPs to pf tables without manually modifying /etc/pf.conf on each of the firewalls every time.

This task becomes quite tedious, specifically when you want to distribute different types of changes to different systems (eg administrative IPs to a firewall and spammer IPs to a mail server), or if you need to distribute real time blacklists to a large number of systems.

The following post outlines one a method of distributing such lists using OpenBGP to deliver them into your pf tables.

The background

We are responsible for a dozen or so firewalls, spread all over the Internet. We utilize a lot of pf tables in order to ease the configuration of pf. Our tables include IPs for admins, blacklists, spammers etc. Different firewalls utilize different tables, for instance a mail server only needs to block spammers on mail related ports, on the other hand the admins table needs to be distributed to all the systems that perform any type of filtering, so that remote administration is always possible, eliminating any risk of lock-out.

After reading the presentation of "Using BGP for real-time import and export of OpenBSD spamd entries" (links at the end of this post), we came to the realization that its not only spam-related white/black lists that one could distribute; we could also achieve the distribution of IPs to remote OpenBSD firewalls, using the same principles.

The idea is that we will utilize BGP communities, with each community representing a single pf table. So for instance the community 65000:1 corresponds to the pf table <admins>.

For our examples we will assume a central OpenBGP router (responsible for distributing the IP/community pairs), an OpenBSD firewall (fw1) and an OpenBSD mail server (mail1).

OpenBGPd sample topology diagram

The following examples have been tested under OpenBSD 5.6.

The router

The central OpenBGP server that will act as router requires a bit more legwork and preparations, mostly for the management part, however it is still simple enough. Before we start, we have to decide what tables we need to distribute and assign them to their corresponding communities. We will be using a separate private AS number (from 64512 - 65534 inclusive) for each of the two clients (fw1 and mail1).

The AS numbers will be

  • 65000: router
  • 65001: fw1
  • 65002: mail1

The BGP AS community to PF tables mapping, that we will use is

  • community 65000:1:<admins> A table that holds the IPs used by us to access the remote servers through ssh.
  • community 65000:2:<spammers> A table that holds the IPs of spammers we collect and actively block

First we create the configuration file for openbgpd, located at /etc/bgpd.conf

routerAS="65000"
AS $routerAS
router-id 10.0.0.1
fib-update no
nexthop qualify via default
group RS {
        announce all
        set nexthop no-modify
        enforce neighbor-as no
        multihop 64
        ttl-security no
        holdtime min 60
        softreconfig in no
        neighbor 0.0.0.0/0 {
                passive
        }
}
deny from any
allow to any

Activate the service by editing /etc/rc.conf.local

bgpd_flags=""

Start the service and check that its running

/etc/rc.d/bgpd start
bgpctl status

Add a sample IP for each table, we will add the IP 1.1.1.1 to the <admins> table community and the IP 2.2.2.2 to the <spammers> table community

bgpctl network add 1.1.1.1/32 community 65000:1
bgpctl network add 2.2.2.2/32 community 65000:2

Verify the entries got added by executing

bgpctl show rib community 65000:1
bgpctl show rib community 65000:2

The output of the command will look something like the following

flags: * = Valid, > = Selected, I = via IBGP, A = Announced, S = Stale
origin: i = IGP, e = EGP, ? = Incomplete

flags destination          gateway          lpref   med aspath origin
AI*>  1.1.1.1/32           0.0.0.0            100     0 i

In order to completely remove an IP from all your tables you can execute

bgpctl network delete 1.1.1.1/32

However if the IP you want to delete exists on more than one communities it is safer to use something like the following, assuming the IP 1.1.1.1/32 is already on communities 65000:1 and 65000:2 and we only want to allow the 65000:1 community (eg deleting the 65000:2 entry for the IP) we have to use the network add command

bgpctl network add 1.1.1.1/32 community 65000:1

The clients

Now that you have your main router up and running lets go to the clients and configure the PF and OpenBGP. Since both client configurations are similar we will only outline the mail1 system which better reflects the principles

First prepare the pf configuration by adding something similar to your /etc/pf.conf

table <admins> persist counters
table <spammers> persist counters
. 
. 
. 
# allow admins to connect to any port
pass quick from <admins>

# block connections from <spammers> to port 25
block quick inet proto tcp from <spammers> to port 25

Make sure that you have loaded the pf configuration, since bgp requires the presence of the tables

pfctl -nf /etc/pf.conf && pfctl -vf /etc/pf.conf

We prepare the /etc/bgpd.conf for the client

routerAS="65000"

AS 65002         # Different number for each client
fib-update no    # Mandatory, to not update the local routing table

group "pftabled" {
        remote-as $routerAS
        multihop 64
        announce none	# Do not send Route Server any information

        neighbor 10.0.0.1
}
match from group pftabled community $routerAS:1  set pftable "admins"
match from group pftabled community $routerAS:2  set pftable "spammers"

Configure bgpd to run on startup by editing /etc/rc.conf.local

bgpd_flags=""

Start the service and confirm that you get updates

/etc/rc.d/bgpd start

You should see now the IP's being populated

# pfctl -t admins -T show
   1.1.1.1

# pfctl -t spammers -T show
   2.2.2.2

Keep in mind

There are some things that you need to keep in mind with this method

  • When the bgpd router exits all the pf tables on all clients are wiped clean (i think there is a way around this one)
  • When bgpd client exits the local pf tables are wiped clean (I think there is a way around this one also)
  • In case you need to add an IP to more than one community you would need to combine them into a single command
    bgpctl network add 1.1.1.1/32 community 65000:1 community 65000:2

  • It might be good, depending on your usage to also perform a dump of the tables locally on each system, eg through crontab
    bgpctl show rib community 65000:1 | awk '{print $2}'|sed 's/\/.*$//'|grep '[0-9]' >/etc/pf-admins.conf

  • BGPd is not meant for this type of use
  • We did keep the setups oversimplified for clarity reasons make sure you read the manual pages before copy/pasting

Final words

The benefits of this method become more visible when you have to distribute "real-time" updates, either with regards to security incidents or for syncing purposes (eg authpf tables).

I hope you enjoyed the reading, feel free to contact me with questions/corrections/suggestions through Twitter @PantelisRoditis.

References