This blog post was written a while ago, not published at the time, and is not yet complete. Besides, the setup this post was based on has changed a bit over time as well. Nevertheless, it took a while to collect all the information, so I may as well publish what I have found up until now*.
* feel free to read now as a couple of years ago.
We want to terminate a public IPv4* subnet, offered through a GRE tunnel, on an OpenBSD machine, and expose the available IP addresses through WireGuard to another machine.
* The setup for IPv6 should be analogous, but we will cover that in another post. Yes, IPv6 should have priority, I know, but due to circumstances, etc. etc.
We reserve three** addresses from the public subnet to make this setup possible.
We use the following prefixes for documentation purposes (taken from RFC5737):
WAN_BSD endpoint: 203.0.113.113/24 (default router: 203.0.113.254)WAN_LIN endpoint: 192.0.2.2/27 (default router: 192.0.2.1)WAN_GRE endpoint: 192.0.2.192198.51.100.232/29** Perhaps two would suffice as well.
WAN_GRE --.
|
|- GRE-tunnel (routes 198.51.100.232/29)
|
| .-- gre232 (198.51.100.232/32)
WAN_BSD --+-- [ OpenBSD ]
| `-- wg232 (198.51.100.238/31)
|
|- WireGuard tunnel
|
| .-- wg232 (198.51.100.239/31)
WAN_LIN --+-- [ Linux ]
`-- vlan232 (198.51.100.233/32)
`-- host (198.51.100.234/32)
`-- host (198.51.100.235/32)
`-- host (198.51.100.236/32)
`-- host (198.51.100.237/32)
Before you start typing commands as they appear in this document, please pause for a moment, and take the following into consideration:
In order to properly route traffic of the public subnet to/from to the GRE tunnel without messing with the ‘regular’ routing of our
OpenBSD machine, we use rdomain(4), or routing domain, to separate this traffic. This functionality is similar to Linux’
network namespaces (on Linux: man 7 network_namespaces).
We chose rdomain 232, because that’s the first address in our public /29 subnet. In short, everything that has an address of the
public /29 subnet needs to be in that rdomain to completely* separate it from our default routing domain. In some cases, we will
see this setting on the interface part, and in some cases on the tunnel part. If we do not set rdomain (or equivalent for the
tunnel part), rdomain 0 is implied: the default routing domain.
* It is still possible to transfer traffic from/to routing domains, but this is set through configuration in pf.
/etc/hostname.vio0 (our connection to BSD_WAN, rdomain 0):
inet 203.0.113.113/24
!route add default 203.0.113.254
/etc/hostname.gre232 (our connection to GRE_WAN; the tunnel is setup in rdomain 0 through vio0’s default gateway, but the
traffic inside the tunnel will be put in rdomain 232):
tunnel 203.0.113.113 192.0.2.192 tunnelttl 225 rdomain 232
inet 198.51.100.232/32
/etc/hostname.wg232 (our connection to the Linux machine; the tunnel is setup in rdomain 0 through vio0's default gateway, but the traffic _inside_ the tunnel will be put in rdomain 232`):
wgkey <openbsd-wg-private-key> wgport 51232 rdomain 232
inet 198.51.100.238/31
wgpeer <linux-wg-public-key> wgaip 0.0.0.0/0
!route -T232 add 198.51.100.233 198.51.100.239
!route -T232 add 198.51.100.234 198.51.100.239
!route -T232 add 198.51.100.235 198.51.100.239
!route -T232 add 198.51.100.236 198.51.100.239
!route -T232 add 198.51.100.237 198.51.100.239
Note: we do not set wgendpoint because our Linux machine “connects” to the OpenBSD machine. Otherwise, you could append
wgendpoint 192.0.2.2 51820 to the wgpeer stanza to set the endpoint.
sh /etc/netstart wg232 setups the WireGuard interface with the corresponding interface configuration. After that, you could run
ifconfig wg232 and/or wg to inspect what happened. This could look similar to this:
$ ifconfig wg232
wg232: flags=80c3<UP,BROADCAST,RUNNING,NOARP,MULTICAST> rdomain 232 mtu 1392
index 6 priority 0 llprio 3
wgport 51232
wgpubkey <openbsd-wg-public-key>
wgpeer <linux-wg-public-key>
tx: 0, rx: 0
wgaip 0.0.0.0/0
groups: wg
inet 198.51.100.238 netmask 0xfffffffe
$ wg
interface: wg232
public key: <openbsd-wg-public-key>
private key: (hidden)
listening port: 51232
peer: <linux-wg-public-key>
allowed ips: 0.0.0.0/0
Identically, this works for gre232 (sh /etc/netstart gre232)
pfWe provide a /etc/pf.conf that should give you the ; you can load it using pfctl -f /etc/pf.conf.
ext_if = "vio0"
prod_if = "gre232"
# Bogons: these address ranges should not occur on the internet. Beware that these ranges could change (e.g. 224.0.0.0/4 is
# currently reserved for multicast).
#
# Note that you should exclude 198.51.100.0/24, 203.0.113.0/24 when running this documentation setup, and due to how pf.conf works,
# you should delete those lines; commenting them on this multi-line statement would also comment the last two ranges.
table <bogons_ext> const { 0.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, \
127.0.0.0/8, 169.254.0.0/16, 172.16.0.0/12, 192.0.2.0/24, \
192.88.99.0/24, 192.168.0.0/24, 198.18.0.0/15, \
198.51.100.0/24, \
203.0.113.0/24, \
224.0.0.0/4, 240.0.0.0/4 }
table <prod_usable> const { 198.51.100.233, \
198.51.100.234, 198.51.100.235, 198.51.100.236, \
198.51.100.237, 198.51.100.238, 198.51.100.239 }
set skip on lo
# default block rule
block log all
# failsafe: make sure we do not send from/to bogons to the internet
block out log quick on $prod_if from <bogons_ext>
block out log quick on $prod_if to <bogons_ext>
block out log quick on $ext_if from <bogons_ext>
block out log quick on $ext_if to <bogons_ext>
pass in quick on $ext_if proto { icmp ipv6-icmp }
pass in quick on $ext_if proto tcp to port ssh
pass in quick on $ext_if proto udp to port 51232 # Linux->BSD WireGuard
# Allow GRE traffic from/to our GRE tunnel provider
pass in quick on $ext_if proto gre from 192.0.2.192
pass out quick on $ext_if proto gre to 192.0.2.192
# Allow ICMP to this machine on the GRE tunnel interface
pass in log on $prod_if proto icmp from any to 198.51.100.232/32
# Allow traffic to flow freely from/to GRE and WireGuard tunnels for our usable range.
# Note: you could build in restrictions here as well.
pass on wg232 from <eipprod_usable>
pass on wg232 to <eipprod_usable>
pass on $prod_if from <eipprod_usable>
pass on $prod_if to <eipprod_usable>
block in log quick from urpf-failed
pass out on $ext_if proto { tcp udp icmp ipv6-icmp } all modulate state
sysctlSet the following in /etc/sysctl.conf:
net.inet.gre.allow=1
net.inet.ip.forwarding=1
Next, run sysctl for each line, using the line itself as argument, e.g. sysctl net.inet.gre.allow=1. (Or reboot the machine).
systemd-networkdYeah, sorry.
One of the key points that did not really come forward in the available documentation on wg(4) was
the wgrtable n option for hostname.if(5). rdomain n exists for putting the tunneled part of the wg(4) interface into a
certain routing domain. However, in our case, we wanted to have the setup of the tunnel in a certain routing domain, whereas the
tunneled part should be in rdomain 0.
Fortunately, ifconfig(8) has a WireGuard section that provides us with this info (although
it took me a while to find where it’s defined). In our setup, the
endpoint of the tunnel is actually part of the public subnet we want to route to another machine, in another rdomain than where
the GRE tunnel terminates.
Setting up a WireGuard® client with routing domains on OpenBSD
WireGuard® is a registered trademark of Jason A. Donenfeld.