First i had ipsec/gre for interconnecting my nodes itself, but now i completely switched to WireGuard. As it is really simple to setup (if you can compile an kernel module or use dkms), it is fast, high throughput and you don’t need gre as with ipsec to use for example OSPF on the links.

As i don’t want to setup a full mesh with WireGuard, i decided to only setup a few “direct tunneled” connections between some nodes and announce Node specific Loopback IPs via OSPF. With this, i had every Node reachable by any node, and if i want to setup an direct tunnel between two nodes, OSPF will get aware of this connection and use this instead of indirect connection over an other node.

The Loopback IPs will be used later for iBGP Peering (full mesh) between all 6 Nodes.

Every Node gets a minimum of two Loopback IPs, one as iBGP Peering IP and one as eBGP (or DN42) Peering IP. The Loopback IPs are shown in the drawing below, also the Tunnel setups and iBGP Peerings:

DN42 Basic Network

Wireguard Tunnel Setup

For Installation just look on the WireGuard Website for support. I use an ifupdown approach on Debian to setup my iBGP Tunnels (and most of the eBGP Tunnels, too).

You need a config file in /etc/wirguard/ and a corresponding ifupdown config in /etc/network/interfaces.d/. For example these are my conf files for an WireGuard Tunnel between dn42-gw <-> dn42-uk01:

On dn42-gw:

/etc/network/interfaces.d/wg-ibgp-uk01.conf

auto wg-ibgp-uk01
iface wg-ibgp-uk01 inet static
	address 172.20.175.195
	netmask 255.255.255.255
	pointopoint 172.20.175.198
	pre-up ip link add wg-ibgp-uk01 type wireguard
	pre-up wg setconf wg-ibgp-uk01 /etc/wireguard/wg-ibgp-uk01.conf
	post-up ip -6 addr add fe80::de49:211/64 dev wg-ibgp-uk01
	post-down ip link del wg-ibgp-uk01

/etc/wireguard/wg-ibgp-uk01.conf

[Interface]
ListenPort = 23909
PrivateKey = <NOT SHOWN>

[Peer]
PublicKey = qAr6EBKEQ2D20H5bs+USOOeRJHEijEy/IAxrLxFLBRM=
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = dn42-uk01.weiti.org:23110

On dn42-uk01:

/etc/network/interfaces.d/wg-ibgp-de02

auto wg-ibgp-de02
iface wg-ibgp-de02 inet static
	address 172.20.175.198
	netmask 255.255.255.255
	pointopoint 172.20.175.195
	pre-up ip link add wg-ibgp-de02 type wireguard
	pre-up wg setconf wg-ibgp-de02 /etc/wireguard/wg-ibgp-de02.conf
	post-up ip -6 addr add fe80::de49:225/64 dev wg-ibgp-de02
	post-down ip link del wg-ibgp-de02

/etc/wireguard/wg-ibgp-de02.conf

[Interface]
ListenPort = 23110
PrivateKey = <NOT SHOWN>

[Peer]
PublicKey = zuRXi9t+uMW9CxlN1gy0X8ZCxeQ9Xm8RwlyIr05SbBU=
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = dn42-gw.weiti.org:23909

And on the wg-* links i use OSFP to announce the Loobpack IPs:

protocol ospf O_OSPF {
	table T_OSPF;
	area 0.0.0.0 {
		interface "lo" {
			stub;
		};

		interface "wg-ibgp*" {
		};
	};
}

This is the OSPF Config part of my bird.conf. For IPv6 i use the same config. Within the pipe from T_OSPF to master, i allow only /32 or /128 from the corresponding Loopback Network. An example of my LO Addressing, OSPF Neighbor and Routing Table from dn42-uk01 will look like the following examples:

Loopback IPs

ip addr list lo
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 172.20.175.198/32 scope global lo
       valid_lft forever preferred_lft forever
    inet 172.20.175.225/32 scope global lo
       valid_lft forever preferred_lft forever
    inet6 fdf7:17d5:de49:4::1/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::666/128 scope link 
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

OSPF Neighbor Table

birdc 'show ospf neighbor'
BIRD 1.6.0 ready.
O_OSPF:
Router ID       Pri          State      DTime   Interface  Router IP   
172.20.175.197    1     Full/PtP        00:38   wg-ibgp-fr01 172.20.175.197 
172.20.175.199    1     Full/PtP        00:32   wg-ibgp-au01 172.20.175.199 
172.20.175.193    1     Full/PtP        00:35   wg-ibgp-de01 172.20.175.193 
172.20.175.195    1     Full/PtP        00:36   wg-ibgp-de02 172.20.175.195 

OSPF Routing Table

birdc 'show route table T_OSPF'
BIRD 1.6.0 ready.
20.175.215/32  via 172.20.175.197 on wg-ibgp-fr01 [O_OSPF 2017-12-31] * I (150/10) [172.20.175.197]
20.175.210/32  via 172.20.175.193 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
20.175.211/32  via 172.20.175.195 on wg-ibgp-de02 [O_OSPF 09:12:02] * I (150/10) [172.20.175.195]
20.175.220/32  via 172.20.175.195 on wg-ibgp-de02 [O_OSPF 09:12:41] * I (150/20) [172.20.175.196]
20.175.196/32  via 172.20.175.195 on wg-ibgp-de02 [O_OSPF 09:12:41] * I (150/20) [172.20.175.196]
20.175.197/32  via 172.20.175.197 on wg-ibgp-fr01 [O_OSPF 2017-12-31] * I (150/10) [172.20.175.197]
20.175.198/32  dev lo [O_OSPF 2017-06-01] * I (150/0) [172.20.175.198]
20.175.199/32  via 172.20.175.199 on wg-ibgp-au01 [O_OSPF 2017-12-17] * I (150/10) [172.20.175.199]
20.175.193/32  via 172.20.175.193 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
20.175.195/32  via 172.20.175.195 on wg-ibgp-de02 [O_OSPF 09:12:02] * I (150/10) [172.20.175.195]
20.175.252/32  via 172.20.175.193 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
20.175.253/32  via 172.20.175.193 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
20.175.254/32  via 172.20.175.193 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
20.175.249/32  via 172.20.175.195 on wg-ibgp-de02 [O_OSPF 09:12:02] * I (150/10) [172.20.175.195]
20.175.250/32  via 172.20.175.195 on wg-ibgp-de02 [O_OSPF 09:12:02] * I (150/10) [172.20.175.195]
20.175.251/32  via 172.20.175.193 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
20.175.230/32  via 172.20.175.199 on wg-ibgp-au01 [O_OSPF 2017-12-17] * I (150/10) [172.20.175.199]
20.175.225/32  dev lo [O_OSPF 2017-06-01] * I (150/0) [172.20.175.198]

OSPFv3 Neighbor Table

birdc6 'sh ospf neighbors'
BIRD 1.6.4 ready.
O_OSPF:
Router ID       Pri          State      DTime   Interface  Router IP   
172.20.175.196    1     Full/PtP        00:37   wg-ibgp-us01 fe80::de49:220                         
172.20.175.197    1     Full/PtP        00:34   wg-ibgp-fr01 fe80::42                               
172.20.175.195    1     Full/PtP        00:34   wg-ibgp-gw fe80::de49:211                         
172.20.175.198    1     Full/PtP        00:33   wg-ibgp-uk01 fe80::de49:225                         
172.20.175.199    1     Full/PtP        00:30   wg-ibgp-au01 fe80::de49:230 

OSPFv3 Routing Table

birdc6 'show route table T_OSPF'
BIRD 1.6.0 ready.
fdf7:17d5:de49::250/128 via fe80::de49:211 on wg-ibgp-de02 [O_OSPF 09:11:57] * I (150/10) [172.20.175.195]
fdf7:17d5:de49::251/128 via fe80::1 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
fdf7:17d5:de49::249/128 via fe80::de49:211 on wg-ibgp-de02 [O_OSPF 09:11:57] * I (150/10) [172.20.175.195]
fdf7:17d5:de49::42/128 via fe80::1 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
fdf7:17d5:de49::43/128 via fe80::1 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
fdf7:17d5:de49:5::1/128 via fe80::de49:230 on wg-ibgp-au01 [O_OSPF 2017-12-17] * I (150/10) [172.20.175.199]
fdf7:17d5:de49:4::1/128 dev lo [O_OSPF 2017-06-01] * I (150/0) [172.20.175.198]
fdf7:17d5:de49:1::1/128 via fe80::de49:211 on wg-ibgp-de02 [O_OSPF 09:11:57] * I (150/10) [172.20.175.195]
fdf7:17d5:de49::1/128 via fe80::1 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]
fdf7:17d5:de49:3::1/128 via fe80::de49:215 on wg-ibgp-fr01 [O_OSPF 2017-11-16] * I (150/10) [172.20.175.197]
fdf7:17d5:de49:2::1/128 via fe80::de49:211 on wg-ibgp-de02 [O_OSPF 09:12:47] * I (150/20) [172.20.175.196]
fdf7:17d5:de49::5222/128 via fe80::1 on wg-ibgp-de01 [O_OSPF 2017-12-29] * I (150/10) [172.20.175.193]