Learning about slightly more advanced networking with linux

Paul Warren

Mastodon: @pwarren@mastodon.thewarrens.name

email: paul at thewarrens.name

© 2021 Paul Warren



I am not a network engineer, there are gaps in my knowledge

The words I use might not be the correct ones

Please don't blame me if your network stops networking

Maybe don't go and try building an ISP based off this talk :)

What even is a network?


Token Ring?



What even is a network?




Allows connections between computers

Managed networks

Controls connections between computers

Scales connections between computers


Like an onion!

That 7 layer thing from the OSI

Ethernet -> IP -> ICMP/TCP/UDP

Administrative overlays

VPNs and Tunnels


This means the Physical layers do not have to match the Logical connections

Networks are also a bit fractal in nature

For the purposes of this talk, we can ignore a lot of complexity

An easy one!


Ethernet layer

Designed to limit broadcast storms

Can also separate IP networks on the same Ethernet

Tagged and Untagged/Default

IEEE 802.1Q

VLANs in linux

Are pretty easy!

	    ip link add link enp0s2 name enp0s2.2501 vlan id 2501
	    ip -6 addr add 2001:db8:2501::10/64 dev enp0s2.2501
	    ip link set dev enp0s2.2501 up

VLANs in linux

Various different methods of making it go in different distributions

Debian: in /etc/network/interfaces

	    auto enp1s0.2502
iface enp1s0.2502 inet dhcp
iface enp1s0.2502 inet6 auto

OpenWRT has a nice GUI

VLANs in linux

Here's what I've done

What even is a router?

Two or more interfaces

Some logic to determine where a packet goes

Linux Routing

Add to sysctl config:

	      net.ipv4.ip_forward = 1
	      net.ipv6.conf.all.forwarding = 1
	      net.ipv6.conf.default.forwarding = 1

And we're done!

Yeah, not really :)

Static Routes

Manually added, or scripted routes that don't change

	      ip route add via dev wg0

Static Routes

ip -6 route add 2001:db8:2501::/64 via 2001:db8:2500::1 dev enp0s1
ip -6 route add 2001:db8:2502::/64 via 2001:db8:2500::1 dev enp0s1
ip -6 route add 2001:db8:2600:beef:/64 via 2001:db8:face::1 dev wg0

Static Routes

Enter dynamic routes

Advertise your LANs

Discovering routes to other LANs

Optional: Do some logic

Add them to the routing table

Userspace daemons

talking to the kernel networking stack

Dynamic Routing

Open Shortest Path First (OSPF)

Uses Dijkstra's algorithm



For: Dynamic routes inside your networks

Dynamic Routing

Border Gateway Protocol (BGP)


For: Sharing dynamic routes outside your network with other Autonomous Sytstems (AS)

"AS" is used a lot in the ISP world, and your ISP will have an assigned AS Number

The Userspace Daemons

BIRD: Berkeley Internet Routing Daemon

Quagga: Fork of Zebra

OpenBGPd/OpenOSPFd: from the OpenBSD project

Significant overlap, but they are different

I've not used Quagga, or the OpenBSD ones

There are probably others!

intro to bird


Complex to configure

Use version control

Intro to bird configuration


router id;

log "/var/log/bird/bird.log" all;
debug protocols { states, routes, filters, interfaces }

protocol kernel {
        import none;
        export all;

protocol device {
        # defaults...

Intro to bird configuration


Each router needs a unique ID

Protocol blocks, There are lots of types

They are how bird knows what to talk to and how

Intro to bird configuration

protocol ospf {
        area 0 {
                interface "lo" {
                interface "vlan1001" {
                interface "vlan1034" {

There are a lot more options than stub or default

BIRD documentation is really good

Intro to bird configuration

Import vs Export

Master Table

Logic in Pipes

A funky thing you can do with OSPF

A necessary service

Contained within a network

Data more efficiently obtained across local links

Can still get the costly versions if local goes down

A funky thing you can do with OSPF

An IP for a host that provides that service

Can use that same IP in two different places in the network

Confused routing?

OSPF to the rescue!

A funky thing you can do with OSPF

Why use a 'real' router

Linux based router:

NIC Hardware

-> whole packet to Driver memory

-> various kernel subsystems

Routing decision made

Then back out the same path

Why use a 'real' router

Enterprise level router

NIC Hardware

Just Enough Packet to get the address

-> routing silicon

Routing decision made

Silicon switched to forward packet out correct NIC Hardware

Why use a 'real' router


Some work being done on 'zero copy' networking in linux

io_uring also has some smarts for this

Why use a 'real' router

Other Considerations

Network Engineers are

Not familiar with linux networking

will have significant skills using commercial routing software

Hardware ruggedness, suitability for datacentre

power consumption

¿Por qué no los dos?

OpenWRT is a linux based distribution for real networking hardware


Sorry, didn't get to explore this as much as I'd like

Not too confident talking about it as yet

Is how the internet is put together

Bonus homework

Why did I use 2001::db8::/32?

Why did I use /64s?

I highly recommend going through KNorrie's network examples: https://github.com/knorrie/network-examples

OSPF Areas

Auth* for OSPF

Automatic transfers of internal aggregate networks to BGP (Something I've not investigated much)


These slides: https://gitea.pwarren.id.au/pwarren/SysAdmin2022

bird: https://bird.network.cz

Openwrt: https://openwrt.org/

The OpenWRT IRC channel, currently on the OFTC network

BenEater's "Why was facebook down for five hours"