Linux Networking From Go

Manipulating network interfaces, firewalling, and forwarding from Go.

The last few years of development on Linux have been exciting on the network front:

• NFTables - A high performance replacement for IPTables, NFTables provides a sophisticated (bytecode-based) rules engine, and the ability to make atomic rule changes (something that IPTables sorely lacked).
• Netlink - Netlink is a generic interface to features inside the kernel, but ends up being prevalent (and very convenient) for manipulating the network stack.
• Network Namespaces - Okay this isn't new, but network namespaces in-part enabled the containerization of server workloads, network sandboxing, and are just awesome in general.

Best of all, we can automate everything using Go.

The Basics: Configuring your interfaces

A pre-requisite to any funky networking is bringing the interface up. This is trivial from Go, thanks to the rtnetlink API, codified by vishvananda's fabulous netlink package.

// Get a structure describing the network interface.
localInterface, err := netlink.LinkByName("eth0")
if err != nil {
	// handle error
}

// Give the interface an address of 192.168.1.1, on a
// network with a 255.255.255.0 mask.
ipConfig := &netlink.Addr{IPNet: &net.IPNet{
  IP: net.ParseIP("192.168.1.1"),
  Mask: net.CIDRMask(24, 32)
}}
if err = netlink.AddrAdd(localInterface, ipConfig); err != nil {
	// handle error
}

// Setup the default route, so traffic that doesn't hit
// 192.168.1.(1-255) can be routed.
if err = netlink.RouteAdd(&netlink.Route{
	Scope:     netlink.SCOPE_UNIVERSE,
	LinkIndex: localInterface.Attrs().Index,
	Dst:       &net.IPNet{IP: gatewayIP, Mask: net.CIDRMask(32, 32)},
}); err != nil {
	// handle error
}

// Lastly, bring up the interface.
if err = netlink.LinkSetUp(localInterface); err != nil {
	// handle error
}

Under the hood, the following is occurring:

1. A socket of family AF_NETLINK is opened.
2. Using the NETLINK_ROUTE protocol and RTM_GETLINK / RTM_SETLINK messages, the package queries and sets link information.
3. Lastly, the RTM_NEWROUTE message is used to set the default gateway.

If you want to read about the wire encoding for this & other netlink interfaces, have a look at Matt Layher's posts on the subject.

Using NFTables for packet filtering

NFTables, the successor to IPTables, is a highly-configurable rules engine for processing packets. It is configured via a netlink interface, much like the example above.

A primer on NFTables

First, some terminology:

• Tables - Tables in NFTables are simply a container. A table can be associated with a single address family (ie: IPv4, IPv6, inet. I use inet for most of my tables as it works on both IPv4 & IPv6 traffic). Rules are contained within chains, and chains are contained within a Table.
• Chains - An NFTables chain is a hook into Linux's packet processing pipeline. You attach your rules to a specific chain, which will cause the rules to be evaluated whenever a packet hits a certain point in Linux's pipeline. For instance, if you attach rules to a chain of type prerouting, the rules will be evaluated before Linux looks at routes & makes a routing decision.
  When you specify a chain, you need to provide a type. I don't know exactly what this means, but its pretty logical. If you want to filter packets, use the type filter. If you want to re-write routing information (IPTables calls this mangling) use type route. If you are using NAT features, use type NAT.
• Rules - Rules are where your filtering/forwarding/whatever logic go. We talk about building these in the next section.

NFTables packet pipeline (abridged version)

The triangles represent chains. Conceptually, it makes sense to think of them as hooks.

Putting it all together

So, if you want to filter out certain IPv4 traffic from the internet from everything, you would put your rules in a prerouting chain with hook type filter, in a table with a name of your choosing. The table would have an address family of IPv4.
Hopefully this section gives you enough information to fill in the opaque-sounding fields. The rest is pretty straightforward.

Setting up NFTables from Go

I've talked for long enough, lets get into some code shall we? We use the nftables package by stapelberg, which is shaping up to be the canonical Go package for nftables (I'm biased thou, I'm sending PRs there too).

c := &nftables.Conn{}

// Lets create our table first.
myTable := c.AddTable(&nftables.Table{
	Family: nftables.TableFamilyIPv4,
	Name:   "myFilter",
})

// Now, we create a chain which we will add our filter
// rules to.
myChain := c.AddChain(&nftables.Chain{
	Name:     "myChain",
	Table:    myTable,
	Type:     nftables.ChainTypeFilter,
	Hooknum:  nftables.ChainHookInput,
	Priority: nftables.ChainPriorityFilter,
})

// Lets add a rule to the chain that loads the source
// address, and compares it to a hardcoded IP.
c.AddRule(&nftables.Rule{
  Table: myTable,
  Chain: myChain,
  Exprs: []expr.Any{
    // payload load 4b @ network header + 12 => reg 1
    &expr.Payload{
      DestRegister: 1,
      Base:         expr.PayloadBaseNetworkHeader,
      Offset:       12,
      Len:          4,
    },
    // cmp eq reg 1 0x0245a8c0
    &expr.Cmp{
      Op:       expr.CmpOpEq,
      Register: 1,
      Data:     net.ParseIP("192.168.1.53").To4(),
    },
    // [ immediate reg 0 drop ]
    &expr.Verdict{
      Kind: expr.VerdictDrop,
    },
  },
})

// Apply the above (commands are queued till a call to Flush())
if err := c.Flush(); err != nil {
  // handle error
}

Lets break this down.

1. First, we construct a literal nftables.Conn. This connection contains all the state of our table.
2. Next, we define and create our table. Aside from naming, the only field we have to choose here is the family. As this example highlights dropping traffic based on an IPv4 address, it makes sense for it to be in the IPv4 family.
3. Creating the chain is where the fields get confusing. We create a chain in the table we previously created, with the following fields:
   • Filter type - Because we want to filter packets
   • Input chain (hook) - Because we want to drop packets before they enter the system (but not interfere with forwarded traffic)
   • Filter priority - I havent talked about priority yet, because you should seldom need to change it. Priorities other than Filter allows you to make your rules run before/after internal nftables operations. See the nftables wiki for more details.
4. Next, we add a rule to drop packets where the source IP is 192.168.1.53. We break down the composition of rules in the next section.
5. Lastly, we call Flush() to apply everything in one go (The table, chain, nor rules are not created till you call Flush). While this may seem like a confusing API design decision, batching NFTables changes into atomic transactions allows you to avoid strange behaviour, when packets are recieved between two NFTables changes.

Congrats! You've just applied your first NFTables rule to the system!

NFTables Rules

I've avoided writing about these till now as they are a little confusing at first. That said, the rule engine is one of the most powerful features NFTables has on offer.

An NFTables rule is composed of a series of expressions, which are evaluated from beginning to end. If the expression checks some condition, and the condition evaluates to false, the remainder of the expressions are not evaluated (and hence any actions in later expressions are not applied).

Other NFTables rules perform some action. For instance expr.Drop instructs NFTables to drop the packet if it is evaluated (conversely, expr.Accept tells NFTables to accept the packet). There are also actions that increment counters, actions that mangle packets, actions that mark the connection or NAT it, and many more. Lets break down the example we saw before:

// payload load 4b @ network header + 12 => reg 1
&expr.Payload{
  DestRegister: 1,
  Base:         expr.PayloadBaseNetworkHeader,
  Offset:       12,
  Len:          4,
},

This expression loads 4 bytes (the size of an IPv4 address) from offset 12 of the packet's network header. If we look at the wikipedia page for an IPv4 packet, we can see that offset 12 is the source IP address. Neat!

There are 20, 32bit registers you can use to store state as your rule is evaluated. Register 0 specifies the verdict (ie: accept/drop packet) code, so don't use it unless you know what you are doing.

// cmp eq reg 1 0x0245a8c0
  &expr.Cmp{
    Op:       expr.CmpOpEq,
    Register: 1,
    Data:     net.ParseIP("192.168.1.53").To4(),
  },

The next blob of code compares the contents of register 1, permitting the rule to continue evaluation if the register contents match (CmpOpEq) the data (192.168.1.53). This has the effect of aborting rule evaluation if the source IP of the packet is not 192.168.1.53.

The last expression expr.Verdict simply drops the packet if the rule gets that far in evaluation (internally, this sets register 0). Since only packets with a source IP of 192.168.1.53 get this far, the expressions in this rule collectively add up to drop packets from 192.168.1.53.

Future posts

There's heaps more to cover but this post is getting long.

• Building advanced NFTables rules - I'll explain how to come up with more complex rules, and provide a load of copypasta for the common use cases.
• NFTables counters & sets - NFTables introduced the ability to use in-kernel lookup datastructures called sets, which makes most rules drastically simpler, & better support the common use-cases (blocking/allowing/forwarding sets of IPs/ports). Counters also let you track the throughput of your rules, both in terms of the number of packets and the number of bytes.
• Filtering process traffic with net namespaces - In this post I'll go over network namespaces, how to use them, and walk through a network filter for select processes.

Post Tags linux Go golang nftables