Advanced NFTables With Go

In my last post we went over the basics of NFTables, and using it in Go. Now, we get funky with a raft of different use-cases:

• Dynamic blacklists based on IP/Ports
• Packet counters
• NATs

NFTables Sets

Before we get into some code, its time to introduce a powerful NFTables feature I've alluded to before; sets.

Every time you want the network to do something differently based on a set of values, you should represent those values in a set.

Sets are in-kernel datastructures with efficient lookup semantics. They allow you to keep the metadata your rule operates on separate from the logic of the rule itself, in a similar way that code and configuration are separate (and not hardcoded) in mature programs.

Using them looks something like this:

1. The programmer creates the NFTables table & chain in the usual fashion.
2. The programmer creates a set with a given name, and populates it with some values. In the case of an IP blacklist/whitelist, these values would be the IPs on the blacklist/whitelist.
3. The programmer uses the lookup expression in their NFTables rule, such that the rule changes behaviour based on the presence of a datapoint in the set or not.

Dynamic blacklisting Example

c := &nftables.Conn{}

// Basic boilerplate; create a table & chain.
table := &nftables.Table{
	Family: nftables.TableFamilyIPv4,
	Name:   "ip_filter",
}
table = c.AddTable(table)

myChain := c.AddChain(&nftables.Chain{
	Name:     "filter_chain",
	Table:    table,
	Type:     nftables.ChainTypeFilter,
	Hooknum:  nftables.ChainHookInput,
	Priority: nftables.ChainPriorityFilter,
})

set := &nftables.Set{
	Name:    "whitelist",
	Table:   table,
	KeyType: nftables.TypeIPAddr, // our keys are IPv4 addresses
}

// Create the set with a bunch of initial values.
if err := c.AddSet(set, []nftables.SetElement{
  {Key: net.ParseIP("8.8.8.8")},
}); err != nil {
	// handle error
}

c.AddRule(&nftables.Rule{
	Table: table,
	Chain: myChain,
	Exprs: []expr.Any{
		// [ payload load 4b @ network header + 16 => reg 1 ]
		&expr.Payload{
			DestRegister: 1,
			Base:         expr.PayloadBaseNetworkHeader,
			Offset:       16,
			Len:          4,
		},
		// [ lookup reg 1 set whitelist ]
		&expr.Lookup{
			SourceRegister: 1,
			SetName:        set.Name,
			SetID:          set.ID,
		},
		//[ immediate reg 0 drop ]
		&expr.Verdict{
			Kind: expr.VerdictDrop,
		},
	},
})
if err := c.Flush(); err != nil {
  // handle error
}

Theres a lot going on here, so lets dig into it piece-by-piece.

• First, the table and chain are created - nothing new here.
• Second, the set structure & datatype are defined. Sets are typically named (though they can be anonymous), and the length in bytes of the data values must be specified (nftables.TypeIPAddr is sufficient for the library to understand you want to store 4-byte sequences representing IP addresses). Under the hood, the set is assigned a unique ID by the library, which is used to reference it inside rules.
• Lastly, we create our rule, which reads the destination IP out of the packet in question, and halts/continues evaluation of the expressions in the rule based on whether the IP matches data contained in our set. Specifically, the code in the linux kernel looks like this:

  found = set->ops->lookup(nft_net(pkt), set, &regs->data[priv->sreg], &ext) ^ priv->invert;
  if (!found) {
  	regs->verdict.code = NFT_BREAK;
  	return;
  }

  There are two possible outcomes from evaluating this expression:
  • If the destination IP is not present in our set, rule execution stops (and hence the packet is accepted).
  • If the IP is present in our set (in our example, if we send a packet to 8.8.8.8), rule execution continues, meaning the drop packet action expr.VerdictDrop is applied and the packet is dropped.

&expr.Lookup{
	SourceRegister: 1,
	SetName:        set.Name,
	SetID:          set.ID,
	Invert:         true,
},

Want to add stuff to the blacklist dynamically? No problem:

if err := c.SetAddElements(set, []nftables.SetElement{/* Your values here */}); err != nil {
  // handle error
}
if err := c.Flush(); err != nil {
  // handle error
}

Packet counters

Much like IPTables, NFTables has the ability to count packets/bytes which get far enough into a rule to hit a counter.

ourCounter := c.AddObj(&nftables.CounterObj{
	Table: table,
	Name:  "countyboi",
}).(*nftables.CounterObj)

c.AddRule(&nftables.Rule{
	Table: table,
	Chain: myChain,
	Exprs: []expr.Any{
		// payload load 4b @ network header + 16 => reg 1
		&expr.Payload{
			DestRegister: 1,
			Base:         expr.PayloadBaseNetworkHeader,
			Offset:       16,
			Len:          4,
		},
		// cmp eq reg 1 0x0245a8c0
		&expr.Cmp{
			Op:       expr.CmpOpEq,
			Register: 1,
			Data:     net.ParseIP("8.8.8.8").To4(),
		},
		// counter 'countyboi'
		&expr.Objref{
			Type: 1,
			Name: ourCounter.Name,
		},
	},
})

counter, err := c.GetObj(ourCounter)
if err != nil {
  // handle error
}
co, ok := counter[0].(*nftables.CounterObj)
if !ok {
  // handle error
}
fmt.Printf("Bytes: %d, Packets: %d\n", co.Bytes, co.Packets)

Network Address Translation (NAT)

NAT is a feature that rewrites source/destination IP addresses, so multiple hosts appear to be coming from/to a single IP address. This is broadly useful for assigning private addresses to different hosts/programs, but translating those addresses so the traffic can still hit the internet.

The NFTables wiki mentions a number of peculiarities for getting NAT working:

• NAT rules must be added to a NAT chain (adding to a filter chain will result in an error).
• NAT chains currently only work on IPv4 & IPv6 tables - the inet table is not yet supported.
• Both prerouting & postrouting chains must be registered for NAT to work.
• Once a connection has been marked for NAT (or not), the packets of that connection no longer traverse rules in the NAT chain.

// [ meta load iifname => reg 1 ]
&expr.Meta{Key: expr.MetaKeyIIFNAME, Register: 1},
&expr.Cmp{
	Op:       expr.CmpOpEq,
	Register: 1,
	Data:     []byte("eth0\x00"),
},
// [ meta load l4proto => reg 1 ]
&expr.Meta{Key: expr.MetaKeyL4PROTO, Register: 1},
// [ cmp eq reg 1 0x00000006 ]
&expr.Cmp{
	Op:       expr.CmpOpEq,
	Register: 1,
	Data:     []byte{unix.IPPROTO_TCP},
},
// [ payload load 2b @ transport header + 2 => reg 1 ]
&expr.Payload{
	DestRegister: 1,
	Base:         expr.PayloadBaseTransportHeader,
	Offset:       2,
	Len:          2,
},
&expr.Cmp{
	Op:       expr.CmpOpEq,
	Register: 1,
	Data:     binaryutil.BigEndian.PutUint16(22),
},
&expr.Immediate{
	Register: 1,
	Data:     net.ParseIP("1.2.3.4").To4(),
},
&expr.Immediate{
	Register: 2,
	Data:     binaryutil.BigEndian.PutUint16(22),
},
// [ nat dnat ip addr_min reg 1 addr_max reg 0 proto_min reg 2 proto_max reg 0 ]
&expr.NAT{
	Type:        expr.NATTypeDestNAT,
	Family:      unix.NFPROTO_IPV4,
	RegAddrMin:  1,
	RegProtoMin: 2,
},

c.AddRule(&nftables.Rule{
  Table: table,
  Chain: postrouteChain,
  Exprs: []expr.Any{
    // [ meta load iifname => reg 1 ]
    &expr.Meta{Key: expr.MetaKeyIIFNAME, Register: 1},
    &expr.Cmp{
      Op:       expr.CmpOpEq,
      Register: 1,
      Data:     []byte("veth0\x00"),
    },
    // masq
    &expr.Masq{},
  },
})