Flow based network accounting with Linux

von Harald Welte (netfilter core team)

Donnerstag, 23.06.2005, UG1: Thoma, 13:30-15:00 Uhr

Many networking scenarios require some form of network accounting that goes beyond some simple packet and byte counters as available from the 'ifconfig' output.

When people want to do network accouting, the past and current Linux kernel didn't provide them with any reasonable mechanism for doing so.

Network accounting can generally be done in a number of different ways. The traditional way is to capture all packets by some userspace program. Capturing can be done via a number of mechanisms such as PF_PACKET sockets, mmap()ed PF_PACKET, ipt_ULOG, or ip_queue. This userspace program then analyzes the packets and aggregates the result into per-flow data structures.

Whatever mechanism used, this scheme has a fundamental performance limitation, since all packets need to be copied and analyzed by a userspace process.

The author has implemented a different approach, by which the accounting information is stored in the in-kernel connection tracking table of the ip_conntrack stateful firewall state machine. On all firewalls, that state table has to be kept anyways - the additional overhead introduced by accounting is minimal.

Once a connection is evicted from the state table, it's accounting relevant data is transferred to userspace to a special accounting daemon for further processing, aggregation and finally storage in the accounting log/database or output via IETF-standardized IP Flow Exchange (ipfix) protocol.

Über den Autor Harald Welte:

Harald Welte is the chairman of the netfilter/iptables core team.

His main interest in computing has always been networking. In the few time left besides netfilter/iptables related work, he's writing obscure documents like the "UUCP over SSL HOWTO" or "A packet's journey through the Linux network
stack". Other kernel-related projects he has been contributing are user mode linux, the international (crypto) kernel patch, device drivers and the neighbour cache.

He has been working as an independent IT Consultant working on projects for various companies ranging from banks to manufacturers of networking gear. During the year 2001 he was living in Curitiba (Brazil), where he got sponsored for his Linux related work by Conectiva Inc.

Starting with February 2002, Harald has been contracted part-time by Astaro AG, who are sponsoring him for his current netfilter/iptables work.

Aside from the Astaro sponsoring, he continues to work as a freelancing kernel developer and network security consultant.

He licenses his software under the terms of the GNU GPL. He is determined to bring all users, distributors, value added resellers and vendors of netfilter/ip tables based products in full compliance with the GPL, even if it includes raisi
ng legal charges.

Harald is living in Berlin, Germany.