A curated list of awesome projects related to eBPF.
BPF, as in Berkeley Packet Filter, is an in-kernel virtual machine running programs passed from user space. Initially implemented on BSD, then Linux, the (now legacy) "classic BPF" or cBPF machine would be used with tools like tcpdump for filtering packets in the kernel to avoid useless copies to user space. More recently, the BPF infrastructure in Linux has been completely reworked and gave life to the "extended BPF", or eBPF, which gained new features (safety and termination checks, JIT-compiling for programs, persistent maps, a standard library, hardware offload support, etc.) and is now used for many tasks. Processing packets at a very low level (XDP), tracing and monitoring events on the system, or enforcing access control over cgroups are but a few examples to which eBPF brings performance, programmability and flexibility.
Recently Cilium launched a great website about eBPF called ebpf.io. It serves a similar purpose to this list, with an introduction to eBPF and links to related projects.
Note: eBPF is an exciting piece of technology, and its ecosystem is constantly evolving. We'd love help from you to keep this awesome list up to date, and improve its signal-to-noise ratio in anyway we can. Please feel free to leave any feedback.
bpf(2)
- Manual page about the bpf()
system call, used to manage BPF programs and maps from userspace.tc-bpf(8)
- Manual page about using BPF with tc, including example commands and samples of code.bpf-helpers(7)
man page - Description of the in-kernel helper functions forming the BPF standard library.IO Visor's Unofficial eBPF spec - Summary of eBPF syntax and operation codes.
Jesper Dangaard Brouer's documentation - Work in progress, contributions welcome.
Emails from David Miller to the xdp-newbies mailing list:
If you are new to eBPF, you may want to try the links described as "introductions" in this section.
A brief introduction to XDP and eBPF - An accessible introduction providing context, history, and details about the functioning of eBPF.
An eBPF Overview - Blog series by Adrian Ratiu, covering many aspects of the eBPF infrastructure:
Ferris Ellis's blog posts about eBPF - They have a few posts about eBPF:
A BPF reference guide - About BPF C and bcc Python helpers, from bcc repository.
Making the Kernel's Networking Data Path Programmable with BPF and XDP - A set of slides covering all the basics about eBPF and XDP (mostly for network processing).
The BSD Packet Filter - An introduction mostly covering the tracing aspects.
BPF: tracing and more - An introduction mostly covering the tracing aspects.
Linux BPF Superpowers - An introduction mostly covering the tracing aspects, first part with flame graphs.
IO Visor - Also introduces IO Visor project.
BPF -- in-kernel virtual machine - Presentation by the author of eBPF.
Extending extended BPF - A blog post from 2014 on the development of BPF and demonstrating what can be done with it, using an example of stateful socket filtering by attaching an eBPF program to a socket.
Greg Marsden made some documentation about eBPF:
Linux Kernel Observability through eBPF - A blog post covering the basics of eBPF as well as code samples in Go on how to build and load a minimal eBPF program into the kernel.
eBPF - From a Programmer's Perspective - A short paper describing the fundamentals of eBPF and how to get started with writing eBPF programs.
Cloudflare's blog posts on eBPF - Different blog posts about networking use cases and low-level aspects of eBPF.
Linux Extended BPF (eBPF) Tracing Tools - An in-depth collection of information around examples of performance analysis tools using eBPF. Contains also a section at the end of the page about other resources.
Beginner's guide to eBPF - A set of live-coding talks and the accompanying code examples, introducing eBPF programming using a variety of libraries and program types.
Daniel Borkmann has made several presentations and papers covering the internals of eBPF, in particular about its use with tc.
Linux Networking Explained - Linux networking internals, with a part about eBPF.
The eXpress Data Path - A very accessible introduction to XDP, providing sample code to show how to process packets.
All XDP details in a technical paper: The eXpress Data Path: Fast Programmable Packet Processing in the Operating System Kernel, by Toke Høiland-Jørgensen, Jesper Dangaard Brouer, Daniel Borkmann, John Fastabend, Tom Herbert, David Ahern and David Miller, all being essential eBPF and XDP contributors.
BPF and XDP Reference Guide - Guide from the Cilium project.
eXpress Data Path (XDP) - The first presentation about XDP.
eXpress Data Path - Contains some benchmark results obtained with the mlx4 driver.
Jesper Dangaard Brouer has several sets of slides describing the internals of XDP:
XDP workshop -- Introduction, experience, and future development (Video)
High Speed Packet Filtering on Linux - About packet filtering on Linux, DDoS protection, packet processing in the kernel, kernel bypass, XDP and eBPF.
How to drop 10 million packets per second - Cloudflare's blog post talking about their move to using XDP for packet filtering.
xt_bpf
module for iptables.tc
, used to manage eBPF filters and actions, and ip
, used to manage XDP programs. Most of the code related to BPF is in lib/bpf.c.LLVM - Contains several tools used in eBPF workflows. Snapshots of the latest versions for Ubuntu/Debian can be retrieved from here.
cargo generate
.bpftool - Also some other tools in the kernel tree, under linux/tools/net/ for versions earlier than 4.15, or linux/tools/bpf/ after that:
bpftool
- A generic utility that can be used to interact with eBPF programs and maps from userspace, for example to show, dump, load, disassemble, pin programs, or to show, create, pin, update, delete maps, or to attach and detach programs to cgroups.bpf_asm
- A minimal cBPF assembler.bpf_dbg
- A small debugger for cBPF programs.bpf_jit_disasm
- A disassembler for both BPF flavors and could be highly useful for JIT debugging.P4 has some interactions with eBPF:
Cilium project (GitHub repository) is a technology relying on BPF and XDP to provide "fast in-kernel networking and security policy enforcement for containers based on eBPF programs generated on the fly". Many presentations available (with overlap):
Open vSwitch (OvS), and its related project Open Virtual Network (OVN, an open source network virtualization solution) are considering using eBPF at various level:
Katran - A layer 4 load-balancer based on XDP, open-sourced by Facebook.
XDP in practice: integrating XDP in our DDoS mitigation pipeline - Protection against DDoS with XDP at Cloudflare.
Droplet: DDoS countermeasures powered by BPF + XDP - Protection against DDoS with XDP at Facebook.
CETH for XDP - Common Ethernet Driver Framework for faster network I/O, a technology initiated by Mellanox.
Suricata, an open source intrusion detection system, relies on eBPF components for its "capture bypass" features:
Project Calico - Calico is an open source networking and network security solution for containers, virtual machines, and native host-based workloads. Calico's eBPF data plane delivers a low latency, high throughput data plane with a rich network security policy model.
merbridge - Use eBPF to speed up your Service Mesh. Merbridge replaces iptables rules with eBPF to intercept traffic. It also combines msg_redirect to reduce latency with a shortened datapath between sidecars and services.
linux/include/linux/bpf.h - with linux/include/uapi/bpf.h: definitions related to eBPF, to be used respectively in the kernel and to interface with userspace programs.
linux/include/linux/filter.h - with linux/include/uapi/filter.h: information used to run the BPF programs themselves.
linux/kernel/bpf/ - This directory contains most of BPF-related code. In particular, those files are worth of interest:
syscall.c
- Different operations permitted by the system call, such as program loading or map management.core.c
- BPF interpreter.verifier.c
- BPF verifier.linux/net/core/filter.c - Functions and eBPF helpers related to networking (TC, XDP etc.); also contains the code to migrate cBPF bytecode to eBPF (all cBPF programs are translated to eBPF in recent kernels).
linux/kernel/trace/bpf_trace.c - Functions and eBPF helpers related to tracing and monitoring (kprobes, tracepoints, etc.).
The JIT compilers are under the directory of their respective architectures, such as file linux/arch/x86/net/bpf_jit_comp.c for x86. Exception is made for JIT compilers used for hardware offload, sitting in their drivers, such as linux/drivers/net/ethernet/netronome/nfp/bpf/jit.c for Netronome NFP.
linux/net/sched/ - and in particular in files act_bpf.c
(action) and cls_bpf.c
(filter): code related to BPF actions and filters with TC.
linux/net/core/dev.c - contains the function dev_change_xdp_fd()
that is called through a Netlink command to hook a XDP program to a device, after is has been loaded into the kernel from user space. This function in turns uses a callback from the relevant driver.
Thank you to Quentin Monnet and Daniel Borkmann for their original work on Dive into BPF: A List of Reading Material which became the basis for this list.
Contributions welcome! Read the contribution guidelines first.
To the extent possible under law, zoidbergwill has waived all copyright and related or neighboring rights to this work.