A curated list of awesome projects related to eBPF.
BPF, as in Berkeley Packet Filter, is an in-kernel virtual machine running programs passed from user space. Initially implemented on BSD, then Linux, the (now legacy) "classic BPF" or cBPF machine would be used with tools like tcpdump for filtering packets in the kernel to avoid useless copies to user space. More recently, the BPF infrastructure in Linux has been completely reworked and gave life to the "extended BPF", or eBPF, which gained new features (safety and termination checks, JIT-compiling for programs, persistent maps, a standard library, hardware offload support, etc.) and is now used for many tasks. Processing packets at a very low level (XDP), tracing and monitoring events on the system, or enforcing access control over cgroups are but a few examples to which eBPF brings performance, programmability and flexibility.
Recently Cilium launched a great website about eBPF called ebpf.io. It serves a similar purpose to this list, with an introduction to eBPF and links to related projects.
Note: eBPF is an exciting piece of technology, and its ecosystem is constantly evolving. We'd love help from you to keep this awesome list up to date, and improve its signal-to-noise ratio in anyway we can. Please feel free to leave any feedback.
bpf(2)
- Manual page about the bpf()
system call, used to manage BPF programs and maps from userspace.tc-bpf(8)
- Manual page about using BPF with tc, including example commands and samples of code.bpf-helpers(7)
man page - Description of the in-kernel helper functions forming the BPF standard library.IO Visor's Unofficial eBPF spec - Summary of eBPF syntax and operation codes.
Jesper Dangaard Brouer's documentation - Work in progress, contributions welcome.
Emails from David Miller to the xdp-newbies mailing list:
If you are new to eBPF, you may want to try the links described as "introductions" in this section.
A brief introduction to XDP and eBPF - An accessible introduction providing context, history, and details about the functioning of eBPF.
An eBPF Overview - Blog series by Adrian Ratiu, covering many aspects of the eBPF infrastructure:
Ferris Ellis's blog posts about eBPF - They have a few posts about eBPF:
A BPF reference guide - About BPF C and bcc Python helpers, from bcc repository.
Making the Kernel's Networking Data Path Programmable with BPF and XDP - A set of slides covering all the basics about eBPF and XDP (mostly for network processing).
The BSD Packet Filter - An introduction mostly covering the tracing aspects.
BPF: tracing and more - An introduction mostly covering the tracing aspects.
Linux BPF Superpowers - An introduction mostly covering the tracing aspects, first part with flame graphs.
IO Visor - Also introduces IO Visor project.
BPF -- in-kernel virtual machine - Presentation by the author of eBPF.
Extending extended BPF - A blog post from 2014 on the development of BPF and demonstrating what can be done with it, using an example of stateful socket filtering by attaching an eBPF program to a socket.
Greg Marsden made some documentation about eBPF:
Linux Kernel Observability through eBPF - A blog post covering the basics of eBPF as well as code samples in Go on how to build and load a minimal eBPF program into the kernel.
eBPF - From a Programmer's Perspective - A short paper describing the fundamentals of eBPF and how to get started with writing eBPF programs.
Cloudflare's blog posts on eBPF - Different blog posts about networking use cases and low-level aspects of eBPF.
Linux Extended BPF (eBPF) Tracing Tools - An in-depth collection of information around examples of performance analysis tools using eBPF. Contains also a section at the end of the page about other resources.
Beginner's guide to eBPF - A set of live-coding talks and the accompanying code examples, introducing eBPF programming using a variety of libraries and program types.
Daniel Borkmann has made several presentations and papers covering the internals of eBPF, in particular about its use with tc.
Linux Networking Explained - Linux networking internals, with a part about eBPF.
The eXpress Data Path - A very accessible introduction to XDP, providing sample code to show how to process packets.
All XDP details in a technical paper: The eXpress Data Path: Fast Programmable Packet Processing in the Operating System Kernel, by Toke Høiland-Jørgensen, Jesper Dangaard Brouer, Daniel Borkmann, John Fastabend, Tom Herbert, David Ahern and David Miller, all being essential eBPF and XDP contributors.
BPF and XDP Reference Guide - Guide from the Cilium project.
eXpress Data Path (XDP) - The first presentation about XDP.
eXpress Data Path - Contains some benchmark results obtained with the mlx4 driver.
Jesper Dangaard Brouer has several sets of slides describing the internals of XDP:
XDP workshop -- Introduction, experience, and future development (Video)
High Speed Packet Filtering on Linux - About packet filtering on Linux, DDoS protection, packet processing in the kernel, kernel bypass, XDP and eBPF.
How to drop 10 million packets per second - Cloudflare's blog post talking about their move to using XDP for packet filtering.
xt_bpf
module for iptables.tc
, used to manage eBPF filters and actions, and ip
, used to manage XDP programs. Most of the code related to BPF is in lib/bpf.c.LLVM - Contains several tools used in eBPF workflows. Snapshots of the latest versions for Ubuntu/Debian can be retrieved from here.
cargo generate
.bpftool - Also some other tools in the kernel tree, under linux/tools/net/ for versions earlier than 4.15, or linux/tools/bpf/ after that:
bpftool
- A generic utility that can be used to interact with eBPF programs and maps from userspace, for example to show, dump, load, disassemble, pin programs, or to show, create, pin, update, delete maps, or to attach and detach programs to cgroups.bpf_asm
- A minimal cBPF assembler.bpf_dbg
- A small debugger for cBPF programs.bpf_jit_disasm
- A disassembler for both BPF flavors and could be highly useful for JIT debugging.P4 has some interactions with eBPF:
Cilium project (GitHub repository) is a technology relying on BPF and XDP to provide "fast in-kernel networking and security policy enforcement for containers based on eBPF programs generated on the fly". Many presentations available (with overlap):
Open vSwitch (OvS), and its related project Open Virtual Network (OVN, an open source network virtualization solution) are considering using eBPF at various level:
Katran - A layer 4 load-balancer based on XDP, open-sourced by Facebook.
XDP in practice: integrating XDP in our DDoS mitigation pipeline - Protection against DDoS with XDP at Cloudflare.
Droplet: DDoS countermeasures powered by BPF + XDP - Protection against DDoS with XDP at Facebook.
CETH for XDP - Common Ethernet Driver Framework for faster network I/O, a technology initiated by Mellanox.
Suricata, an open source intrusion detection system, relies on eBPF components for its "capture bypass" features:
gobpf - utilizing eBPF from Go - A library to create, load and use eBPF programs from Go.
ply - A small but flexible open source dynamic tracer for Linux, with features similar to the bcc tools, but with a simpler language inspired by awk and DTrace.
bpftrace - A tool for tracing with its own high-level tracing language. It is flexible enough to be envisioned as a Linux replacement for DTrace and SystemTap.
kubectl trace - A kubectl plug-in for executing bpftrace programs in a Kubernetes cluster.
inspektor-gadget - A collection of eBPF-based tools to debug and inspect Kubernetes resources and applications.
bpfd - Framework for running BPF programs with rules on Linux as a daemon. Container aware.
BPFd - A distinct BPF daemon, trying to leverage the flexibility of the bcc tools to trace and debug remote targets, and in particular devices running with Android.
adeb - A Linux shell environment for using tracing tools on Android with BPFd.
greggd - System daemon to compile and load eBPF programs into the kernel, and forward program output to socket for metric aggregation.
FUSE - Considers using eBPF.
DEEP-mon - Helps with measuring power consumption for servers and uses eBPF programs for in-kernel aggregation of data.
upf-bpf - An in-kernel solution based on XDP for 5G UPF.
redbpf - Tooling and framework to write eBPF code in Rust efficiently.
pixie - Observability for Kubernetes using eBPF. Features include protocol tracing, application profiling, and support for distributed bpftrace deployments.
Project Calico - Calico is an open source networking and network security solution for containers, virtual machines, and native host-based workloads. Calico's eBPF data plane delivers a low latency, high throughput data plane with a rich network security policy model.
Falco - A cloud-native runtime security project used as a Kubernetes threat detection engine.
SkyWalking Rover - Apache SkyWalking is an open-source Application Performance Monitoring (APM) platform specially designed for distributed systems with microservices, cloud-native and container-based (Kubernetes) architectures. SkyWalking Rover is an eBPF-based profiler and metrics collector for C, C++, Golang, and Rust applications.
Sysmon for Linux - A security monitoring tool. It depends on SysinternalsEBPF.
Red Canary Linux Agent - Red Canary has started to incorporate eBPF to their Linux security sensor.
Tracee - A runtime security and forensics tool for Linux which uses eBPF technology to trace the system and applications at runtime, and analyze collected events to detect suspicious behavioral patterns.
eBPF for Windows - This project is a work-in-progress that allows using existing eBPF toolchains and APIs familiar in the Linux ecosystem to be used on top of Windows.
redcanary-ebpf-sensor - A set of BPF programs that gather security relevant event data from the Linux kernel. The BPF programs are combined into a single ELF file from which individual probes can be selectively loaded, depending on the running operating system and kernel version.
bpflock - Lock Linux machines - An eBPF driven security tool for locking and auditing Linux machines.
merbridge - Use eBPF to speed up your Service Mesh. Merbridge replaces iptables rules with eBPF to intercept traffic. It also combines msg_redirect to reduce latency with a shortened datapath between sidecars and services.
parca-agent - eBPF based always-on continuous profiler for analysis of CPU and memory usage, down to the line number and throughout time.
rbperf - Sampling profiler and tracer for Ruby.
Tetragon - Kubernetes-aware, eBPF-based security observability and runtime enforcement.
Hubble - Network, service and security observability for Kubernetes using eBPF.
Caretta - Instant Kubernetes service dependency map generated by eBPF, right to a Grafana instance.
linux/include/linux/bpf.h - with linux/include/uapi/bpf.h: definitions related to eBPF, to be used respectively in the kernel and to interface with userspace programs.
linux/include/linux/filter.h - with linux/include/uapi/filter.h: information used to run the BPF programs themselves.
linux/kernel/bpf/ - This directory contains most of BPF-related code. In particular, those files are worth of interest:
syscall.c
- Different operations permitted by the system call, such as program loading or map management.core.c
- BPF interpreter.verifier.c
- BPF verifier.linux/net/core/filter.c - Functions and eBPF helpers related to networking (TC, XDP etc.); also contains the code to migrate cBPF bytecode to eBPF (all cBPF programs are translated to eBPF in recent kernels).
linux/kernel/trace/bpf_trace.c - Functions and eBPF helpers related to tracing and monitoring (kprobes, tracepoints, etc.).
The JIT compilers are under the directory of their respective architectures, such as file linux/arch/x86/net/bpf_jit_comp.c for x86. Exception is made for JIT compilers used for hardware offload, sitting in their drivers, such as linux/drivers/net/ethernet/netronome/nfp/bpf/jit.c for Netronome NFP.
linux/net/sched/ - and in particular in files act_bpf.c
(action) and cls_bpf.c
(filter): code related to BPF actions and filters with TC.
linux/net/core/dev.c - contains the function dev_change_xdp_fd()
that is called through a Netlink command to hook a XDP program to a device, after is has been loaded into the kernel from user space. This function in turns uses a callback from the relevant driver.
Thank you to Quentin Monnet and Daniel Borkmann for their original work on Dive into BPF: A List of Reading Material which became the basis for this list.
Contributions welcome! Read the contribution guidelines first.
To the extent possible under law, zoidbergwill has waived all copyright and related or neighboring rights to this work.