I my previous post, Basic eBPF forwarding, I asked the question:
“What happens if the destination IP isn’t in the lxc_map?”
The answer in that post was basically: the packet gets dropped.
To summarize: The lxc_map is a node-wide map that maintains a mapping from a destination-ip to an EndpointInfo. The EndpointInfo contains an interface index. So, when the desination-ip is known, the eBPF program simply redirects it to that interface.
This post is about (very) basic pod-to-pod networking using eBPF. The goal is to create two endpoints, wire them together into an eBPF dataplane, and then send a UDP packet from one to the other to prove that the whole path works using the bare minimum.
The scope is deliberately narrow: same-L2-segment, same-host pod-to-pod forwarding. There is no overlay, cross-host routing, IPv6, policy, conntrack, or upstream connectivity. Those are future work. For now, the goal is to build the smallest possible dataplane that can get one packet from one pod to another.
This post is about a design problem that shows up in many eBPF-based networking project sooner or later, and largely independent of what the dataplane itself actually forwards: how do you attach, track, and tear down kernel-resident programs from a userspace control plane, in a way that survives your own process dying at the worst possible moment?
The project this is drawn from is an early-stage eBPF/XDP/TC dataplane written in Rust on top of aya. The packet-forwarding logic itself — routing, ARP, policy — is still mostly stubbed out. What is built out, and what this post is about, is the layer underneath all of that: the loader that manages the lifecycle of eBPF programs and their pinned kernel state, independent of whatever those programs eventually do.
This framework is heavily inspired by Cilium’s eBPF testing approach. The overall testing architecture comes from Cilium; this post focuses on implementing those ideas in Rust using Aya. If you’re interested in the original design, I recommend taking a look at Cilium’s testing framework. The implementation described in this post is available in my GitHub repository.
When exploring/implementing an eBPF/XDP/TCX dataplane, a question arose:
“How do we actually test this?” For example, when we have an eBPF program
called process_ipv4(), how can we test its functionality? Granted, these
programs are written in Rust (when using Aya), so it looks straightforward
to test them like any other Rust function. However, there are a few
caveats:
Anyone writing eBPF programs for long enough will eventually have a conversation with the verifier. While developing several eBPF programs, I ran into a number of verifier rejections that were not immediately obvious from the source code. In this article I discuss two examples that required looking beyond the Rust source and into the generated BPF bytecode.
The kernel verifier performs static analysis on every BPF program before it is loaded. Its job is to prove that the program is safe to execute under the rules of the eBPF execution model.
I have been working through the implementation of the Raft consensus algorithm in Rust. Part of doing that properly is being able to test it: you need to throw concurrent histories at your implementation and ask whether they are consistent with what a correct register (or log, or key-value store) is supposed to do. That question has a precise name — linearizability — and answering it algorithmically turns out to be a genuinely interesting problem.