15 Million Web Servers on One VM With smoltcp

Diagram of running 15 million web servers on one VM with a userspace network stack

One engineer has squeezed more than 15 million web servers onto a single virtual machine with 10GB of RAM and one CPU core. The trick to running 15 million web servers on that little hardware was dropping the Linux network stack and writing the memory-hungry parts in userspace.

The project is a proof of concept, not a product. It still shows how much of a server’s cost sits in the kernel.

Why the obvious approach fails

A minimal C web server needs only a few syscalls: listen, accept, recv, send and close. That version handles around 65,000 requests per second. Launching it 15 million times fails almost at once, at about 28,000 processes, because the kernel runs out of ports.

Linux hands out local ports from a range of roughly 32768 to 60999. The range can be widened to nearly 65,535, but the TCP port number is a 16-bit field, so that is the ceiling. The way out is addresses: a listener is identified by protocol, IP and port, so each server can bind its own IP. Linux’s AnyIP feature lets a machine listen on a whole subnet, and a single IPv6 route supplies 18 quintillion addresses.

Ports were solved. Memory was not. Each server process used 1.5MB of RAM, which works out to about 20TB for 15 million of them.

network card
Image: kapteko

Moving everything into one runtime

The author’s earlier projects ran millions of processes inside one custom runtime, using a sandbox called LFI to isolate guests and replace their syscalls with calls to a host handler. The scheduler only has to save a guest’s program counter and two file descriptors, about 16 bytes each. For 15 million guests that comes to roughly 220MB.

The first estimate assumed idle connections cost nothing and active ones about 5KB. With 1 million active connections, the total came to about 5GB. A test of 14 million idle listeners ran out of memory anyway.

wolfram
Image: kapteko

The kernel’s per-socket cost

The reason is that the socket syscall itself allocates around 3,300 bytes, whether the socket is active, listening or idle. Ten million sockets overflow a 10GB machine without any traffic. Little can be trimmed without rewriting half the kernel.

So the author went down the OSI model. TCP/IP is stateful, but the layers beneath it are stateless and don’t care about the payload. That means the kernel’s TCP/IP stack can be replaced.

osi_model
Image: kapteko

Three ways to bypass Linux

The replacement is smoltcp, a Rust TCP/IP stack built for bare-metal systems that needs no heap allocation. It only parses packets, so it needs a device to talk to the network. The options:

  • TUN, a virtual interface at the IP layer.
  • TAP, a virtual interface at the Ethernet layer.
  • DPDK, which reads and writes the network card directly and skips the kernel.

AF_XDP is another route that lets a program work with the network card’s traffic directly. DPDK has a cost: it runs a busy loop that occupies a whole CPU core. In return it avoids syscalls and the interrupt storms that can lock up a loaded system. Implementing smoltcp’s Device trait takes three methods, and existing crates for DPDK and AF_XDP can help.

The results

With the final design, the runtime loads all the guests, intercepts their syscalls, and answers them with smoltcp. Each guest listens on its own address, and the scheduler polls for incoming data and wakes the right one.

The kernel stack used up all RAM at about 2.5 million servers, and requests per second dropped as the count grew. With smoltcp, an idle server needed 620 bytes and an active one about a kilobyte, which allowed 16 million servers. DPDK was the fastest in the benchmark, with AF_XDP just behind.

What this does not prove

Building a proof of concept is easy, the author admits. A compatible Linux runtime that reimplements syscalls with full POSIX compatibility is far harder, and it is left for a future article. The code lives in the AI10M repository on GitHub. The takeaway is narrower than the headline: when the network stack moves to userspace, a developer controls memory use and can run guests faster.

Source: kapteko

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.