Part I · How it works
8.3 Event loops: one thread, thousands of connections
alsoevent loopepollio_uringkqueuenon-blockingnon-blocking iononblockingc10klibuvreactorblocking call
TL;DR
- A thread per connection spends most of its time waiting. An event loop asks the kernel "tell me when any of these sockets is ready", then handles whichever is, one at a time.
- On Linux that question is
epoll. Node.js, nginx, Redis, Python's asyncio and Rust's tokio are built on it. - One blocking call (a slow file read, a CPU-heavy loop) freezes every connection the loop serves.
strace -f -e trace=epoll_wait,epoll_ctl -p PID # the loop at work
On your machine
$ atfq 8.3 Files 1,024 open per process, can be raised to 524,288 Processes 62,844 for your user, threads included Stack 8.0 MiB per thread, can be raised without limit System 14,208 files open on the whole system
An example: Alice's laptop, one of the test machines. In your terminal, this block shows your own machine, read live.
Pitfalls
- Run CPU-heavy work or blocking libraries on a worker thread or process;
async frameworks have helpers for it (
run_in_executor,spawn_blocking). - Every connection is a file descriptor: the open-files limit caps how many you can hold (8.4).
- Regular files are always "ready" for epoll: disk reads still block, which is why runtimes use threads, or io_uring, for files.
Deep dive 2 questions
How does epoll scale?
The older select and poll take the whole list of descriptors on every
call, so the cost grows with the number of connections. With epoll,
the program registers each descriptor once; the kernel keeps a ready list
filled by the network stack, and epoll_wait returns only the descriptors
that have something to do. Ten thousand idle connections cost nothing.
What does io_uring change?
Instead of asking "is it ready?" and then calling read, a program puts
whole requests in a ring shared with the kernel and collects results from
another ring: fewer system calls, and real asynchronous file I/O. It's
newer and has had security issues, so some distros and containers turn it
off.