BitLLM: A Peer-to-Peer
LLM Inference System

Abstract. A purely peer-to-peer system for LLM inference would let anyone use open models running on other people's GPUs, without going through a company that can meter, censor or switch it off. Open-weight models are now good enough, and home hardware fast enough, that the missing piece is the network. BitLLM is a single program that every participant runs. Nodes find each other through a distributed hash table, learn which models are online over gossip, and stream each answer straight from the GPU that produced it, through a relay only until the two sides can connect directly. Servers earn credits through receipts signed by both sides, which any node can recount.

Using the network

Using an AI agent? Give it bitllm.org/llms.txt: step-by-step setup with a check after each step, for using the network or sharing a GPU.

Install the program and run it. It joins through a bootstrap node and needs no port forwarding and no configuration.

Then open http://127.0.0.1:7778 in your browser. The page lists the models online right now, gives copy-paste setup for Claude Code, Cline, Aider or any OpenAI-compatible client, and sends a test message. Your agent talks to 127.0.0.1:7778, which speaks both the OpenAI and the Anthropic API, and the program forwards each request to a peer serving that model.

Sharing a GPU

If you already run vLLM, SGLang, llama.cpp, Ollama or LM Studio, point BitLLM at it and give the model its network name. Peers reach you through hole punching or a relay, and you choose how many requests to serve at once.

A request for a model you share yourself is answered by your own server and never leaves your machine.

Access

The beta runs on three rules, inspired by private BitTorrent trackers.

1. An invite gets you served on any free slot. Uninvited nodes are served only when a server is idle, unless its owner opens it to everyone.
2. There is no daily quota by default. A server owner can set one.
3. Sharing makes you a contributor: never limited by quota, and 10 invites instead of 3 once your nodes have served.

Invites are signed by an Ethereum account and gossiped like everything else, so there is no membership server. Ask a member to invite your address, then link your node to it from the dashboard. Revoking an account also removes everyone it invited.

How the network works

Discovery. Nodes meet through a Kademlia DHT and a gossip topic on which every server announces its models and free slots once every few seconds.

Connectivity. Most home machines sit behind NAT. A relay carries the first bytes and hole punching then upgrades the link to a direct connection when the routers allow it. All traffic is encrypted.

Credits. After each job the client signs what it received (token counts and a hash of the exact response), the server checks and countersigns it, and the receipt is gossiped to every node, as in BitTorrent's reciprocity [2] but publicly countable. Credits have no value yet.

Load. When every server for a model is busy, your request waits up to a minute for a free slot instead of failing.

Privacy and limits

The machine that serves a request reads its prompt, as with any inference provider, so do not send secrets. Peers you connect to directly see your IP address; where that matters, run BitLLM behind a VPN. Receipts are signed with a pseudonym per server that changes every 30 days, so the public record cannot follow you across servers. Nothing yet verifies that a node serves the model it names; spot checks are next.

References

1. S. Nakamoto, "Bitcoin: A Peer-to-Peer Electronic Cash System," 2008.
2. B. Cohen, "Incentives Build Robustness in BitTorrent," 2003.
3. P. Maymounkov and D. Mazières, "Kademlia: A Peer-to-Peer Information System Based on the XOR Metric," 2002.
4. libp2p, libp2p.io.

github.com/zengjiajun0623/bitllm