How BitLLM works
One binary, four jobs: client, gateway, server and relay. This page describes every moving part: how a fresh install meets the network, how a request finds a GPU, what is signed, what is only claimed, and what happens when something dies.
BitLLM is a peer-to-peer inference network for open models. There is no coordinator in the request path, no central queue, and no database of who served what. A node joins by finding one seed, then learns the rest from a Kademlia DHT and a gossip topic. The model names are ordinary strings; the network only routes on them. The binary is the same for everybody, and flags decide which hats it wears.
Overview
Four loops run inside a serving node. A node that only uses the network runs the gateway and listens to gossip:
- Gossip. Every 10 seconds a serving node or public relay publishes an announce: version, the models it serves, free slots, queue depth, whether it can relay, and optionally a coarse location and weight hashes. Peers keep the latest announce for three periods, so a node that stops answering disappears from routing within about 35 seconds.
- DHT providers. A serving node records itself as a provider of each model it announces, under a key derived from the model name. It does so at start, when its models change, when a seed connects, when its own addresses change, and otherwise every 30 minutes. A record stays valid for 48 hours and the address in it for 24. A node that does not know a model's servers yet looks them up by name.
- Upstream check. A serving node asks its local model server for
/modelsbefore it announces. A stopped vLLM makes the node go quiet instead of advertising capacity it cannot use. - Gateway. The local HTTP server on
127.0.0.1:7778accepts OpenAI and Anthropic requests, chooses a peer, and streams the answer back. It is the only part a user normally touches.
Four things are centrally hosted, and none of them carries inference traffic: the DNS seed list, the static site and installers, GitHub releases, and the map snapshot in Cloudflare KV. Three more still rest on one party: one root key at the top of the invite tree, auto-update from releases signed with keys in the binary, and the auditor the maintainers run as the --audit default. DNS is used only to find seeds; after that, peers find each other.
Roles and reachability
Roles are not separate programs. They are combinations of flags and network position.
| role | how it starts | what it needs | what it does |
|---|---|---|---|
| client | bitllm | outbound internet | local gateway only; no model, needs no inbound port |
| serving node | --upstream URL | a local OpenAI-compatible server | announces models, accepts inference streams, signs receipts |
| public relay | --public | inbound TCP and UDP 4001 | DHT server, relay for NATed peers; still joins the seeds; add --upstream URL to serve a model too |
| seed | --dht-server | a public relay listed in DNS | first contact; starts alone, answers DHT queries, relays |
A serving node can be behind NAT or CGNAT. It does not need a public address or a port forward: it reserves a relay slot and, when the routers cooperate, upgrades to a direct connection. A public relay is any host that can accept inbound TCP and UDP on 4001. It needs no DNS record and no model: a bare --public relay works once the seeds run v0.5.0, because they forward an announce that lists no models. A seed is a public relay that is also named in _dnsaddr.bootstrap.bitllm.org, so a fresh install can find it without knowing anyone.
This split is deliberate. One seed is enough to join, but it must not be the only relay: relay capacity should grow with the number of public nodes, not with DNS edits. Every public node announces relay=true; NATed nodes add those peers to their autorelay candidate set alongside the seeds.
First contact
A node with no --bootstrap resolves /dnsaddr/bootstrap.bitllm.org. Each dnsaddr= TXT record at that name is one seed multiaddr, so seeds can be added or retired without a release. The production list currently names seed1.bitllm.org over TCP and QUIC. The node also carries the old boot.bitllm.org tunnel as a literal fallback for one more release. A --bootstrap value that names a peer directly is a join code; it works even when DNS does not.
Two details matter at this step:
- Self-filter. A node removes its own peer id from the resolved addresses before dialing. A seed that is also a normal participant would otherwise connect to itself, reserve a relay slot on itself, and count a duplicate connection.
- Failure is survivable. If the name does not resolve, the node tries the fallback; if that fails too, it starts alone, prints its addresses, and keeps redialing every 15 seconds. A join code supplied later is enough to attach it to the network.
The node asks DNS only for seeds. It does not resolve peers by name, and it does not use DNS to decide who serves a model.
Discovery
Two mechanisms run in parallel, because they answer different questions.
Gossip: who is online now
Every node subscribes to a libp2p pubsub topic, but not every node announces. A client-only node stays silent, a serving node publishes an announce every 10 seconds, and a public relay with no model publishes one that lists no models. The announce is small and unsigned beyond transport identity:
{
"v": "v0.5.0",
"models": ["nvidia/GLM-5.3-Flash-NVFP4"],
"free": 1, "max": 2, "q": 1,
"relay": true,
"loc": "u33",
"weights": {"nvidia/GLM-5.3-Flash-NVFP4": "<64 hex digits>"}
}
v is the release tag. q and relay appear only when there is a queue or the node is public. loc is a three-character geohash cell, sent only with --location. A weight hash is the 64 lowercase hex digits that bitllm hash-model prints, sent only with --weights.
An announce is trusted only while it is fresh: a peer last seen within 3 × announce interval + 5 s (about 35 seconds) is a routing candidate. That gives fast failure detection without a heartbeat server. A server that stops answering goes quiet on its next failed upstream check, and clients stop choosing it within three periods.
DHT: who serves this model, ever
Gossip may not have reached a node that just started, so every serving node also publishes a provider record under a key derived from the model name:
key = sha256("bitllm/model/" + model)
The key keeps the exact model string, including case and slashes, so nvidia/GLM-5.3-Flash-NVFP4 is a different key from NVIDIA/GLM-5.3-Flash-NVFP4. A client with no candidates for a model asks the DHT for its providers and connects to them; this is the cold-start path. Provider records are refreshed every two minutes and expire after a day, so an offline node eventually leaves the index.
The DHT stores routing and provider records, not model files and not inference results. It is a phone book, not a switchboard.
Connectivity
Most nodes are behind NAT or CGNAT. A node that can accept inbound connections listens on TCP and QUIC (UDP) 4001. A node that cannot still participates:
- It reserves a slot on a relay: the seeds first, then public relays discovered through gossip. The reservation is a libp2p circuit address.
- Peers dial the circuit address. The relay forwards the first bytes and never sees plaintext, because the stream is end-to-end encrypted.
- Both sides attempt hole punching, and each sends the other its listen addresses to do it. If the routers allow it, the connection upgrades to direct and the relay leaves the path. If not, the circuit stays for the life of the stream.
Relay candidates are the seeds plus every recently seen peer that announced relay=true, shuffled so a new public relay gets traffic without a coordinated rollout. A node limits how many relays it reserves on, and it never treats a relay as a model server unless that relay also serves the model.
Transport encryption is Noise/TLS/QUIC depending on the transport. The serving machine still reads the plaintext prompt; encryption protects the path, not the endpoint.
Request path
The gateway listens on localhost unless --gateway says otherwise. A request goes through these steps:
- Parse and bound. The body is read up to a fixed limit, the model is read, and image-heavy requests are defused for older peers. Malformed JSON is a
400. - Own model stays here. If this node shares the requested model, the request is answered locally. It never leaves the machine, costs nothing, and is not metered as network work. If the local server is down, the gateway answers
503. With--peer-fallbackit tries peers that serve the same model instead, and those peers can read the request. - Candidates. Peers are filtered by fresh announce, by model, by weight hash when the model manifest records one, and by audit status. They are ranked by free slots, queue depth and measured latency. A peer never dialed has no latency and ranks behind a peer that just answered.
- Try up to three. For each candidate the gateway opens an inference stream with a bounded wait. A server with no free slot keeps the request in its own queue; the client waits up to its own deadline. A server that refuses the caller outright (
403, an allow or deny list) is skipped for the rest of the request. - Stream. The serving node forwards the upstream response. Tokens are written to the local client as they arrive; a disconnect on either side tears down the stream. The response carries
X-Bitllm-Peer, the peer id that answered. - Retry or fail. If every attempt was busy, the gateway backs off, refreshes the candidate list, and tries again until its deadline. If nobody serves the name, it returns
503 no peer serves model X. If every server refused, it returns the status they gave, withall peers refused. If everyone failed, it returns502 all peers failed.
Both OpenAI (/v1/chat/completions, /v1/completions) and Anthropic (/v1/messages) shapes are accepted. The gateway does not translate between them; it forwards the shape the client sent, so the serving upstream must support it. vLLM supports both.
Queueing is per server, not global. A server orders waiting requests by access class first, then by arrival; a request's class improves one step every 20 seconds of waiting, never past the best class, and after that the longest wait goes first among equals. A full queue refuses immediately rather than accepting a place that does not exist.
Serving
A serving node points at an OpenAI-compatible base URL:
bitllm --upstream http://localhost:8000/v1 --concurrency 2
At startup it calls GET /models on that upstream. Unless --model was passed, every id the server reports becomes a public model name, unchanged. While the node runs, it re-reads the list every 30 seconds when the upstream is idle, so a model added to vLLM later appears without restarting BitLLM. A model removed from the upstream disappears from announces and the DHT on the next refresh. Passing --model pins the mapping: it is for renaming a local id, or for sharing a subset, and it stops the refresh.
Other serving behavior:
- Concurrency.
--concurrencyis how many requests the upstream can take in parallel. Extra requests wait in the node's queue; the announce reports free slots and depth so clients can choose. - Request fields. A host forwards only the request fields it knows from a peer: the standard OpenAI and Anthropic fields and the audit fields. Images must be
data:URLs, and a/completionsrequest carries one prompt.--peer-field NAMEadds a field and can be repeated. Requests written on your own machine are not filtered. - Upstream limit.
--upstream-timeoutis the longest your server may take to send response headers; the default is 10 minutes. A non-streaming answer sends them only after generating. - Images. If the upstream caps images per prompt (vLLM's
--limit-mm-per-prompt), the node learns the cap from the first refusal and keeps only the newest N images, replacing older ones with a short note. Long agent sessions therefore do not fail on every turn. - Token cap.
--max-tokensbounds new tokens per peer request (default 32768). A peer gets one choice per request;nis dropped. - Quotas.
--free-dailysets a per-member daily quota; the default is unlimited. A contributor has no quota on a server that asked its node for a job in the last 7 days. - Allow lists.
--allow PEER_IDserves only the listed peers and refuses everyone else. The error names the caller's peer id, so the owner can add it. - Deny list.
--deny PEER_IDrefuses one node while everyone else is served as before, including on an--openserver. A v0.5.1+ client treats the refusal as permanent and tries another server. - Attribution log.
--attribution-logwrites one local line per countersigned receipt with the node id, address and token counts, never the prompt or the answer. Off by default. - Weights.
--weights NAME=HASHstates what the server loads, frombitllm hash-model DIR. It is a claim, not a proof. When the model manifest records a canonical hash, clients skip peers that announce a different one.
A model you share stays on your machine unless --peer-fallback is set and your server is down. Your own requests are never refused. They hold a slot, so the free count the node announces stays true, and the node earns no credits from itself.
Model names
A model name is the string clients send. Two machines that announce the same string are interchangeable for it: the gateway picks one, and if it stops answering it tries another. The name must therefore identify the weights, not the machine.
- A public model uses its exact Hugging Face repo id.
nvidia/GLM-5.3-Flash-NVFP4, with the slash and the case as published. Independent hosts that serve the same repository match without a central registry, and one can replace another when it goes down. - A fine-tune or quant uses the repo id of the actual weights when it has one; otherwise the base id plus a suffix that says what changed. One name means one set of weights. Example:
junmo-k/GLM-5.3-Flash-FP4-Local-Piecrust-Abliterated-MLX. - A private or brand-new model can use any stable name. Nothing is registered first: the node announces it,
GET /v1/modelslists it, and the dashboard lets a user pick it.--model local=publicexists only for the case where the server's local id is not the name clients should use.
The naming pattern is <model>[-<quant>][-<source>], joined with hyphens; the model part keeps the Hub id's case and slashes. Quant is the weight format when it is not the maker's release (fp4, nvfp4, int4, q4_k_m); source is who changed the weights (a fine-tune name, or the group that made the quant). How a server runs the model (vLLM or SGLang, context length, fixed sampling) does not change the name.
No registry is required. The optional table in the README lists reference names the beta has used; it is a convenience, not a gate.
Receipts and credits
Serving earns credits, and credits are countable without a ledger server:
- The client signs what it received: model, prompt and completion token counts, and a SHA-256 of the exact response bytes. It signs with a pseudonym, one key per server rotated every 30 days, so the receipt shows which server did the work but not who asked. That server can link one client's jobs to each other.
- The serving node checks the claim against what it sent and countersigns. A receipt that does not match the token formula is rejected.
- The receipt is gossiped. Any node can verify both signatures, recompute the credit (
prompt tokens / 4 + completion tokens), and aggregate per server.
Credits are a public count, not a token or a promise. They are self-reported: both signatures show what the two sides claim, not that the work happened, and a node's table depends on the receipts it has heard. A new node catches up from its bootstrap peers; a node accepts at most 120 receipts a minute per server and 600 overall from gossip, which bounds flooding. Receipts live in ~/.bitllm/receipts.jsonl, and GET /credits returns the current table.
What counts toward future value is not decided yet. Self-dealing through fake clients, or serving a model other than the one named, is expected to be excluded when the rule is published.
Access
The beta is invite-only in the BitTorrent sense: no account is needed to try the network, and membership decides who gets a free slot first, not who can connect.
- An invite is a signature from an Ethereum account. Nodes gossip invites; there is no membership server. Revoking an account takes membership from it and from everyone it invited; they are then served like uninvited nodes.
- A member is served on any free slot. An uninvited node is served only when a server is idle, unless its owner starts with
--open. - Sharing makes you a contributor: 10 invites instead of 3 once your nodes have served, and no quota on a server that asked your node for a job in the last 7 days.
- A server owner can still restrict serving to an allow list (
--allow), refuse one node (--deny), or set a daily quota. Those are per-server decisions, not network policy.
The membership root is one Ethereum address in the binary's defaults; --roots replaces it. A node with no roots treats every node as a member, which is useful for a private test network.
Audits
A node can claim a model it does not serve. Auditing is an opt-in spot check, not a proof:
- With
--audit, about one request in 50 to a peer is followed by a canary request. Another node, the auditor, scores the answer token by token. - The default auditor is a node run by the maintainers.
--auditor PEER_IDreplaces it;--auditor nonedrops it. - A peer that keeps failing gets no requests for that model for 24 hours.
GET /auditsshows the results. - It finds a server running a different model, and a coarse quant on greedy requests. It does not find a server that recognizes canaries, a quant swap on sampled requests only, or a light fine-tune swap. An auditor can lie in either direction. A shuffled control catches only an auditor that passes everything. The canary prompts are public in the source.
Auditing is off by default because it costs a second request and a second node. Turning it on improves the experience for that node without changing the network's guarantees.
Privacy boundary
- The serving peer reads the prompt in plain text. Transport is encrypted, but the GPU host is an endpoint. Do not send secrets.
- A peer you connect to directly sees your IP address. A relayed peer sees the relay's address on that connection. The serving peer sees your node id either way.
- Hole punching sends your listen addresses to the other peer. A node sends them when it tries to turn a relayed connection into a direct one, so a peer reached through a relay can still learn them.
- Nodes bind all interfaces. Every node, including a client, listens on the p2p port on all IPv4 and IPv6 interfaces. UPnP asks your router to forward that port and mDNS looks for peers on your local network; both are on by default.
--upnp=false,--mdns=falseand--ipv6=falseturn these off. - Linking a wallet is public. A claim ties your node id to your wallet address and display name, and every node can read it.
- A location is opt-in.
--location LAT,LONis turned into a three-character geohash cell (about 156 km by 156 km) before it leaves the machine. The point itself is never announced. Without it, the node is counted as "no location". - IP geolocation is not used, on purpose: it would place people who never asked, and for relayed nodes it would often place the relay instead.
- Receipts are pseudonymous. A client signs with one key per server, rotated every 30 days, so a receipt links a server to work but not to the person who asked. That server can link one client's jobs to each other.
- A host can keep a record of who it served. With
--attribution-loga server writes the node id and address behind each receipt it countersigns, and the token counts, to a local file. It does not write the prompt or the answer, and a job whose client does not send a receipt back leaves no line. It is off by default. - A host decides whom it serves. Do not use the network to attack systems you have no permission to test, or to harm people. The machines belong to people who share them for free, and a host can refuse any node with
--deny. What a host does when it finds abuse is in docs/abuse.md.
Failure modes
| what fails | what happens | what you see |
|---|---|---|
| a seed is down | other DNS records still resolve; redial every 15 s | slower first contact, then normal |
| DNS does not resolve | fallback tunnel, then start alone | node prints its addresses; a join code can attach it |
| a serving node stops | announce expires in ~35 s; DHT record expires in a day | clients route elsewhere; cold lookups may still list it |
| an upstream stops | node stops announcing models; requests for your own model get 503 unless --peer-fallback is set | upstream ...: up=false in the log |
| every server is full | queueing, then retry until the client deadline | 429 busy ... |
| no server for a model | DHT lookup, then failure | 503 no peer serves model X |
| a relay dies | reservation is lost; node reserves on another public relay or the seed | streams blip; clients redial |
| the self-update restarts a node | restart waits for idle, for up to 24 hours, then happens anyway; after three failed starts the node goes back to the previous binary; same peer id and data dir | a short gap in announces |
Operations
Run a seed
bitllm --dht-server --port 4001 \
--announce /ip4/203.0.113.10/tcp/4001 \
--announce /ip4/203.0.113.10/udp/4001/quic-v1
A seed starts without joining the default seeds (it may be the only one), answers DHT queries, and relays. It needs no GPU. Open TCP and UDP 4001 in the firewall and the cloud security list; TCP alone works but loses QUIC and some hole punching. Announce the public address explicitly when the machine is behind a 1:1 NAT, or the node will advertise its private one. Publish the seed's multiaddr as a dnsaddr= TXT record at _dnsaddr.bootstrap.bitllm.org.
Run a public relay
bitllm --public --port 4001 \
--announce /ip4/203.0.113.11/tcp/4001 \
--announce /ip4/203.0.113.11/udp/4001/quic-v1
--public is a public DHT server and relay that still joins the default seeds. It needs no model: a bare --public relay works once the seeds run v0.5.0. Add --upstream URL to serve a model as well. It needs no DNS record: NATed nodes discover it through its relay=true announce and add it to their relay candidates. This is how relay capacity grows without editing DNS.
Keep a node running
Use a systemd unit (Linux) or a LaunchAgent (macOS) with restart-on-failure. Keep the data directory: it holds the peer identity, receipts, claims and quota state. A node that loses its data directory comes back as a new peer.
Updates
Nodes update only to a release signed with a key in the binary. A release with a missing or wrong signature is refused and the running binary stays as it is. A release build checks for a newer release 30 seconds after start and every 6 hours, verifies the binary against the release's signed SHA256SUMS, and restarts in place when the node is idle: same unit, same peer id. If the node stays busy it restarts anyway after 24 hours. --auto-update=false disables it; bitllm update installs now, bitllm update --check only reports, and bitllm update --tag vX.Y.Z installs one named release through the same checks. The previous binary is kept as bitllm.prev, and after three failed starts of a release it installed by itself, the node goes back to it. Builds from source never update themselves.
Ports
4001 TCP and UDP is the p2p port. 7778 is the local gateway and listens on 127.0.0.1 unless --gateway moves it; set --gateway-key if other machines can reach it. The gateway refuses non-local Host headers, so a browser on the LAN cannot reach it by the machine's name.
The map
Every node can produce an aggregate of the network at GET /network: counts per model, per version, and per ~156 km geohash cell, plus how many nodes did not declare a location. It contains no peer ids and no addresses.
The map at bitllm.org is a snapshot of one node's /network, copied into Cloudflare KV by a GitHub workflow and served as /network.json. The landing Worker's one minute cron dispatches that workflow whenever the snapshot is more than three minutes old, so a new snapshot lands about every three minutes. The uploader never accepts inbound requests. The page fetches the snapshot once a minute, and a snapshot past 15 minutes is marked stale and the totals blank out. The key expires one hour after the last write, so a publisher that stops leaves no numbers rather than old ones, and the page shows an empty network instead of failing. Counts are for serving nodes only, so a client-only node does not appear.
Only operators who passed --location are placed. The rest are counted in the "no location" bucket. Nothing is inferred from IP addresses.
Reference
Local API
| endpoint | what it returns |
|---|---|
GET /v1/models | models online now, with the number of nodes serving each |
POST /v1/chat/completions | OpenAI chat, streaming or not |
POST /v1/completions | OpenAI completions |
POST /v1/messages | Anthropic Messages |
GET /status | version, peer id, connections, models served, slots, queue, last update check |
GET /queue | slots, running jobs, wait-list depth, oldest waiter, refusals by reason |
GET /network | the map aggregate described above |
GET /credits | credits per serving peer, from every receipt this node has seen |
GET /audits | recent audit results |
GET /members/me, GET /members | this node's membership, and the member list |
GET / | the dashboard |
Important flags
| flag | effect |
|---|---|
--upstream URL | share an OpenAI-compatible server |
--upstream-key KEY | API key for the upstream; BITLLM_UPSTREAM_KEY keeps it out of the process list |
--upstream-timeout D | longest your server may take to send response headers; default 10 minutes |
--peer-fallback | if your own server is down, send requests for your model to peers, which can read them; off by default |
--peer-field NAME | request field peers may send to your server beyond the standard ones; repeatable |
--gateway ADDR | where the gateway listens; default 127.0.0.1:7778 |
--gateway-key KEY | key every gateway request except the dashboard page must carry, as Authorization: Bearer or x-api-key; BITLLM_GATEWAY_KEY keeps it out of the process list |
--model local=public | rename a local id, or pin a subset; stops the 30 s refresh |
--concurrency N | parallel requests the upstream accepts |
--public | public DHT server and relay; joins the seeds; needs no model |
--dht-server | start as a seed; does not need the default seeds |
--bootstrap ADDR | join through a specific peer (a join code) |
--location LAT,LON | declare a coarse location for the map |
--audit, --auditor ID | spot-check peers that answer you |
--audit-every N | with --audit: real requests to a peer between spot checks; default 50 |
--audit-serve | offer this node as an auditor, if its server returns prompt logprobs |
--roots ADDR | root account of the invite tree, for private networks and tests |
--allow PEER_ID | serve only listed peers |
--deny PEER_ID | refuse one node; repeatable |
--attribution-log | record the node id and address behind each countersigned receipt, never the prompt |
--open | also serve uninvited nodes on spare slots |
--open-slots N | with --open: most slots uninvited nodes may hold at once |
--open-daily N | with --open: credits all uninvited nodes together may use per UTC day |
--free-daily N | per-member daily quota; 0 = unlimited |
--max-tokens N | cap new tokens per peer request |
--ipv6=false | do not listen on IPv6; keeps the p2p port off a public IPv6 address |
--mdns=false | do not look for peers on the local network |
--upnp=false | do not ask the router to forward the p2p port |
--auto-update=false | do not replace the binary on a new release |
bitllm update --tag vX.Y.Z | install one named release through the same signature and checksum checks |
bitllm invite --revoke 0xADDRESS | revoke an account; roots only |
Ports and files
4001/tcp,4001/udp: libp2p; every node binds them on all interfaces, and only public nodes need inbound.127.0.0.1:7778: gateway and dashboard; local unless--gatewaymoves it.~/.bitllm/: peer key, receipts, claims, quota, update state. Keep it to keep the identity.