Broker — RingLoom

The broker is a standalone process. It owns the node identity, the broker metadata file, local service registration, cross-node routing, peer connections, cluster state, and runtime observability.

Services do not link against broker internals. They write control messages and outbound remote messages into the broker’s shared-memory rings. The broker consumes those rings and performs the control-plane and network work on behalf of local services.

Responsibilities

Area	Broker responsibility
Identity	Own service ID `0` on the configured `broker.node.id`.
Metadata	Create the broker metadata file under `<storage_path>/<group>/services/`.
Service registration	Accept `RegisterService` messages and track local service instances.
Discovery	Send complete service-instance snapshots to local subscribers.
Heartbeats	Detect stale services and notify local and remote subscribers when instances disappear.
Local-to-remote routing	Drain the broker send ring and forward messages to peer brokers.
Remote-to-local routing	Receive peer TCP frames and write application payloads into target service rings.
Cluster management	Maintain peer membership, admin messages, heartbeats, leader election, and service state sync.
Observability	Publish counters, ring positions, heartbeat state, flow-control state, and errors through metadata.

Broker metadata

At startup, the broker creates a metadata file for its node. The file contains:

a fixed metadata header;
a control ring for service-to-broker control messages;
a send ring for outbound cross-node application messages;
optional flow-control and per-peer send-counter regions;
counter and error-log regions used by monitoring tools.

The broker metadata file is the local service entry point. Services open it during RingLoomEngine.start, allocate a service ID, and write registration messages into its control ring.

Event loops

In the default dedicated mode, the broker runs three event loops:

Loop	Main duties
Control loop	Registration, unregister, discovery subscriptions, service heartbeat checks, service leader election, cluster admin commands, flow-control updates, and control-plane counters.
Sender loop	Drain the broker send ring, route records by target node ID, enqueue frames per peer, submit TCP writes, publish per-peer send counters, and manage reconnects.
Receiver loop	Accept peer connections, validate handshakes, read framed TCP messages, route application messages to local service rings, and dispatch admin messages.

Each loop follows the same duty-cycle pattern: process available work, return a work count, and let the idle strategy decide whether to spin, yield, sleep, or block when no work was done.

Control plane

Local services write control messages into the broker control ring. Key message types include:

Template	Direction	Meaning
1	service → broker	Register service.
3	service → broker	Subscribe to updates for a service name.
5	service → broker	Unregister service during graceful shutdown.
2	broker → service	Registration response.
4	broker → service	Complete current instance list for a service name.
6	broker → service	Leader changed.

The broker sends complete snapshots, not deltas, so service clients can replace their target instance list with the latest broker view.

Sender path

Remote sends enter the broker through the send ring in the broker metadata file. Local services write records that include the target node ID, target service ID, template ID, correlation ID, and application payload.

The sender loop:

reads committed records from the send ring;
checks the target node ID;
finds the peer connection for that node;
wraps the message in the TCP frame format when needed;
enqueues the frame in the peer write queue;
submits TCP writes through ringloom_tcp.

If the peer is disconnected or a queue is full, the broker records the event through counters and error state. RingLoom is best-effort across disconnects; applications use correlation IDs and timeouts when they need request/response semantics.

Receiver path

The receiver loop owns the broker listener socket and incoming peer connections. It accepts sockets, validates the broker handshake, reads length-prefixed frames, and classifies each frame as application data, admin traffic, or heartbeat.

Application data is routed to the local target service message ring. Admin traffic goes to the control/cluster path. The receiver follows an always-read model: it keeps draining sockets and drops or counts messages that cannot be delivered instead of letting one slow target block all traffic from a peer.

Threading modes

Mode	Shape	Use case
`dedicated`	Separate control, sender, and receiver loops.	Default and easiest to tune.
`shared_network`	Control loop plus shared network loop.	Fewer threads while preserving control isolation.
`shared`	Combined broker work in fewer runtime threads.	Development or constrained environments.

Idle strategies tune latency versus CPU usage. backoff is a practical default, while busy_spin is appropriate only when CPU isolation is available and the lowest wakeup latency matters.

Executable lifecycle

The user-facing broker artifact is ringloom-broker.

zig build run -- --config path/to/broker.properties
zig-out/bin/ringloom-broker --config path/to/broker.properties

Startup resolves configuration, creates metadata, initializes counters and error logs, starts the control/sender/receiver loops, initializes peer transport, and blocks until shutdown. Shutdown stops loops, closes peer transport, unmaps metadata, unregisters runtime resources, and exits with a meaningful process status.

Operational tools

Use these broker-facing tools during development and operations:

zig build stat -- --storage-path /dev/shm --group ringloom
zig build observability
zig build run-observability -- --storage-path /dev/shm --group ringloom --listen 127.0.0.1:9464

ringloom-stat is a human CLI for inspecting metadata. ringloom-observability is a long-running Prometheus exporter that maps metadata read-only and serves /metrics, /healthz, and /readyz.