The first viewer is easy. You put a TrackLocalStaticRTP on the server, you write the publisher's packets into it, and the tile lights up. Then a second viewer joins and you find out that "forward the RTP" was never a complete design.
The single-viewer case hides five decisions. Each one is small, and each one is the difference between a demo and a session that behaves the same way for viewer number two.
The whole thing in one line: the publisher sends one stream, the server's egress scales with viewers, and every viewer needs its own negotiation plus its own keyframe request.
In a mesh, three viewers means the publisher encodes and uploads three streams. In an SFU it uploads one. The server reads each incoming packet once and writes it to a local track that is bound to every viewer's sender:
The consequence people miss is where the bandwidth lands. The field technician on a phone keeps a single modest uplink no matter how many people are watching, and the server pays the multiplication. That is the right trade for a support call — the person on site has the worst connection in the system, and the server has the best one.
It is also the number you have to watch when you plan capacity: egress, not ingress, and it grows linearly with viewers per session.
The first viewer connects while the encoder is producing frames, so a keyframe arrives quickly. A viewer that joins ten minutes into a session receives inter-frames that reference a keyframe it never saw. RTP flows, framesReceived climbs, and the tile is black until the publisher's encoder happens to emit the next keyframe — which, for a long GOP, can be many seconds.
The fix is to relay the viewer's feedback upstream. Browsers already send a Picture Loss Indication when they cannot decode; the server's job is to carry it to the publisher. But a PLI carries the SSRC it is about, and a PLI that arrives from a viewer is about the server's local track, not the publisher's upstream track:
Forward the packet unchanged and the publisher sees feedback about an SSRC it does not own, ignores it, and the new viewer waits out the GOP. Rewrite MediaSSRC to the publisher's value and the keyframe arrives in milliseconds. This one line is the difference between "the second viewer is broken" and "the second viewer is instant", and nothing in the browser tells you which one you are looking at.
A publisher's ontrack fires once per track. Audio and video often arrive within milliseconds of each other, and a naive implementation answers each one immediately, which puts two setLocalDescription cycles in flight for the same viewer. That is offer glare on your own connection, and the symptoms are intermittent in the way that wastes an afternoon.
Batch them. A short timer that coalesces whatever arrived into a single offer per viewer is enough:
Two details make it safe: the timer is stored per room so a burst of track arrivals collapses into one callback, and the callback re-takes the lock and re-checks the state, because 250 milliseconds is plenty of time for a viewer to leave.
The control channel looks trivial until it does not. Read it with a buffer sized for the largest datagram you can receive, not for the packet you expect:
An undersized buffer turns into a short read, Unmarshal fails, and the feedback is silently dropped — which surfaces as poor quality that only appears on networks sending large compound RTCP packets.
If the publisher disconnects or republishes, the viewers' PeerConnections are holding transceivers that belong to a session that no longer exists. Patching them is not worth it; tear them down so the next negotiation starts clean:
This pairs with the client-side ladder: the viewer's full-rejoin fallback exists precisely because the server rebuilt its side, and a viewer that reuses its old connection negotiates successfully and receives nothing.
Being explicit about the edges is more useful than pretending they are not there: One publisher per room. Multiple publishers and stream mixing are a different product; this fan-out assumes a single source that everyone watches. The codec is forwarded as-is. The server does not transcode, so if the publisher switches codec mid-session, viewer connections have to be rebuilt. No simulcast or SVC in this version. Every viewer receives the same encoding, which is right for a support call with a handful of viewers and wrong for a 500-person webinar.
Knowing which of those you need is most of the architecture decision. For one technician showing a fault to two or three people, the simple version is not a compromise.