IoT SPOFs: Gateways, Brokers and SIM Cards

A client's fleet dashboard can be green across every device and still be one router reboot from a total blackout. That's the pattern I keep finding when I audit IoT deployments: nobody has mapped what sits between the devices and the dashboard. That's exactly where the single points of failure live.

The SPOF map most fleets don't see

Most teams stress-test the sensor to death — battery life, calibration drift, enclosure rating. But they never ask the same question about the infrastructure every reading passes through. That infrastructure includes one MQTT broker, one site gateway, one SIM or ISP uplink, one NTP source, one OTA server.

Each one is invisible on a normal day, and their impact depends on the deployment: one broker can stop telemetry, while an OTA outage may prevent updates without interrupting readings. A dashboard that only watches devices shows 200 green dots until the broker dies, then 200 gray dots that explain nothing.

Broker as SPOF: clustering vs. bridged failover

A single Mosquitto instance handling an entire fleet is the SPOF I see most often — easiest to stand up. Then the VM reboots for a kernel patch and every device queues messages against a socket that isn't answering.

Clustering distributes the broker itself. EMQX splits nodes into Core (writes, replicated via Mria/RLOG) and Replicant (read-only session and routing) roles — if one or more nodes go offline, the cluster keeps operating. HiveMQ takes a masterless approach instead: every node is a peer, and if one drops out the remaining nodes absorb its traffic. Clustering earns its complexity when one broker serves many sites.

Bridged failover is the lighter answer for a single site. Mosquitto's bridge config accepts multiple broker addresses on one connection. With round_robin false (the default), the first address is primary, and the others are tried in order if it fails.

You're not distributing load — you're giving one broker a documented escape hatch to a standby. Don't choose clustering because it sounds correct and then run it as a single node anyway; that's just a broker with extra steps.

Site gateway as SPOF: buffer first, watchdog second

The gateway translates Modbus, BACnet, or serial into MQTT and pushes it upstream. When its WAN link blips — and at industrial sites it will — a naive gateway drops every reading from the gap. Nobody notices until an audit finds the hole.

The fix is store-and-forward: write every reading to local persistent storage, typically SQLite, before attempting to send it. Think of it like a local mailbox at a branch office. If the courier from headquarters can't pick up yet, mail piles up in the box instead of getting thrown out. Azure IoT Edge's edgeHub does this by default, queuing telemetry locally on a configurable time-to-live until reconnect.

The pattern holds regardless of platform: the buffer sits near-empty normally, grows during an outage, and drains in chronological batches once the link returns. That turns a WAN blip into a delay instead of data loss. It won't save you from the gateway process itself hanging on a deadlocked library, though.

That's what a watchdog is for: a separate, dumber process that expects a heartbeat and power-cycles the gateway when one doesn't arrive. Buffering handles network failure; watchdogs handle software failure.

When a cold-standby gateway is enough

Not every site needs active-active gateways. If you're running one site, minutes of downtime are tolerable, and the gateway is commodity hardware, the lazy-correct answer is simple. Keep a second identical unit powered off in the rack, with a documented swap procedure. Same config, labeled cables, a runbook a technician can follow without calling you.

Active-active buys back minutes at the cost of ongoing config sync and split-brain handling. That money is worth spending at a cold-chain site, where ten minutes means spoiled inventory. It's not worth spending at a retail site, where local buffering already absorbs the gap.

Uplink as SPOF: why single-carrier SIMs fail with the carrier

Dual-SIM or multi-carrier modems can protect against one carrier's radio or core-network outage when the alternate path is independent. They do not protect against infrastructure shared by both paths. The FCC's report on AT&T's February 2024 outage found one misconfigured network element caused a nationwide outage. Voice and 5G data went down for roughly 12 hours across all AT&T devices. Over 92 million calls were blocked, including more than 25,000 attempts to reach 911.

The 2022 Rogers Communications outage in Canada had a similar cause at a different layer. A maintenance change deleted a routing filter. Rogers ran wireless and wireline traffic over one shared IP core. The failure took down debit terminals and a quarter of Toronto's traffic signals too — not a radio-tower problem; an independent carrier path could help, while a second SIM using the same failed core would not.

This is exactly the gap in multi-IMSI SIM cards. They store several carrier profiles and switch when one radio network is unreachable. But the profiles commonly route through one shared mobile core, so a core failure takes every profile down at once. Real redundancy means multi-core SIMs or a second physical uplink, not just a second carrier name on the same SIM.

The overlooked pair: NTP and OTA

A single NTP source looks harmless until a drifted clock fails TLS certificate validation, or cross-site log correlation falls apart because timestamps no longer agree. NTP best practice predates IoT: use at least four independent, diverse time sources when available. With only two disagreeing servers there's no way to tell which one drifted, while three lets the outlier be excluded.

A single OTA server is the SPOF nobody budgets for. It only matters on the worst day: pushing a patch to 500 devices when the update endpoint is what's down. A fleet that can't get telemetry out during an outage can't get patches in either. The fix matches NTP's — a mirrored endpoint instead of one origin box, plus staged rollout to reduce the number of devices exposed before an update fault is detected.

A decision framework: match redundancy to consequence

Redundancy costs money and complexity, so spend it where failure actually costs something. That's not whether a layer can fail, but what the device controls and what an hour of silence costs. A soil-moisture sensor reporting hourly tolerates a bridged broker and a buffer; full clustering is money spent on a risk that doesn't exist at that stakes level.

For cold-chain monitoring or an alarm pipeline, assess the tolerated gap and test independent communication paths. Broker clustering and multi-core uplinks are options, not substitutes for local alarm behavior. An undetected gap there is spoiled product, not a dashboard hole. It's the same discipline behind single points of failure in business systems generally. IoT fleets just hide more of them. A green dashboard quietly stops meaning a healthy fleet once any one of these five layers has exactly one path.

Closing checklist

• Broker: clustered, or bridged with a documented failover address
• Gateway: store-and-forward buffering plus a separate watchdog
• Standby gateway: cold spare and swap runbook, sized to downtime cost
• Uplink: multi-carrier or multi-core SIM, or a second physical path
• NTP: at least four independent, diverse sources when available
• OTA: redundant delivery endpoint plus staged rollout

If your fleet needs this scrutiny applied to a real deployment, that's the audit we run as part of IoT backend development engagements.