Three questions in thirty seconds
Observability in a plant is not the same as in a cloud-native service. A useful test is whether an operator or engineer can answer three questions within about half a minute:
- Is the line running?
- What is abnormal right now?
- What changed in the last hour?
Everything else is detail that should appear only when someone drills down.
Two audiences, two views
Keep plant-operations metrics (state, rate, quality, alarms) apart from IT infrastructure metrics (servers, switches, brokers, disks). Operators do not need a server dashboard, and infrastructure teams do not need a throughput chart. Do keep them linkable, because the interesting incidents sit between the two.
Measure the data path, not only the machines
A dashboard that looks calm may simply be showing old data. Monitor the pipeline itself:
- Data freshness: the age of the latest value per source. Silence is a signal.
- Connection status of every PLC, gateway and broker session.
- Queue depth and backlog in store-and-forward buffers and brokers.
- Message and error rates per gateway.
- Clock offset per device (see time synchronization).
- Edge device health: CPU, memory, disk, temperature.
- Historian write lag.
Layer the dashboards
Start with a single overview screen, then offer line, cell and asset detail on demand. Avoid decoration that conveys nothing: no rainbow palettes and no gauges that only look impressive. The practical build is described in industrial dashboards with Grafana.
Design alerts like alarms
- Every alert has an owner and a documented action. If there is nothing to do, it is not an alert.
- Alert on symptoms that matter (no fresh data for 5 minutes) rather than every cause.
- Group and suppress duplicates so one fault produces one notification.
- Review noisy alerts regularly and retire them.
The same principles apply to process alarms: see alarm management.
Correlate IT and OT events
A switch reboot and a loss of PLC communication are one incident seen from two sides. Show events from both worlds on the same time axis, as annotations on the charts, and make sure their clocks agree. That single view turns hours of guesswork into a short list of hypotheses.
Start small
Pick one line and three signals: is it running, is its data fresh, is the gateway healthy. Get those right and trusted before widening the scope.