Historian vs Data Lake: Where Should Industrial Data Actually Live?

Manufacturers are told to put everything in a data lake and simultaneously that the historian is the system of record. Both statements are true in their own layer - the mistake is treating one as a replacement for the other.

What Each Is For

Historians are optimized for high-throughput time-series writes, fast retrieval of recent and ranged data, and compression tuned for process values. They are the operational system of record: SCADA, HMIs, and operators read from them.

Data lakes (object storage with structured tables, query engines, notebooks) are optimized for flexible, exploratory analytics over large volumes, joining many sources, and machine learning. They are where engineering, data science, and management query without impacting operations.

The Common Boundary Pattern

• Acquisition and short-term storage happen in the historian (minutes to months depending on resolution).
• A scheduled pipeline exports aggregated or event-relevant data to the lake - summaries, batches, alarm events, quality results.
• Analytics run on the lake; dashboards needing operational freshness read from the historian.

Design Rules

• Define which system is authoritative per dataset. Dual-write without a designated source creates reconciliation fights.
• Export with a schema and a data dictionary; a lake full of undocumented exports is a junkyard.
• Keep raw high-frequency data in the historian where compression and retrieval are optimized; do not bulk-copy raw streams to the lake just because you can.