Now more than ever, industrial operations and manufacturers are racing to ‘unlock’ process data to leverage their plant-floor information. It’s the challenge of the decade. In order to address this challenge, we need to be looking at what time-series databases and data historians can do.Correct use of these database technologies will mean the difference between simply having data, and being able to analyse and use data to maintain operations and improve the performance of the plant.

So which is better for your plant? We compare the strengths and limitations of the more traditional data historians and the newer open-source time-series databases to find out.

Strengths of data historians

Data historians are built for niche industries, such as pharmaceuticals, oil and gas, manufacturing, and any kind of process engineering. Those historians specific to manufacturing are designed to measure specific production processes, such as:

total defects for a particular shift
vibrations of a motor fan on a production line
pH levels for a water treatment plant
current speed of a conveyor
when a human-entered data event occurred
total amount of ingredient added to tank

Plus, they have:

compatibility with industrial software such as HMIs PLCs DCSs, OPC drivers
engineering specific analysis and data visualisation
algorithms that compress data to save disk space
interfaces that sit with data collection points and relay information back to the central historian
‘store and forward’ capability that guarantees no data gets lost (interfaces will start storing data locally if they detect a lost connection with the central historian)

Limitations of data historians

For decades, industrial data historians have collected vast amounts of data. But the full potential of this data has been largely unrealised. In many cases, it’s remained ‘locked’ in the historians, never to be used by operations and maintenance staff. Since historians exist within niche markets, the software development and skillset required is highly specialised, making them expensive and hard to break into.

Other limitations of traditional enterprise data historians include:

Licensing and cost can be prohibitive. Most enterprise software licences limit the amount of data tags that can be captured, so organisations must be tactical about what data tags they need vs what they can afford.

It’s difficult to link the data to context. For example, a power meter on your house measures energy consumption over time. But what if you wanted to correlate the data with other events inside the house, such as the dinner party you had last week? It’s hard for data historians to relate sensor data to other data sets like this.

Older technology limits scale and analysis. Data historian technology has remained unchanged for the last few decades. This means it’s challenging to embed and integrate historian data into modern web-based applications.

Strengths of open source time-series

Most time-series databases are open source, as they were developed by Google and other internet companies. Though they haven’t been designed with industrial process data in mind, these databases were built to handle any non-relational data (or NoSQL). Think of tracking the likes, comments, scrolls and uploads of billions of Facebook users in real-time across time, and you’ll get an idea of the mammoth amount of data these databases can deal with.

Here’s what these open-source products can offer:

Scale: These databases are designed for linear scalability. That is, they can keep growing forever and their performance doesn’t derogate.
Performance: An open-source database like Influx can handle a thousand more tags than the industry-leading proprietary data historian. And they can do this 10 times faster.

Ease of integration: Because Google and other tech giants created NoSQL databases for a range of applications, they’re designed to cope with unstructured data and connect with other modern web-based applications. The NoSQL database approach says, “I need to be able to mix all sorts of data”.

Easy to try before you buy: With open source, you can start with no licence, then scale up to a level suitable for the business.

Data handling and analysis: These databases can integrate multiple complex data sources, including relating the time data with other data types. So while the data historian is unable to tell you how the dinner party you held last week affected your power consumption, a time-series database, on the other hand, would enable your energy consumption data to be integrated with a calendar application that details the party date, time, attendees and menu.

Resource: Open source time series database comparison (open source Google document).

Limitations of open-source time-series databases

The biggest limitation of open source time-series databases is in their lower install base for industrial industries. While the technology is safe, reliable and faster than traditional historians (and has been used by major tech companies for more than a decade), it is fairly new to industrial industries.

This means these companies who want to adopt open-source time-series databases will need to have a willingness to be innovative, and a desire to be at the leading edge of data technology in their industry. This approach may not suit everyone, but those who choose time-series will undoubtedly remain competitive and future-proof their operations.

Harnessing the right data disruption tools

There’s nothing particularly bad about data historians, but the current technology certainly limits an organisation’s ability to grow into a more integrated operation, link data to other softwares and harness data more powerfully. What’s more, time-series databases such as NoSQL have provided the foundation for machine learning and other disruptive technologies.

As a Mckinsey & Company report highlights, business leaders need to update their strategies to account for these disruptive technologies and ‘not be afraid to look beyond established models’. Otherwise, it will be too late for businesses to capture the benefits.

Catch up on database technologies for industrial in our other blog posts and case studies:

1. Data analysis: why process historians and time-series are essential

2. How Google gave rise to time-series databases

3. How TasWater reduced data costs with Nukon's time series configuration sync tool