The Internet of Things (IoT) is already entrenched in our everyday lives – from wearables and smart watches through to connected TVs and smart home appliances.
Businesses, too, are utilising the technology; in a B2B context, says Sean Kandel, CTO and co-founder of Trifacta, connected devices refer to machines and sensors that are used to track everything from machine performance to maintenance requirements.
For instance, sensor devices might be found on a production line to track the readiness of the machines and automate predictive maintenance. Or, a hospital might use IoT devices for remote patient monitoring, robotic surgery or dispensing medication.
All of these growing sensors, devices, and other connected “things” ultimately mean more data. And lots of it. But with more data come more complex challenges in preparing it. To harness the value of IoT and big data—and deliver innovation-driving insights— industrial organisations must quickly prepare all of this disparate, unstructured data. Below, we’ve named some of the top three challenges in preparing IoT data to leverage it for analysis.
1. Huge volumes of data
International Data Corporation (IDC) market research estimates that IoT devices will create 40,000 exabytes of data by 2020. To keep this in perspective, in the year 2000, three exabytes of information were created globally. That is a lot of data to prepare—and under many current processes, organisations won’t be able to keep up. This is particularly challenging in the industrial world, where manufacturers and other large industrial organisations typically collect billions of data sets from machines, sensors and internal business applications.
Data preparation still accounts for up to 80% of the time and resources involved in any data project, and the more data you add, the more time-intensive that process will become. As organisations take on new IoT data initiatives, it’s important for them to consider new technologies and processes that will allow them to keep up with this huge influx of data.
Another challenge in preparing IoT data is its complex nature. Often, organisations must not only prepare timestamp or geotag data, but combine it with more structured sources, such as csv files. This complexity is only multiplied when factoring in the rate at which this data is being generated.
Finding a solution to this problem is tricky. The technical resources within an organisation that could handle this complexity are typically limited, and scaling out those resources are costly. Using common data preparation tools like Excel can’t handle this complexity, which leaves skilled analysts locked out of working with this data. Today’s organisations must figure out a way to leverage the resources they have in order to prepare the increasingly complex IoT data.
Business computer systems—both hardware and software—aren’t made to exchange or process the vast amounts of complex information pulled from sensors and connected devices. It’s difficult to quickly integrate and enrich machine generated data with data from business applications such as Salesforce and Marketo for example, and other data repositories. Therefore today’s organisations must look for solutions that better allow data to talk to each other, so that the entirety of an organisation’s […]
Read more: iot-now.com