By Natalia Regalado, Data Science Leader at Nubiral.
Data without structure, use or visibility, which companies collect in large quantities and which, only in relation to aspects related to storage and security, usually generate more costs and greater risks than value. This is the broad definition of the concept of dark data. Some examples? Recordings of calls made in customer service centers, videos recorded by security cameras, geolocation data and social media posts related to products, among others.
In a data-driven world, in which the differentiator is given by the ability to base all decisions on data, companies have to take on the challenge of starting to monetize their dark data.
The first obstacles
There are some barriers that complicate the use of dark data. According to data collected by Nubiral, 85% of companies consider that they do not have a tool to capture and store this data, while another 39% mention there is so much data that their BI teams cannot analyze it.
These are obstacles that can be avoided. Today, there are different tools that help us to store and process this information. There are different architectures when it comes to developing and assembling an appropriate solution.
The key tools
We should not lose sight of the fact that when we talk about dark data we are probably also referring to unstructured data, including audios, videos, blogs, social media posts or emails, etc., so it would be ideal to start by building a data lake that enables its storage in a single centralized repository.
The next step will be to recognize the different tools that could allow, through the analysis of this data, to obtain insights that add value to the business. The range of possibilities is wide and goes from no-code or low-code alternatives (such as Excel or Tableau) to those that require programming knowledge, such as SQL or Python. There is an option that is the most suitable for each work team: it is essential to find it.
Unlimited use cases
The number of use cases is also extensive. The processing of security cameras located at the doors of commercial premises, for example, could tell us how many people enter the business and, of that total, how many leave with a bag (i.e., they made a purchase).
Cameras housed in the checkout area can help quantify the average time customers wait to be served, information that could be used to improve the user experience.
The analysis of tweets, on the other hand, provides key information about the level of satisfaction of a person with a product, their habits and their preferences so that a company can put together personalized promotions. As we indicated, the number of examples is too extensive to try to cover them all in this article.
For now, it is time to take the first step: to recognize what would be the dark data of our business and start storing all this information in order to later generate value from it.