How third-party data can enhance analytics
The benefits and challenges of external data sources
Companies are increasingly seeking better insights by tapping into third-party data. Outside sources can bring opportunity, but using them effectively can be challenging.
- Data volume and insights
- Deriving value from external data
- The challenges of external data
- Tap into a data ecosystem
- Connecting to the ecosystem
Many executives already know the data generated from their operations can yield valuable insights. Relying solely on internally generated information can leave gaps, however, so many companies are looking to incorporate new, nontraditional, and external sources of data into their analyses. This data can include almost anything, from historical demographic and weather data to satellite imagery and private company information.
Companies increasingly operate within networks of business partners such as suppliers, resellers, channel partners, regulators, and other stakeholders. These networks are often globally distributed and potentially affected by economic, political, or environmental factors. Analyzing external data can help companies see risks and opportunities they might otherwise miss from internal operations, customers, and first-tier suppliers. Outside data can also illuminate how factors such as shifting consumer behaviors, competitor initiatives, or geopolitical events might affect a business.
As most business and technology professionals know, the volume of data being created, shared, and stored is increasing at an exponential pace. According to one study, the data stored in data centers will nearly quintuple to reach 1.3 zettabytes globally by 2021. (One zettabyte is equivalent to one trillion gigabytes.) Along with the volume of information available, the potential value of analyzing this data grows bigger by the day.
It’s not surprising, then, that companies on the leading edge of data and analytics are more likely to make use of external sources. According to a recent MIT Sloan Management Review report, the companies making the most innovative use of data and analytics were more likely than others to leverage external data sources, including social, mobile, and publicly available data. A separate study found faster-growing companies were more likely to be considering the use of external data than those with lower growth rates.
Deriving value from external data
Third-party data sources can help businesses personalize marketing offers, improve HR decisions, gain new revenue streams by launching new products or services, enhance risk visibility and mitigation, and better anticipate shifts in demand for products and services. In one case, a major semiconductor manufacturer used third-party data to build predictive customer models. The models identified potential targets that fit profiles similar to the company’s most engaged customers. These lookalikes helped the organization optimize marketing spend, reducing a major campaign’s cost per engagement by 75 percent.
There are numerous other examples of analytics programs generating value with external data. Several startups monitor social networking data to predict patterns of job-seeking behavior and retention risk, believing outside sources are more predictive of an employee’s likelihood of leaving than any internal data available. Others use geolocation and weather data to predict crop yields, helping farmers optimize fertilizer use. Many retailers are using economic data and forecasts, data from suppliers, and geolocation information to better predict demand and reduce stock shortages.
The challenges of external data
Access to external data is getting easier in some ways, but incorporating it can still be daunting. Organizations report a wide variety of business and technical challenges in deriving insights from external data. Among the business challenges are the size and complexity of the data-provider market, which can make it hard to identify the right data sources and partners. Negotiating data acquisition can be arduous, depending on factors such as whether ongoing access is needed for refreshing machine learning models, as well as determining usage restrictions, vendor revenue-sharing needs, and liability if the data proves to be inaccurate or tainted. Negotiation can also involve lengthy risk and legal reviews of vendor contracts and licensing agreements. The ongoing management of a growing roster of data-sharing relationships and partnerships can also be taxing.
Technical challenges can include fundamentals such as data quality and accuracy: A variety of studies have demonstrated that third-party data can be riddled with inaccuracies. There can also be inconsistencies between external and internal recording to resolve before performing analysis. Preprocessing—such as cleansing and formatting—is time-consuming, accounting for as much as 80 percent of the analysis effort by some estimates. Securely storing and cataloging data in an easily accessible manner can require updating information management processes and capabilities designed to handle only internal data. The longer it takes to work through these challenges, the less time available to react to market trends and external events with agility.
Tap into a data ecosystem
Research suggests most companies haven’t yet developed the capabilities necessary to use external data effectively. To close this gap, executives may want to think of their companies as part of a data ecosystem—a network of participants that directly or indirectly consume, produce, or provide data and other related resources.
To be good at using external data can mean being competent in identifying, evaluating, procuring, and preparing external data in a consistent and timely manner. Companies will want to set up a continual process for identifying, engaging with, and evaluating new external data sources and partners and, when appropriate, integrating these data sources into analytics processes or product offerings. Maximizing the value of external data often requires integrating it with internal data for a more insightful analysis.
Companies also may want to form a cross-functional group that acts as the organization’s interface to the broader data ecosystem. This group could draw on competencies from multiple areas—including product management, business analysis, data science, legal, and procurement—to address the previously identified organizational and technical challenges. Some organizations have created special roles charged with scanning the third-party data market and matching business requests with relevant sources. These data curators can help companies quickly identify and assess data sources matched to business needs while reviewing external data sets for quality and accuracy using consistent evaluation processes. Companies with effective tech-scouting capabilities may look within this group for inspiration.
Connecting to the ecosystem
Organizations looking to connect to a data ecosystem can turn to a wide and growing variety of data and insights providers. Gartner Group categorizes data services, for instance, by the level of insight they provide:
Simple data services. Data brokers collect information from multiple sources and offer it in collected and conditioned form. The data is used as additional input to a decision process by a person, an application system, or a device in an IoT ecosystem.
Smart data services. Data is enhanced by applying analytical rules and calculations. The results often take the form of scores or the tagging of objects, as in services from marketing data providers and credit ratings agencies.
Adaptive data services. Customers submit data pertaining to specific analytical requests. Providers combine that data with data from other sources.
There are other ways to segment this dynamic marketplace. Some providers specialize in serving industry sectors such as hedge funds or health care organizations. Many consulting and systems integration services providers, meanwhile, are leveraging publicly available data or information from third-party partners and then integrating those sources with their clients’ internal data records to perform custom analysis.
Companies are making growing use of data from third parties. To get more value from their data analytics efforts, executives may want to consider enhancing their ability to identify, evaluate, and contract for these new external sources by creating a data ecosystem management program under a chief data office that links to business, IT, and legal teams. The pressure on companies to innovate and to improve the efficiency and effectiveness of their operations is unrelenting; they cannot afford to ease off in their pursuit of insights. For many companies, the effective use of external data is a critical new frontier.
Authors of Driving industry through advanced technology include: David Schatsky, Craig Muraskin, and Jonathan Camhi.