Data Mining Across Digital Threads: Analytics, Searches and Effective Collaboration

Study shows how leaders are leveraging search-powered tools to drive speed, productivity and digital transformation.

Searching for information implies knowing where to look across multiple apps and data sources, and having the right digging tool to mine for the relevant content in an effective and timely manner.

Searching for information implies knowing where to look across multiple apps and data sources, and having the right digging tool to mine for the relevant content in an effective and timely manner.

It isn’t new to see an analyst study or survey refer to time wasted by engineers, wider teams and management functions, across departments and business units, when simply searching for data. It is typically referred to as “essential non-value added” (ENVA), or could also link to non-value added (NVA) activities in given cases. Data access and searchability is core to every business in their day-to-day activities, whether it be for:

  • Building analytics and intelligence (application or cross-application level).
  • Driving product or service performance (business operations).
  • Planning and delivering continuous improvements, including more complex business and digital transformations.
  • Learning from data and building automations or new business models.

Data exchange and enterprise platform integration is core to building digital threads across business functions and wider ecosystems. This implies implementing data interfaces across PDM, CAD, PLM, ERP, MES, MRP, MOM, CRM and other digital platforms, as well as tools and processes that enable data mining and interpretation across multiple point solutions.

As elaborated in the 2022 Forrester study commissioned by Elastic, titled “Search-Powered Technologies: A Mission-Critical Enabler for The Digital Future of Business,” searching for data covers multiple use cases, from data reuse, business analytics, data visualization, quality assessment, performance monitoring, improvement planning, discovering new operating patterns, learning, cybersecurity, data migration, storage, IT maintenance and more.

In this post, I discuss the findings of the survey and expand on how data searches relate to digital threads, PLM, ERP and MES integration.

Searching for data is often a matter of productivity. How long does it take to find the relevant information across single, or most likely multiple data sources, considering internal and external functions and teams? What tools are available to search? How do people consume data? How do they transform it into value? Is it easily interpreted by users or other enterprise systems?

Per the survey run in April 2022 and reported on Business Wire, Forrester highlighted the following trends and findings:

  • A shift towards integrated search platforms (across apps and clouds).
  • Eight in ten data leaders believe search-powered technology—in other words, tools enabling the search of data across multiple sources—drives results that matter for their business.
  • There are multiple business use cases for enterprise-wide search platforms.
  • A focus on observability and managed operations with input/output controllability (towards more predictive capabilities).
  • A drive towards better analytics to improve operational health (covering both business, OT and IT operational governance).

The Forrester report seems to have focused on IT and cybersecurity leaders across Europe, Asia-Pacific and the Americas. It also focusses on three industries: Financial services and/or insurance, Telecommunications and Public sector/government).

“Search-powered technologies drive the future of business. Data leaders rely on them to improve cybersecurity, drive digital transformation initiatives and help with cloud migration and usage. Firms will respond to unstructured data and interoperability challenges with fully featured, integrated search platforms that comprise critical point-solution capabilities. Data leaders turn to search-powered technologies to drive digital transformation initiatives through improvements in data quality, access and usability.”—Forrester (2022)

This is not to say that similar conclusions won’t be derived from other industries, though there are obvious nuances in terms of industry processes, data and operations maturity and complexity. There are also business and IT/OT centric perspectives when judging from the quality of data, the efficiency and effectiveness of given IT platforms and the lack of integration of digital threads across digital systems.

The “Data Quality” Dilemma

Clearly, search engines must be effective at monitoring metrics, discovering existing and potentially new data patterns and extracting the relevant trends and intelligence across both structured and unstructured data. As highlighted by Forrester, most digital transformation challenges are held back by lack of data access, knowledge or understanding (including data that can be trusted).

Data quality, data source quality (process-driven) and data traceability are considered core to solving business challenges. Now, one of the key questions needs to be asked: Should the data be improved at the source (where it is created in the first place) or through additional analytics layers (overlaying legacy sources)? Should data strategies involve transforming search capabilities across platforms, or replacing legacy systems by modern platforms with embedded advanced search capabilities?

The answer is perhaps in the middle, with different trade-offs required based on the type of data and data sources involved.

Data Evolves, Search Tools and Analytics Must Also Evolve

Interestingly, the Forrester study refers to a forecasted increase in the use of search platforms to drive integrated maintenance and observability across tools. This could easily apply in a DevOps-PLM combined context. For example, leveraging correlated analytics across apps, repositories and database levels. This is something that already resonates with software development and technical maintenance teams.

As such, data searchability and associated analytics can certainly become much more dynamic, taking into consideration process and data mining, associated intelligence and feedback loops into improvement roadmaps. Digital transformation can obviously leverage data profiling tools to assess suitability and quality prior to data migration or process improvement.

Beyond this, it would be interesting to explore how such feedback loops can be implemented across the enterprise, possibly by segregating application-driven and integration-driven search capabilities. As a matter of fact, there is a strong interdependency across master data management and integrated search strategies when prioritizing how to search effectively within and across data sets. This would also be part of an essential data searchability governance to understand data access restrictions and security matters, and possibly looking at fixing issues or concerns at source where relevant.

What are your thoughts?