Your data lives at the edge and determining how to connect processes can improve manufacturing and monitoring.

Real-time and near real-time processing at the edge is more common than ever, thanks to improvements in chips and batteries. Yet a variety of logistical and technical problems present challenges for companies engaging in such processing. Fortunately, every instance of such work presents opportunities for these businesses to learn more from themselves and one another.

Implementing industry 4.0 practices in real-time and near real-time processing at the edge requires evaluating how current procedures can be improved. Beneficial changes enable companies to handle numerous scenarios that relate to interconnected procedures. For example, ensuring there is adequate security at the edge is best accomplished as a team goal between business partners. This goal can utilize two or more tools, such as encryption and two-factor authentication.

Recent changes that have increased the amount of real and near real-time processing at the edge include a current capability of up to 20 trillion operations per second (TOPS) for standard semiconductors, as opposed to a single TOPS a few years ago; faster speed and lower power consumption in different networks, from Long Range Wide Area Network (LoRaWAN) to 5G; and better software, including more Artificial Intelligence (AI) models, as well as new data sets and tools.

“The edge is where the data come from. Bringing the processing to companies working in these spaces is the goal. Such action can bring deployment time down by as much as a third, like from 18 months to six months. That presents cost savings and better opportunities to leverage AI,” says Pete Bernard, Executive Director of tinyML Foundation.

tinyML is a Seattle-based nonprofit that focuses on low power AI at the edge of the cloud. Its members include large corporations, including Qualcomm and Sony, academic institutions like Johns Hopkins University and nongovernmental organizations.

“tinyML holds frequent events to build community around the concept of the edge. We educate people about the potential of working at it. Our programs include contests, conferences, hackathons and workshops. One of the concepts we are considering now is data provenance,” says Bernard.

This idea relates to the watermarking of data sets and models. AI models must be trained on data sets. Stamping provenance helps users identify sources of data and the developers behind them. Such work makes it easier to integrate different data sets and models.

Software for the edge

Simplifying edge operations is easier to accomplish with software designed for that purpose, like Dell’s NativeEdge platform. 

“With NativeEdge, a client can build an AI model to operate at the edge. They can retrain the model onsite at the edge. This saves money and gives them the ability to scale up the solution as needed,” says Pierluca Chiodelli, Vice President, Edge Engineering and Product Management at Dell Technologies.

Dell sees security as the biggest challenge for clients.

A company that tries to do everything itself runs the risk of exposing information. Any entity that generates data must protect the data at the points where the data is created and stored.

Dell is enhancing security by working closely with NVIDIA, which developed the AI Enterprise software integrated with NativeEdge’s engine.

“Inference at the edge, which involves gathering data with AI techniques, is really important. Everybody needs to have a way to deploy and secure that. Also a company has to maintain its AI stack, the tools and services to use AI correctly. It must have a blueprint to update all the pieces of the puzzle,” says Chiodelli.

As the different components of an AI stack can change, a company must be aware of all of them and how they interact. This helps the company make the necessary adjustments in proportion and on the appropriate timeline. Such work prevents deviations in manufactured products and slowdowns in production time. It also minimizes the time needed to retrain AI models and workers.

The market for the edge is growing

Nvidia is working on numerous hardware and software applications to meet the needs of companies utilizing edge computing. The company sees this market as expanding. A March 2024 forecast from the International Data Corp. stated worldwide spending on edge computing is expected to be $232 billion this year.

One of Nvidia’s platforms for the edge is the Nvidia IGX Orin with NVIDIA Holoscan, which is designed for real-time AI computing in industrial and medical environments. This platform provides high performance hardware and enterprise AI software. The platform is for companies working in robotics, healthcare, scientific research, video analytics and broadcasting.

In scientific computing, the Nvidia IGX Orin with Holoscan platform has the power to stream high-bandwidth sensor data to the GPU. It can use AI to detect anomalies, drive sensor autonomy and lower the time to scientific insights. In the medical space, Magic Leap has already integrated Holoscan in its extended reality (ER) software stack to enhance the capabilities of customers. This has allowed one of its clients in software development to provide real-time support for minimally invasive treatments of stroke.

It’s difficult to establish interoperability across systems, says Chen Su, Senior Technical Product Marketing Manager of Edge AI and Robotics for Nvidia.

 “Today there are numerous installed legacy systems that weren’t originally designed with AI capabilities in mind. Integrating AI into those systems and still achieving real-time performance continues to pose a significant challenge. This can be overcome by developing industry-wide standards that can meet the complex connectivity requirements across sensors, actuators, control systems and interfaces,” says Su.

Once the task above is accomplished, the entire edge AI system will have no bottleneck in communication. It can then act in a software-defined manner, making the system more flexible and easier to manage.

STMicroelectronics (ST), a global manufacturer and designer of semiconductors, meets the needs of companies that process data in real-time and near real-time with a variety of edge AI tools and products.

These include STM32 and Stellar-E for microcontrollers (MCU) edge AI pure software solutions; the incoming STM32N6, a high-performance STM32 MCU with ST proprietary Neural Processing Units (NPU) and the STM32MP2 microprocessor series.

Danilo Pau, Technical Director in System Research and Applications at STMicroelectronics, says advances in embedded AI computing that enable processing at the edge require higher energy efficiency. The task is made possible by a mix of assets, including super-integrated NPU accelerators, In Memory Computing (IMC) and 18nm Fully Depleted Silicon On Insulator (FD-SOI) ST technologies. Such resources can be super integrated close to standard MCU and Memory Processing Unit (MPU) cores for viable, high volume low-cost manufacturing.

“There is also the super-integration of heterogeneous technologies in a single package achieved by Intelligent Sensor Processing Unis (ISPU) and Machine-Learning Core (MLC) product families. In a tiny package, micro-electromechanical systems (MEMs) sensors, analog and digital technologies are stacked for large and cheap sensor volumes. They engage in microwatt power consumption. This is a fundamental contribution that enables the incoming trillion of sensor economies envisaged by many IoT experts,” says Pau.

Organizations like tinyML Foundation play an important role in the business community. Since 2018, tinyML has encouraged many companies to invest in generative AI at the edge (edgeGenAI).

Pau says there is need of even greater energy efficiency and super integration of heterogeneous technologies, including NPU, IMC, deep submicron technologies and sensors.

“The vision is to design embedded systems that match the energy efficiency of the human brain,” says Pau.

He adds companies will increasingly need more education about edge AI technologies, tools and mastery of skills.

That fact explains why ST, which is currently Europe’s largest designer and manufacturer of custom semiconductors, is an active part of the tinyML community.

“ST works with many actors in the edgeGenAI ecosystem. We’re eager to see this ecosystem expand and serve AI developers in the best and most productive way. That will ease their path in bringing innovation to the edge AI market,” says Pau.