Computing

Tweet

AI Processing Moves From Cloud Into 'Smart' Edge

Its Capabilities are Providing Intelligent Ways to Enable Tomorrow's IoT

Image Source: Phonlamai Photo/shutterstock.com

By Paul Golata for Mouser Electronics

Edited December 28, 2020 (Originally published January ‎28, ‎2020)

Technological breakthroughs in semiconductor processors have enabled a host of artificial intelligence (AI) capabilities within the cloud-computing domain. AI brings intelligence to devices allowing them to behave in more performative ways. Theoretically, AI is not bound by human biological limitations regarding capacity or response times because it is in the electronic domain. The cloud provides storage and necessary processing power to handle large amounts of data and output innovative solutions previously unobtainable. Processors, accelerators, graphics processing units (GPUs), and field-programmable gate arrays (FPGAs) are semiconductor products that provide the necessary computational-processing power to the cloud (Figure 1).

Figure 1: AI and the cloud computation made available with processors, accelerators, graphics processing units, and field-programmable gate arrays. (Source: metamorworks/Shutterstock)

For tomorrow’s applications, it is no longer sufficient to keep artificial intelligence (AI) only in the cloud. The Internet of Things (IoT) requires more and more of the processing work that happens closer to the end nodes—the location of data (sensors) and actuation (control). The ability to collect data, store it, and then perform the analysis provides a myriad of new ways to do business. The demand for low-latency, real-time decision-making and response, which is imperceptible to humans (<0.05s), demands that AI processing moves from the cloud to what we’ll call a “smart” edge to transform our present business processes and products to those that the market wants tomorrow.

The edge is becoming “smart” because, more and more, it is incorporating intelligence in the form of edge computing AI.

The edge exists in several contexts of which the IoT layer is one application. The edge is intermediate between the end-node device deployment and the top-level cloud-computing system. As an intermediate layer, it is now being provisioned with real-time AI capabilities, enabling applications that do not require computational power or cannot sustain cloud computing latency. Edge computing AI can experience use in various locations and stages between the cloud-computing layer and the end nodes layer, including gateways, access points, and metro edges.

Inferencing at the Edge

Motion pictures provide the illusion of reality because multiple frames pass before the eye faster than the eye responds, creating the illusion of ongoing motion and no discontinuity. IoT applications, particularly those directly interacting with humans, require the same kind of ongoing continuity to avoid wait periods. Humans want the devices that they electronically interface with to work with them in real-time.

To enable future IoT applications, including autonomous vehicles, real-time decision-making is required. Data at the edge moves to the cloud, and what comes back is features or responses to that data. In that small moment, the decision of whether a particular object in front of a traveling vehicle is a blowing piece of litter, a ball, another vehicle, or a person can arrive back at the vehicle too late. Voice-enabled applications now found in many homes respond within the response time expected of a normal human conversation. Detecting errors and defects within the industrial contexts can help prevent accidents and hazard occurrences. These specific applications are only the tip of the iceberg among numerous IoT applications coming along in the future that will call for immediate real-time decision-making.

One way to make this happen is to enable inferencing at the “smart” edge layer instead of the higher-level cloud-computing layer. Inferencing means to draw or reach a conclusion based on the evidence by way of reasoning. AI at the cloud-computing level can process large amounts of data over time and draw conclusions from the patterns it discerns. It is possible to conceive of ways of pushing these categorized conclusions down into the “smart” edge layer so that upon initialization, the “smart” edge has a wide variety of highly intelligent categories from which to start its computational processes.

This “smart” edge effectively provides a shortcut, greatly expanding the “smart” edge’s capability to decide something faster than a human’s biological response time.

To enable this, the “smart” edge must process fresh data against these categories and quickly perform inferences toward a correct conclusion. “Smart” edge inferencing happens through machine learning (ML), an application approach utilizing AI. Machine learning is a process or set of rules that occur through calculations or other problem-solving operations to empower the extraction of structured categories and representations from received input data. Processors, accelerators, GPUs, and FPGAs at the “smart” edge can be programmed to employ algorithms that draw these inferences. The cloud layer supports ML by ensuring that the “smart” edge always contains the most appropriate algorithms based upon the computational power it exerts upon the vast raw data it studies and processes to gain experience. It is also the best layer level to train ML algorithms because it is the informational knowledge that passes on to the smart edge. Trained ML algorithms at the “smart” edge enable the “smart” edge to process the real-time data it receives in a structured manner and compare these data-to-data models: This process is a recent development called federated learning that is an alternative to centralized training.

Training ML algorithms at the cloud layer requires the highest performance processors, such as graphic procession units (GPUs). However, these higher performance processors might not necessarily be a requirement at the “smart” edge. The reason is that the highly intensive computational processes occur at the cloud layer before moving to the “smart” edge. Engineers must decide what level of computational quality metrics are necessary for learning and determine how quickly the model needs updates with new data providing input regarding where processors should reside.

This deductive methodology allows for the proper optimization of a necessary performance level relative to power efficiency for action at the edge, thereby enabling a greater variety of intelligent applications to perform without the necessity of the most costly, power-consuming processing chips (Figure 2).

Figure 2: AI and ML depend on high-performance semiconductor processors. (Source: Sittipong Phokawattana/Shutterstock)

One common example of an AI and ML performance at the “smart” edge of IoT is image and video analysis. In this example, AI, by way of ML, processes a large amount of raw data and extracts usable data content to assist with future decision-making when data arrives through new observations, an ML model processes this data to produce a classification or decision (Figure 3).

Figure 3: AI and ML in the smart edge can analyze videos and images to make decisions. (Source: Alexander Supertramp/Shutterstock)

Processing Chip Architecture

The “smart” edge generates, handles, and utilizes large amounts of data. Because of this large amount of data, high-computing processors with excellent efficiency are ideal. Arm^® technologies enable the world’s most popular AI platform—the smartphone—along with ML features such as predictive text, speech recognition, and computational photography. Within Arm’s line of high-performance, 64-bit (with full 32-bit compatibility), Arm v8-A processors are several devices such as the multicore Arm^® Cortex^®-A53 and Arm^® Cortex^®-A72. World-leading semiconductor firms can employ these Arm cores coupled with vector-processing engines in the form of a system-on-chip (SoC) to support AI processing at the edge.

An important aspect of SoCs is that they frequently integrate with things commonly found outside of the processor, such as accelerators for applications, busses, interfaces, etc. Through this extended integration, SoCs provide flexibility and scalability, allowing designers to match their connected end node devices with the data, security, and performance characteristics essential to their IoT systems and applications at the “smart” edge.

Processing chip architectures in the “smart” edge want to take advantage of high, next-generation speeds (≥100Gbps) with excellent packet-processing abilities. Standard and open programming models that employ software-aware architecture frameworks make it easier for designers to configure the product to match their specific network requirements. By providing a core-agnostic architecture, designers can select the optimum core for their particular application. This concept takes advantage of the multicore trend and extends it by allowing an increased performance by incorporating either related cores or diversified cores. This “smart” edge computing generally contains provisions that support cybersecurity requirements inherent in IoT applications throughout their entire lifecycle. They also support virtualization—the abstraction of computing resources—permitting lower costs and complexities to prevail in designs. Virtualization achieves these benefits by treating computational and storage resources as separate entities, then redirecting and managing them for optimal utilization (Figure 4).

Figure 4: High-performance processors work with AI, electronics, and the IoT to enable the smart edge. (Source: metamorworks/Shutterstock)

Another processing chip architecture is the x86 complex instruction set computer (CISC) microprocessor. The x86 architecture has been around for more than 40 years. Similar in architecture to central processing units (CPUs) found in desktop computers, they incorporate a more advanced feature set to meet workstations and networks’ requirements, making them suitable for smart-edge applications. They work in smart-edge applications to provide computational processing power, enhanced connectivity, and storage on a high-speed product that does not compromise on keeping every piece of data secure. The x86 architecture allows for an intelligent workload placement, low latency, scalability, and extreme responsiveness. These processors contribute to optimizing performance, providing analytics, and offering accelerated data compressions. These microprocessors come in several performance levels, with the highest performance level being suitable for demanding smart-edge applications, including real-time analytics, AI, and ML. They perform well when provisioned to give real-time AI-inferencing results at the “smart” edge. They come with up to 28 CPU cores and can support up to 12TB of address space.

Conclusion

The “smart” edge is emerging from the clouds. AI processing is enabling the “smart” edge to do various tasks, which were previously relegated to the cloud-computing layer. The demand for flexible, low-latency, real-time IoT solutions is part of the ongoing requirements that are now getting addressed at the “smart” edge. The ability to constantly adapt and provide data collections and analysis at the edge will allow AI and ML to provide better drive, business transformations, and performance. By managing information from a series of end node devices and applying ML algorithms on data business assets, IoT applications will be connected in real-time, optimized way to decrease costs and increase value deliverables to the customer. The future of IoT includes soaring into the clouds only when necessary, for much of what we will accomplish will be performed at–what else?–the “smart” edge.

Paul Golata joined Mouser Electronics in 2011. As a Senior Technical Content Specialist, Paul is accountable for contributing to the strategic leadership, tactical execution, and overall product line marketing direction for advanced technology related products. Paul provides design engineers with the newest and latest information delivered through the creation of unique and valuable technical content that facilitates and enhances Mouser Electronics as the preferred distributor of choice.

Before Mouser Electronics, he served in various manufacturing, marketing, and sales related roles for Arrow Electronics, JDSU, Balzers Optics, Piper Jaffray, Melles Griot, and Hughes Aircraft Company. Paul holds a BSEET from DeVry Institute of Technology—Chicago, IL; an MBA from Pepperdine University—Malibu, CA; an MDiv w/BL from Southwestern Baptist Theological Seminary—Fort Worth, TX; and a PhD from Southwestern Baptist Theological Seminary—Fort Worth, TX.