Can Intel’s XPU imaginative and prescient information the trade into an period of heterogeneous computing?

This text is a part of the Know-how Perception collection, made attainable with funding from Intel.

As knowledge sprawls out from the community core to the clever edge, more and more numerous compute sources comply with, balancing energy, efficiency, and response time. Traditionally, graphics processors (GPUs) had been the offload goal of alternative for knowledge processing. Right now subject programmable gate arrays (FPGAs), imaginative and prescient processing models (VPUs), and software particular built-in circuits (ASICs) additionally convey distinctive strengths to the desk. Intel refers to these accelerators (and anything to which a CPU can ship processing duties) as XPUs.

The problem software program builders face is figuring out which XPU is greatest for his or her workload; arriving at a solution typically entails a number of trial and error. Confronted with a rising checklist of architecture-specific programming instruments to assist, Intel spearheaded a standards-based programming model called oneAPI to unify code throughout XPU sorts. Simplifying software program improvement for XPUs can’t occur quickly sufficient. In spite of everything, the transfer to heterogeneous computing—processing on one of the best XPU for a given software—appears inevitable, given evolving use instances and the numerous gadgets vying to deal with them.

KEY POINTS

• Intel sees heterogenous computing (the place a number machine sends compute duties to totally different accelerators) as inevitable.
• An XPU will be any offload goal commanded by the CPU, constructed on any structure from any {hardware} vendor.
• The oneAPI initiative is an open, standards-based programming mannequin that permits builders to focus on a number of XPUs with a single code base.

Intel’s technique faces headwind from NVIDIA’s incumbent CUDA platform, which assumes you’re utilizing NVIDIA graphics processors solely. That walled backyard might not be as impenetrable because it as soon as was. Intel already has a design win with its upcoming Xe-HPC GPU, code-named Ponte Vecchio. The Argonne National Laboratory’s Aurora supercomputer, for instance, will characteristic greater than 9,000 nodes, every with six Xe-HPCs totaling greater than 1 exa/FLOP/s of sustained DP efficiency.

Time will inform if Intel can ship on its promise to streamline heterogenous programming with oneAPI, reducing the barrier to entry for {hardware} distributors and software program builders alike. A compelling XPU roadmap definitely provides the trade a cause to look extra carefully.

Heterogeneous computing is the long run, however it’s not straightforward

The full quantity of information unfold between inside knowledge facilities, cloud repositories, third-party knowledge facilities, and distant places is predicted to extend by greater than 42% from 2020 to 2022, in response to The Seagate Rethink Data Survey. The worth of that data will depend on what you do with it, the place, and when. Some knowledge will be captured, categorized, and saved to drive machine studying breakthroughs. Different purposes require a real-time response.

The compute sources wanted to fulfill these use instances look nothing alike. GPUs optimized for server platforms devour lots of of watts every, whereas VPUs in the single-watt range would possibly energy sensible cameras or pc vision-based AI home equipment. In both instance, a developer should resolve on one of the best XPU for processing knowledge as effectively as attainable. This isn’t a brand new phenomenon. Quite, it’s an evolution of a decades-long pattern towards heterogeneity, the place purposes can run management, knowledge, and compute duties on the {hardware} structure greatest suited to every particular workload.

“Transitioning to heterogeneity is inevitable for a similar causes we went from single core to multicore CPUs,” says James Reinders, an engineer at Intel specializing in parallel computing. “It’s making our computer systems extra succesful, and capable of remedy extra issues and do issues they couldn’t do previously — however throughout the constraints of {hardware} we will design and construct.”

As with the adoption of multicore processing, which pressured builders to start out interested by their algorithms by way of parallelism, the most important impediment to creating computer systems extra heterogenous right now is the complexity of programming them.

It was that builders programmed near the {hardware} utilizing low-level languages, offering little or no abstraction. The code was typically quick and environment friendly, however not moveable. Nowadays, higher-level languages prolong compatibility throughout a broader swathe of {hardware} whereas hiding a number of pointless particulars. Compilers, runtimes, and libraries beneath the code make the {hardware} do what you need. It is sensible that we’re seeing extra specialised architectures enabling new performance by way of abstracted languages.

oneAPI goals to simplify software program improvement for XPUs

Even now, new accelerators require their very own software program stacks, gobbling up the {hardware} vendor’s money and time. From there, builders make their very own funding into studying new instruments to allow them to decide one of the best structure for his or her software.

As an alternative of spending time rewriting and recompiling code utilizing totally different libraries and SDKs, think about an open, cross-architecture mannequin that can be utilized emigrate between architectures with out leaving efficiency on the desk. That’s what Intel is proposing with its oneAPI initiative.

oneAPI helps high-level languages (Knowledge Parallel C++, or DPC++), a set of APIs and libraries, and a {hardware} abstraction layer for low-level XPU entry. On high of the open specification, Intel has its own suite of toolkits for various development tasks. The Base Toolkit, for instance, contains the DPC++ compiler, a handful of libraries, a compatibility device for migrating NVIDIA CUDA code to DPC++, the optimization oriented VTune profiler, and the Advisor evaluation device, which helps determine one of the best kernels to dump. Different toolkits residence in on extra particular segments, equivalent to HPC, AI and machine studying acceleration, IoT, rendering, and deep studying inference.

“Once we discuss oneAPI at Intel, it’s a fairly easy idea,” says Intel’s Reinders. “I need as a lot as attainable to be the identical. It’s not that there’s one API for all the things. Quite, if I need to do quick Fourier transforms, I need to be taught the interface for an FFT library, then I need to use that very same interface for all my XPUs.”

Intel isn’t placing its clout behind oneAPI for purely selfless causes. The corporate already has a wealthy portfolio of XPUs that stand to profit from a unified programming mannequin (along with the host processors tasked with commanding them). If every XPU was handled as an island, the trade would find yourself caught the place it was earlier than oneAPI: with impartial software program ecosystems, advertising sources, and coaching for every structure. By making as a lot frequent as attainable, builders can spend extra time innovating and fewer time reinventing the wheel.

What is going to it take for the trade to start out caring about Intel’s message?

An unlimited variety of FLOP/s, or floating-point operations per second, come from GPUs. NVIDIA’s CUDA is the dominant platform for normal function GPU computing, and it assumes you’re utilizing NVIDIA {hardware}. As a result of CUDA is the incumbent know-how, builders are reluctant to vary software program that already works, even when they’d desire extra {hardware} alternative.

If Intel desires the group to look past proprietary lock-in, it must construct a greater mousetrap than its competitors, and that begins with compelling GPU {hardware}. At its recent Architecture Day 2021, Intel disclosed {that a} pre-production implementation of its Xe-HPC structure is already producing greater than 45 TFLOPS of FP32 throughput, greater than 5 TB/s of material bandwidth, and greater than 2 TB/s of reminiscence bandwidth. A minimum of on paper, that’s greater single-precision efficiency than NVIDIA’s quickest knowledge heart processor.

The world of XPUs is extra than simply GPUs although, which is exhilarating and terrifying, relying on who you ask. Supported by an open, standards-based programming mannequin, a panoply of architectures would possibly allow time-to-market benefits, dramatically decrease energy consumption, or workload-specific optimizations. However with out oneAPI (or one thing prefer it), builders are caught studying new instruments for each accelerator, stymying innovation and overwhelming programmers.

Luckily, we’re seeing indicators of life past NVIDIA’s closed platform. For example, the staff chargeable for RIKEN’s Fugaku supercomputer just lately used Intel’s oneAPI Deep Neural Community Library (oneDNN) as a reference to develop its personal deep studying course of library. Fugaku employs Fujitsu A64FX CPUs, based mostly on Armv8-A with the Scalable Vector Extension (SVE) instruction set, which didn’t have a DL library but. Optimizing Intel’s code for Armv8-A processors enabled an up to 400x speed-up compared to simply recompiling oneDNN without modification.  Incorporating these adjustments into the library’s predominant department makes the staff’s positive aspects accessible to different builders.

Intel’s Reinders acknowledges the entire thing sounds quite a bit like open supply. Nevertheless, the XPU philosophy goes a step additional, affecting the best way code is written in order that it’s prepared for several types of accelerators operating beneath it. “I’m not nervous that that is some kind of fad,” he says. “It’s one of many subsequent main steps in computing. It isn’t a query of whether or not an thought like oneAPI will occur, however moderately when it can occur.”