Bookmark and Share

Programmable Devices for Embedded Vision

Click HERE for additional resources


Executes embedded vision algorithms and can be reprogrammed.

This technology category includes any device that executes embedded vision algorithms or vision system control software.  The following diagram shows a typical computer vision pipeline, and programmable devices are often optimized for the compute-intensive portions of the software workload. The EVA analysis primarily focuses on semiconductor components, but board-level (embedded) products are included in this definition.  The definition also encompasses reprogrammable hardware, including FPGAs and reconfigurable logic.

ev pipeline




General-purpose CPUs, GPUs, DSPs and FPGAs

These examples represent distinctly-different types of programmable architectures for embedded vision computing, and each has advantages and trade-offs that depend on the workload. For this reason, many programmable devices combine multiple processor types into a heterogeneous computing environment, often integrated into a single semiconductor component. In addition, a programmable device can be accelerated by dedicated hardware that improves performance on computer vision algorithms.

General-purpose CPUs:

While computer vision algorithms can run on most general-purpose CPUs, desktop processors (and systems) may not meet the design constraints of embedded systems (See: Challenges to Embedding Computer Vision).  However, embedded x86 processors and system boards can leverage the PC infrastructure for low-cost hardware and broadly-supported software development tools. Several EVA Member companies offer programmable devices that integrate an ARM RISC CPU core.  A general-purpose CPU (x86 or RISC) is best-suited for heuristics, complex decision-making, network access, user interface, storage management, and overall control. A general purpose CPU may be paired with a specialized programmable device for better performance on pixel-level processing.  A widely-used example of a general purpose CPU is the ARM Cortex-A9. The Xilinx  Zynq-7000 Extensible Processor Platform has paired a dual-core version of this ARM CPU with a 28nm FPGA fabric.

Graphics Processing Units:

High-performance GPUs deliver massive amounts of parallel computing, and graphics processors can be used to accelerate the portions of the computer vision pipeline that perform parallel processing on pixel data. While General Purpose GPUs (GPGPUs) have primarily been used for high-performance computing (HPC), even mobile graphics processors and integrated graphics cores are gaining GPGPU capability—meeting the power constraints for a wider range of embedded vision applications. In applications that require 3D processing in addition to computer vision, a GPU will already be part of the system and can be used to assist a general-purpose CPU with many of the computer vision algorithms.  Many examples exist for x86-based embedded systems with discrete GPGPUs. While it doesn’t yet support GPGPU, the nVidia Tegra family is an example of the integration of ARM CPUs with a high-performance GPU.

Digital Signal Processors:

DSPs are much more efficient for processing streaming data, since the bus and memory architecture is optimized to process high-speed data as it traverses the system. This architecture makes DSPs an excellent solution for processing image pixel data as it streams from a sensor source. Many DSPs for embedded vision have been enhanced with additional co-processors that are better optimized for processing video inputs and accelerating computer vision algorithms.  The specialized nature of DSPs makes these devices inefficient for processing general-purpose software workloads, so DSPs are usually paired with a RISC processor to create a heterogeneous computing environment that offers the best of both worlds.  An example is the Texas Instruments DM8186 that integrates ARM CPUs with a DSP and specialized co-processors.

Field Programmable Gate Arrays (FPGAs):

Instead of incurring the high cost and long lead-times for a custom ASIC to accelerate computer vision systems, designers can implement an FPGA to offer a reprogrammable solution for hardware acceleration. With millions of programmable gates, hundreds of I/O pins, and compute performance in the trillions of multiply-accumulates/sec (tera-MACs), high-end FPGAs offer the potential for best performance in an embedded vision system. Unlike a CPU, which has to time-slice or multi-thread tasks as they compete for compute resources, an FPGA has the advantage of being able to accelerate multiple portions of a computer vision pipeline simultaneously. Since the parallel nature of FPGAs offers so much advantage for accelerating computer vision, many of the algorithms are available as optimized libraries from semiconductor vendors like Xilinx. These computer vision libraries also include preconfigured interface blocks for connecting to other embedded vision devices, such as IP cameras. An example of the latest FPGA device is the Virtex-7 FPGA family from Xilinx.

For a deeper look at architectural trade-offs in programmable devices, see the EVA technical article, "Implementing Vision Capabilities in Embedded Systems".


Additional Resources:

Member Product Information: CogniVue Array Processor for Automotive

Member Product Information: Xilinx Platforms