Design of Low-Power and High Performance Embedded Vision SoC

Embedded vision (EV) systems rely on advanced computer vision (CV) algorithms implemented on programmable embedded computing platforms, where the optimal resource utilization is of paramount importance. Many of these algorithms deliver state-of-the-art accuracy in a myriad of applications, however, they require high computational power and large amounts of memory, and thus can benefit greatly from parallelization on multi-core processors. While general-purpose-based machines, especially graphics processing units (GPUs), have performed well for CV processing due to their ease of programmability and high parallelism, their excessive power consumption hinders deployment in embedded or low-power portable systems. Increasingly, there is interest in providing more specialized hardware acceleration of the vision computation. Accelerators can be in the form of fixed-function hardware (e.g. ASICs), programmable cores (e.g. GPUs) or dynamically programmable logic (e.g. FPGAs) for delivering a high performance requirement. FPGAs receive a significant attention due to its low power consumption, flexibility, reconfigurability. This research proposal aimed at developing a faster execution hardware design and energy-efficient object detector using deep learning (DL) techniques based on dynamically programmable logic acceleration. The proposed framework builds on the FPGA architecture, it shows that joint DL algorithm and hardware design can be used to reduce the energy consumption of object detection while delivering real-time and robust performance. The approach uses appearance-based method, contains parallelisable building blocks and flexible data types, mainly consists of two basic models, a model for object descriptor and a detector based on DL. The hardware resource consumption and performance estimation for the architecture will be synthesized using the High-Level Synthesis (HLS) design methodology considering Xilinx Virtex6 and Nvidia Jetson TX2 FPGA as the target device. Model functionality will be tested and experimentally verified. It is anticipated, the proposal will gain speedups over conventional methods and high performance in term of detection accuracy compared to state-of-the-art CPU and GPU-based implementations.

Embedded vision (EV) systems rely on advanced computer vision (CV) algorithms implemented on programmable embedded computing platforms, where the optimal resource utilization is of paramount importance. Many of these algorithms deliver state-of-the-art accuracy in a myriad of applications, however, they require high computational power and large amounts of memory, and thus can benefit greatly from parallelization on multi-core processors. While general-purpose-based machines, especially graphics processing units (GPUs), have performed well for CV processing due to their ease of programmability and high parallelism, their excessive power consumption hinders deployment in embedded or low-power portable systems. Increasingly, there is interest in providing more specialized hardware acceleration of the vision computation. Accelerators can be in the form of fixed-function hardware (e.g. ASICs), programmable cores (e.g. GPUs) or dynamically programmable logic (e.g. FPGAs) for delivering a high performance requirement. FPGAs receive a significant attention due to its low power consumption, flexibility, reconfigurability. This research proposal aimed at developing a faster execution hardware design and energy-efficient object detector using deep learning (DL) techniques based on dynamically programmable logic acceleration. The proposed framework builds on the FPGA architecture, it shows that joint DL algorithm and hardware design can be used to reduce the energy consumption of object detection while delivering real-time and robust performance. The approach uses appearance-based method, contains parallelisable building blocks and flexible data types, mainly consists of two basic models, a model for object descriptor and a detector based on DL. The hardware resource consumption and performance estimation for the architecture will be synthesized using the High-Level Synthesis (HLS) design methodology considering Xilinx Virtex6 and Nvidia Jetson TX2 FPGA as the target device. Model functionality will be tested and experimentally verified. It is anticipated, the proposal will gain speedups over conventional methods and high performance in term of detection accuracy compared to state-of-the-art CPU and GPU-based implementations.