This paper proposes HW/SW Co-design of an automatic classification system of Khalas, Khunaizi, Fardh, Qash, Naghal, and Maan dates fruit varieties in Oman. The system implements pre-processing, segmentation of the colored input images, color and shape-size features extraction followed by ANN-tansig classification. The performance of the proposed system is experimented and 97.26% highest classification accuracy are achieved. The proposed system is prototyped using a selected Zynq 7020 SoC platform featuring, on the same chip, a dual-core ARM Cortex A9 processing System (PS) interconnected with FPGA logic (PL) though high-throughput communication channels. The original classification algorithm is profiled and then a HW/SW Co-design is developed achieving 10.9 fps real time classification performance. This performance is acceptable and represents almost 14 times speedup acceleration comparatively to the original program implementation.