Investigating Existing Techniques for Designing Convolutional Neural Network Filters

In the last decades, Deep Learning (DL) has played an essential role in extracting automatically high-level features and performing complex computer vision tasks. A Convolutional Neural Network (CNN) is one of the deep learning methods that is currently attaining state-of-the-art accuracy in image classification and object recognition. CNN has three types of layers: convolutional layer, pooling layer and fully connected layer. The first two types of layers extract different levels of useful features, while the latter type is dedicated to perform classification. Traditional CNNs like AlexNet, VGGNet, GoogLeNet, ResNet use back-propagation approach in their training. Such approach requires huge amount of datasets which leads to high computational cost and suffers from vanishing gradient problem that deteriorates the quality of learning. Recently, a forward propagation approach was proposed to tackle these problems. In this paper, we summarize the results of a comparative study which we conducted on the two learning approaches.