This work addresses the feature selection problem using a wrapper approach to select a feature subset to distinguish between the classes of newborn heart rate variability (HRV) corresponding to seizure and non-seizure. The method utilizes a filter as a pre-step to remove the irrelevant and redundant features from the original set of features to provide a starting feature subset for the wrapper. This reduces the computation load and the severity of the search operations involved in a wrapper approach. The goodness of the feature subset selected is compared over 3 different classifiers, namely linear classifier, quadratic classifier and k-Nearest Neighbour (k-NN) statistical classifiers in a leave-one-out (LOO) cross validation. It was found that the 1-NN outperformed the other classifiers resulting in significant reductions in feature dimensionality and achieving 85.7% sensitivity and 84.6% specificity.