Hyper-Parameter Search for Convolutional Neural Networks - An Evolutionary Approach

Abstract

Convolutional neural networks is one of the most popular neural network classes within the deep learning research area. Due to their specific architecture they are widely used to solve such challenging tasks as image and speech recognition, video analysis etc. The architecture itself is defined by a number of (hyper-)parameters that have major impact on the recognition rate. Although much significant progress has been made to improve the performance of convolutional networks, the typical hyper-parameter search is done manually, taking therefore a long time and likely to disregard some very good values. This paper solves the problem by proposing two different evolutionary algorithms for automated hyper-parameter search in convolutional architectures. It will be shown that in case of image recognition these algorithms are capable of finding architectures with nearly state of the art performance automatically, sparing the scientists from much tedious effort.