Object detection is still considered a difficult task in the field of computer vision. Specifically, earlobe detection has become a popular application as the interest in human identification using earlobe biometry has increased. So far earlobe detection problem has been solved using a combination of skin detection, edge detection, segmentation by fusion of histogram-based k-means, and template matching algorithms. In this work we present a method of earlobe detection without template matching by using a convolutional neural network, performing image segmentation. With this method, which is invariant to angle at which the photo was taken, earlobe shape, skin color, illumination, occlusions, and earlobe accessories, we were able to accurately detect the area of the image, where an earlobe is present. Moreover, detection time was significantly improved when compared to other methods for solving the same task. We expect our method to be used in Annotated Web Ears Toolbox.