The pitfalls of transfer learning in computer vision for agriculture

Abstract

Computer vision applications based on modern AI methods are becoming increasingly important in agriculture, supporting and automating common processes. These applications are usually based on well-established architectures and pre-trained models. However, our prior experience has shown that applying the concept of transfer learning to AI tasks in agriculture repeatedly resulted in systematic issues. The structure of agricultural images, containing objects similar in shape, color and texture, makes the reuse of well-established applications more challenging. To give a more detailed insight into the expected challenges, we trained two different networks, which are well-established in the literature: Mask R-CNN and YOLOv5 [He18; Jo21] and investigated them in two different learning setups. First, we applied the concept of transfer learning to these models by pre-training each on the COCO dataset and subsequently continued expanding the available target set with classes of the sugar beets dataset [Ch17]. In the second setup, we skipped pre-training and only trained the models on the given agriculture dataset. Furthermore, we describe the reasons for the results in more detail and highlight possible causes for the identified differences. Finally, the different performances of the networks allowed us to improve on best practices for the agricultural domain and give some advice for future computer vision tasks in this area.