Deep Phenotyping: Deep Learning for Temporal Phenotype/Genotype Classification

Abstract High resolution and high throughput, genotype to phenotype studies in plants are underway to accelerate breeding of climate ready crops. Complex developmental phenotypes are observed by imaging a variety of accessions in different environment conditions, however extracting the genetically heritable traits is challenging. In the recent years, deep learning techniques and in particular Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs) and Long-Short Term Memories (LSTMs), have shown great success in visual data recognition, classification, and sequence learning tasks. In this paper, we proposed a CNN-LSTM framework for plant classification of various genotypes. Here, we exploit the power of deep CNNs for joint feature and classifier learning, within an automatic phenotyping scheme for genotype classification. Further, plant growth variation over time is also important in phenotyping their dynamic behavior. This was fed into the deep learning framework using LSTMs to model these temporal cues for different plant accessions. We generated a replicated dataset of four accessions of Arabidopsis and carried out automated phenotyping experiments. The results provide evidence of the benefits of our approach over using traditional hand-crafted image analysis features and other genotype classification frameworks. We also demonstrate that temporal information further improves the performance of the phenotype classification system.