基于图卷积网络的多标签食品原材料识别

当前,食品图像的营养成分识别主要还是集中在食品类别的识别以及作为多标签任务的识别.但是这两种方法并不具备很好的判别性,因为它们忽略了原材料之间的潜在关系.因此,本文在前期工作的基础上引入了原材料之间的关系.具体地说,我们的工作主要分为图像特征提取和原材料关系学习两部分.图像特征提取通过卷积神经网络提取到图像的低维特征向量.图卷积网络通过使用图数据（图的每个节点表示原材料的词嵌入,边表示节点之间的相关性）,将图数据直接映射到一组相互依赖的分类器中,并与图像的低维特征向量融合,最后进行分类.通过在Food-101和VireoFood-172两个食品数据集上进行实验,并与当前最好的实验模型进行对比,发现基于图卷积的食品多标签分类方法可以有效地提高食品图像的分类性能.

Multi-label food ingredient recognition via graph convolution network

People''s awareness about their nutrition habits is increasing.Keeping track of what we eat will be helpful for us to follow a healthier diet.Currently,nutrient recognition of food images is mainly focused on food categories recognition,or is tackled as multi-label task recognition.These two approaches,however,are not very discriminative owing to their neglect of potential relationship between ingredients.In this paper,we introduce the relationship between ingredients to identify food nutrients based on previous work.The recognition approach includes two modules,namely the image feature extraction module and the ingredients relationship learning module.The low-dimensional image feature vectors are extracted by convolutional neural network (CNN),and the relationship between ingredients is learned through a graph convolutional network (GCN).Specifically,GCN uses graph data where nodes represent food ingredients as word embedding and edges represent the correlation between nodes.Then the GCN directly map the graph data into a set of interdependent classifiers.Finally,the low-dimensional image feature vectors are fused to make detailed classification.We conducted experiments on food data sets of Food-101 and VireoFood-172.Compared with state of the art food recognition methods,our GCN-based multi-label food image classification method offers very promising results and can effectively improve the recognition performance.