基于BERT嵌入和对抗训练的农业领域命名实体识别

doi:10.14088/j.cnki.issn0439-8114.2022.18.035

湖北农业科学 ›› 2022, Vol. 61 ›› Issue (18): 196-202.doi: 10.14088/j.cnki.issn0439-8114.2022.18.035

基于BERT嵌入和对抗训练的农业领域命名实体识别

费凡, 杨林楠

云南农业大学大数据学院,昆明 650201

收稿日期:2021-07-14 出版日期:2022-09-25 发布日期:2022-10-21
通讯作者: 杨林楠（1964-）,男,云南保山人,教授,博士,主要从事农业信息化研究,（电话）13888263241（电子信箱）lny5400@163.com。
作者简介:费凡（1994-）,男,安徽安庆人,在读硕士研究生,研究方向为自然语言处理,（电话）17555121687（电子信箱）1962288205@qq.com。
基金资助:
云南省重大科技项目专项计划项目（202002AD080002）

Named entity recognition in agriculture based on BERT embedding and adversarial training

FEI Fan, YANG Lin-nan

College of Big Data, Yunnan Agricultural University,Kunming 650201, China

Received:2021-07-14 Online:2022-09-25 Published:2022-10-21

摘要/Abstract

摘要： 面向农业领域的命名实体识别任务是农业领域中信息提取及问答系统的关键步骤,目标是从海量非结构化农业文本中找出需要的命名实体,通常存在诸如实体名称多样和上下文语义缺失等挑战。为完成复杂语境下的农业命名实体识别任务,首先构建一个农业领域的标注语料库,其中包含6类实体共16 048个样本;接着将BERT预训练语言模型作为词嵌入层,相比较传统词嵌入模型,很好地解决了不同语境下同一词语的不同语义以及指代问题;然后利用BiGRU网络模型进行上下文编码;最后利用CRF优化输出标注。同时,对输入数据引入一定的噪声,以此进行对抗训练来提高模型的泛化性和鲁棒性。经过试验,提出模型的准确率、召回率、F值分别为92.75%、91.53%、92.49%,与基线模型相比,该方法的性能表现更好,可以有效地识别出农业领域命名实体。

关键词: 农业, 自然语言处理, 命名实体识别, 信息抽取, BERT, BiGRU, 对抗训练

Abstract: Named entity recognition task for the agricultural domain is a key step in information extraction and question and answer system in the agricultural domain. The goal of this step is to find out the required named entities from a huge amount of unstructured agricultural texts, which usually have challenges such as diverse entity names and contextual semantics missing. To accomplish the task of recognizing agricultural named entities in complex contexts, this paper first constructs an annotated corpus in the agricultural domain, which contains 16 048 samples of six types of entities; then uses the BERT pre-trained language model as the word embedding layer, which can well solve the problem of different semantics and referents of the same word in different contexts compared with the traditional word embedding model; then uses the BiGRU network model for context encoding; finally, the output sequence is annotated using CRF. At the same time, this paper introduces a certain amount of noise to the input data, which is used for adversarial training to improve the generalization and robustness of the model. After the experiments, the accuracy, recall, and F-value of the proposed model are 92.75%, 91.53%, and 92.49%, respectively. Compared with the baseline model, this method has better performance and can effectively identify named entities in the agricultural field.

Key words: agriculture, natural language processing, named entity recognition, information extraction, BERT, BiGRU, adversarial training

中图分类号:

TP391.1

费凡, 杨林楠. 基于BERT嵌入和对抗训练的农业领域命名实体识别[J]. 湖北农业科学, 2022, 61(18): 196-202.

FEI Fan, YANG Lin-nan. Named entity recognition in agriculture based on BERT embedding and adversarial training[J]. HUBEI AGRICULTURAL SCIENCES, 2022, 61(18): 196-202.

参考文献

[1] CHINCHOR N, ROBINSON P.MUC-7 named entity task definition[C].Proceedings of the 7th conference on message understanding,1997.1-21.
[2] DRURY B, ROCHE M.A survey of the applications of text mining for agriculture[J]. Computers and electronics in agriculture,2019, 163: 104864.
[3] LIU W, YU B, ZHANG C, et al.Chinese named entity recognition based on rules and conditional random field[A]. Proceedings of the 2018 2nd international conference on computer science and artificial intelligence[C].United States,2018.268-272.
[4] WANG X, JIANG X, LIU M, et al.Bacterial named entity recognition based on dictionary and conditional random field[A]. IEEE international conference on bioinformatics and biomedicine (BIBM)[C]. 2017.439-444.
[5] WALLACH H M.Conditional random fields: An introduction[R]. University of Pennsylvania CIS Technical Report, 2004.22.
[6] LI X, WEI X H, JIA L, et al.Recognition of crops, diseases and pesticides named entities in Chinese based in conditional random fields[J]. Trans Chin Soc Agric Mach, 2017, 48: 178-185.
[7] MALARKODI C S,LEX E, DEVI S L.Named entity recognition for the agricultural domain[J]. Res Comput Sci,2016,117:121-132.
[8] ZHOU G D, SU J.Named entity recognition using an HMM-based chunk tagger[A].Proceedings of the 40th annual meeting of the association for computational linguistics[C].2002.473-480.
[9] LECUN Y,BENGIO Y,HINTON G.Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[10] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[11] HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[J/OL].arXiv preprint,https://arxiv.org/abs/1508.01991,2015-08-09.
[12] ESPEJO-GARCIA B,LOPEZ-PELLICER F J,LACASTA J, et al. End-to-end sequence labeling via deep learning for automatic extraction of agricultural regulations[J]. Computers and electronics in agriculture, 2019, 162: 106-111.
[13] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[C].Advances in neural information processing systems. 2017. 5998-6008.
[14] DEVLIN J,CHANG M W,LEE K,et al.Bert: Pre-training of deep bidirectional transformers for language understanding[DB/OL]. https://aclanthology.org/N19-1423, 2019.
[15] GOODFELLOW I J,SHLENS J,SZEGEDY C. Explaining and harnessing adversarial examples[J/OL]. arXiv preprint,https://doi.org/10.48550/arXiv.1412.6572, 2014.
[16] MIYATO T,DAI A M,GOODFELLOW I. Adversarial training methods for semi-supervised text classification[J/OL]. arXiv preprint,https://doi.org/10.48550/arXiv.1605.07725,2016.
[17] CHEN X, CARDIE C. Multinomial adversarial networks for multi-domain text classification[J/OL]. arXiv preprint,https://doi.org/10.48550/arXiv.1802.05694,2018.
[18] ZHOU J T,ZHANG H,JIN D, et al.Dual adversarial neural transfer for low-resource named entity recognition[A].Proceedings of the 57th annual meeting of the association for computational linguistics[C]. 2019.3461-3471.
[19] 赵鹏飞,赵春江,吴华瑞,等. 基于注意力机制的农业文本命名实体识别[J].农业机械学报,2021,52(1):185-192.
[20] 李林,周晗,郭旭超,等. 基于多源信息融合的病虫害命名实体识别研究[J/OL].农业机械学报,http://kns.cnki.net/kcms/detail/11.1964.S.20210915.0933.006.html,2021-11-20.
[21] CHUNG J, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J/OL]. arXiv preprint,https://doi.org/10.48550/arXiv.1412.3555, 2014.
[22] MIYATO T,DAI A M,GOODFELLOW L.Adversarial training methods for semi-supervised text classification[J/OL]. arXiv preprint,https://doi.org/10.48550/arXiv.1605.07725, 2017.
[23] DONG C, ZHANG J, ZONG C, et al.Character-based LSTM-CRF with radical-level features for Chinese named entity recognition[A].Natural language processing and Chinese computing[C]. Kunming,2016,2016.239-250.

基于BERT嵌入和对抗训练的农业领域命名实体识别

Named entity recognition in agriculture based on BERT embedding and adversarial training

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李志芳, 沈新磊, 王锐. 基于水足迹的漯河市农业水资源空间特征分析[J]. 湖北农业科学, 2022, 61(9): 24-26.
[2]	朱强, 鲍晨. 大数据赋能农业：从底层逻辑到顶层设计[J]. 湖北农业科学, 2022, 61(9): 136-140.
[3]	赵俊. 塔里木河流域农业经济发展的多维度分析——基于灰色关联度模型[J]. 湖北农业科学, 2022, 61(9): 176-179.
[4]	李成媛, 李敬锁. 山东省农业科技资源配置效率研究[J]. 湖北农业科学, 2022, 61(9): 186-191.
[5]	曾德鹏, 马瑛, 李婷, 赵和萍, 杨宏伟, 杨童波. 中国农业绿色发展研究的特征与趋势分析——基于CiteSpace的数据分析[J]. 湖北农业科学, 2022, 61(8): 13-19.
[6]	张盉儆. 安康市生态环境保护与农业经济耦合协调发展评价研究[J]. 湖北农业科学, 2022, 61(8): 140-143.
[7]	刘晓敏, 王煜. 江苏省农业现代化与农业保险耦合协调性研究[J]. 湖北农业科学, 2022, 61(8): 174-178.
[8]	张庶, 向俊, 李卫东, 向何龙, 瞿勇, 胡百顺. 特色产业地区实施乡村振兴过程中的农业科技政策调研——以湖北省恩施州为例[J]. 湖北农业科学, 2022, 61(8): 200-203.
[9]	刘震, 纪明妹, 郭志顶, 黄素芳, 赵忠祥, 闫旭东, 滕霄, 石秘, 岳明强, 刘青松, 徐玉鹏. 河北省农业害虫图像识别系统建设[J]. 湖北农业科学, 2022, 61(7): 135-139.
[10]	卢东宁, 张雨. 陕西省农业绿色发展水平评价研究[J]. 湖北农业科学, 2022, 61(6): 201-206.
[11]	宁夏, 李小红, 黄艳芳, 容建波, 孔令孜. 广西现代农业园区发展历程与经验启示[J]. 湖北农业科学, 2022, 61(6): 207-211.
[12]	柳开楼, 万长艳, 胡丹丹, 胡惠文, 余喜初, 李大明, 叶会财, 胡志华, 宋惠洁. 江西省不同地市化肥用量分析及对农业产值的贡献[J]. 湖北农业科学, 2022, 61(5): 11-13.
[13]	梁天财, 王德全. 农业废弃物还田对小白菜农艺性状的影响[J]. 湖北农业科学, 2022, 61(5): 41-44.
[14]	王万茹, 何秉宇. 孤立绿洲传统农业生产对自然灾害的响应研究——以新疆和田地区为例[J]. 湖北农业科学, 2022, 61(5): 120-126.
[15]	张瀚亓, 李璐骥, 高坤. 东北地区农业水足迹的空间差异及其影响因素分析[J]. 湖北农业科学, 2022, 61(5): 214-221.