基于少样本学习的农业领域短文本分类研究

doi:10.14088/j.cnki.issn0439-8114.2022.13.028

湖北农业科学 ›› 2022, Vol. 61 ›› Issue (13): 151-155.doi: 10.14088/j.cnki.issn0439-8114.2022.13.028

基于少样本学习的农业领域短文本分类研究

麻之润^1,2, 费凡¹, 黎芬¹, 董慧洁¹, 彭琳¹

1.云南农业大学大数据学院,昆明 650000;
2.绿色农产品大数据智能信息处理工程研究中心,昆明 650000

收稿日期:2021-06-03 出版日期:2022-07-10 发布日期:2022-08-10
通讯作者: 彭琳（1978-）,女,河南邓州人,副教授,主要从事问答系统和知识图谱研究,（电子信箱）dapengjiao@163.com。
作者简介:麻之润（1992-）,男,安徽合肥人,在读硕士研究生,研究方向为文本分类,（电话）19942414389（电子信箱）rudolph0ma@126.com
基金资助:
云南省重大科技专项（202002AD080002）

Research on short text classification in agricultural field based on few-shot learning

MA Zhi-run^1,2, FEI Fan¹, LI Fen¹, DONG Hui-jie¹, PENG Lin¹

1. College of Big Data, Yunnan Agricultural University, Kunming 650000, China;
2. Green Agricultural Products Big Data Intelligent Information Processing Engineering Research Center, Kunming 650000, China

Received:2021-06-03 Online:2022-07-10 Published:2022-08-10

摘要/Abstract

摘要： 为了便捷、准确、高效地识别海量信息中所描述问题的所属类别,解决农业领域短文本分类存在数据稀疏性、高度依赖上下文等问题,爬取了10 000多条农业问答领域的短文本,经过清洗、过滤和标注等处理后形成一个5分类的短文本数据集;构建了基于BERT和ERNIE预训练模型的农业短文本分类算法,并与基于决策树模型的农业短文本分类算法进行对比分析。结果表明,随着数据集样本的减少,3种模型的准确率、精确率和召回率均呈下降趋势;基于ERNIE预训练模型的准确率、F₁值处于较高水平,远高于同数据的决策树模型,表明构建的农业短文本分类算法能够在数据量不足的情况下依然获得较高的分类效果。

关键词: 预训练模型, 文本分类, BERT, ERNIE

Abstract: In order to conveniently, accurately and efficiently identify the categories of the problems described in the massive information, and to address the problems of data sparsity and high dependence on context in agricultural short text classification, this paper crawled more than 10,000 short texts in the field of agricultural Q&A and formed a 5-classified short text dataset after cleaning, filtering and annotation processes. Then an agricultural short text classification algorithm based on BERT and ERNIE pre-training models was constructed, and compared with the agricultural short text classification algorithm based on the decision tree model. The results indicated that with the reduction of data set samples, the accuracy, precision and recall of the three models all showed a downward trend. The accuracy and F₁ value of the ERNIE pre-training model were at a high level, which was much higher than those of the decision tree model with the same data, showing that the constructed agricultural short text classification algorithm could still achieve a high classification effect in the case of insufficient data.

Key words: pre-training model, text classification, BERT, ERNIE

中图分类号:

TP391.1

麻之润, 费凡, 黎芬, 董慧洁, 彭琳. 基于少样本学习的农业领域短文本分类研究[J]. 湖北农业科学, 2022, 61(13): 151-155.

MA Zhi-run, FEI Fan, LI Fen, DONG Hui-jie, PENG Lin. Research on short text classification in agricultural field based on few-shot learning[J]. HUBEI AGRICULTURAL SCIENCES, 2022, 61(13): 151-155.

参考文献

[1] 邓丁朋,周亚建,池俊辉,等.短文本分类技术研究综述[J].软件,2020,41(2):141-144.
[2] MARON M E.Automatic indexing: An experimental inquiry[J]. ACM,1961,8(3): 404-417.
[3] 李静梅,孙丽华,张巧荣,等.一种文本处理中的朴素贝叶斯分类器[J].哈尔滨工程大学学报,2003(1):71-74.
[4] 余芳. 一个基于朴素贝叶斯方法的web文本分类系统:WebCAT[J].计算机工程与应用,2004(13):195-197.
[5] COVER T,HART P.Nearest neighbor pattern classification[J]. IEEE transactions on information theory,1967,13(1):21-27.
[6] 庞剑锋. 基于向量空间模型的自反馈的文本分类系统的研究与实现[D].北京:中国科学院研究生院(计算技术研究所),2001.
[7] 湛燕. K-近邻、K-均值及其在文本分类中的应用[D].河北保定:河北大学,2003.
[8] JOACHIMS T.Text categorization with Support Vector Machines: Learning with many relevant features[J/OL].Machine learning: ECML-98, https://doi.org/10.1007/BFb0026683.
[9] LIU L, ÖZSU M T. Term frequency by inverse document frequency[J/OL].https://doi.org/10.1007/978-0-387-39940-9_3784.
[10] MIKOLOV T, CHEN K, CORRADO G, et al.Efficient estimation of word representations in vector space[J].Computation and language,2013,1:3781.
[11] PENNINGTON J,SOCHER R,MANNING C D.Glove: Global vectors for word representation[J/OL].https://nlp.stanford.edu/pubs/glove.pdf.
[12] KIM K.Convolutional neural networks for sentence classification[A].Proceedings of the 2014 conference on empirical methods in natural language processing(EMNLP)[C].Doha, Qatar:Association for computational linguistics,2014.1746-1751.
[13] IYYER M, MANJUNATHA V, BOYD-GRABER J L, et al. Deep unordered composition rivals syntactic methods for text classification[A].Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the Asian federation of natural language processing[C].Beijing, China:Association for computational linguistics, 2015.1681-1691.
[14] TAI K S, SOCHER R, MANNING C D.Improved semantic representations from tree-structured long short-term memory networks[A].Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing of the Asian federation of natural language processing[C].Beijing, China:Association for computational linguistics, 2015.1556-1566.
[15] GRAVE E, MIKOLOV T, JOULIN A, et al.Bag of tricks for efficient text classification[A].Proceedings of the 15th conference of the european chapter of the association for computational linguistics[C].Valencia,Spain:Association for computational linguistics,2017.427-431.
[16] VASWANI A, SHAZEER N, PARMAR N, et al.Attention is all you need[J].Computation and language,2017,1:5998-6008.
[17] 刘川. 面向小样本的文本分类模型及算法研究[D].成都:电子科技大学,2017.
[18] SONG G, YEY M, DU X L,et al.Short text classification: A survey[J]. Journal of multimedia,2014,9(5):635-643.
[19] HAN X,ZHU H,YU P F, et al.FewRel: A Large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation[A].Proceedings of the 2018 conference on empirical methods in natural language processing[C].Brussels, Belgium:Association for computational linguistics, 2018.4803-4809.
[20] VINYALS O, BLUNDELL C, LILLICRAP T, et al.Matching networks for one shot learning[A].30th Conference on neural information processing systems (NIPS 2016)[C].Barcelona, Spain:Association for computational linguistics,2016.
[21] WANG Y X, GIRSHICK R, HEBERT M, et al.Low-shot learning from imaginary data[A].Proc. of the IEEE Conf. on computer vision and pattern recognition[C]. Salt Lake City,UT,USA:IEEE,2018.
[22] LU J,GONG P H,YE J P, et al.Learning from very few samples: A survey [J/OL].https://doi.org/10.48550/arXiv.2009.02653.
[23] PETERS M E,NEUMANN M,IYYERM,et al.Deep contextualized word representations[A].Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics:Human language technologies[C].New Orleans,Louisiana:Association for computational linguistics,2018.2227-2237.
[24] HOWARD J,RUDER S.Universal language model fine-tuning for text classification[A].Proceedings of the 56th annual meeting of the association for computational linguistics[C]. Melbourne,Australia:Association of computational linguistics,2018. 328-339.
[25] DEVLIN J, CHANG M, LEE K, et al.BERT: pre-training of deep bidirectional transformers for language understanding[A].Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies[C]. Minneapolis,Minnesota:Association for computational linguistics, 2019, 1:4171-4186.
[26] YANG Z, DAI Z, YANG Y, et al.XLNet: Generalized autoregressive pretraining for language understanding[A].Advances in neural information processing systems 32: Annual conference on neural information processing systems 2019[C]. Vancouver, Canada:NeurIPS ,2019.5754-5764.

基于少样本学习的农业领域短文本分类研究

Research on short text classification in agricultural field based on few-shot learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价