U-Net:用于包含无答案问题的机器阅读理解的轻量级模型

被引：3

作者：

孙付 ^{[1
]}

李林阳 ^{[1
]}

邱锡鹏 ^{[1
]}

刘扬 ^{[2
]}

黄萱菁 ^{[1
]}

机构：

[1] 复旦大学计算机学院

[2] 流利说硅谷人工智能实验室

来源：

中文信息学报 | 2021年 / 35卷 / 02期

关键词：

机器阅读理解; SQuAD; 注意力机制;

D O I：

暂无

中图分类号：

TP391.1 [文字信息处理];

学科分类号：

081203 ; 0835 ;

摘要：

处理机器阅读理解任务时,识别其中没有答案的问题是自然语言处理领域的一个新的挑战。该文提出U-Net模型来处理这个问题,该模型包括3个主要成分:答案预测模块、无答案判别模块和答案验证模块。该模型用一个U节点将问题和文章拼接为一个连续的文本序列,该U节点同时编码问题和文章的信息,在判断问题是否有答案时起到重要作用,同时对于精简U-Net的结构也有重要作用。与基于预训练的BERT不同,U-Net的U节点的信息获取方式更多样,并且不需要巨大的计算资源就能有效地完成机器阅读理解任务。在SQuAD 2.0中,U-Net的单模型F1得分72.6、EM得分69.3,U-Net的集成模型F1得分74.9、EM得分71.4,均为公开的非基于大规模预训练语言模型的模型结果的第一名。

引用

页码：99 / 106

页数：8

共 14 条

[1]

Bidirectional attention flow for machine comprehension . Seo Minjoon,Kembhavi Aniruddha,Farhadi Ali,et al. Proceedings of the International Conference on Learning Representations . 2015

[2]

Read + Verify: Machine Reading Comprehension with Unanswerable Questions[J] . Minghao Hu,Furu Wei,Yuxing Peng,Zhen Huang,Nan Yang,Dongsheng Li. roceedings of the AAAI Conference on Artificial Intelligence . 2019

[3]

Machine Comprehension Using Match-LSTM and Answer Pointer[J] . Shuohang Wang,Jing Jiang. oRR . 2016

[4]

Dropout: a simple way to prevent neural networks from overfitting[J] . Nitish Srivastava,Geoffrey E. Hinton,Alex Krizhevsky,Ilya Sutskever,Ruslan Salakhutdinov. ournal of Machine Learning Research . 2014 (1)

[5] Long short-term memory [J].

Hochreiter, S ;

Schmidhuber, J .

NEURAL COMPUTATION, 1997, 9 (08) :1735-1780

[6]

FusionNet:Fusing via fully-aware attention with application to machine comprehension . Hsin-Yuan Huang,Chenguang Zhu,Yelong Shen,et al. . 2018

[7]

QANet:Combining local convolution with global self-attention for reading comprehension . Adams Wei Yu,David Dohan,Minh Thang Luong,et al. Proceedings of the International Conference on Learning Representations . 2018

[8]

BERT:Pre-training ofdeep bidirectional transformers for Language Understanding . Jacob Devlin,Mingwei Chang,Kenton Lee,et al. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics . 2019

[9]

Stochastic answer networks for machine reading comprehension . Xiaodong Liu,Yelong Shen,Kevin Duh,et al. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics . 2018

[10]

Reinforced mnemonic reader for machine reading comprehension . Minghao Hu,Yuxing Peng,Zhen Huang,et al. Proceedings of the 27th International Joint Conference on Artificial Intelligence . 2018

← 1 2 →