标题:选择,提取和生成:具有语法指导的神经关键字生成

摘要】近年来,深度神经序列到序列框架已经证明了关键短语生成中的有希望的结果。但是,使用这种深度神经网络处理长文档需要大量的计算资源。为了降低计算成本,通常在将文档作为输入之前将其截断。结果,模型可能会遗漏文档中传达的要点。而且,大多数现有方法要么是提取性的(从文档中识别出重要的短语),要么是生成性的(逐个单词地生成短语),因此它们无法从两种建模技术的优点中受益。为了解决这些挑战,我们提出了\ emph {SEG-Net},这是一种神经密钥短语生成模型,由两个主要部分组成:(1)选择器,用于选择文档中的突出句子;(2)提取器-生成器共同从选定的句子中提取并生成关键短语。 SEG-Net使用一种自我注意的体系结构,称为\ emph {Transformer}作为具有几个唯一性的构造块。首先,SEG-Net结合了新颖的\ emph {layer-wise}覆盖注意事项,以总结目标文档中讨论的大多数要点。其次,它使用\ emph {informed}复制注意机制来鼓励在短语提取和生成过程中集中于文档的不同部分。此外,SEG-Net联合学习关键短语的生成及其词性标签预测,而后者则为前者提供语法监督。来自科学和网络文档的七个关键短语生成基准的实验结果表明,SEG-Net在两个领域中都大大优于最新的神经生成方法。

Title: Select, Extract and Generate: Neural Keyphrase Generation with Syntactic Guidance

[abstract]  In recent years, deep neural sequence-to-sequence framework has demonstrated promising results in keyphrase generation. However, processing long documents using such deep neural networks requires high computational resources. To reduce the computational cost, the documents are typically truncated before given as inputs. As a result, the models may miss essential points conveyed in a document. Moreover, most of the existing methods are either extractive (identify important phrases from the document) or generative (generate phrases word by word), and hence they do not benefit from the advantages of both modeling techniques. To address these challenges, we propose \emph{SEG-Net}, a neural keyphrase generation model that is composed of two major components, (1) a selector that selects the salient sentences in a document, and (2) an extractor-generator that jointly extracts and generates keyphrases from the selected sentences. SEG-Net uses a self-attentive architecture, known as, \emph{Transformer} as the building block with a couple of uniqueness. First, SEG-Net incorporates a novel \emph{layer-wise} coverage attention to summarize most of the points discussed in the target document. Second, it uses an \emph{informed} copy attention mechanism to encourage focusing on different segments of the document during keyphrase extraction and generation. Besides, SEG-Net jointly learns keyphrase generation and their part-of-speech tag prediction, where the later provides syntactic supervision to the former. The experimental results on seven keyphrase generation benchmarks from scientific and web documents demonstrate that SEG-Net outperforms the state-of-the-art neural generative methods by a large margin in both domains.

【作者】Wasi Uddin Ahmad, Xiao Bai, Soomin Lee, Kai-Wei Chang

点击查看原文

头像

By szf