Representing syntax and composition with geometric transformations

Bertolini, Lorenzo, Weeds, Julie, Weir, David and Peng, Qiwei (2021) Representing syntax and composition with geometric transformations. The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), Bangkok, Thailand, August 1-6, 2021. Published in: Findings of the Association for Computational Linguistics: ACL 2021. 3343-3353. Association for Computational Linguistics

[img] PDF - Accepted Version
Available under License Creative Commons Attribution.

Download (1MB)
[img] PDF - Published Version
Available under License Creative Commons Attribution.

Download (1MB)

Abstract

The exploitation of syntactic graphs (SyGs) as a word's context has been shown to be beneficial for distributional semantic models (DSMs), both at the level of individual word representations and in deriving phrasal representations via composition. However, notwithstanding the potential performance benefit, the syntactically-aware DSMs proposed to date have huge numbers of parameters (compared to conventional DSMs) and suffer from data sparsity. Furthermore, the encoding of the SyG links (i.e., the syntactic relations) has been largely limited to linear maps. The knowledge graphs' literature, on the other hand, has proposed light-weight models employing different geometric transformations (GTs) to encode edges in a knowledge graph (KG). Our work explores the possibility of adopting this family of models to encode SyGs. Furthermore, we investigate which GT better encodes syntactic relations, so that these representations can be used to enhance phrase-level composition via syntactic contextualisation.

Item Type: Conference Proceedings
Keywords: Distributional Semantics, Composition, Syntax, Representation Learning
Schools and Departments: School of Engineering and Informatics > Informatics
SWORD Depositor: Mx Elements Account
Depositing User: Mx Elements Account
Date Deposited: 01 Jun 2021 06:59
Last Modified: 26 Nov 2021 13:22
URI: http://sro.sussex.ac.uk/id/eprint/99426

View download statistics for this item

📧 Request an update