Skip to Main content Skip to Navigation
Poster communications

Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri?

Abstract : While contextual language models are now dominant in the field of Natural Language Processing, the representations they build at the token level are not always suitable for all uses. In this article, we propose a new method for building word or type-level embeddings from contextual models. This method combines the generalization and the aggregation of token representations. We evaluate it for a large set of English nouns in the perspective of the building of distributional thesauri for extracting semantic similarity relations. Moreover, we analyze the differences of static embeddings and type-level embeddings according to features such as the frequency of words or the type of semantic relations these embeddings account for, showing that the properties of these two types of embeddings can be complementary and exploited for further improving distributional thesauri.
Document type :
Poster communications
Complete list of metadata

https://hal-cea.archives-ouvertes.fr/cea-03745322
Contributor : Contributeur MAP CEA Connect in order to contact the contributor
Submitted on : Thursday, August 4, 2022 - 9:02:59 AM
Last modification on : Saturday, August 6, 2022 - 3:26:37 AM

File

2022.lrec-1.276.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : cea-03745322, version 1

Citation

Olivier Ferret. Building Static Embeddings from Contextual Ones: Is It Useful for Building Distributional Thesauri?. 13th Conference on Language Resources and Evaluation (LREC 2022), Jun 2022, Marseille, France. pp.2583‑2590, 2022, LREC 2022. ⟨cea-03745322⟩

Share

Metrics

Record views

13

Files downloads

3