Skip to Main content Skip to Navigation
Conference papers

Using Distributional Principles for the Semantic Study of Contextual Language Models

Abstract : Many studies were recently done for investigating the properties of contextual language models but surprisingly, only a few of them consider the properties of these models in terms of semantic similarity. In this article, we first focus on these properties for English by exploiting the distributional principle of substitution as a probing mechanism in the controlled context of SemCor and WordNet paradigmatic relations. Then, we propose to adapt the same method to a more open setting for characterizing the differences between static and contextual language models.
Document type :
Conference papers
Complete list of metadata

https://hal-cea.archives-ouvertes.fr/cea-03648645
Contributor : Contributeur MAP CEA Connect in order to contact the contributor
Submitted on : Thursday, April 21, 2022 - 5:29:34 PM
Last modification on : Saturday, April 23, 2022 - 3:38:41 AM
Long-term archiving on: : Friday, July 22, 2022 - 7:35:07 PM

File

 Restricted access
To satisfy the distribution rights of the publisher, the document is embargoed until : 2022-10-12

Please log in to resquest access to the document

Identifiers

  • HAL Id : cea-03648645, version 1
  • ARXIV : 2111.12174

Citation

Olivier Ferret. Using Distributional Principles for the Semantic Study of Contextual Language Models. 35th Pacific Asia Conference on Language, Information and Computation, Nov 2021, Shanghai, China. pp.189-200. ⟨cea-03648645⟩

Share

Metrics

Record views

20