izpis_h1_title_alt

Cross–lingual mappings of contextual word embedding ELMo
MILOSHESKI, LJUPCHE (Author), Robnik Šikonja, Marko (Mentor) More about this mentor... This link opens in a new window

.pdfPDF - Presentation file, Download (663,62 KB)

Abstract
To work with textual data, machine learning algorithms, in particular, neural networks, require word embeddings – vector representations of words in high-dimensional space. There are languages with a small amount of available resources. Exploiting the knowledge from the well-resourced languages for under-resourced languages is possible with cross-lingual embeddings by aligning the embeddings of one language with the vector space of another language. Existing methods for aligning embeddings are intended for context-independent embeddings, where every word has one representation. We propose a method, based on a dictionary and a parallel corpus aligns contextual embeddings, which capture more information about the context in which words appear. The proposed method requires a small amount of bilingual data, which is available for many language pairs. We empirically show that the proposed method outperforms the baseline obtained by alignment of context-independent embeddings.

Language:English
Keywords:cross-lingual word embeddings, contextual word embeddings, vector word embeddings, word translation, parallel corpus, vector space mappings, singular value decomposition
Work type:Bachelor thesis/paper (mb11)
Organization:FRI - Faculty of computer and information science
Year:2019
Views:152
Downloads:53
Metadata:XML RDF-CHPDL DC-XML DC-RDF
 
Average score:(0 votes)
Your score:Voting is allowed only to logged in users.
:
Share:AddThis
AddThis uses cookies that require your consent. Edit consent...

Secondary language

Language:Slovenian
Title:Medjezikovne poravnave kontekstne vložitve besed ELMo
Abstract:
Da bi algoritmi strojnega učenja, še posebej nevronske mreže, delali z besedilnimi podatki, potrebujejo vložitve besed – predstavitev besed v visokodimenzionalnem vektorskem prostoru. Za nekatere jezike je na voljo le majhna količina jezikovnih virov. Zanje je pomembno izkoriščanje znanja iz tehnološko bolj razvitih jezikov, kar omogočajo medjezikovne vložitve. Te vektorski prostor besed enega jezika preslikajo v vektorski prostor drugega jezika. Obstoječe metode za poravnavo vektorskih vložitev so namenjene kontekstno neodvisnim vložitvam, kjer ima vsaka beseda le eno predstavitev. Obstoječe metode za poravnavo vektorskih vložitev so namenjene kontekstno neodvisnim vložitvam, kjer ima vsaka beseda le eno predstavitev. Predstavljamo pristop, ki na podlagi slovarja in paralelnega korpusa poravna kontekstne vložitve, ki zajemajo več informacij o kontekstu, v katerem so besede uporabljene. Predlagani pristop zahteva majhno količino dvojezičnih virov, ki so na voljo za mnogo parov jezikov. Empirično pokažemo, da je predlagani pristop opazno boljši od izhodiščnega, s katerim poravnavamo kontekstno neodvisne vložitve.

Keywords:medjezikovne vložitve besed, kontekstne vložitve besed, vektorske vložitve besed, prevajanje besed, paralelni korpus, preslikave vektorskih prostorov, razcep z metodo singularnih vrednosti

Similar documents

Similar works from RUL:
Similar works from other Slovenian collections:

Comments

Leave comment

You have to log in to leave a comment.

Comments (0)
0 - 0 / 0
 
There are no comments!

Back