Translation Aligned Sentence Embeddings for Turkish Language
Eren Unlu; Unver Ciftci
arXiv2023
19
ciftci2023translation
Abstract
Due to the limited availability of high quality datasets for training
sentence embeddings in Turkish, we propose a training methodology and a regimen
to develop a sentence embedding model. The central idea is simple but effective
: is to fine-tune a pretrained encoder-decoder model in two consecutive stages,
where the first stage involves aligning the embedding space with translation
pairs. Thanks to this alignment, the prowess of the main model can be better
projected onto the target language in a sentence embedding setting where it can
be fine-tuned with high accuracy in short duration with limited target language
dataset.