english

A Word-in-Context (WiC) encoder trained with contrastive learning on top of FacebookAI/xlm-roberta-large. It embeds a target word in its sentence context: the target span is wrapped in <t> … </t> markers, and two usages are compared by cosine similarity. A pair is predicted to share a sense when cosine similarity exceeds the stored threshold.

The default decision threshold is 0.7810 (stored in threshold.json).

Usage

# from https://github.com/... (contrastive_encoder_training)
from word_transformer import load_model, predict_pairs

model, threshold_info = load_model("english")   # local dir or hub id
pairs = [{
    "sentence1": "They walked along the bank of the river.",
    "sentence2": "She deposited the check at the bank.",
    "start1": 22, "end1": 26,     # exclusive end: sentence1[22:26] == "bank"
    "start2": 21, "end2": 25,
}]
predictions, cosine_sims = predict_pairs(model, threshold_info["threshold"], pairs)
# predictions[i] == 1  -> same sense,  0 -> different sense

Training


base model	FacebookAI/xlm-roberta-large
language	English
paper	Multilinguality Does not Make Sense (EMNLP 2025)
threshold	0.7810
validation accuracy	0.9146

This model uses a custom encode() (target-word embedding), so it is not a drop-in SentenceTransformer; load it with the word_transformer.py helper above.