Language Organizes Color Geometry in Multilingual Encoders

Across six languages and five multilingual encoders, each of Berlin and Kay’s (1969) eleven basic color terms is represented by the mean embedding of its corpus contexts, giving one representational dissimilarity matrix (RDM) for each of the 30 language–encoder pairs. RDMs of the same language correlate at a mean Spearman ρ = .495 across encoders, compared with ρ = .231 for different languages within the same encoder.

Method

For each of the eleven basic color terms (black, blue, brown, gray, green, orange, pink, purple, red, white, yellow) in English, Russian, Spanish, Italian, Greek and Turkish, about 1,200 key-word-in-context lines are mined from CC-100. Five multilingual encoders (mBERT, XLM-R base and large, mDeBERTa-v3 and mmBERT) each read every line, and one hidden layer at about two-thirds depth is kept. Each occurrence of a word is represented by the mean over its subword tokens, and a word’s centroid is the mean of its occurrence vectors. Where a language has two words for blue, as Russian does with синий and голубой, the BLUE representation is the mean of the two. For each language–encoder cell, the pairwise cosine distances among the eleven centroids give an 11 × 11 RDM, and pairs of RDMs are compared by Spearman correlation.

blueсинийголубойazulμπλεmavi

1Term list

…увидеть голубое небо…

Usually light blue…

2KWIC lines

3Encoder

гол##уб##ое

4Term centroid

5RDM

ρ = 0.61

6Compare

Figure 1. How each RDM is made. Real examples throughout: mBERT's wordpieces for голубое; the mBERT English RDM; mBERT vs. XLM-R English (ρ = 0.61). Before the RDM, each language's vectors are centered and the two blues averaged into one BLUE. Layers: 8 for most models, 16 for XLM-R-L, 15 for mmBERT.

Results

RDMs from the same language computed by different encoders agree far more strongly than RDMs from the same encoder across languages (mean Spearman ρ = .495 versus .231). Language identity is also recoverable from a held-out RDM. Classifying each cell by its nearest language prototype succeeds in 27 of 30 cells, against 11 of 30 for encoder identity.

−0.250.85Spearman correlation between RDMs

Figure 2. Spearman correlations between all 30 RDMs (deeper red = higher correlation), with rows grouped by language (five encoders per block; diagonal masked). The weaker mmBERT rows correspond to its low-reliability cells.

The language signature is not driven by any single term, and it replicates on five disjoint samples of sentences per word.

Discussion

Across five encoders and six languages, the relational geometry of basic color terms behaves as a stable property of the language as used. It is nearly invariant to the sentences that instantiate it and largely invariant to the encoder that computes it, yet distinctive enough to identify its language from held-out data.

These findings are exploratory. The encoders’ pretraining data overlap, so shared text may contribute to same-language stability, and nothing here shows the effect is specific to color. Testing other lexical domains, and comparing against human color-similarity judgments, is a natural next step.