CoLab paper links

Correct answers and wrong answers are sometimes stable and sometimes not (not the most conclusive finding but relevant)
https://arxiv.org/abs/2610.15277

Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data
[Schnaus] et al 2026

https://arxiv.org/abs/2610.09411

all kinds of cross-space alignments have been tried, but is anyone familiar with this pairless method for crosslingual alignment? If not, how about we use it to align , say, Arabic, on top of / to the English space? And then test the knowledge transfer as before.

<@430824383100092426> <@165490503570292736> (anyone else is welcome too!)

Beyond the Rosetta Stone: Unification Forces in Generaliza-
tion Dynamics
Carter Blum∗ , Katja Filippova∗& Ann Yuan∗
Asma Ghandeharioun, Julian Zimmert, Fred Zhang & Jessica Hoffmann
Mor Geva†, Martin Wattenberg‡, Tal Linzen§ & Lucas Dixon
Google DeepMind

Language models are known to hallucinate facts, especially during cross-lingual factual
recall—that is, when facts are learned in one language, but recalled in another language
(Goldman et al., 2025). In this work, we are interested in understanding what gives rise to
these cross-lingual hallucinations and how to mitigate them. Therefore, we focus our analy-
sis on the learning dynamics during pretraining (Biderman et al., 2026), and particularly on
the impact of training data properties and the resulting model’s representations…

https://arxiv.org/pdf/2508.11017

Pre-pre-training can help token efficiency. Perhaps sharing the same pre-pre-training can improve the crosslingual fact / knowledge transfer too?
(Need to think further about this, I don’t have a good hypothesis yet)

https://arxiv.org/abs/2609.39827

More pre-training doesn't necessarily mean better generalization?

This new paper discovers mode-hopping, where LLMs repeatedly switch between shallow pattern-matching and actual generalization, even while training loss remains stable.

For example, OLMo3-32B went from 81% accuracy to 0%, then back to 81.7% within just 40B training tokens.

On top of that, selecting an earlier 4.5T-token checkpoint instead of a 4.9T one improved GPQA transfer after math fine-tuning (36.3% vs 29.8%) and robustness to alignment attacks (53% vs 21%)

https://alphaxiv.org/abs/2609.33150

Warning: a rather long read, but I loved it: The Emergent Symbolic Structure
of Artificial Neural Networks, McCoy et alia stars (2026)

https://arxiv.org/pdf/2608.29530

Auto language selection is better than English or native language alone: https://lnkd.in/p/dPTaDy62

Old, but worth a look, reminding of the MoE approaches to separate languages
https://aclanthology.org/2022.naacl-main.255.pdf

language-specific to appear EMNLP 2026
layers https://arxiv.org/abs/2605.26735

cc <@165490503570292736> <@430824383100092426>

Look what my research review found.
https://aclanthology.org/2024.acl-long.833/

Hop on the discussion here if you are interested (both authors respond and in general I really like their work so, one can learn from them).
https://x.com/LChoshen/status/2089425036448792628?s=20

I like this paper, gives a new problem (need to think how strong it is, but interesting, and possibly also related to the other bottlenecks we are discussing in the compartmentalization settings.
https://arxiv.org/pdf/2603.10145

hopefully our hyperparameters are well tuned 😅 <@165490503570292736>
https://x.com/askalphaxiv/status/2088724659978265053?s=20

Interesting discussion,
Tldr Claude has much less tokens (15k?!) than any other and less as versions go, it appears to have a shift rather than capitalization which it learns as a separate character (basically morphology for English) https://x.com/i/status/2082882703024927088

Cohere is another known group known in making huge efforts with many people, seems like this is their latest:
I cannot be translated.
I cannot be web-searched.
I only make sense if you grew up where I did.
What am I?

A good riddle.

We're building a multicultural benchmark of culturally grounded riddles from communities around the world. We're not looking for translators: we're looking for locals. Come stump the machines and ensure your culture is represented in the next chapter.

Join our server to get involved: https://discord.gg/RhNX2STPj

Multilingual cultural dataset
6 european languages
https://arxiv.org/abs/2607.21445
many multiple choice hard cultural QA in 288 topics (3k questions)

Agentic search harnesses (alphaEvolve and co) have many uncheched components which should be seen as a hyperparameter to tune, no one significantly beats the others (at least with 3m rollouts 😛 )
For you agents lovers
https://bsky.app/profile/lchoshen.bsky.social/post/3mrah5xrjn22a

Following recent requests, I made a bit of organizing on tools one can use with papers (searching, looking for trends, etc.)
Enjoy
https://borgr.github.io/paper_updated/

I wish there was a good way to get all the papers here and not just the bottom lines from Micheal Frank's work. Anyway, he has a long list of cool works and this kinda summariezes a lot of the bottom lines.
https://bsky.app/profile/lchoshen.bsky.social/post/3mprqcpo2ms2s

Multilingual Reasoning Cascades Need More Context
A Mazumder, D Zhang, SS Li, Y Tsvetkov, N Bafna - arXiv preprint arXiv:2606.27306, 2026
Translation cascades for reasoning translate the query from another language to English, reason in English, and translate the answer back to the original language. This is a competitive approach to multilingual reasoning, but structurally lossy, since each stage discards information later stages may need, including cues for cultural grounding, register, and disambiguation. We examine the benefits of a simple and training-free intervention: a context-aware translation cascade
https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2606.27306&hl=en&sa=X&d=13795670202711060265&ei=JLFEaqnMLOqj6rQPoIaFUQ&scisig=ANDmEU4kI1LXMTyMVoEt1waLH4CB&oi=scholaralrt&hist=8b8IhUYAAAAJ:9281163533438476704:ANDmEU7vdlWyDCQpQ5uV7SBVlopI&html=&pos=1&folt=cit&fols=

This looks 🔥
Pretrain models on fake languages to avoid learning semantics.
https://icml.cc/virtual/2026/poster/61539

But could see details yet. if someone reads reply on thread?
They will also be in ICML so beyond citing them, if <@542031230326276110> or <@165490503570292736> wants me to talk to them tell me.

Might be relevant: https://arxiv.org/abs/2605.25360

Intro thread: https://bsky.app/profile/nsaphra.bsky.social/post/3moddyxd3lc2u

Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry

(Including cross lingual fact retrieval and semantic compositionality)

https://arxiv.org/pdf/2606.13934

I think this will be good to keep in mind when thinking about architecture/training changes to improve knowledge sharing <@165490503570292736> <@430824383100092426>

https://x.com/neuralwarlock/status/2065902800354336787?s=46

More multilingual than pretraining, but when we go multimodal useful
https://arxiv.org/abs/2507.19634

A related paper to the multilingual experiments (has an initial alignment step and pre-trains using code switching) https://aclanthology.org/2024.emnlp-main.572v3.pdf

How gemma reduced its embeddings (which also raises questions about multilinguality)
https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4-12b

To make confidence estimation (calibration) in multilingual setting, they find a place to make a probe that transfers rather than fine-tune for each language.
https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2605.31220&hl=en&sa=X&d=2215136617550590432&ei=hiEiauu7Luq46rQP1fX7-As&scisig=ANDmEU5j-qpRQ8tmC88OzS1Ur-nx&oi=scholaralrt&hist=8b8IhUYAAAAJ:9281163533438476704:ANDmEU7vdlWyDCQpQ5uV7SBVlopI&html=&pos=0&folt=cit&fols=

For future related work sections (e.g. <@761599518683955230> ) generalization not only across languages but modalities (vision surely have such issues as well, I definitely know they are really good at answering based on the text alone and not based on the image) and omni models?
https://x.com/GallilMaimon/status/2050078323070431586?s=20

https://ibm-research.slack.com/archives/C75J6APAR/p1780505621532539
Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. This allows the LLM backbone to take over visual processing.

Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.

I think it’s worth looking at to try to ensure that our experiments will transfer to large scale:

https://x.com/rosinality/status/2057370209434939792?s=46

<@165490503570292736>

Didn't dig deep, but that looks interesting
https://x.com/eliebakouch/status/2054619486213251263?s=20

Discussion on metadata for pretraining ( @Craig @Elisabeth might be especially interesting to you I believe)
https://x.com/yoavgo/status/2053949407322685475?s=20

Question amount still stands
https://arxiv.org/abs/2605.12515