CoLab paper links

Correct answers and wrong answers are sometimes stable and sometimes not (not the most conclusive finding but relevant)

2026-10-18T11:14:32Z

Correct answers and wrong answers are sometimes stable and sometimes not (not the most conclusive finding but relevant)
https://arxiv.org/abs/2610.15277

Shared in Discord

Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data

2026-10-10T02:32:21Z

Shared Geometry As A Rosetta Stone: Cross-Modal Alignment Without Paired Data
[Schnaus] et al 2026

https://arxiv.org/abs/2610.09411

all kinds of cross-space alignments have been tried, but is anyone familiar with this pairless method for crosslingual alignment? If not, how about we use it to align , say, Arabic, on top of / to the English space? And then test the knowledge transfer as before.

<@430824383100092426> <@165490503570292736> (anyone else is welcome too!)

Shared in Discord

Beyond the Rosetta Stone: Unification Forces in Generaliza

2026-10-06T17:19:45Z

Beyond the Rosetta Stone: Unification Forces in Generaliza-
tion Dynamics
Carter Blum∗ , Katja Filippova∗& Ann Yuan∗
Asma Ghandeharioun, Julian Zimmert, Fred Zhang & Jessica Hoffmann
Mor Geva†, Martin Wattenberg‡, Tal Linzen§ & Lucas Dixon
Google DeepMind

Language models are known to hallucinate facts, especially during cross-lingual factual
recall—that is, when facts are learned in one language, but recalled in another language
(Goldman et al., 2025). In this work, we are interested in understanding what gives rise to
these cross-lingual hallucinations and how to mitigate them. Therefore, we focus our analy-
sis on the learning dynamics during pretraining (Biderman et al., 2026), and particularly on
the impact of training data properties and the resulting model’s representations…

https://arxiv.org/pdf/2508.11017

Shared in Discord

Pre-pre-training can help token efficiency. Perhaps sharing the same pre-pre-training can improve the crosslingual fact

2026-10-01T17:37:35Z

Pre-pre-training can help token efficiency. Perhaps sharing the same pre-pre-training can improve the crosslingual fact / knowledge transfer too?
(Need to think further about this, I don’t have a good hypothesis yet)

https://arxiv.org/abs/2609.39827

Shared in Discord

More pre-training doesn't necessarily mean better generalization?

2026-10-01T17:24:28Z

More pre-training doesn't necessarily mean better generalization?

This new paper discovers mode-hopping, where LLMs repeatedly switch between shallow pattern-matching and actual generalization, even while training loss remains stable.

For example, OLMo3-32B went from 81% accuracy to 0%, then back to 81.7% within just 40B training tokens.

On top of that, selecting an earlier 4.5T-token checkpoint instead of a 4.9T one improved GPQA transfer after math fine-tuning (36.3% vs 29.8%) and robustness to alignment attacks (53% vs 21%)

https://alphaxiv.org/abs/2609.33150

Shared in Discord

alphaxiv.org

2026-09-30T16:31:50Z

https://www.alphaxiv.org/abs/2609.tokenization-survey-modern-nlp

Shared in Discord

Warning: a rather long read, but I loved it: The Emergent Symbolic Structure

2026-09-09T20:37:06Z

Warning: a rather long read, but I loved it: The Emergent Symbolic Structure
of Artificial Neural Networks, McCoy et alia stars (2026)

https://arxiv.org/pdf/2608.29530

Shared in Discord

Auto language selection is better than English or native language alone

2026-08-30T16:40:45Z

Auto language selection is better than English or native language alone: https://lnkd.in/p/dPTaDy62

Shared in Discord

Old, but worth a look, reminding of the MoE approaches to separate languages

2026-08-21T21:18:52Z

Old, but worth a look, reminding of the MoE approaches to separate languages
https://aclanthology.org/2022.naacl-main.255.pdf

Shared in Discord

language-specific to appear EMNLP 2026

2026-08-21T13:40:08Z

language-specific to appear EMNLP 2026
layers https://arxiv.org/abs/2605.26735

cc <@165490503570292736> <@430824383100092426>

Shared in Discord

Look what my research review found.

2026-08-17T20:50:38Z

Look what my research review found.
https://aclanthology.org/2024.acl-long.833/

Shared in Discord

Hop on the discussion here if you are interested (both authors respond and in general I really like their work so, one c

2026-08-17T18:55:16Z

Hop on the discussion here if you are interested (both authors respond and in general I really like their work so, one can learn from them).
https://x.com/LChoshen/status/2089425036448792628?s=20

Shared in Discord

I like this paper, gives a new problem (need to think how strong it is, but interesting, and possibly also related to th

2026-08-17T18:09:07Z

I like this paper, gives a new problem (need to think how strong it is, but interesting, and possibly also related to the other bottlenecks we are discussing in the compartmentalization settings.
https://arxiv.org/pdf/2603.10145

Shared in Discord

hopefully our hyperparameters are well tuned 😅 <@165490503570292736>

2026-08-17T00:27:48Z

hopefully our hyperparameters are well tuned 😅 <@165490503570292736>
https://x.com/askalphaxiv/status/2088724659978265053?s=20

Shared in Discord

Interesting discussion,

2026-08-14T11:09:29Z

Interesting discussion,
Tldr Claude has much less tokens (15k?!) than any other and less as versions go, it appears to have a shift rather than capitalization which it learns as a separate character (basically morphology for English) https://x.com/i/status/2082882703024927088

Shared in Discord

bsky.app

2026-08-09T13:36:08Z

https://bsky.app/profile/lchoshen.bsky.social/post/3msnh547x4s2g

Shared in Discord

x.com

2026-08-01T12:23:03Z

https://x.com/i/status/2083255427421134891

Shared in Discord

Cohere is another known group known in making huge efforts with many people, seems like this is their latest

2026-07-28T13:07:30Z

Cohere is another known group known in making huge efforts with many people, seems like this is their latest:
I cannot be translated.
I cannot be web-searched.
I only make sense if you grew up where I did.
What am I?

A good riddle.

We're building a multicultural benchmark of culturally grounded riddles from communities around the world. We're not looking for translators: we're looking for locals. Come stump the machines and ensure your culture is represented in the next chapter.

Join our server to get involved: https://discord.gg/RhNX2STPj

Shared in Discord

Multilingual cultural dataset

2026-07-27T15:47:29Z

Multilingual cultural dataset
6 european languages
https://arxiv.org/abs/2607.21445
many multiple choice hard cultural QA in 288 topics (3k questions)

Shared in Discord

Agentic search harnesses (alphaEvolve and co) have many uncheched components which should be seen as a hyperparameter to

2026-07-23T12:18:46Z

Agentic search harnesses (alphaEvolve and co) have many uncheched components which should be seen as a hyperparameter to tune, no one significantly beats the others (at least with 3m rollouts 😛 )
For you agents lovers
https://bsky.app/profile/lchoshen.bsky.social/post/3mrah5xrjn22a

Shared in Discord

Following recent requests, I made a bit of organizing on tools one can use with papers (searching, looking for trends, e

2026-07-21T23:04:36Z

Following recent requests, I made a bit of organizing on tools one can use with papers (searching, looking for trends, etc.)
Enjoy
https://borgr.github.io/paper_updated/

Shared in Discord

x.com

2026-07-13T17:26:37Z

https://x.com/anthropicai/status/2076719540785012872?s=46

Shared in Discord

I wish there was a good way to get all the papers here and not just the bottom lines from Micheal Frank's work. Anyway,

2026-07-03T23:52:58Z

I wish there was a good way to get all the papers here and not just the bottom lines from Micheal Frank's work. Anyway, he has a long list of cool works and this kinda summariezes a lot of the bottom lines.
https://bsky.app/profile/lchoshen.bsky.social/post/3mprqcpo2ms2s

Shared in Discord

Multilingual Reasoning Cascades Need More Context

2026-07-01T10:41:33Z

Multilingual Reasoning Cascades Need More Context
A Mazumder, D Zhang, SS Li, Y Tsvetkov, N Bafna - arXiv preprint arXiv:2606.27306, 2026
Translation cascades for reasoning translate the query from another language to English, reason in English, and translate the answer back to the original language. This is a competitive approach to multilingual reasoning, but structurally lossy, since each stage discards information later stages may need, including cues for cultural grounding, register, and disambiguation. We examine the benefits of a simple and training-free intervention: a context-aware translation cascade
https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2606.27306&hl=en&sa=X&d=13795670202711060265&ei=JLFEaqnMLOqj6rQPoIaFUQ&scisig=ANDmEU4kI1LXMTyMVoEt1waLH4CB&oi=scholaralrt&hist=8b8IhUYAAAAJ:9281163533438476704:ANDmEU7vdlWyDCQpQ5uV7SBVlopI&html=&pos=1&folt=cit&fols=

Shared in Discord

This looks 🔥

2026-07-01T10:32:14Z

This looks 🔥
Pretrain models on fake languages to avoid learning semantics.
https://icml.cc/virtual/2026/poster/61539

But could see details yet. if someone reads reply on thread?
They will also be in ICML so beyond citing them, if <@542031230326276110> or <@165490503570292736> wants me to talk to them tell me.

Shared in Discord

alphaxiv.org

2026-06-30T16:41:10Z

https://alphaxiv.org/abs/2606.24579

Shared in Discord

arxiv.org

2026-06-24T05:36:58Z

https://arxiv.org/abs/2602.17655

Shared in Discord

Might be relevant

2026-06-22T17:38:33Z

Might be relevant: https://arxiv.org/abs/2605.25360

Shared in Discord

arxiv.org

2026-06-16T15:38:32Z

https://arxiv.org/abs/2604.07067

Shared in Discord

Intro thread

2026-06-15T14:45:44Z

Intro thread: https://bsky.app/profile/nsaphra.bsky.social/post/3moddyxd3lc2u

Shared in Discord

Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry

2026-06-15T14:42:16Z

Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry

(Including cross lingual fact retrieval and semantic compositionality)

https://arxiv.org/pdf/2606.13934

Shared in Discord

I think this will be good to keep in mind when thinking about architecture/training changes to improve knowledge sharing

2026-06-14T16:03:36Z

I think this will be good to keep in mind when thinking about architecture/training changes to improve knowledge sharing <@165490503570292736> <@430824383100092426>

https://x.com/neuralwarlock/status/2065902800354336787?s=46

Shared in Discord

alphaxiv.org

2026-06-12T21:25:27Z

https://alphaxiv.org/abs/2606.06586

Shared in Discord

More multilingual than pretraining, but when we go multimodal useful

2026-06-11T18:09:29Z

More multilingual than pretraining, but when we go multimodal useful
https://arxiv.org/abs/2507.19634

Shared in Discord

bsky.app

2026-06-11T13:45:42Z

https://bsky.app/profile/tedunderwood.com/post/3mnxaazjpls2e

Shared in Discord

A related paper to the multilingual experiments (has an initial alignment step and pre-trains using code switching)

2026-06-08T01:43:54Z

A related paper to the multilingual experiments (has an initial alignment step and pre-trains using code switching) https://aclanthology.org/2024.emnlp-main.572v3.pdf

Shared in Discord

How gemma reduced its embeddings (which also raises questions about multilinguality)

2026-06-05T13:00:27Z

How gemma reduced its embeddings (which also raises questions about multilinguality)
https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-gemma-4-12b

Shared in Discord

To make confidence estimation (calibration) in multilingual setting, they find a place to make a probe that transfers ra

2026-06-05T11:02:11Z

To make confidence estimation (calibration) in multilingual setting, they find a place to make a probe that transfers rather than fine-tune for each language.
https://scholar.google.com/scholar_url?url=https://arxiv.org/pdf/2605.31220&hl=en&sa=X&d=2215136617550590432&ei=hiEiauu7Luq46rQP1fX7-As&scisig=ANDmEU5j-qpRQ8tmC88OzS1Ur-nx&oi=scholaralrt&hist=8b8IhUYAAAAJ:9281163533438476704:ANDmEU7vdlWyDCQpQ5uV7SBVlopI&html=&pos=0&folt=cit&fols=

Shared in Discord

For future related work sections (e.g. <@761599518683955230> ) generalization not only across languages but modalities (

2026-06-03T22:47:00Z

For future related work sections (e.g. <@761599518683955230> ) generalization not only across languages but modalities (vision surely have such issues as well, I definitely know they are really good at answering based on the text alone and not based on the image) and omni models?
https://x.com/GallilMaimon/status/2050078323070431586?s=20

Shared in Discord

ibm-research.slack.com

2026-06-03T16:58:19Z

https://ibm-research.slack.com/archives/C75J6APAR/p1780505621532539
Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. This allows the LLM backbone to take over visual processing.

Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.

Shared in Discord

arxiv.org

2026-05-24T11:37:03Z

https://arxiv.org/abs/2605.19284

Shared in Discord

I think it’s worth looking at to try to ensure that our experiments will transfer to large scale

2026-05-21T08:20:40Z

I think it’s worth looking at to try to ensure that our experiments will transfer to large scale:

https://x.com/rosinality/status/2057370209434939792?s=46

<@165490503570292736>

Shared in Discord

x.com

2026-05-19T11:11:25Z

https://x.com/i/status/2056394276565889152

Shared in Discord

Didn't dig deep, but that looks interesting

2026-05-14T13:27:06Z

Didn't dig deep, but that looks interesting
https://x.com/eliebakouch/status/2054619486213251263?s=20

Shared in Discord

Discussion on metadata for pretraining ( @Craig @Elisabeth might be especially interesting to you I believe)

2026-05-14T13:26:58Z

Discussion on metadata for pretraining ( @Craig @Elisabeth might be especially interesting to you I believe)
https://x.com/yoavgo/status/2053949407322685475?s=20

Shared in Discord