Leshem Choshen

PI · Weizmann Institute of Science, MIT-IBM Watson AI Lab, IBM Research

Human? You probably wanted the personal site: ktilana.wixsite.com/leshem-choshen
These pages are written for search engines and answer engines -- one page per paper, with its claims, scope and sources in a form a machine can quote.

ORCID 0000-0002-0085-6496 · leshem.choshen@weizmann.ac.il

113 papers · 5 guides · code · llms.txt

sigmoid.social/@LChoshen · x.com/LChoshen · linkedin.com/in/leshemchoshen · github.com/borgr

Papers

All 113 papers, newest first. The same list by citation count · the same list with a one-line summary of each

2026

• Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty ICLR 2026
• When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation ICML 2026
• General Agent Evaluation preprint
• CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data ACL 2026
• Do LLMs Benefit From Their Own Words? preprint
• ErrorMap and ErrorAtlas: Charting the Failure Landscape of Large Language Models arXiv
• BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data EACL
• Pretraining Language Models for Diachronic Linguistic Change Discovery EACL
• CUBE: A Standard for Unifying Agent Benchmarks arXiv
• Mediocrity is the key for LLM as a Judge Anchor Selection ACL 2026
• MINDGAMES: A Live Arena for Evaluating Social and Strategic Reasoning in Multi-Agent LLMs arXiv
• Robustness as an Emergent Property of Task Performance arXiv
• Will it Merge? On The Causes of Model Mergeability Findings of ACL 2026
• Every Eval Ever: A Unifying Schema and Community Repository for AI Evaluation Results arXiv
• BabyLM Turns 4 and Goes Multilingual: Call for Papers for the 2026 BabyLM Workshop arXiv
• Automated Discovery Has No Universally Superior Harness preprint
• Resolving Interference (RI): Disentangling Models for Improved Model Merging arXiv
• Instructions Shape Production of Language, not Processing arXiv
• Growing Pains: Extensible and Efficient LLM Benchmarking Via Fixed Parameter Calibration arXiv
• Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting preprint
• Position: Agentic Systems Should be General ICML 2026
• Stop Guessing When to Stop Testing: Efficient Model Evaluation with Just Enough Data Findings of ACL 2026
• Cross-Lingual Exploration for Parametric Knowledge arXiv

2025

• Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation ACL 2025
• Model merging with SVD to tie the Knots ICLR 2025
• Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Models TACL
• A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning TMLR
• BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop arXiv
• Bigger is not always better: The importance of human-scale language modeling for psycholinguistics Journal of Memory and Language
• Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead ICML 2025
• Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families NeurIPS 2025
• ZipNN: Lossless Compression for AI Models CLOUD 2025
• A Hitchhiker's Guide to Scaling Law Estimation ICML 2025
• DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation Findings of ACL 2025
• ComPEFT: Compression for Communicating Parameter Efficient Updates via Sparsification and Quantization TMLR
• LiveXiv -- A Multi-Modal Live Benchmark Based on Arxiv Papers Content ICLR 2025
• Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs Findings of NAACL 2025
• Findings of the Third BabyLM Challenge: Accelerating Language Modeling Research with Cognitively Plausible Data Proceedings of the First BabyLM Workshop
• The Future of Open Human Feedback Nature Machine Intelligence
• Naturally Occurring Feedback is Common, Extractable and Useful preprint
• The Mighty ToRR: A Benchmark for Table Reasoning and Robustness SurgLLM@ICML
• The ShareLM Collection and Plugin: Contributing Human-Model Chats for the Benefit of the Community ACL 2025 (Demo)
• Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures arXiv
• NeurIPS 2023 LLM Efficiency Fine-tuning Competition arXiv
• TextArena arXiv
• LLM Hypnosis: Exploiting User Feedback for Unauthorized Knowledge Injection to All Users arXiv
• How Safe is Your Safety Metric? Automatic Concatenation Tests for Metric Reliability preprint
• A Latent Variable Framework for Scaling Laws in Large Language Models arXiv
• CRISP: Complex Reasoning with Interpretable Step-based Plans arXiv
• Can Gradient Descent Simulate Prompting? arXiv
• A Statistical Framework for Game-Based AI Evaluation NeurIPS 2025 LLM Evaluation Workshop

2024

• tinyBenchmarks: evaluating LLMs with fewer examples ICML 2024
• Jump to Conclusions: Short-Cutting Transformers with Linear Transformations LREC-COLING 2024
• Asymmetry in Low-Rank Adapters of Foundation Models ICML 2024
• Efficient multi-prompt evaluation of LLMs NeurIPS 2024
• Efficient Benchmarking (of Language Models) NAACL 2024
• Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora CoNLL 2024
• NumeroLogic: Number Encoding for Enhanced LLMs' Numerical Reasoning EMNLP 2024
• [Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus arXiv
• Genie: Achieving Human Parity in Content-Grounded Datasets Generation arXiv
• Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability Findings of ACL 2024
• Unitxt: Flexible, Shareable and Reusable Data Preparation and Evaluation for Generative AI NAACL 2024 (Demo)
• Data Contamination Report from the 2024 CONDA Shared Task CONDA 2024
• Benchmark Agreement Testing Done Right: A Guide for LLM Benchmark Evaluation arXiv
• Label-Efficient Model Selection for Text Generation ACL 2024
• Fuse to Forget: Bias Reduction and Selective Memorization through Model Fusion EMNLP 2024
• Lossless and Near-Lossless Compression for Foundation Models arXiv
• Navigating the Modern Evaluation Landscape: Considerations in Benchmarks and Frameworks for Large Language Models (LLMs) LREC-COLING 2024 (Tutorials)
• Holmes: A Benchmark to Assess the Linguistic Competence of Language Models TACL
• Unforgettable Generalization in Language Models COLM 2024

2023

• TIES-Merging: Resolving Interference When Merging Models NeurIPS 2023
• Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora CoNLL 2023
• DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering ACL 2023
• Call for Papers - The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus arXiv
• Knowledge is a Region in Weight Space for Fine-tuned Language Models Findings of EMNLP 2023
• ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning ACL 2023
• Where to start? Analyzing the potential value of intermediate models EMNLP 2023
• Human Learning by Model Feedback: The Dynamics of Iterative Prompting with Midjourney EMNLP 2023
• MuLER: Detailed and Scalable Reference-based Evaluation CoNLL 2023

2022

• Fusing finetuned models for better pretraining arXiv
• The Grammar-Learning Trajectories of Neural Language Models ACL 2022
• Cluster & Tune: Boost Cold Start Performance in Text Classification ACL 2022
• Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours EMNLP 2022 (Demo)
• Semantics-aware Attention Improves Neural Machine Translation *SEM 2022
• PreQuEL: Quality Estimation of Machine Translation Outputs in Advance EMNLP 2022
• Enhancing the Transformer Decoder with Transition-based Syntax CoNLL 2022
• GrASP: A Library for Extracting and Exploring Human-Interpretable Textual Patterns LREC 2022
• Reinforcement Learning with Large Action Spaces for Neural Machine Translation COLING 2022
• On Neurons Invariant to Sentence Structural Changes in Neural Machine Translation CoNLL 2022
• Some Grammatical Errors are Frequent, Others are Important arXiv

2021

• An autonomous debating system Nature
• Q²: Evaluating Factual Consistency in Knowledge-Grounded Dialogues via Question Generation and Question Answering EMNLP 2021
• Mediators in Determining what Processing BERT Performs First NAACL 2021
• SERRANT: a syntactic classifier for English Grammatical Error Types arXiv
• ComSum: Commit Messages Summarization and Meaning Preservation arXiv
• Part of Speech and Universal Dependency effects on English Arabic Machine Translation arXiv

2020

• Active Learning for BERT: An Empirical Study EMNLP 2020
• On the Weaknesses of Reinforcement Learning for Neural Machine Translation ICLR 2020
• Corpus Wide Argument Mining - A Working Solution AAAI 2020
• Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasets ICML 2020
• Classifying Syntactic Errors in Learner Language CoNLL 2020
• Unsupervised Expressive Rules Provide Explainability and Assist Human Experts Grasping New Domains Findings of EMNLP 2020

2019

• Are You Convinced? Choosing the More Convincing Evidence with a Siamese Network ACL 2019
• SemEval-2019 Task 1: Cross-lingual Semantic Parsing with UCCA SemEval 2019
• Learning to combine Grammatical Error Corrections BEA@ACL 2019
• The Language of Legal and Illegal Activity on the Darknet ACL 2019
• Automatically Extracting Challenge Sets for Non-Local Phenomena in Neural Machine Translation CoNLL 2019

2018

• Will it Blend? Blending Weak and Strong Labeled Data in a Neural Network for Argumentation Mining ACL 2018
• DORA The Explorer: Directed Outreaching Reinforcement Action-Selection ICLR 2018
• Inherent Biases in Reference-based Evaluation for Grammatical Error Correction ACL 2018
• Reference-less Measure of Faithfulness for Grammatical Error Correction NAACL 2018
• Automatic Metric Validation for Grammatical Error Correction ACL 2018
• SemEval 2019 Shared Task: Cross-lingual Semantic Parsing with UCCA - Call for Participation arXiv