Selected research

Science of language models and in-context learning

Data Distributional Properties Drive Emergent In-Context Learning in Transformers
SCY Chan, A Santoro, AK Lampinen, JX Wang, A Singh, PH Richemond, J McClelland, F Hill
Paper / Twitter (Oral, NeurIPS 2022)

Transformers generalize differently from information stored in context vs in weights
SCY Chan*, I Dasgupta*, J Kim, D Kumaran, AK Lampinen, F Hill
Paper / Twitter (NeurIPS MemARI Workshop 2022)

The Transient Nature of Emergent In-Context Learning in Transformers
AK Singh*, SCY Chan*, T Moskovitz, E Grant, AM Saxe, F Hill
Paper / Twitter (NeurIPS 2023)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation
AK Singh, T Moskovitz, F Hill, SCY Chan*, AM Saxe*
Paper / Twitter (Spotlight, ICML 2024)

Strategy Coopetition Explains the Emergence and Transience of In-Context Learning
AK Singh, T Moskovitz, S Dragutinovic, F Hill, SCY Chan*, AM Saxe*
Paper / Twitter (ICML 2025)

On the generalization of language models from in-context learning and finetuning: a controlled study
AK Lampinen*, A Chaudhry*, SCY Chan*, C Wild, D Wan, A Ku, J Bornschein, R Pascanu, M Shanahan, JL McClelland
Paper / Twitter (2025)

AI for Education

Towards responsible development of generative AI for education: An evaluation-driven approach
Paper / Blog (2024)

LearnLM: Improving Gemini for Learning
Paper / Gemini API (2025)

Full list of publications

Google Scholar

Posts

subscribe via RSS