Positional Description Matters for Transformers Arithmetic
Ruoqi Shen, Sébastien Bubeck, Ronen Eldan, Yin Tat Lee, Yuanzhi Li, Yi Zhang
November 2023
Physics of AGI
Understanding and improving emergent intelligence in large language models
Publications
Publications
Publications by Year
-
Who’s Harry Potter? Approximate Unlearning in LLMs
October 2023
The AI Revolution in Medicine GPT-4 and Beyond
Peter Lee, Carey Goldberg, Isaac S. Kohane, Sébastien Bubeck
October 2023
Textbooks Are All You Need II: phi-1.5 technical report
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, Yin Tat Lee
September 2023
Textbooks Are All You Need
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio Cesar, Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, Yuanzhi Li
June 2023
TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Ronen Eldan, Yuanzhi Li
May 2023
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Peter Lee, Sébastien Bubeck, Joseph Petro
New England Journal of Medicine | March 2023, Vol 388(13): pp. 1233-1239
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang
March 2023
-
Unveiling Transformers with LEGO: a synthetic reasoning task
Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Tal Wagner
June 2022
-
A Universal Law of Robustness via Isoperimetry
Sébastien Bubeck, Mark Sellke
NeurIPS 2021 | December 2021
Outstanding Paper Award
-
Convex Optimization: Algorithms and Complexity
Foundations and Trends in Machine Learning | January 2015, Vol 8(4): pp. 231-357
Publications by Research Area
-
Positional Description Matters for Transformers Arithmetic
Ruoqi Shen, Sébastien Bubeck, Ronen Eldan, Yin Tat Lee, Yuanzhi Li, Yi Zhang
November 2023
Who’s Harry Potter? Approximate Unlearning in LLMs
October 2023
The AI Revolution in Medicine GPT-4 and Beyond
Peter Lee, Carey Goldberg, Isaac S. Kohane, Sébastien Bubeck
October 2023
Textbooks Are All You Need II: phi-1.5 technical report
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, Yin Tat Lee
September 2023
Textbooks Are All You Need
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio Cesar, Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, Yuanzhi Li
June 2023
TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Ronen Eldan, Yuanzhi Li
May 2023
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Peter Lee, Sébastien Bubeck, Joseph Petro
New England Journal of Medicine | March 2023, Vol 388(13): pp. 1233-1239
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang
March 2023
Unveiling Transformers with LEGO: a synthetic reasoning task
Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Tal Wagner
June 2022
A Universal Law of Robustness via Isoperimetry
Sébastien Bubeck, Mark Sellke
NeurIPS 2021 | December 2021
Outstanding Paper Award
-
The AI Revolution in Medicine GPT-4 and Beyond
Peter Lee, Carey Goldberg, Isaac S. Kohane, Sébastien Bubeck
October 2023
-
Unveiling Transformers with LEGO: a synthetic reasoning task
Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Tal Wagner
June 2022
Convex Optimization: Algorithms and Complexity
Foundations and Trends in Machine Learning | January 2015, Vol 8(4): pp. 231-357
Publications by Type
-
Positional Description Matters for Transformers Arithmetic
Ruoqi Shen, Sébastien Bubeck, Ronen Eldan, Yin Tat Lee, Yuanzhi Li, Yi Zhang
November 2023
Who’s Harry Potter? Approximate Unlearning in LLMs
October 2023
Textbooks Are All You Need
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio Cesar, Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, Yuanzhi Li
June 2023
TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Ronen Eldan, Yuanzhi Li
May 2023
-
The AI Revolution in Medicine GPT-4 and Beyond
Peter Lee, Carey Goldberg, Isaac S. Kohane, Sébastien Bubeck
October 2023
-
Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine
Peter Lee, Sébastien Bubeck, Joseph Petro
New England Journal of Medicine | March 2023, Vol 388(13): pp. 1233-1239
Convex Optimization: Algorithms and Complexity
Foundations and Trends in Machine Learning | January 2015, Vol 8(4): pp. 231-357
-
Textbooks Are All You Need II: phi-1.5 technical report
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, Yin Tat Lee
September 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, Yi Zhang
March 2023
Unveiling Transformers with LEGO: a synthetic reasoning task
Yi Zhang, Arturs Backurs, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Tal Wagner
June 2022
-
A Universal Law of Robustness via Isoperimetry
Sébastien Bubeck, Mark Sellke
NeurIPS 2021 | December 2021
Outstanding Paper Award