-->

On the changing role of artificial intelligence in mathematics

On the changing role of artificial intelligence in mathematics

Department of mathematics

University of Southern California

May 14th, 2025

On the changing role of artificial intelligence in mathematics

Department of mathematics

University of Southern California

May 14th, 2025

On the changing role of artificial intelligence in mathematics

Department of mathematics

University of Southern California

May 14th, 2025

  • Goal: initiate an open-ended discussion on the rapidly changing role of artificial intelligence in mathematics
  • Disclaimer: I claim absolutely no authority on this topic. These slides are nothing more than my attempt to get the ball rolling.

A few impetuses:

  • jacuzzi incident
  • FrontierMath symposium
  • DARPA expMath, Renaissance Philanthropy AI for Math Fund, ...

A few impetuses:

  • Jacuzzi incident
  • FrontierMath symposium
  • DARPA expMath, Renaissance Philanthropy AI for Math Fund, ...
  • Steadily growing discourse within mathematics community
    • "Some thoughts on automation and mathematical research"
    • Bulletin of the AMS vol 61 no.2 2024
    • "AI Will Become Mathematicians’ ‘Co-Pilot’"
    • Quanta series
    • various conferences and workshops (IAS, UCI, ...)
    • I'm sure much more

2015 Breakthrough Prize panel at Stanford

2015 Breakthrough Prize panel at Stanford

2015 Breakthrough Prize panel at Stanford

2015 Breakthrough Prize panel at Stanford (paraphrased)

  • Yuri Milnor: Could any of you imagine computers surpassing humans in mathematics like they did for chess, in a hundred or maybe a thousand years?
  • Terry Tao: Well computers could act like a really powerful search engine, but it will still be humans driving the show, I'm pretty sure.
  • Maxim Kontsevich: Actually, I don't think creating artificial intelligence is that hard. I think we'll have it pretty soon (nervous laughter from panel).
  • Yuri Milnor: You are sort of a contrarian here saying it will happen so quickly. What makes you so optimistic?
  • Maxim Kontsevich: Actually it's pessimistic (more nervous laughter). I thought a little about it myself and I don't see any fundamental difficulty.
  • Yuri Milnor: Then why don't you just work on that?
  • Maxim Kontsevich: I think it would be immoral (more nervous laughter).
  • Richard Taylor: But isn't the way a computer plays chess just a big combinatorial check? I would be very surprised to see a computer win a Fields medal in our lifetime.
  • Terry Tao: Well I think a computer could discover just by brute force a connection between two fields of mathematics, and then the person using the computer could actually flesh it out and maybe collect the medal.
  • Yuri Milnor: So Jacob, are you terrified at all by what Maxim is saying?
  • Jacob Lurie: I'm not too worried...

Recent (highly abbreviated, possibly inaccurate) timeline

  • 2012: AlexNet wins ImageNet competition
  • 2015(??): recurrent neural network trained on the Stacks project

Recent (highly abbreviated, possibly inaccurate) timeline

  • 2012: AlexNet wins ImageNet competition
  • 2015(??): recurrent neural network trained on the Stacks project
  • June 2016: transformer architecture introduced in by Vaswani et al (Google)
  • November 2022: OpenAI releases ChatGPT (based on GPT-3.5)
  • March 2023: GPT-4 released
  • July 2024: DeepMind AI system solves four out of six IMO problems (silver medal?)
  • September 2024: openAI releases o1 reasoning model
  • November 2024: EpochAI introduces FrontierMath benchmark with contributions from over 60 mathematicians (including Fields medalists). Leading models score $<2\%$
  • December 2024: o3 released, claims $25\%$ accurate on FrontierMath benchmark (some controversy)
  • January 2025: DeepSeek released R1, supposedly competitive with o1 but much cheaper and open source

Example from graduate level complex analysis final exam (May 2025)

Formalization of mathematics?

  • Lean and mathlib is rapidly growing in popularity
  • 2021: Scholze's Liquid Tensor Experiment
  • 2022: sphere eversion
  • supposedly undergraduate mathematics in now entirely formalized

Forecasting?

  • Kevin Roose: one or more AI companies will claim AGI by 2026 or 2027
  • AI2027: world ends by mid 2030

Forecasting?

  • Kevin Roose: one or more AI companies will claim AGI by 2026 or 2027
  • AI2027: world ends by mid 2030
  • Christian Szegedy (xAI, formerly worked on automating mathematics at Google): a computer system will match or exceed the problem-solving ability of the best human mathematicians by 2026
  • Terry Tao (June 2024):
    "I think in three years AI will become useful for mathematicians. It will be a great co-pilot. You’re trying to prove a theorem, and there’s one step that you think is true, but you can’t quite see how it’s true. And you can say, “AI, can you do this stuff for me?” And it may say, “I think I can prove this.” I don’t think mathematics will become solved. If there was another major breakthrough in AI, it’s possible, but I would say that in three years you will see notable progress, and it will become more and more manageable to actually use AI. And even if AI can do the type of mathematics we do now, it means that we will just move to a to a higher type of mathematics. So right now, for example, we prove things one at a time. It’s like individual craftsmen making a wooden doll or something. You take one doll and you very carefully paint everything, and so forth, and then you take another one. The way we do mathematics hasn’t changed that much. But in every other type of discipline, we have mass production. And so with AI, we can start proving hundreds of theorems or thousands of theorems at a time. And human mathematicians will direct the AIs to do various things. So I think the way we do mathematics will change, but [Christian's] time frame is maybe a little bit aggressive."

Reasons NOT to have this discussion

  • Objection #1: This is all just overblown hype.
  • Rebuttal: There's a lot of hype, but see e.g. o3.
  • Objection #2: Current AI systems are lousy at advanced mathematics, so we should be safe for a long time.
  • Rebuttal: Look at progress over the last few years.
  • Objection #3: There are bigger concerns for mathematicians right now (e.g. loss of NSF funding).
  • Rebuttal: The policital pendulum always swings back and forth, but empirically technology moves only in one direction.
  • Objection #4: If the above predictions are remotely true, so many aspects of society will change that it seems silly to worry about academic mathematics of all things.
  • Rebuttal: Who else will be shepherd the fate of mathematics?
  • Objection #5: There are already plenty of articles, conferences, etc about all of this.
  • Rebuttal: Given the existential gravity of the situation, more seems reasonable.
  • Objection #6: At any rate, there's not much we can do.
  • Rebuttal: There are always way to influence the future. However, it is important to first get a better sense of what the future holds, and what are the possible outcomes, and which outcomes we wish to achieve.

Questions for the future

  • Should we start teaching Lean to our students (and ourselves)?
  • Should mathematicians take control of their own AI benchmarks?
  • Should we try to codify the goals and values of mathematical inquiry?
  • Will it be necessary for mathematics to become more outward facing?
  • What would we DO with a superhuman mathematical machine?