Yeah. My other favorite example are books. Why do nonfiction books exist? There are some pathologies and corner cases, but fundamentally: to develop and share new ideas. Downstream from that, if it reads well and if you're lucky, you make some money.
But now, LLMs can generate hundreds of books per hour. They make up 80-90% of new arrivals in many nonfiction categories on Amazon. They short-circuit the system, allowing their "authors" to extract money from the system with zero effort by crowding out human work. And it's not even the question of whether these books are good or bad (although overwhelmingly, they're terrible). It's whether it's actually accomplishing anything worthwhile, or just destroying incentives for humans to write or go into any other sort of intellectual work.
In fact, I see many professions push back. Artists, writers, now mathematicians. And I'm amazed that our profession doesn't and that we have so many people who are hooked on vibecoding. I'm still waiting for that 10x payoff. All this velocity and somehow, the landscape of the software I want to use still looks the same as it did in 2021.
It doesn't, that's the whole point. You can produce output that passes the smell test with naive buyers in a matter of hours. If you want to write good LLM books, then you gotta work closer to human speed - weeks, months - which allows others to produce 100 slop-books in the same timeframe. You still lose.
If an author let AI do 90 to 95% of the work, cleaned up the remaining 5 to 10% themselves, would you say AI or the author wrote it? Because this is roughly the split between an author and an editor, with human writing, now.
The work is either: (1) the act of concealment: generating an AI book that cannot be detected as AI generated; or (2) actually just writing the book yourself.
Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube, even if there was no doubt that humanity would in this way survive until the sun expands and makes the earth uninhabitable.
> There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube
Even in a hypothetical world where AGI can do everything that humans do better, this would still be a choice to make, not an inevitable consequence for everyone.
And life doesn't require suffering to have meaning.
> Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
I'd argue that this extremely extreme scenario is the only one in which it kind of makes sense to not have understanding. But let's be honest: no one knows if we'll be there (and it seems unlikely since everything reaches a plateau eventually). So, what happens if we allow ourselves to forget everything and then we don't reach the ideal scenario?
I grew up in a cult. Based on my experience, I believe that the most dangerous thing a human can do is to allow someone else to do their thinking for them.
It’s not going to be ubiquitous? There hasn’t been a single frontier model where generation n costs less than generation n-1 to run. So the reasonable thing is to assume that GPT-7 will cost even more than GPT-6, and more and more of the frontier of knowledge will be locked behind a giant paywall. Participating in any field will mean ponying up to the oligarchs that own the infrastructure that runs the model.
At the individual level any mathematicians can, but the social group of “mathematicians” will not unless drastic social changes are made, because that's not how the system works today. Drastic social changes usually take decades, and funerals, to occur, so the reasonable hypothesis is that it's not going to change fast enough.
Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot. Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.
This is a litmus test for the effective altruists in AI labs: fashion or conviction? The social pressure is to slow down and let the guild keep its norms a few more years. But what's clearly best for humanity is to push forward violently and leave them in the dust. A Fields medalist's feelings are not more important than the wellbeing of the planet.
> Mathematics belongs to all of us. If mathematicians get depressed about it, I don't give a hoot.
They are depressed because what's being destroyed is mathematics, actually.
> Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.
This is complete BS, math doesn't solve climate change or save life, that's a category error. The purpose of math is to help us build understanding of the mathematical objects and that's it.
Sometimes, after much distillation to the academic world, this understanding is later being leverage by other scientific field, but that's a very rare event when it happens. And it's not how other science progress.
> This is a litmus test for the effective altruists
EA has always been a lie to justify short term individual greed by hypothetical shared future benefits.
youre not wrong, but i think you are missing his base point. tao is saying that mathematics is a practice that the group collectively builds. students and new methods are built on top of what has previously been done, so its a community effort of research, experimentation, and knowledge sharing.
his core argument that is LLMs (and specifically LLMs owned and gated by corporations was my read), while able to solve problems, do not contribute to this practice of knowledge building. solving problems is just one piece, and the mathematics community ingests problems and new methods, iterates and thinks on them, and then produces new ideas, methods, etc. this is what he is defining as progress, and solving things like millenium problems are markers of this progress.
I published a substack about this just a few days ago [1], my core theory here is that we will absolutely have what I call a "highly productive dark age" in mathematics where knowledge vastly outpaces understanding driven by publish-or-perish incentives, but additionally this will lead to the loss of the skills necessary to understand.
The hopeful note is that I do think we are entering a golden age for the curious casual/semi-pro mathematician and for niche mathematics areas that won't get the attention of the top labs. Everyone is sprinting to solve the millennium problems, but this is a very exciting time to be in a sub-sub-field where you and 4 others are keeping things alive.
The existence of solutions to problems does not prevent you from solving the problems yourself anyway, if your goal truly is personal development. You're free to go solve Navier-Stokes yourself right now. You're free to manually do all of the AI work in any field, actually. You won't get grant money or prestige (which are not a part of conceptual understanding and insight) but you will get all of the conceptual understanding and insight you're after. You have not been deprived of it.
Except most people that get actually good at a field do so through their day job. By removing mathematician as a viable job, most can't afford to spend the necessary hours to become a good mathematician at all. We're going back to only rich people or those with a rich patron having a chance of developing themselves.
I experienced mild brain freeze as a child but it went away in adolescence. It doesn't seem pleasant so I don't envy those of you who get intense versions of it.
I'm surprised it's so few! I'm in the 60-70% here -- I've never gotten it (despite many deliberate efforts), but I've never known anyone else who seems to be immune.
In case it wasn't obvious - it wasn't obvious to me from the 43 mentions of it in the article - beads is a product by the same author. There was an HN post on it 8 months ago https://news.ycombinator.com/item?id=46075616 (68 comments). I've not tried it.
Yup. I followed his terrible behavior since his 2021 crashout, when he caused a third of his employees to leave his company, and he's only gotten louder and worse since. Looking back further, I can't see him actually supporting anything pro-social.
This is the effect of AI on most intellectual disciplines, and it’s a real worry.
reply