There are many conjectures that we know are almost surely true, but we don't know why, and explaining why is the main purpose of the mathematician when publishing a proof.
The fact that we don't know why is a clue pointing at some area of math that we haven't discovered yet. The hope is always that it will uncover some hidden fertile valley that will lead to lots of new discoveries. But the proof of the conjecture itself, without understanding, is really not that valuable.
My point is that even if AI discovers many new truths, there's still plenty to do for the mathematical community, in dissecting it and building useful abstractions to understand it, abstractions that can be leveraged for further exploration and uncovering new questions.
> 1. People will publish so much frontier mathematics, humans won't be able to understand it all
That already happened before AI.
> 2. Frontier mathematics will all be kept secret
Gauss kept lots of frontier mathematics in his drawer. In the 20th centuries government spy agencies developed public key cryptography long before that was known to the public. To give just two examples.
It's not the end of the world.
And what do you care, if someone keeps frontier mathematics a secret, if you can ask DeepSeek version 10 in 2030 to prove the Riemann hypothesis for you?
About 4-6 years earlier, depending what you want to count, although the inventors may also have been less clear on its importance or applications compared to the later public inventors.
It's been in use far longer than LLMs. Thought I'm not exactly sure when it came into common use. I'm pretty sure I've heard the term on Nova decades ago.
1. People will publish AI generated frontier math making it difficult to identify frontier mathematicians.
2. Trained frontier mathematicians will become scarce
Seems perfectly possible to get the worst of both worlds: more mathematics than anyone can read, and less access to the mathematics people actually care about.
That's also what the gerrymandering discourse is like. Gerrymandering is a threat to civilization because it means political parties will minimize their electoral margins, and because it means politicians will maximize their electoral margins.
The whole point of gerrymandering is to split districts so that votes for the other party are diluted. The whole point is to create many districts where your party is majority and then a few districts for the rest of the opposition.
If you are good at it, end result is that minority can keep majority of the seats and power. So, as there are two parties, the groups are "likely to voted republicans" and "likely to vote democrats".
I think "final" here just means the unit that currently gets treated as publishable and credit-worthy: a theorem with a proof, a solved conjecture, a completed paper, etc
If it's not worth publishing, it's not a result at all. That being said, the problem really is that since LLMs excel at sifting through piles or papers, publishing more would aggravate the problem instead of alleviating it.
> Despite the never-ending thirst for knowledge of all branches of the order, most Tech-priests of the Adeptus Mechanicus have lost the ability to innovate or carry out basic scientific research.
> No longer the master of its creations, the Cult Mechanicus is enslaved to the past. It maintains the glories of yesteryear with rite, dogma and edict instead of true discernment and comprehension.
> For instance, even the theoretically simple process of activating a vehicle's engine is preceded by the application of ritual oils, the burning of sacred resins and the chanting of long and complex hymns.
I misinterpreted the article. Its more about sharing ideas within the company, because teams and team members are apt to steal ideas and implement them with AI faster than the originator. I guess this was always theoretically a problem but its especially pronounced now because of the commonality of layoffs, and exacerbated at my company due to the failing stock price.
Talking about frontier mathematics? Even the common for-mere-mortals version of mathematics is totally inaccessible to most people. Even to those who want to really learn it.
Take a good look at the textbooks that teach mathematics and tell me I am wrong.
Prior to LLMs there was some minimal effort required to snipe someone and possibly your reputation was attached otherwise there would have been no point in publishing to begin with.
Now anyone with a few dollars can do it, many who don't have a reputation to worry about.
It's the same problem as YouTube AI slop, AI-generated music, and everything else. Don't you dare tweet or blog about a video idea or hum a few bars from a song - within an hour 27 people will have posted AI slop rip-offs. There's something uniquely depressing about being beaten to the punch by a thief that doesn't deliver the same soul-crushing impact as having someone copy you after the fact.
Definitly seeing this with video games. Especially ones that are easy to make and have generic names. I mean you could already easily clone those before, but now 10 people will easily clone them.
I think making these type of games simply becomes an unviable source of income in the current era.
I'm sure there's a named concept for this: the idea of the game is its selling point, and has to be revealed to consumers to market itself. But the market has low friction (via Steam distribution, and by copies being cheap to make w/ LLMs), so you can't reliably establish your product before masses take over, especially if you don't have reputation built up from previous games.
You have to be bringing in something else for this not to happen, trade secrets not present to the consumer being one such factor.
I wonder how many fields will have to suffer through their collective work being sloppily copied or used as a basis for generative slop before we figure out we aren't getting AGI just Artificial General Slop.
Universities should be providing university hosted llms to their faculty and students. No student or faculty member should be using public llms for their work. They can share info directly with other people or in non-public forums to keep it away from commercial llms. It is going to have to be against the rules to submit anyone else's work to a commercial llm. Businesses are going to have similar policies.
Universities can host the llms just like they hosted any other computer lab or web service on campus. This is what universities are supposed to be doing. Universities should be involved in open model research and should offer models that are not datamined.
Or if you are afraid of scooping, just don't use public LLMs? At universities, apart from admin stuff, nobody will tell you what you must or musn't use.
And I really doubt university ITs would be able to run these, at least in an up to date and stable manner. High performance computing is typically run totally separately from everyday user systems, and tend to be extremely painful to use (hello slurm!).
I welcome the era of secrecy. After living so much in this era of open information where everyone seems to know everything, secrets may be a way to make things more interesting again.
Like we can prevent the rest of our society from devolving into the Medieval Era of secrecy: by treating individuals with respect and dignity and not as the ore from which resources can be profitably extracted.
Those were fought for, with large strikes and sadly bloodshed and conflict.
The pressing issue is that in the industrial revolution, capital needed labour so strikes are ineffective. How do people fight for their rights in a system that sees no need for them? Especially in one where power is concentrating and politics is frequently on sale to the highest bidder.
I know lot of folks on Hacker News hates patents but that was the whole point of them: the patent holder got exclusive rights to the invention for 20 years. But the trade off was that specific details about the invention, how it worked and how it's made were entered openly for anyone to look up. The patent holder would basically benefit it for the majority of his or her life but then after that it was permanently part of the public domain.
AFAIK it was the reason why the patent system was invented, otherwise the best option jealous guarding of trade secrets that ideally (for the secret holder) died with them.
Why is socialism a pipe dream? Unions and co-ops aren't pipe dreams, and if we had nation wide unions or converted most businesses into co-ops, we would arguably be living under a socialist economy. If people had direct voting power over economic and business issues that would be socialism.
Some methods are more realistic than others, but I don't see a requirement for any outlandish ideas.
It’s telling that you think treating people with respect and dignity is “socialist”.
In a sane society it should be easy to argue against that claim, but looking around in the US at least, it turns out you’re correct: that doesn’t seem to be the norm under US-style capitalism.
Luckily we have other countries to provide an example here.
This is a simple matter of intellectual property. A tool or technique for doing a thing can be patented. A legal monopoly is granted to the originator. The public is made aware of the technique but is legally forbidden from using it for a period of years/decades.
There is a financial incentive to selling access to the leading LLMs needed to find whatever secret result there is out there. See the play station hypervisor 0day from the other day. If a LLM can find someone's secret 0day they are flaunting around they can find a math proof someone else says they have.
This is about cooperation before publishing results. And they will keep everything medieval secret, else some big company steals it and claims it their own.
So then the solution is to publish more often (e.g. on a public blog) even if your ideas are not fully developed in order to establish priority and show you are doing something.
Analogously with software development, it's always been good practice to write things down, but since the start of this year it's become dramatically more important for everyday work.
The problem here is that AI isn't just a problem, it also revealed a problem: too much emphasis on publishing papers and churning out new results. The name of the game right now is just that: make something new and significant. AI itelf is a problem around the world right now because we've set up some seriously bad incentives. Same with art - it devolved into content creation for money, so AI snaps that up.
What we need to do is make mathematics about understanding, rather than churning out results. People should be rewarded for reaching an ability to explain mathematics without the aid of computers, to teach people for the sake of their learning.
I do think AI also needs to be eradicated because of its destructive properties, but I think that at the same time it also is a manifestation of the sickness in our society to go after the wrong incentives that are detrimental in the long run.
I wonder how life was while the Butlerian Jihad was raging through the universe.
Jihadis were fighting opponents with thinking machines. Did they win because the machines were not capable enough?
The idea that this genie is going back in the bottle is, imo, very wishful thinking. The thinking machines can rip off your software ideas completely, in days.
This. The value of intelligence and execution is plummeting towards zero for knowledge work. Right now, software and mathematics are the most affected, but the transition to every other industry is going to happen with shocking speed.
I respect your opinion, and I was with you all the way until your last sentence.
"Eradicated" is a pretty strong word. It's also completely unfeasible.
Wrong incentives can maybe be adjusted for. Shutting down the pursuit of one of the most astonishing things we've ever created is simply not going to happen short of a cataclysm, and I'm not a big fan of those.
> The problem here is that AI isn't just a problem, it also revealed a problem: too much emphasis on publishing papers and churning out new results.
I wonder how it revealed it... oh, right, because AI labs rushed to churn out new results for marketing purposes. Mathematicians didn't make them do it. So I'm not sure it's really an indictment of the field.
In fact, how often does mathematics feature in university press releases, how often do mathematicians compete for multi-million grants, how many of them are interviewed on TV?... This discipline is less afflicted by weird incentives than most other fields. It's mostly just a small clique of nerds publishing abstract "open source" work.
> Regardless of the true cost, it seems that professional mathematicians now need to wary about what they put into a LLM and think hard about how to disclose and publish a result.
This is all but guaranteed now.
Mathematicians/Scientists/Researchers need to stop sharing freely with "AI Companies" and have explicit clauses in place in their publications about not using their research without their explicit consent.
There should be a clear legal distinction between using research data for AI model-training vs. another researcher using it.
Come up with a legal framework, establish procedures for sharing and using others work and have a single scientific body in charge of enforcing it.
Just putting a clause in a publication won't prevent it from being used as training data. Information wants to be free.
The frontier LLM vendors do sell enterprise licenses which contractually guarantee that your prompts won't be used for training. (Maybe they'll secretly violate the agreement but in principle it's legally enforceable.) Scholars and universities who care about credit and attribution will either have to purchase those licenses or run their own private open-weight LLM instances.
Even the $20 tier of ChatGPT has privacy settings that forbid using the user's data to be used for training. The question is, whether this setting is respected.
minus two or three things, corporate data is classified as public/goverment data. we just saw something about earmarking domain last month? two being imminent domain. three being natsec.
risk of prescient theory is more important than dismissive ablation.
edit: to wit, facebook google and anything else not e2e.
I don't like this and I wish it weren't true, but I think the period of "information wants to be free" is coming to an end, it was a relic of a bygone era. Increasingly, making your information free means you're the sucker who is doing free labor for AI companies, or worse, you're helping your competitors. Paywalls, login walls, and rate-limits are going up everywhere: there's the GitLab news on the home page right now, and sites like Twitter, Reddit etc. which used to be publicly-readable are now gated (and Xitter is using the legal system to shut down any bypasses).
I hate this but I don't think there's any going back now that LLMs exist.
"Information wants to be free" never meant that people want to release their information; it meant that information is very hard to keep secret, and that everything leaks like a sieve, and especailly that once it's out, it's out forever.
Exactly. While there are a few academics who work in private for years and then surprise the world with an amazing breakthrough, most of modern science and mathematics is a collaborate process. Researchers make gradual progress on hard problems, and discuss issues with colleagues and students along the way. Some of those collaborators will then pass on the information to social media or public discussion forums or free-tier LLM prompts or whatever and it gets incorporated into the next round of training runs.
Two people can keep a secret if one of them is dead.
Would this legal framework cut both ways? When AI companies use AI to make and publish mathematical discoveries, would they be able to legally prevent professional mathematicians from using them?
Stop treating AI companies and their software as somehow unconstrained, above-the-law actors. It's delusional that anyone buys that. Regulate them appropriately.
At the same time, mathematicians should be using sophisticated, specialized LLM tools in much more sophisticated ways than lay people. There should be no way lay people can compete. There are new tools to master and if you use your slide rule, you won't keep up. It's a chance for mathematics productivity to boom.
With apologies to Baudelaire: The greatest trick exploitative powers ever played was convincing the people that it was impossible to imagine anything else.
Sometimes blinkering people so that they never ask the question "why is this being done to me" or "why is justice not possible" is much easier than finding an answer that will get them to go away.
If in 2019 someone wrote a program that generated photorealistic child porn on demand, the FBI would break down their door the day after hearing about it.
All of life proceeds effectively with a right brain and a left brain. A fast loop and a slow loop. A general and a scout. A melody and a base. The evolutionary moments involve moving the melody into the base and making way for a new melody. But there is always an overseeing element and a work element. A manager or coach and an employee or athlete. It's just an effective pattern of growth. You see it everywhere in nature. In generations of animals and plant life (the more experienced parent, the growing child; the central 'brain' and the rest of the body; the queen ant and the ant colony; the trunk and the leaves).
The friction in math right now is you have the fast moving melodic bits racing faster than the base can understand always or keep up with. So you start to need AI for both the left brain part that is executing and the right brain part that is synthesizing. That's just the friction right now. You reduce the friction by using AI to help with understanding AI and getting back to a more normal rhythm of scouting out (with AI) and (what is still developing more and more) synthesizing with AI.
I have seen two worries recently:
1. People will publish so much frontier mathematics, humans won't be able to understand it all
2. Frontier mathematics will all be kept secret
Fortunately, these seem like they can't both happen at once.
There are many conjectures that we know are almost surely true, but we don't know why, and explaining why is the main purpose of the mathematician when publishing a proof.
The fact that we don't know why is a clue pointing at some area of math that we haven't discovered yet. The hope is always that it will uncover some hidden fertile valley that will lead to lots of new discoveries. But the proof of the conjecture itself, without understanding, is really not that valuable.
My point is that even if AI discovers many new truths, there's still plenty to do for the mathematical community, in dissecting it and building useful abstractions to understand it, abstractions that can be leveraged for further exploration and uncovering new questions.
This seems right as long as the AI output is legible enough to reverse-engineer
> 1. People will publish so much frontier mathematics, humans won't be able to understand it all
That already happened before AI.
> 2. Frontier mathematics will all be kept secret
Gauss kept lots of frontier mathematics in his drawer. In the 20th centuries government spy agencies developed public key cryptography long before that was known to the public. To give just two examples.
It's not the end of the world.
And what do you care, if someone keeps frontier mathematics a secret, if you can ask DeepSeek version 10 in 2030 to prove the Riemann hypothesis for you?
> long before that was known to the public
About 4-6 years earlier, depending what you want to count, although the inventors may also have been less clear on its importance or applications compared to the later public inventors.
https://en.wikipedia.org/wiki/Public-key_cryptography#Classi...
I guess that's kind of a long time in computer technology terms.
Was the term "frontier mathematics" always in use, or did it just become a thing after the advent of contemporary AI?
It's been in use far longer than LLMs. Thought I'm not exactly sure when it came into common use. I'm pretty sure I've heard the term on Nova decades ago.
Google trends shows it spiking last year: https://trends.google.com/explore?q=frontier%20mathematics&d...
I think the worry is:
1. People will publish AI generated frontier math making it difficult to identify frontier mathematicians. 2. Trained frontier mathematicians will become scarce
Seems perfectly possible to get the worst of both worlds: more mathematics than anyone can read, and less access to the mathematics people actually care about.
You don't think frontier mathematics is already secret?
How to admit you're in finance without admitting you're in finance?
Or Cryptanalysis.
" The National Security Agency is the largest employer of mathematicians in the U.S. "
https://www.scientificamerican.com/article/mathematicians-an...
Or signals intelligence.
unfortunately, the worse of both can be true at once.
That's also what the gerrymandering discourse is like. Gerrymandering is a threat to civilization because it means political parties will minimize their electoral margins, and because it means politicians will maximize their electoral margins.
Like yeah, result is that one group politicians are minimized and others maximized.
Which makes total sense unlike the "too much frontier math" nonsense.
> Like yeah, result is that one group politicians are minimized and others maximized.
Which groups are those?
The whole point of gerrymandering is to split districts so that votes for the other party are diluted. The whole point is to create many districts where your party is majority and then a few districts for the rest of the opposition.
If you are good at it, end result is that minority can keep majority of the seats and power. So, as there are two parties, the groups are "likely to voted republicans" and "likely to vote democrats".
In science and math, too often only the final result gets credited or published.
Maybe we should start crediting and publishing our intermediate results and attempts.
Humans would gain credit for providing a part of the solution to tough problems, LLMs would profit from the resulting data flywheel.
> In science and math, too often only the final result gets credited or published.
> Maybe we should start crediting and publishing our intermediate results and attempts.
All mathematical results are "intermediate results". What would it mean for something to be a final result?
I think "final" here just means the unit that currently gets treated as publishable and credit-worthy: a theorem with a proof, a solved conjecture, a completed paper, etc
Then it's circular: "only the units that are publishable are published"
If it's not worth publishing, it's not a result at all. That being said, the problem really is that since LLMs excel at sifting through piles or papers, publishing more would aggravate the problem instead of alleviating it.
anyone else already experiencing this in a corporate environment? I know I am. Its not just between teams either, its within them.
The age of a Warhammer 40k Tech Priest is upon us! Praise to the Machine God.
Just, without the aliens, space travel, or mech-suits.
Got half the other downsides though!
More on the upcoming priesthood and monasteries: https://news.ycombinator.com/item?id=49743400
What a fantastic article! Thank you for sharing it!
A Tech Priest is way smarter than any software developer you could ever think of.
Your joke overestimates the intelligence of real life humans.
> A Tech Priest is way smarter than any software developer you could ever think of.
https://warhammer40k.fandom.com/wiki/Tech-Priest:
> Despite the never-ending thirst for knowledge of all branches of the order, most Tech-priests of the Adeptus Mechanicus have lost the ability to innovate or carry out basic scientific research.
> No longer the master of its creations, the Cult Mechanicus is enslaved to the past. It maintains the glories of yesteryear with rite, dogma and edict instead of true discernment and comprehension.
> For instance, even the theoretically simple process of activating a vehicle's engine is preceded by the application of ritual oils, the burning of sacred resins and the chanting of long and complex hymns.
this isn't a new thing in corporate
I would see stuff like this even before AI.
Would you expand on your experiences? Can you not share ideas now? What if you enhance them with an LLM before sharing them?
I misinterpreted the article. Its more about sharing ideas within the company, because teams and team members are apt to steal ideas and implement them with AI faster than the originator. I guess this was always theoretically a problem but its especially pronounced now because of the commonality of layoffs, and exacerbated at my company due to the failing stock price.
Huh, and I thought "stealth-mode" for startup founders was a meme. Turns out a form of it started percolating around.
Talking about frontier mathematics? Even the common for-mere-mortals version of mathematics is totally inaccessible to most people. Even to those who want to really learn it.
Take a good look at the textbooks that teach mathematics and tell me I am wrong.
If there is any barrier, it is not intentional, I assure you. Math is just hard.
There are many amazing Youtube channels that explain complex math well, for example https://www.youtube.com/c/3blue1brown
... and quaternions?
The books exist and are possible to understand. They teach maths at most universities. You should try it, learning is fun.
you don't worry about it for a couple years and check back to see if its a real problem or a panic induced engagement generator
We don't.
Prior to LLMs there was some minimal effort required to snipe someone and possibly your reputation was attached otherwise there would have been no point in publishing to begin with.
Now anyone with a few dollars can do it, many who don't have a reputation to worry about.
It's the same problem as YouTube AI slop, AI-generated music, and everything else. Don't you dare tweet or blog about a video idea or hum a few bars from a song - within an hour 27 people will have posted AI slop rip-offs. There's something uniquely depressing about being beaten to the punch by a thief that doesn't deliver the same soul-crushing impact as having someone copy you after the fact.
Definitly seeing this with video games. Especially ones that are easy to make and have generic names. I mean you could already easily clone those before, but now 10 people will easily clone them.
See this amusing reddit post. https://www.reddit.com/r/IndieDev/comments/1wfl64l/game_went...
I think making these type of games simply becomes an unviable source of income in the current era.
I'm sure there's a named concept for this: the idea of the game is its selling point, and has to be revealed to consumers to market itself. But the market has low friction (via Steam distribution, and by copies being cheap to make w/ LLMs), so you can't reliably establish your product before masses take over, especially if you don't have reputation built up from previous games.
You have to be bringing in something else for this not to happen, trade secrets not present to the consumer being one such factor.
Copycats and rip-offs are really fast. Recently outbid.lol was the target.
I wonder how many fields will have to suffer through their collective work being sloppily copied or used as a basis for generative slop before we figure out we aren't getting AGI just Artificial General Slop.
Universities should be providing university hosted llms to their faculty and students. No student or faculty member should be using public llms for their work. They can share info directly with other people or in non-public forums to keep it away from commercial llms. It is going to have to be against the rules to submit anyone else's work to a commercial llm. Businesses are going to have similar policies.
Universities can host the llms just like they hosted any other computer lab or web service on campus. This is what universities are supposed to be doing. Universities should be involved in open model research and should offer models that are not datamined.
Or if you are afraid of scooping, just don't use public LLMs? At universities, apart from admin stuff, nobody will tell you what you must or musn't use.
And I really doubt university ITs would be able to run these, at least in an up to date and stable manner. High performance computing is typically run totally separately from everyday user systems, and tend to be extremely painful to use (hello slurm!).
Their main cash cow is disappearing(students) Also from what I've seen of University IT it's still very very siloed by department.
Similar principle applies to open source or any other creative endeavor put in the public domain.
I welcome the era of secrecy. After living so much in this era of open information where everyone seems to know everything, secrets may be a way to make things more interesting again.
that would be the pendulum swinging too far back
Maybe wise ones will return to paper.
Like we can prevent the rest of our society from devolving into the Medieval Era of secrecy: by treating individuals with respect and dignity and not as the ore from which resources can be profitably extracted.
Might as well go tell depressed people to just stop being depressed
Or homeless people to just buy a house.
Here's a thought for sweat shop owners: Air Conditioning. Problem solved.
— Mitch Hedberg
Any solutions to this issue that aren't unrealistic socialist pipe dreams?
Yep.
Muddling through with markets, democracy and welfare.
Global life expectancy was 34 in 1913. Today it's 70.
https://ourworldindata.org/life-expectancy
Global GDP was less than 1Trn until about 1800. In 1950 it was 11.7Trn . Today it's 157 Trn. In a life time it's gone up by more than 10 fold.
https://ourworldindata.org/grapher/global-gdp-over-the-long-...
People are living longer, richer lives all over the world.
The world's largest country grew at 7.8% last year.
https://www.reuters.com/world/india/why-indias-strong-gdp-gr...
Things are getting better. Look at the data, work at what you do and ignore the doom sayers.
In 2060 it will be 34
I don't doubt that AI might increase the GDP but it will happen at the expense of the many.
You could've asked that about people saying we want 8 hour work days, sick leave, and safer work 100 years ago, and yet we got them.
Those were fought for, with large strikes and sadly bloodshed and conflict. The pressing issue is that in the industrial revolution, capital needed labour so strikes are ineffective. How do people fight for their rights in a system that sees no need for them? Especially in one where power is concentrating and politics is frequently on sale to the highest bidder.
In the margins, guerilla-style. A system having no need for me means it doesn't need to know I exist, or can be placated with fake data about me.
Patent system?
I know lot of folks on Hacker News hates patents but that was the whole point of them: the patent holder got exclusive rights to the invention for 20 years. But the trade off was that specific details about the invention, how it worked and how it's made were entered openly for anyone to look up. The patent holder would basically benefit it for the majority of his or her life but then after that it was permanently part of the public domain.
AFAIK it was the reason why the patent system was invented, otherwise the best option jealous guarding of trade secrets that ideally (for the secret holder) died with them.
Why is socialism a pipe dream? Unions and co-ops aren't pipe dreams, and if we had nation wide unions or converted most businesses into co-ops, we would arguably be living under a socialist economy. If people had direct voting power over economic and business issues that would be socialism.
Some methods are more realistic than others, but I don't see a requirement for any outlandish ideas.
Only realistic capitalist nightmares.
It’s telling that you think treating people with respect and dignity is “socialist”.
In a sane society it should be easy to argue against that claim, but looking around in the US at least, it turns out you’re correct: that doesn’t seem to be the norm under US-style capitalism.
Luckily we have other countries to provide an example here.
I'd guess universities might starting hosting open source models. They can probably actually afford to, unlike individual mathematicians.
Though maybe if there's a flurry of math-optimized agents coming up, like there are small coding agents, those might be feasible to host personally.
This is a simple matter of intellectual property. A tool or technique for doing a thing can be patented. A legal monopoly is granted to the originator. The public is made aware of the technique but is legally forbidden from using it for a period of years/decades.
The algorithm cannot be patented, a specific use for it can. Which on cutting edge math may not exist for decades or even hundreds of years.
There is a financial incentive to selling access to the leading LLMs needed to find whatever secret result there is out there. See the play station hypervisor 0day from the other day. If a LLM can find someone's secret 0day they are flaunting around they can find a math proof someone else says they have.
Simple: if you don't publish your work, we don't fund you. Why is this even a question?
This is about cooperation before publishing results. And they will keep everything medieval secret, else some big company steals it and claims it their own.
So then the solution is to publish more often (e.g. on a public blog) even if your ideas are not fully developed in order to establish priority and show you are doing something.
Analogously with software development, it's always been good practice to write things down, but since the start of this year it's become dramatically more important for everyday work.
I don’t follow… if you publish your underdeveloped ideas, these companies will develop them for you, which is the problem
The problem here is that AI isn't just a problem, it also revealed a problem: too much emphasis on publishing papers and churning out new results. The name of the game right now is just that: make something new and significant. AI itelf is a problem around the world right now because we've set up some seriously bad incentives. Same with art - it devolved into content creation for money, so AI snaps that up.
What we need to do is make mathematics about understanding, rather than churning out results. People should be rewarded for reaching an ability to explain mathematics without the aid of computers, to teach people for the sake of their learning.
I do think AI also needs to be eradicated because of its destructive properties, but I think that at the same time it also is a manifestation of the sickness in our society to go after the wrong incentives that are detrimental in the long run.
> I do think AI also needs to be eradicated
I wonder how life was while the Butlerian Jihad was raging through the universe.
Jihadis were fighting opponents with thinking machines. Did they win because the machines were not capable enough?
The idea that this genie is going back in the bottle is, imo, very wishful thinking. The thinking machines can rip off your software ideas completely, in days.
This. The value of intelligence and execution is plummeting towards zero for knowledge work. Right now, software and mathematics are the most affected, but the transition to every other industry is going to happen with shocking speed.
It actually won't. Math/Coding feedback loop is closed and can be automated, most other knowledge work can't.
Automated spreadsheets and PPTs won't get people fired.
I respect your opinion, and I was with you all the way until your last sentence.
"Eradicated" is a pretty strong word. It's also completely unfeasible.
Wrong incentives can maybe be adjusted for. Shutting down the pursuit of one of the most astonishing things we've ever created is simply not going to happen short of a cataclysm, and I'm not a big fan of those.
> The problem here is that AI isn't just a problem, it also revealed a problem: too much emphasis on publishing papers and churning out new results.
I wonder how it revealed it... oh, right, because AI labs rushed to churn out new results for marketing purposes. Mathematicians didn't make them do it. So I'm not sure it's really an indictment of the field.
In fact, how often does mathematics feature in university press releases, how often do mathematicians compete for multi-million grants, how many of them are interviewed on TV?... This discipline is less afflicted by weird incentives than most other fields. It's mostly just a small clique of nerds publishing abstract "open source" work.
wouldn't it be dystopian secrecy cause a flock camera is equally capable of watching the mathematicians as it is the public citizens.
Maybe mathematicians are smart enough to never ever buy such piece of shit on higher principle, regardless of their actual fiasco?
I am a (former) mathematician and know many more.
> Maybe mathematicians are smart enough to never ever buy such piece of shit on higher principle
They are not. Mathematicians are as human as most of the rest of us here.
Individuals aren’t the ones buying Flock cameras.
> Regardless of the true cost, it seems that professional mathematicians now need to wary about what they put into a LLM and think hard about how to disclose and publish a result.
This is all but guaranteed now.
Mathematicians/Scientists/Researchers need to stop sharing freely with "AI Companies" and have explicit clauses in place in their publications about not using their research without their explicit consent.
There should be a clear legal distinction between using research data for AI model-training vs. another researcher using it.
Come up with a legal framework, establish procedures for sharing and using others work and have a single scientific body in charge of enforcing it.
Just putting a clause in a publication won't prevent it from being used as training data. Information wants to be free.
The frontier LLM vendors do sell enterprise licenses which contractually guarantee that your prompts won't be used for training. (Maybe they'll secretly violate the agreement but in principle it's legally enforceable.) Scholars and universities who care about credit and attribution will either have to purchase those licenses or run their own private open-weight LLM instances.
Even the $20 tier of ChatGPT has privacy settings that forbid using the user's data to be used for training. The question is, whether this setting is respected.
I assume the data is laundered into a format that qualifies as no longer being the "user's" data, then trained on.
the existence of the triplets NSA/CIA/GRU implies an imperitive no.
that is a whole different thing though... AI labs are not government intelligence agencies
minus two or three things, corporate data is classified as public/goverment data. we just saw something about earmarking domain last month? two being imminent domain. three being natsec.
risk of prescient theory is more important than dismissive ablation.
edit: to wit, facebook google and anything else not e2e.
it's not like the ai is homomorphic.
oh, haha. another point to make: what do you think they've been building out for 25 years with fusion centers and maryland/utah?
"government intelligence agencies" ARE 'AI'.
I don't like this and I wish it weren't true, but I think the period of "information wants to be free" is coming to an end, it was a relic of a bygone era. Increasingly, making your information free means you're the sucker who is doing free labor for AI companies, or worse, you're helping your competitors. Paywalls, login walls, and rate-limits are going up everywhere: there's the GitLab news on the home page right now, and sites like Twitter, Reddit etc. which used to be publicly-readable are now gated (and Xitter is using the legal system to shut down any bypasses).
I hate this but I don't think there's any going back now that LLMs exist.
"Information wants to be free" never meant that people want to release their information; it meant that information is very hard to keep secret, and that everything leaks like a sieve, and especailly that once it's out, it's out forever.
Exactly. While there are a few academics who work in private for years and then surprise the world with an amazing breakthrough, most of modern science and mathematics is a collaborate process. Researchers make gradual progress on hard problems, and discuss issues with colleagues and students along the way. Some of those collaborators will then pass on the information to social media or public discussion forums or free-tier LLM prompts or whatever and it gets incorporated into the next round of training runs.
Two people can keep a secret if one of them is dead.
Would this legal framework cut both ways? When AI companies use AI to make and publish mathematical discoveries, would they be able to legally prevent professional mathematicians from using them?
The USA doesn't have legal frameworks any more, you just buy and sell the right to do what you want. Even our supreme court is disingenuous now.
Stop treating AI companies and their software as somehow unconstrained, above-the-law actors. It's delusional that anyone buys that. Regulate them appropriately.
At the same time, mathematicians should be using sophisticated, specialized LLM tools in much more sophisticated ways than lay people. There should be no way lay people can compete. There are new tools to master and if you use your slide rule, you won't keep up. It's a chance for mathematics productivity to boom.
> Stop treating [them as] unconstrained, above-the-law actors.
With apologies to Baudelaire: The greatest trick exploitative powers ever played was convincing the people that it was impossible to imagine anything else.
Sometimes blinkering people so that they never ask the question "why is this being done to me" or "why is justice not possible" is much easier than finding an answer that will get them to go away.
If in 2019 someone wrote a program that generated photorealistic child porn on demand, the FBI would break down their door the day after hearing about it.
Seems rather a dramatic notion for a single incident...
The simple short-term solution is just don't feed all your work into the IP theft machine?
Mathematical ideas and proofs are not IP.
All of life proceeds effectively with a right brain and a left brain. A fast loop and a slow loop. A general and a scout. A melody and a base. The evolutionary moments involve moving the melody into the base and making way for a new melody. But there is always an overseeing element and a work element. A manager or coach and an employee or athlete. It's just an effective pattern of growth. You see it everywhere in nature. In generations of animals and plant life (the more experienced parent, the growing child; the central 'brain' and the rest of the body; the queen ant and the ant colony; the trunk and the leaves).
The friction in math right now is you have the fast moving melodic bits racing faster than the base can understand always or keep up with. So you start to need AI for both the left brain part that is executing and the right brain part that is synthesizing. That's just the friction right now. You reduce the friction by using AI to help with understanding AI and getting back to a more normal rhythm of scouting out (with AI) and (what is still developing more and more) synthesizing with AI.