I think at this point it is safe to say that the whole bubble is having deep mental health problems - and I'm not saying that as some sort of covert insult trying to skirt around HN rules.
Which isn't to say that there is no substance worth exploring in George's post there, but it should be treated as a cry for help and not an assessment of the situation.
(Though the cry for help - in classic intellectualizing fashion - is wrapped in a lot of real genuine thoughts)
I’d expect George to weather this future better than most of us. I don’t see evidence of mental health issues in his post? Sounds pretty spot on to me. But then maybe I have issues..
He definitely has the smarts to figure stuff out, yes.
It will just be necessary to separate the emotions from the reality to enable one to react properly and decisively against a threat that primarily attacks emotions, stories, etc.
LLMs are story engines. Stories are the exploit vector. Language is stories.
___
There are very real things wrapped in stories, but one can only truly tackle them by unwrapping first.
Not sure if that helps you or makes it worse, but you can probably relatively safely assume that you personally have little (if at all) control over the future at large.
You do however have control over your personal future, your friend- and relationships, your neighbors and your local social circles.
Unless you're in the bay area I guess, in which case I am part sorry part full of schadenfreude.
I am not confident that healthy local social relationships are possible there.
I would say that these quotes hold a lot of stuff one could explore after resolving the emotional undercurrents.
> AI that can play a whole society in hours can manufacture entire histories, years of archives of daily newspapers, and an Internet that never happened. The dead Internet theory is coming true before our eyes. Now I can still see the bots, soon I won’t be able to.
> A media landscape that is botted and microtargeted to all hell, nonsensical elections where the only thing people agree on is the names of candidates. Massive dumps of corporate slacks and intranets. (come on Russian kids, grab an abliterated AI and do a hacking!)
> The Internet era is over. Your life will never be the same. The chaos will come for the resistance as well as the fascists; it’s unclear who it will hurt more.
> I think at this point it is safe to say that the whole bubble is having deep mental health problems
I'm starting to arrive at the same conclusion. It's very, very difficult to see the transition from intelligence to madness. You really do need to be connected (and close!) to people outside of tech, especially in the humanities, to fully grasp the insanity of this industry's development and evolution. The tech industry is, broadly, starting to be reviled in developed countries. I'm regularly having to qualify that I don't work in big tech when I share what I do.
When I connect with people who largely only have connections in tech, I'm finding I relate less and less with their thinking. I think "AI psychosis" is a very apt term.
> And remember, chaos is the good outcome. The alternative is that you are a dancing monkey in a cage you won’t be smart enough to see.
I'd think we're actually going to see a split. Some will choose order - and end up in some sort of cage, seeking more control, order, and staying connected. Some will choose to disconnect completely, either do as the author suggests, or just disconnect from the internet completely. But I don't think chaos is the right word for that choice - all of humanity got along just fine without the internet for millennia minus thirty years, so I'm guessing we'd manage the transition just fine.
The internet will be around, it just won't be open anymore.
Already the trolls,etc were a nuicance enough that anonymous social-media,etc is starting to become outlawed. Sure people will hack,etc but the easy exploitation of today will be a fleeting thing once we patch the fundamentally broken stuff.
Remember how Windows machines were wormed in the early 00s? MS got their act together and today the primary target is via browsers or Excel-files,etc. The recent Linux RCE while bad and probably not alone, is still not the common case.
The history of human scientific discovery and innovation has been a repeated practice in humility, reinforcing over and over that we are not the center of the universe, and everything we hold of value is a transient condition of the times we exist within.
Put your hands up. The chain-lift is nearing its big peak.
I did say that, yes, but I also think that this should be flagged, because HN will otherwise run with the panic, 5 "rationalists" will show up and argue for 50 comments about how their LARP is certainly real, and, generally, the world will be worse off.
Which is not George's fault, but it is the fault of the uncaring machine that is people on the internet.
George probably needs a hug and a friend that is grounded and sane. He definitely does not need more people fueling this fire.
(I do not know the guy. I'm just claiming this with fake authority)
I think it degrades the level of discourse on HN when people insist on treating eschatonic navel-gazing as a significant contribution to a separate and ongoing debate.
Just because an insular group of people who share the same unsupported apocalyptic beliefs have labelled themselves as rationalists doesn't mean they are actually being rational; the actual debates around AI and its uses are very far away from the cult-like beliefs espoused here, but the focus turns to them again and again.
It's as if a board for discussing NASA's advances kept getting sidetracked by Raëlists. Discussing these beliefs as though they were serious positions is bad for the field, bad for the discourse, and bad for the writers themselves.
> Anthropic cannot wait until they have the leverage to ban all their users.
Huh? Their power is precisely in the users they have.
Let’s say they build a godlike superintelligence. Now they ban all their users. Let’s further assume that they are fine with all their users rotting and dying (easy to believe). Now what exactly is Anthropic going to do in that world? Invent Millennium Problems and solve them?
This is exactly what happened when manufactures could offshore production. They didn't care (sure there was PR "caring" but they still outsourced) how it would impact their former workers or company towns.
https://www.sfgate.com/business/article/levi-strauss-aims-to...
You've made the mistake of thinking Anthropic just wants to sell everybody a subscription to their models, so everyone can make their little spreadsheets or whatever.
That's not what Anthropic wants. They plan to be The Company. The only one. The one that does everything and makes every decision.
I'm pretty confident all hyperscalers only need their users because they are RL environments for them, this likely will mutate as they reach a ceiling.
My prediction is once they reach this ceiling (or find a better RL environment), they will cancel free plans and shift towards the next scaling laws challenge (robotics, etc) by buying out whoever started early and repeating the playbook.
Canceling the paid plans would only make sense if they solve for control, which right now isn't yet clear who has it and in what capacity.
> Their power is precisely in the users they have.
False. There is a brilliant quote from The Blacklist series: "every information is worth something to someone". Their strength comes from the fact that they convinced everyone to willingly let user-facing computers collect data, write and execute arbitrary code on people's computers with a simple "trust me bro". In other words they have near complete monopoly over the world's infrastructure.
> Now what exactly is Anthropic going to do in that world
Dario already said it according to multiple sources: he believes Anthropic will be the last surviving company in the world, where all the value will accumulate
Elon Musk has another vision: his AI is going to build the Dyson swarm around the Sun. Money doesn't matter in that world, only how many robots you have, and how much mass you can put into orbit.
> Dario already said it according to multiple sources: he believes Anthropic will be the last surviving company in the world, where all the value will accumulate
Given that LLMs are practically commodities and open models are a dime a dozen, this is pure insanity. There's no way there from here.
It's pretty easy to imagine them creating a new AI that is an order of magnitude better than the current models with no one else having access to that technology.
And it's pretty easy to imagine me creating a castle made of cake and ice cream.
That doesn't mean there's any actual mechanism for me to do so.
What you're talking about requires a genuine breakthrough in AI research, not just iterative improvements on what we've already got. By their very nature, those are not something that can be predicted or relied upon; you cannot even be guaranteed that a given type of breakthrough will ever happen.
Without order nothing can exist, without chaos nothing can evolve (Oscar Wilde apparently)
I think at this point it is safe to say that the whole bubble is having deep mental health problems - and I'm not saying that as some sort of covert insult trying to skirt around HN rules.
Which isn't to say that there is no substance worth exploring in George's post there, but it should be treated as a cry for help and not an assessment of the situation. (Though the cry for help - in classic intellectualizing fashion - is wrapped in a lot of real genuine thoughts)
I’d expect George to weather this future better than most of us. I don’t see evidence of mental health issues in his post? Sounds pretty spot on to me. But then maybe I have issues..
He definitely has the smarts to figure stuff out, yes.
It will just be necessary to separate the emotions from the reality to enable one to react properly and decisively against a threat that primarily attacks emotions, stories, etc.
LLMs are story engines. Stories are the exploit vector. Language is stories.
___
There are very real things wrapped in stories, but one can only truly tackle them by unwrapping first.
Truth. And I am very nervous about the future, so of course it resonated with me for that reason alone
Not sure if that helps you or makes it worse, but you can probably relatively safely assume that you personally have little (if at all) control over the future at large.
You do however have control over your personal future, your friend- and relationships, your neighbors and your local social circles.
Unless you're in the bay area I guess, in which case I am part sorry part full of schadenfreude. I am not confident that healthy local social relationships are possible there.
What´s the substance? Seems like we see these minor differences in the enthusiasms of comfortable people pop up here every week
I would say that these quotes hold a lot of stuff one could explore after resolving the emotional undercurrents.
> AI that can play a whole society in hours can manufacture entire histories, years of archives of daily newspapers, and an Internet that never happened. The dead Internet theory is coming true before our eyes. Now I can still see the bots, soon I won’t be able to.
> A media landscape that is botted and microtargeted to all hell, nonsensical elections where the only thing people agree on is the names of candidates. Massive dumps of corporate slacks and intranets. (come on Russian kids, grab an abliterated AI and do a hacking!)
> The Internet era is over. Your life will never be the same. The chaos will come for the resistance as well as the fascists; it’s unclear who it will hurt more.
> I think at this point it is safe to say that the whole bubble is having deep mental health problems
I'm starting to arrive at the same conclusion. It's very, very difficult to see the transition from intelligence to madness. You really do need to be connected (and close!) to people outside of tech, especially in the humanities, to fully grasp the insanity of this industry's development and evolution. The tech industry is, broadly, starting to be reviled in developed countries. I'm regularly having to qualify that I don't work in big tech when I share what I do.
When I connect with people who largely only have connections in tech, I'm finding I relate less and less with their thinking. I think "AI psychosis" is a very apt term.
> And remember, chaos is the good outcome. The alternative is that you are a dancing monkey in a cage you won’t be smart enough to see.
I'd think we're actually going to see a split. Some will choose order - and end up in some sort of cage, seeking more control, order, and staying connected. Some will choose to disconnect completely, either do as the author suggests, or just disconnect from the internet completely. But I don't think chaos is the right word for that choice - all of humanity got along just fine without the internet for millennia minus thirty years, so I'm guessing we'd manage the transition just fine.
The internet will be around, it just won't be open anymore.
Already the trolls,etc were a nuicance enough that anonymous social-media,etc is starting to become outlawed. Sure people will hack,etc but the easy exploitation of today will be a fleeting thing once we patch the fundamentally broken stuff.
Remember how Windows machines were wormed in the early 00s? MS got their act together and today the primary target is via browsers or Excel-files,etc. The recent Linux RCE while bad and probably not alone, is still not the common case.
The history of human scientific discovery and innovation has been a repeated practice in humility, reinforcing over and over that we are not the center of the universe, and everything we hold of value is a transient condition of the times we exist within.
Put your hands up. The chain-lift is nearing its big peak.
Why has this been flagged...?
Hi, flagger here. This is incoherent rambling and top comment pretty much highlights the problem. Posts should hold some substance.
The top comment says there is substance?
I did say that, yes, but I also think that this should be flagged, because HN will otherwise run with the panic, 5 "rationalists" will show up and argue for 50 comments about how their LARP is certainly real, and, generally, the world will be worse off.
Which is not George's fault, but it is the fault of the uncaring machine that is people on the internet. George probably needs a hug and a friend that is grounded and sane. He definitely does not need more people fueling this fire.
(I do not know the guy. I'm just claiming this with fake authority)
https://en.wikipedia.org/wiki/False_dilemma
https://en.wikipedia.org/wiki/Edge_of_chaos
I didnt know George was such a good writer (and rapper!). Some people have all the talent
Even Isaac Newton fell victim to complex financial bubbles.
I think it degrades the level of discourse on HN when people insist on treating eschatonic navel-gazing as a significant contribution to a separate and ongoing debate.
Just because an insular group of people who share the same unsupported apocalyptic beliefs have labelled themselves as rationalists doesn't mean they are actually being rational; the actual debates around AI and its uses are very far away from the cult-like beliefs espoused here, but the focus turns to them again and again.
It's as if a board for discussing NASA's advances kept getting sidetracked by Raëlists. Discussing these beliefs as though they were serious positions is bad for the field, bad for the discourse, and bad for the writers themselves.
> Anthropic cannot wait until they have the leverage to ban all their users.
Huh? Their power is precisely in the users they have.
Let’s say they build a godlike superintelligence. Now they ban all their users. Let’s further assume that they are fine with all their users rotting and dying (easy to believe). Now what exactly is Anthropic going to do in that world? Invent Millennium Problems and solve them?
Users arent buyers. If my employer could fire everyone tomorrow and replace them with a claude subscription we’d be out the door in seconds.
This is exactly what happened when manufactures could offshore production. They didn't care (sure there was PR "caring" but they still outsourced) how it would impact their former workers or company towns. https://www.sfgate.com/business/article/levi-strauss-aims-to...
We need a government program to retrain knowledge workers to flip burgers and make beds.
You've made the mistake of thinking Anthropic just wants to sell everybody a subscription to their models, so everyone can make their little spreadsheets or whatever.
That's not what Anthropic wants. They plan to be The Company. The only one. The one that does everything and makes every decision.
> How what exactly is Anthropic going to do in that world?
Govern it. Every ones end up delegating decision to it, because it will better informed, and anyone using can it out-compete you.
I'm pretty confident all hyperscalers only need their users because they are RL environments for them, this likely will mutate as they reach a ceiling.
My prediction is once they reach this ceiling (or find a better RL environment), they will cancel free plans and shift towards the next scaling laws challenge (robotics, etc) by buying out whoever started early and repeating the playbook.
Canceling the paid plans would only make sense if they solve for control, which right now isn't yet clear who has it and in what capacity.
> Their power is precisely in the users they have.
False. There is a brilliant quote from The Blacklist series: "every information is worth something to someone". Their strength comes from the fact that they convinced everyone to willingly let user-facing computers collect data, write and execute arbitrary code on people's computers with a simple "trust me bro". In other words they have near complete monopoly over the world's infrastructure.
Until they ban their users, which is precisely my point.
My point is that their power comes from the data they already posses, which is likely not far off 100%, including personal and proprietary data.
> Now what exactly is Anthropic going to do in that world
Dario already said it according to multiple sources: he believes Anthropic will be the last surviving company in the world, where all the value will accumulate
Elon Musk has another vision: his AI is going to build the Dyson swarm around the Sun. Money doesn't matter in that world, only how many robots you have, and how much mass you can put into orbit.
> Dario already said it according to multiple sources: he believes Anthropic will be the last surviving company in the world, where all the value will accumulate
Given that LLMs are practically commodities and open models are a dime a dozen, this is pure insanity. There's no way there from here.
It's pretty easy to imagine them creating a new AI that is an order of magnitude better than the current models with no one else having access to that technology.
It's easier to imagine Google doing it. It's equally likely to imagine OpenAI (equally resourced) or Deepseek (pushing novel optimizations) doing it.
And it's pretty easy to imagine me creating a castle made of cake and ice cream.
That doesn't mean there's any actual mechanism for me to do so.
What you're talking about requires a genuine breakthrough in AI research, not just iterative improvements on what we've already got. By their very nature, those are not something that can be predicted or relied upon; you cannot even be guaranteed that a given type of breakthrough will ever happen.
It would be completely realistic if the US was the only player in AI. But that era is over.
LLMs are commodity. The ability to run them at scale is not.
Yes, and Anthropic doesn't have that ability and has to lease data centers from AWS and Google!
Neither of those is a reasonable belief; the two of them are either delusional, or lying.